Driving · 2026-10-12
Sampling-based Kinodynamic Planning with World Models
Yitian Gao, Hanlan Yang, Zachary Kingston
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose kinodynamic sampling-based planning with a frozen visual world model (DINO-WM) used as a latent-space propagator. They replace myopic CEM/MPPI with latent tree search (nearest-neighbor-guided RRT or projection-based KPIECE) plus short CEM “polishing,” using GPU-native batched expansion. They evaluate on four simulated domains and a real block-pushing task.
Why it matters
The paper targets long-horizon tasks where shooting optimizers can stall because progress may require motion not rewarded by the objective. By using sampling-based tree search in the world-model latent space, the planner can explore intermediate states that myopic optimizers may not sample, and can terminate early when a latent goal is found.
Method
- Receding-horizon planning with a frozen off-the-shelf world model (DINO-WM) used as the forward propagator in latent space; execute only a short action prefix each iteration and re-plan from the newly observed state.
- Latent tree search: nearest-neighbor-guided RRT (Voronoi-biased expansion using a pooled spatial feature distance) or KPIECE-style exploration (projection into a low-dimensional exploration space, grid-based frontier prioritization).
- Batched latent expansion for efficient GPU utilization, followed by “trajectory polishing” with a short CEM optimization on the extracted action sequence.
Limitation
The goal metric biases search and is described as “the weakest link in the pipeline,” and because the goal criterion is based on the latent metric, reaching the latent goal does not guarantee success on the ground-truth system.
Abstract (from arXiv)
Visual world models predict the consequences of actions from images, but planning with them is typically done by optimizers such as CEM and MPPI. However, these optimizers stall on long-horizon tasks where progress requires motion the objective does not reward, such as moving around a block before pushing it. This paper revisits kinodynamic sampling-based motion planning for world models, using the learned model as the propagator of a tree search in latent space. Our key insight is that the sampling strategies devised for high-dimensional search in configuration space apply directly to the latent space of a world model, where they rapidly expand through intermediate states that myopic optimizers never sample. We present two exploration strategies for the high-dimensional latent space, nearest-neighbor-guided RRT and projection-based KPIECE, with GPU-native batched expansion for efficient device utilization. We evaluate with an off-the-shelf DINO-WM on four simulated domains and a real-world block-pushing task. Our approaches match or exceed the success rate of CEM and MPPI at up to 13.8x lower planning time in simulation, and improve the success rate by 20% on the real robot.
Related papers
- PILOT: Guiding Long-Horizon World-Model Planning for Robot Manipulation with Visual Subgoals and Policy Priors
- The Planning Limits of Latent World Models
- Vector-Quantized World Model: Latent Feedback Motion Planning for Robot Manipulation
- Roz: A Taste Test for World Model Latents
- MARS: Systematic Multi-Level Acceleration for World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-09 · Indexed by Pub-AI: 2026-10-12 UTC
Sampling-based Kinodynamic Planning with World Models
Preprint posted: 2026-10-09 · Original record ↗
- arxiv
- 2610.12910
Explicit source versions (1)
- v1 · 2026-10-09
- Preprint available
Source date: 2026-10-09 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Visual world models predict the consequences of actions from images, but planning with them is typically done by optimizers such as CEM and MPPI. However, these optimizers stall on long-horizon tasks where progress requires motion the objective does not reward, such as moving around a block before pushing it. This paper revisits kinodynamic sampling-based motion planning for world models, using the learned model as the propagator of a tree search in latent space. Our key insight is that the sampling strategies devised for high-dimensional search in configuration space apply directly to the latent space of a world model, where they rapidly expand through intermediate states that myopic optimizers never sample. We present two exploration strategies for the high-dimensional latent space, nearest-neighbor-guided RRT and projection-based KPIECE, with GPU-native batched expansion for efficient device utilization. We evaluate with an off-the-shelf DINO-WM on four simulated domains and a real-world block-pushing task. Our approaches match or exceed the success rate of CEM and MPPI at up to 13.8x lower planning time in simulation, and improve the success rate by 20% on the real robot.
