Latent Dynamics · 2026-11-02

Hierarchical World Models with Implicit Dynamics

Gaoyue Zhou, Yvonne Wu, Zichen Jeff Cui, Nicolas Ballas, Mido Assran, Lerrel Pinto, Yann LeCun

arXiv:2610.42146PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

ID-WM (Implicit Dynamics World Models) is a hierarchical world model for long-horizon zero-shot planning. A high-level diffusion model generates neighboring reachable latent states using smooth Gaussian noise (implicit dynamics), while a deterministic low-level inverse dynamics model grounds those transitions into executable actions. The authors report 38.5% improvement over state-of-the-art policy and diffusion planners and 135% over single-level world model MPC, with videos at https://id-wm.github.io.

Why it matters

The paper targets two known issues in world-model planning: search complexity growing with horizon and compounding prediction errors in long rollouts. ID-WM’s key idea is to decouple reachability from execution details, using implicit high-level dynamics for subgoal discovery and inverse dynamics for local action grounding, with the goal of enabling robust long-horizon planning without task-specific value functions or pre-trained policies.

Method

  • Hierarchical model: a high-level diffusion-based world model outputs states within a fixed Δ-step radius (neighboring state generator) using initial Gaussian noise as an implicit high-level action; a deterministic low-level model acts as an inverse dynamics solver to produce executable action sequences.
  • Planning as MPC with interleaved stages: (1) high-level subgoal generation via CEM over latent noise to sample feasible latent subgoals, constrained/regularized to avoid adversarial latent regions; (2) low-level subgoal realization via CEM to search actions that minimize distance to the target subgoal state.
  • Low-level inverse dynamics is implemented by a transformer-based deterministic model trained to predict the next state; given a target state, it uses planning over low-level actions to minimize final imagined-state distance to the target.
Abstract (from arXiv)

Planning with world models enables test-time adaptation for embodied agents, offering superior generalization over direct policy learning. However, long-horizon tasks remain hindered by cumulative errors and search complexity. Existing hierarchical methods often use rigid macro-actions or latent skills, leading to brittle representations and limited fine-grained control. We introduce ID-WM (Implicit Dynamics World Models), a hierarchical world model that decouples state-space reachability from local control. Our framework utilizes a high-level model to sample feasible transitions across the state manifold and a low-level world model as an inverse dynamics solver to ground these transitions into precise actions. This allows the planner to identify reachable subgoals without being constrained by low-level execution details. By prioritizing local physical feasibility over global behavioral patterns, ID-WM synthesizes novel long-horizon plans from short segments, transcending the specific behavioral modes seen during training. We demonstrate its effectiveness across three navigation and manipulation suites, achieving a 38.5% improvement over state-of-the-art policy and diffusion-based planners. Crucially, by leveraging this hierarchical structure to mitigate compounding errors, our approach yields a 135% performance gain over single-level world model MPC, particularly in complex, unseen configurations for long-horizon tasks. Videos can be viewed at https://id-wm.github.io

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-30 · Indexed by Pub-AI: 2026-11-02 UTC

preprint

Hierarchical World Models with Implicit Dynamics

Gaoyue Zhou, Yvonne Wu, Zichen Jeff Cui, Nicolas Ballas, Mido Assran, Lerrel Pinto, Yann LeCun

Preprint posted: 2026-10-30 · Original record ↗

arxiv
2610.42146
Explicit source versions (1)
  • v1 · 2026-10-30
  1. Preprint available

    Source date: 2026-10-30 · First detected: 2026-11-02 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Planning with world models enables test-time adaptation for embodied agents, offering superior generalization over direct policy learning. However, long-horizon tasks remain hindered by cumulative errors and search complexity. Existing hierarchical methods often use rigid macro-actions or latent skills, leading to brittle representations and limited fine-grained control. We introduce ID-WM (Implicit Dynamics World Models), a hierarchical world model that decouples state-space reachability from local control. Our framework utilizes a high-level model to sample feasible transitions across the state manifold and a low-level world model as an inverse dynamics solver to ground these transitions into precise actions. This allows the planner to identify reachable subgoals without being constrained by low-level execution details. By prioritizing local physical feasibility over global behavioral patterns, ID-WM synthesizes novel long-horizon plans from short segments, transcending the specific behavioral modes seen during training. We demonstrate its effectiveness across three navigation and manipulation suites, achieving a 38.5% improvement over state-of-the-art policy and diffusion-based planners. Crucially, by leveraging this hierarchical structure to mitigate compounding errors, our approach yields a 135% performance gain over single-level world model MPC, particularly in complex, unseen configurations for long-horizon tasks. Videos can be viewed at https://id-wm.github.io