Latent Dynamics · 2026-11-13

Rethinking Actions: Continuity-Aware Priors for World-Model Planning

Shaobo Yang, Zichen Ai, Ziyang Ma, Qian Yang, Bin Fang

arXiv:2611.16645PDFCode

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors propose LG-DKGP, a locality-gated deep-kernel Gaussian-process prior for visual world-model planning that learns action continuations from the temporal structure of expert demonstrations. Their planner uses a learned neural mean plus a temporal GP residual to form a continuity-aware prior, initializes CEM search with a training-derived exploration scale, and evaluates progress with a frozen world model.

Why it matters

The paper tests whether preserving expert action continuity (ordered actions vs shuffled) improves action proposals for world-model planning. It reports higher goal-directed planning success than several baselines, and isolates which components—prior conditioning, GP residual, gated attention, and search initialization—contribute to gains, including sensitivity to history length and planning horizon.

Method

  • LG-DKGP builds a continuity-aware action prior from executed action history using a locality-gated history encoder with both recency-biased local attention and global attention.
  • A neural mean predicts continuation across forecast leads, and a temporal Gaussian-process (GP) residual models structure left by the nonlinear mean.
  • Search samples candidate action chunks around the combined prior mean using CEM with an exploration scale derived from expert training action increments; a frozen visual world model scores progress toward a visual goal.
Abstract (from arXiv)

Actions in expert demonstrations exhibit temporal structure. An ordered-versus-shuffled analysis across five environments confirms this dependence. We ask whether action proposals that better reflect expert continuity can improve visual world-model planning. Rather than imposing generic smoothness, we learn continuations from the temporal structure of demonstrated actions. Our locality-gated deep-kernel Gaussian-process prior, LG-DKGP, combines recent and longer-range executed action history, predicts a neural continuation, and learns a temporal Gaussian-process (GP) residual. A training-derived exploration scale initializes search around the corrected mean; a frozen world model evaluates progress toward the goal. Training first fits the neural mean, then jointly optimizes the prior with the GP residual using expert future actions. The resulting planner achieves 87.07\% macro success across five simulated tasks spanning native and fragment-level goal attainment, compared with 71.03\% for INTACT, 61.66\% for PRISM, and 58.66\% for LeWM. Component interventions and expert-relative action-structure comparisons support demonstration-informed continuations and reveal task-dependent effects of visual conditioning, history length, and planning horizon.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-13 · Indexed by Pub-AI: 2026-11-13 UTC

preprint

Rethinking Actions: Continuity-Aware Priors for World-Model Planning

Shaobo Yang, Zichen Ai, Ziyang Ma, Qian Yang, Bin Fang

First announced by arXiv: 2026-11-13 · Original record ↗

arxiv
2611.16645
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-13 · First detected: 2026-11-13 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.16645v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Actions in expert demonstrations exhibit temporal structure. An ordered-versus-shuffled analysis across five environments confirms this dependence. We ask whether action proposals that better reflect expert continuity can improve visual world-model planning. Rather than imposing generic smoothness, we learn continuations from the temporal structure of demonstrated actions. Our locality-gated deep-kernel Gaussian-process prior, LG-DKGP, combines recent and longer-range executed action history, predicts a neural continuation, and learns a temporal Gaussian-process (GP) residual. A training-derived exploration scale initializes search around the corrected mean; a frozen world model evaluates progress toward the goal. Training first fits the neural mean, then jointly optimizes the prior with the GP residual using expert future actions. The resulting planner achieves 87.07\% macro success across five simulated tasks spanning native and fragment-level goal attainment, compared with 71.03\% for INTACT, 61.66\% for PRISM, and 58.66\% for LeWM. Component interventions and expert-relative action-structure comparisons support demonstration-informed continuations and reveal task-dependent effects of visual conditioning, history length, and planning horizon.