Driving · 2026-10-14
Directional Pullback Regularization for Long-Horizon Rollout of Latent World Models in Autonomous Driving
Shatadal Mishra, Ahmadreza Moradipari, Nejib Ammar
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study long-horizon open-loop ego-motion rollout for autonomous driving latent world models, noting non-holonomic constraints forbid instantaneous pure lateral motion. They propose Directional Pullback Regularization (DPR), a training-time method that defines a kinematics-aware anisotropic Riemannian metric in decoded space and pulls it back through the decoder Jacobian to shape latent transitions. DPR reduces 10-second final displacement error vs ULR and EPR across U∈{1,3,5,10}.
Why it matters
Latent world models can have small one-step loss yet accumulate kinematically inconsistent drift over long rollouts. DPR improves long-horizon rollout quality by adding a motion-aware geometric bias to latent transitions—without changing the encoder–dynamics–decoder backbone—using a directional penalty on decoded lateral motion.
Method
- Start from Unconstrained Latent Rollout (ULR) trained with prediction loss only; use self-fed multi-step rollout training with U∈{1,3,5,10} and evaluate on 10-second open-loop ego-motion rollouts.
- Introduce Directional Pullback Regularization (DPR): define a kinematics-aware anisotropic Riemannian metric in decoded state space that assigns higher cost to motion outside a feasible (non-lateral) subspace, then pull it back to latent space through the decoder Jacobian.
- Use Euclidean Pullback Regularization (EPR) as an ablation by setting β=0, removing directional preference, and compare against ULR/EPR to test whether directional structure is necessary.
Limitation
The authors state that β is selected by validation and its preferred value varies with training rollout length; they also note the pullback metric acts locally on decoded tangents and does not by itself guarantee global long-horizon accuracy or closed-loop stability, and the experiments are limited to ego-motion rollout.
Abstract (from arXiv)
Learned world models for autonomous driving operate in unconstrained latent spaces, yet the physical systems they represent are governed by strict non-holonomic constraints: ground vehicles cannot realize instantaneous pure lateral motion. We study long-horizon latent rollout modeling for autonomous driving and ask whether rollout quality can be improved by assigning a motion-aware geometry to latent transitions, rather than by redesigning the predictor architecture. Starting from an unconstrained latent rollout baseline, we introduce Directional Pullback Regularization (DPR), a training time method that defines a kinematics-aware anisotropic Riemannian metric in decoded state space and pulls it back through the decoder Jacobian to the latent manifold. The resulting latent-space metric assigns higher cost to latent directions that decode to lateral vehicle motion, while preserving the original encoder-dynamics-decoder backbone. This yields a clean three-way comparison among an Unconstrained Latent Rollout model (ULR), an Euclidean Pullback Regularization ablation (EPR), and the full directional method (DPR). We evaluate all models on open-loop ego-motion rollout over a common 10-second test horizon, using training rollout lengths of 1, 3, 5, and 10 steps. Across all four training regimes, DPR achieves lower 10-second final displacement error than both ULR and EPR. The strongest relative accuracy gain appears for models trained with 3-step rollout supervision, where DPR reduces the 10-second final displacement error by approximately 44 percent relative to ULR. The strongest overall control-relevant tradeoff is obtained for longer rollout-trained models, especially at 10 steps, where DPR simultaneously improves displacement error, lateral motion consistency, and forward motion preservation relative to both ULR and EPR.
Related papers
- What Do Latent Predictive Vehicle Representations Retain? Measuring State, Geometry, and Local Response
- DriveWorld3D Evidence Constrained 3D World Modeling and Generative Simulation for Real-World Autonomous Driving
- Beyond Planning: Differentiable Control-in-the-loop End-to-End Autonomous Driving
- GeoV2V: Geometry-Grounded Video Diffusion Model for Driving Scene Generation
- BridgeGuard: Explicit Safety Drift for Diffusion-based Autonomous Driving
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-14 UTC
Directional Pullback Regularization for Long-Horizon Rollout of Latent World Models in Autonomous Driving
Preprint posted: 2026-10-12 · Original record ↗
- arxiv
- 2610.16139
Explicit source versions (1)
- v1 · 2026-10-12
- Preprint available
Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Learned world models for autonomous driving operate in unconstrained latent spaces, yet the physical systems they represent are governed by strict non-holonomic constraints: ground vehicles cannot realize instantaneous pure lateral motion. We study long-horizon latent rollout modeling for autonomous driving and ask whether rollout quality can be improved by assigning a motion-aware geometry to latent transitions, rather than by redesigning the predictor architecture. Starting from an unconstrained latent rollout baseline, we introduce Directional Pullback Regularization (DPR), a training time method that defines a kinematics-aware anisotropic Riemannian metric in decoded state space and pulls it back through the decoder Jacobian to the latent manifold. The resulting latent-space metric assigns higher cost to latent directions that decode to lateral vehicle motion, while preserving the original encoder-dynamics-decoder backbone. This yields a clean three-way comparison among an Unconstrained Latent Rollout model (ULR), an Euclidean Pullback Regularization ablation (EPR), and the full directional method (DPR). We evaluate all models on open-loop ego-motion rollout over a common 10-second test horizon, using training rollout lengths of 1, 3, 5, and 10 steps. Across all four training regimes, DPR achieves lower 10-second final displacement error than both ULR and EPR. The strongest relative accuracy gain appears for models trained with 3-step rollout supervision, where DPR reduces the 10-second final displacement error by approximately 44 percent relative to ULR. The strongest overall control-relevant tradeoff is obtained for longer rollout-trained models, especially at 10 steps, where DPR simultaneously improves displacement error, lateral motion consistency, and forward motion preservation relative to both ULR and EPR.