Digest

Reset

1180 papers

2026-10-01 · arXiv:2609.38948

DrivingBench: Can Vision-Language Models Drive a Toyota Corolla?

Aditya Ramabadran, Simon Mahns, Tobias Gessler

DrivingBench is a benchmark intended to test general-purpose vision-language models as closed-loop drivers of a real 2022 Toyota Corolla. Models view camera frames and directly command steering and velocity via an MCP tool interface; latency is part of the task because the car continues while the model thinks. GPT-6 Astra is the only model to finish, reaching the finish zone on its second attempt (100% progress).

DrivingAI summary
2026-10-01 · arXiv:2609.38984

Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination

Xinling Xie, Haodong Wang, Jiazhi Mi et al.

Sparse-WAM is a training-free framework for action-guided sparse imagination in world-action models. It uses action-to-future attention to select frame-specific core tokens plus shared spatial anchors, then uses Pilot to execute these selections efficiently via lightweight attention profiling and cross-step reuse, accelerating WAM inference while largely preserving performance.

DrivingAI summary
2026-10-01 · arXiv:2609.39056

SteerQuant: Steering Quantization Error with Action-Guided Scaling in World-Action Models

Yunhan Wang, Haodong Wang, Zhiming Liu et al.

SteerQuant is a 4-bit post-training quantization method for world-action models. It steers quantization errors using an Action-Impact Stream Map so that errors are prioritized toward computations with less effect on final actions. It coordinates shared channel scaling and stream-specific activation modulation across denoising steps, and uses Rudder to fuse scaling into low-bit kernels for efficient inference.

DrivingAI summary
2026-10-01 · arXiv:2609.39101

Beyond Prediction: Steering VLM Agents with Retrospective World Modeling

Yongjiang Liu, Jie Zhang, Haoyue Zhang et al.

The authors propose Retrospective World Modeling (RWM) for VLM agents: instead of only predicting future states from actions, the agent also infers a retrospective attribution P(â_t|s_t,s_{t+1}) for which action best explains an observed transition. They define a Self-Consistency Reward (SCR) that scores how consistent the executed action is with retrospective explanation, and integrate it into PPO for dense intrinsic feedback and improved robustness/generalization over prospective-only world modeling baselines.

AI summary
2026-10-01 · arXiv:2609.39128

GeoGAT: Bidirectional Temporal Sampling Meets Hierarchical Graph Attention for Global Video Geo-localization

Junchao Cui, Xuanzi Ma, Wenqi Shi et al.

GeoGAT addresses global video geo-localization with (1) bidirectional temporal sampling that extracts forward and offset-reversed frame sequences, (2) a graph-attention hierarchy classifier over a predefined geographic hierarchy graph, and (3) dual constraints to eliminate cross-hierarchy conflicts. They build and release GeoGAT10k (9,720 multi-shot edited videos) and report state-of-the-art across city/state/country/continent on CityGuessr68k and GeoGAT10k.

2026-10-01 · arXiv:2609.39135

Asking the World: Generalist Physical Reasoning through Agentic World Modeling and Probing

Shenxiang Zeng, Chen Yang, Peiyao Chen et al.

Asking the World (ATW) is a training-free physical reasoning agent that builds an executable world from video (World Modeling) and then probes it with simulations and interventions to obtain evidence for questions (World Probing). It uses PolyWorld Engine, a Warp-based multiphysics simulator, and CEM system identification to calibrate task-relevant dynamics. Using Gemini-3-Flash, ATW improves accuracy on CLEVRER and ContPhy and on three real-world tasks.

Latent DynamicsAI summary
2026-10-01 · arXiv:2609.39147

MindWorldBench: Evaluating Mental-State-to-Behavior Reasoning in Image-to-Video Generation

Ruiqi Li, Xuanyi Liu, Sijia Li et al.

MindWorldBench evaluates whether image-to-video models generate mental-state-conditioned behavior (belief/desire/perception) using Zero-Action Prompting and counterfactual prompt pairs (744 prompts over 372 images). The authors report that across 11 models, visual quality and commonsense plausibility stay relatively strong, but intention accuracy and world-state maintenance are much weaker, revealing frequent misalignment between latent mental states and generated actions (e.g., Omniscient Bias).

2026-10-01 · arXiv:2609.39151

Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey

F. Olivia Fan, Oliver Obst

The authors study robot air-hockey defense with temporary puck tracking loss and distil a DreamerV3 history-dependent teacher into compact recurrent policies with a 64D state. Across five seeds, a purely linear recurrent update (k=0) matches a GRU-64 and the teacher through 400 ms blackout, while increasing nonlinear innovation rank (k=1,2,4) gives no measured benefit. The linear model uses fewer recurrent parameters and less computation than GRU-64.

DrivingAI summary
2026-10-01 · arXiv:2609.39179

LocoWM: High-Precision Locomotion through World-Model-Guided Residual Adaptation

Zijie Zhao, Shengqian Chen, Xiaoxu Wang et al.

LocoWM is a world-model-guided residual adaptation framework for high-precision locomotion. A base policy provides command-following actions, and an action-conditioned world model predicts future task substates from proprioceptive history and the base action. A residual adapter uses the predicted sequence to output additive action corrections. Two-stage training freezes locomotion and dynamics before training the adapter for precision.

DrivingAI summary
2026-10-01 · arXiv:2609.39182

MEND: Label-Free Detection, Localisation, and Correction of Latent Hallucination in World Models

Ali J Alrasheed, Aryan Yazdan Parast, Basim Azam et al.

The authors identify latent hallucination in latent world models: a predicted next latent can decode to a scene that never occurs, with error that is silent and compounds in autoregressive rollouts. They propose MEND, a label-free inference-time detect–localise–correct loop using a single conditional score network trained by denoising score matching on real transitions. On Wall and PointMaze, MEND detects hallucination (AUROC up to 0.80) and localises it (per-token AUPRC up to 0.87), and correction reduces latent error.

Latent DynamicsAI summary
2026-10-01 · arXiv:2609.39195

Uruqi: Learning Spatial Cognition from Visual Experience

Shichao Li, Meiqi Wang, Fei Su et al.

Uruqi trains vision-language models with dense multi-turn supervision on continuous visual experience. Each training episode is compiled from synthesized motif-driven camera trajectories into interleaved tasks: self-motion tracking (M1), persistent object mapping (M2), and operations over spatial state (M3). URUQI Syn-8B improves accuracy from 15.84% to 47.73% on the Uruqi benchmark; URUQI-SI-Mix-8B reaches 50.41% and transfers to external spatial benchmarks using only synthesized training data.

DrivingAI summary
2026-10-01 · arXiv:2609.39235

The Planning Limits of Latent World Models

Ali Alrasheed, Basim Azam, Naveed Akhtar

The authors report that latent action-conditioned world models guide action selection reliably only when the goal is within, or slightly beyond, the imagined rollout. With five-step rollouts, targets 5–10 control steps ahead are reliable, while typical tasks require 16–53 steps. The limit persists even with perfect prediction: simulator-based success drops from 92% to 41% when the target moves from 5 to 20 steps ahead. Planning needs longer imagination or closer subgoals.

Latent DynamicsAI summary
2026-10-01 · arXiv:2609.39245

ReWAM: Reciprocal World Action Models for Interactive Autonomous Driving

Benshan Ma, Pei Liu, Ruiguo Zhong et al.

ReWAM is a game-theoretic world action modeling framework for interactive autonomous driving that explicitly models reciprocal influence between the ego agent and other agents. It uses a Level-k response hierarchy with role-specific ego/other Action DiTs that exchange strategy tokens via cross-agent attention, grounded in a shared future-world latent representation. The model is learned from demonstrations using conditional flow matching.

DrivingCode availableAI summary
2026-10-01 · arXiv:2609.39324

MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies

Jingqiu Wang, Yan Wang

MotionWeave is a motion-centric future-dynamics framework for vision-language-action action-chunk prediction. It uses AIMG to form horizon-specific queries that localize interaction regions from current visual tokens, supervised with future robot-arm masks via KL loss during training. HRC composes adjacent-horizon interaction differences into motion cues injected into action tokens via a gated residual. On six MetaWorld tasks it reports 75.3% average success, +8.6% over π0.

DrivingCode availableAI summary
2026-10-01 · arXiv:2609.39388

UniWAM Technical Report: Unified Mobile Manipulation via Mixed-Stream World-Action Modeling and Manipulation Anchor Pose Supervision

Wei Xue, Keliang Liu, Mingzhang Cui et al.

UniWAM is a unified mobile-manipulation world-action model that models navigation and manipulation as two independently sampled streams over a shared backbone. It uses separate action encoders and output heads, enabling either stream-only inference or both streams in one batch. The paper introduces Manipulation Anchor Pose (MAP) supervision and constructs MAP-Data (1.5M+ episodes, 7,500 hours) from large-scale 3D scenes to supervise where to stop and how to orient for manipulation.

DrivingCode availableAI summary
2026-10-01 · arXiv:2609.39441

CAST: Causal Advantage-Structured Training with Spatially Grounded Compositional Rewards for Diffusion Models

Shu Yu, Chaochao Lu

CAST is an RL fine-tuning method for pretrained diffusion image generation models (flow-matching). The authors (1) set the SDE stochastic window from each model’s denoising trajectory, (2) decompose prompts with Causal Scene Graphs into independently verifiable atoms and score them separately, and (3) project signed atom-level advantages into pixel space via teacher-forced attention to spatially weight the SDE policy objective.

2026-10-01 · arXiv:2609.39601

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

Qize Yu, Lianrui Fan, Boyu Chen et al.

GroundingPI is a 4B grounding foundation model that outputs points and boxes as quantized coordinate tokens in a shared vocabulary. Trained with multimodal/spatial pretraining, supervised fine-tuning, and GRPO reinforcement learning, it improves grounding accuracy across 34 benchmarks and transfers to physical-intelligence tasks (autonomous driving and robot manipulation) as a visual backbone.

Latent DynamicsCode availableAI summary
2026-10-01 · arXiv:2609.39604

Why Do Conventional World Models Fail to Learn Cellular Automata?

Shaoyang Guo, Ziming Liu

The authors test conventional world models (CNNs, transformers, diffusion) on cellular automata and find they often predict many pixels correctly but rarely complete exact long-horizon rollouts. They attribute failure to three inductive-bias gaps: spatial locality, temporal locality, and temporal stability. Simple information-flow changes (2D rotary positions, momentum induction, causal freezing) lift rollout success dramatically on Game of Life and unseen rules.

Latent DynamicsCode availableAI summary
2026-10-01 · arXiv:2609.39685

RoboCoach: World Models as Active Coaches for Compositional Robot Skills

Jiajun Liu, Yifan Chen, Yichao Liu et al.

RoboCoach is a world-model-guided coaching framework for compositional long-horizon robot skills. It uses a Route–Imagine–Diagnose–Improve (RIDI) loop: imagined rollouts in CoachWorld route control through reusable skill experts, a progress judge attributes the first unresolved subtask on timeouts, and a scorecard selects targeted subtask demonstrations and updates only the corresponding expert adapters.

RoboticsAI summary
2026-10-01 · arXiv:2609.39727

OverForge: Reasoning Through Strategies and Tactics Helps Cooperative Lifelong Adaptation

Oana Madalina Fron, Ojas Shirekar, Chirag Raman

OverForge is a training-free hierarchical architecture for cooperative language-model agents. It separates strategic reasoning over roles/divisions of labour from tactical reasoning over actions inside each agent’s private, partner-conditioned world model, coupled by a metacognitive Prefrontal Cortex Module that imagines consequences and commits when confident. In OvercookedV2, it delivers 7 soups in a connected kitchen vs 3 for flat LLM baselines.

JEPAAI summary