Digest

Reset

1167 papers

2026-10-14 · arXiv:2610.16166

Esports World Model Dataset

Weiyu Ma, Zeyuan Gong, Xuhui Liu et al.

EWM (Esports World Model Dataset) is a multiplayer, multi-genre dataset built from human gameplay in StarCraft II, Dota 2, and Counter-Strike 2. It provides replay-clock-aligned, player-specific rendered views paired with game-native actions, structured state, and events, supporting research on shared dynamics under different information constraints (fog of war, shared team vision, first-person POV).

AI summary
2026-10-14 · arXiv:2610.16242

Same Goal, Different Words: Spatial Instruction Sensitivity in World-Action Models

Ali-Adeeb Abbas, Esmaeil Seraj, Behrad Toghi et al.

The authors test spatial instruction sensitivity in world-action models by running 864 rollouts across four RoboLab scenes with the same initial states and goals but three equivalent goal wordings (direct, target-first, reference-first). Describing relations via the reference object (RF) reduces reaching stable placement vs target-first (TF) by 12.2 percentage points on average. They also show scoring can credit moving the wrong object and that adding images can disrupt text-only instruction interpretation.

DrivingAI summary
2026-10-14 · arXiv:2610.16302

RAR: Rate-Adaptive Rollout for Time-Consistent Interactive World Models

Yaoyang Liu, Dongyu Yan, Yihua Du et al.

RAR (Rate-Adaptive Rollout) addresses temporal-rate coupling in interactive world models by enabling controllable rollout temporal rates. It uses Stride-RoPE to align latent frames sampled at different strides on a shared physical-time coordinate, supports multi-stride training via multi-stride video sampling, and allows online rate switching by varying stride during rollout without discarding history. Experiments report temporally consistent multi-rate rollout and online rate switching across Doom, CS2, and FPV.

2026-10-14 · arXiv:2610.16355

PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation

Xinyu Guo, Chao Jing, Li Yi et al.

PINT is a unified robot manipulation model that shares a learned chunk-level “Predictive Intent” across a vision-language model, a video world model, and an action expert. It inserts learnable Intent Tokens into the VLM, uses them to condition only the VGM (VGM remains active at inference), and trains with a two-stage schedule. The authors report strong ID and improved OOD generalization without embodied pretraining.

RoboticsAI summary
2026-10-14 · arXiv:2610.16365

World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models

Shinseong Kim, Jaeseok Jeong, Jianlong Fu et al.

Physical Registry introduces temporally grounded registry slots inside a video DiT to hold predictive future dynamics. The registry is initialized from an observation-conditioned JEPA forecast, co-evolves with video tokens, and receives future-representation supervision, making predictive dynamics an explicit readable model state. It improves Physics-IQ Verified and WorldModelBench physical adherence, and supports Physical Registry Selector (PRS) for candidate selection during denoising.

2026-10-14 · arXiv:2610.16443

RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations

Yuanpeng He, Ceyao Zhang, Fangjing Li et al.

RAP-WAM learns world-action models for control by combining predictive rehearsal with action perturbations. It uses a future-supervised action interface and directional query attention, then measures video prediction changes under perturbed actions to derive spatiotemporal credit for visual and action supervision. The method avoids extra region scoring/probes and is trained from a video-pretrained backbone. It improves RoboDojo success rates and achieves strong real-robot task-macro success with agent-refined, task-conditioned policy selection.

Latent DynamicsAI summary
2026-10-14 · arXiv:2610.16517

Trust-Calibrated VLA Policy Post-Training with Evolving Imagination

Wenzhan Li, Yiran Qin, Heng Zhou et al.

The authors propose MERL, a trust-calibrated VLA policy post-training method that couples online simulator evolution (a visual world model plus reward proxy head) with two-level trust control. Stage-level trust schedules the imagined-data ratio and rollout horizon, while chunk-level trust selects and weights imagined replay. On perturbed LIBERO-PRO, MERL attains 79.7% average success and improves high-threshold learning speed by 10.9%.

RoboticsAI summary
2026-10-14 · arXiv:2610.16558

G$^2$WAM: World Action Models for Efficient and Geometry Grounded Image-Goal Navigation

Ke Gui, Chenghao Shi, Ziyue Wang et al.

G$^2$WAM (Geometry Grounded World Action Model) targets image-goal navigation where the goal is often out of view. It anchors current, goal, and history frames in a shared scene frame using a learned scene-coordinate localization module, providing stable relative pose. Using the world-action-model paradigm, it jointly predicts latent future states and canonical waypoint increments entirely in latent space, avoiding CEM-style planning and pixel reconstruction.

DrivingAI summary
2026-10-14 · arXiv:2610.16634

Language2Drive: Do Vision-Language-Action Models Really Follow the Instructions in Autonomous Driving?

Zihan You, Zehua Fan, Wenqi Lyu et al.

Language2Drive tests whether vision-language-action driving models follow natural-language instructions in autonomous driving by using Paired Counterfactual Units (PCUs) that keep the driving scene fixed while changing only the instruction. Across environments and multiple models, the authors report a large gap between individual instruction completion and paired counterfactual success, with 93.2% of successful rollouts lacking a successful opposite-instruction counterpart.

DrivingAI summary
2026-10-14 · arXiv:2610.16641

What Limits Cross-Task Generalization in World Action Models?

Chen Zhao, Zipeng Ye, Yihan Lin et al.

The authors show that cross-task generalization fails for world action models (WAMs) because decoded actions from world-model-predicted futures collapse, unlike actions recovered from recorded futures. They introduce AoE (a frozen, camera-space inverse-dynamics model) to evaluate and supervise. Using VITA loss (AoE-guided action discrepancy), execution success of generated futures increases from near zero to 47.0% across two world models and three manipulation benchmarks.

DrivingAI summary
2026-10-14 · arXiv:2610.16763

Relational Learning as World Modeling

Zheyu Zhang, Gjergji Kasneci

The authors introduce TabWM and WorldRel to learn relational dynamics in-context. TabWM predicts the sparse structured edits caused by an action in an unseen relational world using observed executions as context, with no parameter updates. Across 512 held-out worlds, complete-edit F1 rises from 0.016 with no context to 0.685 with 64 executions. Recursive edit predictions enable model-based planning with broader search improving realized return.

Latent DynamicsCode availableAI summary
2026-10-14 · arXiv:2610.16782

LIBERO-Industry: Benchmarking Robotic Manipulation under Industrial Process Constraints

Ming Shin Cindy Sung, Duan Leijiahao, Haoxiang Wang et al.

LIBERO-Industry is a benchmark built on LIBERO to evaluate generalist robot policies under industrial process constraints by translating them into controllable conditions and measurable success criteria. It includes three task suites (S2D loading, D2S unloading, D2D transfer) totaling 216 tasks with variations in object geometry, conveyor speed, interaction-window duration, and geometric acceptance region size. Results show performance degrades as constraints accumulate and combine.

DrivingAI summary
2026-10-14 · arXiv:2610.16967

Proactive Physical AI through Continuous Observation and Timely Intervention

Zhenhua Yang, Jiafeng Liang, Xingyu Zhou et al.

The authors study Proactive Physical AI, where an AI system continuously observes the physical world, detects when intervention is needed, and initiates timely responses or device actions without a predefined task. They develop ProAgent with StreamingJev (proactive timing) and a Response Model (person/device responses), plus an automated video-to-supervision pipeline for training. Experiments show better intervention timing and fewer false triggers than a source-backbone baseline, including real-robot results.

DrivingAI summary
2026-10-14 · arXiv:2610.16969

UniLAP: A Unified Latent Action Policy for Cross-Embodiment Manipulation

Lei Yao, Qiao Feng, Mutian Tong et al.

UniLAP introduces a unified continuous latent-action policy for cross-embodiment manipulation. It models latent actions with flow matching distilled from a frozen LAM teacher, “bypassing reliance on future frames” during inference. It uses label-free motion supervision from cross-frame correspondences in frozen DINOv2 to isolate kinematic cues. With an embodiment-agnostic latent manifold and lightweight adaptation, the authors report improvements on hand pose forecasting, single-arm manipulation, humanoid manipulation, and real-world deployment.

Latent DynamicsAI summary
2026-10-14 · arXiv:2610.16984

From Suboptimal to Optimal: One-Step Latent Flow Policies for Offline Reinforcement Learning

Shenghua Lu, Zixuan Yang, Rongxu Zhao et al.

MeanFlow2Better (MF2B) reformulates offline RL as generative decision-chunk optimization in a high-level latent decision space. It uses a chunk-level latent decision encoder plus a shared conditional mean velocity field with two branches: Gaussian sources to positive behavioral regions and suboptimal decisions to high-value regions. Refinement is done in one-step Euler integration over Δt=1 (no multi-step iterative sampling/backprop through time).

Latent DynamicsAI summary
2026-10-14 · arXiv:2610.16985

AUGUstus: Autonomous Driving VLA Model with Unified Generation and Understanding

Yuqi Ye, Caigao Jiang, Zhuonan Wang et al.

AUGUstus is an autonomous-driving VLA that unifies diffusion-based action generation with autoregressive CoT understanding in one transformer. It uses “AUGUstus-mode” to generate actions first and make CoT supervision training-only, enabling low-latency inference. AUGUstus-GRPO jointly optimizes action and CoT, improving planning performance.

DrivingAI summary
2026-10-14 · arXiv:2610.17059

PatientWM: Towards Generalizable Clinical World Models via Unified Patient States

Ziwen Kan, Jie Peng, Bo Yan et al.

PatientWM is a clinical world-model framework that learns a unified predictive patient state from heterogeneous observations using availability-aware encoders. It conditions next-state prediction on the current state, supplied clinical action, and an ordered bounded history of preceding latent patient-state and action pairs, using typed decoders for task-specific clinical readouts. Experiments on MIMIC-III, MIMIC-IV, and three external datasets report superior clinical prediction, multi-step forecasting, and action-planning evaluation, with transfer/generalization via frozen-state reuse.

DrivingAI summary
2026-10-14 · arXiv:2610.17114

D-World: An Explicit World-Language-Action Model with Recursive Self-Improvement under Dual Control

Xuan Wen, Baichao Wang

D-World is an explicit world-language-action model using Sense-Plan-Act and an executable digital-twin “world” written from robot observations. Plan searches and commits action chunks in simulation, then revises the twin via measurement-grounded recursive self-improvement (RSI): revisions are accepted only if replaying the latest chunk in the revised twin does not increase measured residuals beyond tolerances. Reconstruction is used only for structural mismatches. The LLM stays frozen while the world improves.

DrivingAI summary
2026-10-14 · arXiv:2610.17131

SonicWorld: Source-Centric Latent Dynamics for Spatial Audio Reasoning

Chenlong Wei, Hanling Wang, Mingjin Lou et al.

SonicWorld targets dynamic source-specific spatial audio reasoning by maintaining persistent source states from FOA recordings, then projecting source-state sequences into an Omni LLM for spatial question answering. The paper introduces Sonic-Bench (2.8K FOA clips, 28.8K QA pairs, 480 simulated indoor scenes). On Sonic-Bench, SonicWorld improves direction accuracy and reduces azimuth MAE versus Spatial-Omni; it also reports zero-shot gains on Spatial LibriSpeech.

Latent DynamicsAI summary
2026-10-14 · arXiv:2610.17141

MARS: Systematic Multi-Level Acceleration for World Action Models

Zengyuan Zuo, Yongchang Zhang, Guangyu Zhao et al.

MARS is a training-free, plug-and-play framework for speeding up World Action Models by removing non-uniform redundancy at three levels: selectively skipping low-impact Transformer attention, adaptively accelerating diffusion denoising based on motion granularity (one-step generation for coarse motions; residual reuse for fine motions), and speculatively reusing predicted future visual tokens across consecutive queries with online verification. The authors report up to 4.00× end-to-end speedups while maintaining task success near the original policy.

DrivingAI summary