Digest

Reset

1167 papers

2026-10-16 · arXiv:2610.21798

Cross-Embodiment Causal Transfer from Human Videos to Robot Manipulation

Xingyu Zhang, Yingchao Li, Peizheng Guo et al.

CBST (Causal-Beam-Guided Subtask Transfer) transfers cross-embodiment knowledge from human manipulation videos to robot world-action models. It learns manipulation-related representations from human videos, grounds them into task-level semantic variables using robot demonstrations, then uses causal-beam constraints to select context-dependent retained dependencies. These semantic conditions let the robot generate actions from its own observations without human–robot motion correspondence. Experiments show improved task success over world-action model baselines.

DrivingAI summary
2026-10-16 · arXiv:2610.21910

CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness

Dongjie Cheng, Ruifeng Yuan, Ran Wei et al.

CoWorld Reasoner is an agentic multimodal reasoning framework that uses a Video World Model (VWM) as a unified engine for visual imagination. A Video World Model-Driven Harness exposes view change, interpolation, continuation, and intervention through tools/skills and guides a VLM to decide when and how to imagine and interpret generated video. The authors add VLM-VWM Harness Co-Evolution that refines tools/skills/execution code via feedback while keeping model weights fixed. It improves over direct VLM reasoning on seven benchmarks.

2026-10-16 · arXiv:2610.21915

What Should Robot Policies Learn to Predict? Rethinking Supervision Targets in Vision-Language-Action Learning

Haitao Shen, Hanyu Liu, Yihan Lin et al.

The authors build a unified vision-language-action training framework with a sample-level router that assigns each training sample to either action prediction or an auxiliary latent-action (LA) or future-vision (FV) objective. Under the same fixed training budget on LIBERO, CALVIN, and four real-robot tasks, suitable LA/FV supervision outperforms action-only training, even when action prediction receives half or fewer of the training samples.

DrivingAI summary
2026-10-16 · arXiv:2610.21922

TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control

Menghao Yin, Jiaju Yin, Xiyue Dong et al.

TS-WAM is a lightweight teacher-student vision-language robot control model that learns temporal latent actions and predicts future visual features. It uses vlm-jepa (EMA target branch for future-aware latent-action supervision) coupled with delta-lwm (latent action-conditioned one-step feature-delta prediction) and a flow-matching action head to generate actions from predicted future features. Training is end-to-end on robot demos plus action-free human videos; the EMA teacher is pruned at inference.

RoboticsAI summary
2026-10-16 · arXiv:2610.21985

VeriVLA: Adaptive Test-Time Verification for Flow-Based VLAs with Latent World Models

Xueli Sun, Yuxiang Lu, Yantong Liu et al.

VeriVLA uses a training-free two-axis trigger to decide when to spend extra compute on repeated sampling for flow-based VLA policies. When triggered, a JEPA-based latent world-model verifier ranks candidate action chunks by predicting action-conditioned future latent states and estimated delta progress, avoiding pixel reconstruction. The authors report improved completion/success scores on real-world tasks (pen insertion, pipetting, apple placement) and simulation benchmarks, at added verification latency.

JEPAAI summary
2026-10-16 · arXiv:2610.22041

Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift

Danqi Zhao, Keyu Chen, Wenchao Sun et al.

World-model planners can fail when a deployment controller executes the same planned trajectory differently, creating a planning–execution gap. The authors find that substituting predicted/observed executed trajectories into a frozen world-model planner yields only limited decision gains. They propose ExecutionBridge, which infers controller-specific execution characteristics from planned–executed pairs and adapts the frozen model’s ego latent rollout while keeping the original planned trajectory and the planner/world-model/reward frozen.

DrivingAI summary
2026-10-16 · arXiv:2610.22056

Progress Reward Model for Reinforcement Learning via Large Language Models

Xiuhui Zhang, Ning Gao, Xingyu Jiang et al.

PRM4RL integrates LLM-based task decomposition with a Potential-based Progress Reward Model. The LLM generates a subtask list plus a state-based subtask determination function and fine-grained progress functions per subtask. The overall progress Φ(s) is used to build a dense reward via a temporal-difference/potential-based form, with policy invariance and convergence guarantees. Experiments on Metaworld and Maniskill show improved success rate and reduced LLM calls; code is released.

DrivingCode availableAI summary
2026-10-16 · arXiv:2610.22311

Velox: Spatial Memory Makes World Models Efficient

Qiuhong Shen, Shizun Wang, Xinchao Wang

Velox is a training-free attention router for chunked video world models. It summarizes historical KV context as per-block camera frustum slabs with depth intervals and uses recency-priority multiwarp to estimate query-block depth. Soft frustum-slab overlap modulates layer-specific block importance, with geometric fallback when coverage is unreliable. It targets late-chunk attention slowdowns in long visual rollouts.

2026-10-16 · arXiv:2610.22340

Dexterity from Supervised Play

Botao He, Yiming Dou, Yiannis Aloimonos et al.

DSP (Dexterity from Supervised Play) trains a goal-agnostic play policy by behavior cloning on minutes of human play, then uses it to autonomously collect hours of interaction for an action-conditioned world model. At inference, the play policy proposes candidates for MPC under the learned model to achieve arbitrary goals; optionally a closed-loop goal-conditioned policy is distilled via self-play in the world model. Evaluated in simulation and on real hardware.

RoboticsAI summary
2026-10-16 · arXiv:2610.22343

GeoState: Learning View-Decoupled Geometry States for 3D World Modeling

Hongchang Liu, Jian Liu, Yujuan Ding et al.

GeoState learns a view-decoupled geometry state for 3D world modeling. A scene tokenizer aggregates multi-view GFM features into a fixed-size GeoState in a shared scene coordinate system; a ray-conditioned geometry decoder queries it under a camera only at decoding time. A temporal model predicts future GeoStates while modeling camera evolution separately, enabling decoding under predicted or external cameras and avoiding per-frame scale alignment.

DrivingAI summary
2026-10-16 · arXiv:2610.22362

CoDA: Co-Evolution of Data Engines and Action Policies

Tianjiao Yu, Zhilin Ou, Yifan Shen et al.

CoDA co-evolves a simulation data engine and an action policy via a shared heterogeneous memory of scene-construction traces, rollout-grounded failure diagnoses, and learning progress. A vision-language critic produces validated diagnoses that are compiled into (i) generation programs to recreate difficulties and (ii) Text2potential-based training programs that provide dense potentials for action-chunk value learning. Across four manipulation task families, CoDA improves OOD success to 27.6% from 5.3% after supervised fine-tuning.

RoboticsAI summary
2026-10-16 · arXiv:2610.22363

Perceive, Predict, and Plan: a Unified World Action Model for Egocentric Dexterous Manipulation

Zirui Wu, Chen Guo, Tianjian Jiang et al.

P3-WAM is a unified world action model for egocentric dexterous manipulation with three modes: Perceive (reconstruct 3D hand motion and camera trajectories from video), Predict (generate action-conditioned visual futures), and Plan (jointly generate action trajectories and their anticipated visual consequences from an initial image, language instruction, and optional visual goal). It integrates compact structured 3D action into a pretrained video diffusion transformer via mixture-of-transformers and randomized modality visibility.

RoboticsAI summary
2026-10-16 · arXiv:2610.22364

T2-z: Test-Time Optimization of Environment Latents for Robotic World Models

Chenyuan Zhou, Yining Hong, Huang Huang et al.

T2-z adds an environment latent z_e to an action-conditioned robotic world model and trains it with alternating optimization: learn general action dynamics with z_e fixed, then optimize z_e with the world model frozen. At test time, the world model stays frozen while a new z_e is optimized from observed interactions to adapt to unseen environments. The authors report improved object/physical prediction error and action success in simulation and real-world tasks.

RoboticsCode availableAI summary
2026-10-15 · arXiv:2610.18991

Contact-UAN: Unsupervised Actuator Networks as a Residual World Model for Humanoid Control

Nolan Fey, Khai Nguyen, Yash Prabhu et al.

Contact-UAN is an unsupervised residual world model for humanoid control that preserves a simulator’s physics priors while correcting actuator dynamics to match hardware rollouts. It uses contact-consistent replay by inferring simulator-side stance-foot pose from onboard proprioception so residual learning corrects actuator discrepancies rather than contact/reset artifacts. A UAN trained on 6.5 minutes of real-world walking transfers to running, force control/impedance, and robust jumping.

DrivingAI summary
2026-10-15 · arXiv:2610.19007

Action Jets: Learning the Local Shape of the Action-conditioned Data Manifold

Anton Arapin, Chahyon Ku, Bernadette Bucher

Action Jets estimate the local shape of action-conditioned data manifolds from generator-measured tangents plus higher-order corrections fit to recorded transitions. The resulting Taylor-polynomial “jet” can generate predicted futures for many candidate actions with one extraction per state, avoiding per-candidate generator inference for short-horizon planning. The authors report large speedups and improved held-out short-horizon accuracy on multiple benchmarks.

DrivingAI summary
2026-10-15 · arXiv:2610.19063

WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models

Nan Zhou, Jianxiong Gao, Qiuyu Wang et al.

WorldMemBench is a benchmark for memory in video world models, covering 260 test cases. It evaluates static memory via human identity/appearance, object appearance/geometry, and environment content/geometry, and dynamic memory via event progression and state persistence under controlled camera departure-and-revisit trajectories. The authors report that strong static retention does not imply reliable dynamic evolution, and that repeated revisits reveal cumulative memory degradation.

2026-10-15 · arXiv:2610.19129

Imagining Goals with Geometric Horizon Models for Policy Generalization

Daniel Lawson, Dhruv Sreenivas, Adriana Hugessen et al.

The authors propose TD-Aug, which trains a geometric horizon model (GHM) using Temporal Difference Flows (TD-Flows) to sample counterfactual future outcomes from an offline mixture policy. They then use these generated outcomes as goals to augment goal-conditioned behavior cloning (BC). TD-Aug improves zero-shot compositional goal generalization on stitching-based OGBench tasks.

DrivingAI summary
2026-10-15 · arXiv:2610.19201

Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and Sensors

Henry Che, Xuanchi Ren, Sanja Fidler et al.

LidarVerse is a real-time unified LiDAR world model that unifies generative simulation and reconstruction-grounded resimulation across tasks and sensors. A universal attention-based tokenizer maps LiDAR range sequences into a shared latent space across 32/64/128-beam sensors and supports unseen scanning patterns. A bidirectional diffusion teacher is distilled into a four-step causal student with rolling temporal KV caching and a world-aligned persistent spatial cache for long-horizon rollouts at 11 FPS on a single H100 GPU.

2026-10-15 · arXiv:2610.19212

Decoupled Drift-Diffusion World Models for Consistent Future Frame Prediction

Anton Arapin, Aditya Singh, Yixin Wang et al.

D3WM predicts future image frames from action sequences by separating deterministic “drift” from stochastic “diffusion.” It trains a transformer to output a deterministic next-state mean plus a spatial heteroscedastic aleatoric uncertainty (variance) map. A variance-gated conditional diffusion model then refines uncertain regions while anchoring deterministic regions, yielding improved action-conditioned prediction on robotic manipulation and camera-motion/NVS.

Latent DynamicsAI summary
2026-10-15 · arXiv:2610.19327

SonoWorld: Towards Interactive Pose-Controllable Ultrasound World Models

Zefan Yang, Zheyuan Zhang, Boris Maihe et al.

SonoWorld is a causal ultrasound world model trained primarily from unlabeled freehand clinical ultrasound videos. It learns latent-action dynamics from ~6.9M multi-organ frames, then grounds them to 6-DoF probe control using a small EM-tracked thyroid dataset, and distills a bidirectional diffusion model into a causal long-horizon generator with selective KV-cache retention for interactive rollouts.