Digest

Reset

1167 papers

2026-10-08 · arXiv:2610.09462

TMT: Runtime Backdoor Detection for Vision-Language-Action Policies on Unseen Tasks

Zirun Zhou, Jingfeng Zhang, HaoChuan Xu et al.

TMT is an inference-time runtime backdoor detector for vision-language-action (VLA) policies on unseen tasks. It combines a token manifold branch (checks token representation structure) with a latent transition branch (checks prediction errors in adjacent-layer latent dynamics), both trained on benign rollouts. It can also purify the backdoored policy via self-distillation using paired benign/triggered observations, without a separate clean reference policy.

DrivingAI summary
2026-10-08 · arXiv:2610.09514

STRIKE: Learning Visual State Transitions for Physical World Modeling

Wenbin Teng, Tianshuo Xu, Depu Meng et al.

STRIKE learns physical world modeling via explicit visual state transitions instead of directly generating dense video. It builds event-aligned supervision by extracting observed visual states from training videos and pairing adjacent states with transition descriptions and elapsed times. A transition image model predicts the next state recursively under VLM-planned timed events; a separately trained video dynamics model then generates dense rollouts conditioned on those predicted timed states.

DrivingAI summary
2026-10-08 · arXiv:2610.09734

{\Delta}WAM: Distilling Action Tangent Fields into World Action Models

Ke Wu, Hanwen Huang, Bo Gu et al.

ΔΔWAM introduces Action Tangent Fields (ATFs) to reweight world-action model (WAM) denoising toward residual-dynamics directions that change under local action perturbations. It uses an information-preserving Residual-VAE target (residual relative to a broadcast reference) and probes action-responsive directions with a frozen action-conditioned world model (ACWM). The method improves robustness on LIBERO-Plus and RoboTwin benchmarks, and distills multi-step VideoDiT denoising into one step.

DrivingAI summary
2026-10-08 · arXiv:2610.09759

Unrolled Flow Models for Reasoning

Faissal Izermine, Hanru Bai, Oscar Davis et al.

The authors study when increasing flow integration steps helps reasoning. They prove a two-layer Transformer-defined flow can solve directed graph reachability, with needed steps tied to distance. Standard flow training supervises each time point independently and yields little gains from more steps; they propose training through an unrolled latent rollout with only a terminal loss. On ProsQA this reaches 97% accuracy and improves with more steps; for longer Sudoku/Maze rollouts, sphere retraction stabilizes dynamics and improves scaling.

AI summary
2026-10-08 · arXiv:2610.09763

Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving

Mahmoud Selim, Cristina Cipriani, Karl Henrik Johansson

The authors introduce ICDP, an offline RL method for autonomous driving that explicitly controls “interaction distribution shift” (IDS), where an ego trajectory may be individually supported yet poorly supported jointly with surrounding-agent behavior. ICDP decomposes joint-support degradation into an ego component and an interaction component, estimates the interaction part via contrastive density-ratio estimation, and uses it to regularize diffusion-policy optimization. Evaluations are reported on nuPlan, Interplan, and real-world truck experiments.

DrivingAI summary
2026-10-08 · arXiv:2610.09785

UltraWorld: Learning Interactive Ultrasound World Models from Untracked Clinical Videos with Acoustic Sampling Map

Keke Yang, Erqi Wang, Sainan Guan et al.

UltraWorld learns interactive ultrasound world models from untracked clinical videos without real action annotations. It adapts a video foundation model into a mask-conditioned ultrasound generator, then uses 3D anatomical assets and programmable probe trajectories to synthesize action–video pairs for self-distillation into an observation+action world model. It further introduces the Acoustic Sampling Map (AsMap) to encode pixel-wise 3D sampling geometry for improved action following.

2026-10-08 · arXiv:2610.09940

Juno: Taming Predictive Latents for Vision-Language-Action Models

Yuchen Zhu, Chenyi Xu, Yulin Zhang et al.

Juno is a unified action-conditioned JEPA framework for vision-language-action models. The authors train a control-aligned JEPA backbone with a Dynamic CLS loss, fuse its patch features into a VLA with a decoupled predictive reasoning branch, and run dynamics-first test-time adaptation that adapts the world model on all transitions then re-aligns the policy on verified executions. They report improved success in simulation and robust real-robot performance under distribution shifts.

DrivingAI summary
2026-10-08 · arXiv:2610.10079

RealtimeWAM: How Fast Can I Run My World Action Model?

Huanan Liu, Ye Li, Kangye Ji et al.

RealtimeWAM is a training-free framework for world action models that reduces inference latency for responsive robot control. It overlaps observation processing with prediction using layerwise dependencies, reduces Prefill contention via hardware-aware token reuse guided by observation change and latency plateaus, and allocates refinement using predicted motion. Evaluated on FastWAM and OpenWAM across RoboTwin, LIBERO, and LIBERO-Plus, it reports large latency reductions with near-native success.

DrivingAI summary
2026-10-08 · arXiv:2610.10270

Video Prediction Policy 2: Predict Better, Act Better

Yanjiang Guo, Haodong Yan, Zhide Zhong et al.

Video Prediction Policy 2 (VPP2) is a world action model that targets open-ended manipulation failures of prior WAMs caused by (1) video models not being optimized for manipulation and (2) naively adding action components degrading generalization. VPP2 continues pretraining on manipulation videos with detailed captions and event-level video training, post-trains/distills to a single-step visual planner, and trains an MoT action expert for inverse dynamics.

DrivingAI summary
2026-10-08 · arXiv:2610.10274

Sparse Planning in Visual World Models via Cost Gradients

Yingchen Xu, Edward Grefenstette

The authors introduce CostGrad, a training-free, goal-conditioned token selector for sparse planning in token-based visual world models. It ranks spatial tokens by the gradient norm of the planning cost w.r.t. each input token, then runs CEM planning using only the selected tokens. On AdaLN-conditioned predictors at 50% sparsity, it matches or exceeds full-token planning on three of four continuous-control benchmarks while giving measured 2.6× wall-clock speedup per planning step.

DrivingAI summary
2026-10-08 · arXiv:2610.10390

Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving

Xingtai Gui, Yucheng Zhou, Dongqian Guo et al.

GeoCoTDrive is an explicit geometric chain-of-thought framework for vision-language-action autonomous driving. It “think[s] with 2D first,” by grounding planning-relevant 2D regions, then “drive[s] with dedicated 3D priors” by retrieving localized geometry tokens from a frozen geometric foundation model within those regions and interleaving them into the autoregressive context for trajectory generation. It introduces planning-relevant grounding and the PlanningGrounding dataset (146K annotations).

DrivingCode availableAI summary
2026-10-08 · arXiv:2610.10409

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

Zhiqin Yang, Chenxin Li, Xiaomeng Hu et al.

RobotWorld is a simulation benchmark for “robot use”: turning instructions and observations into physical task execution through robot interfaces. It evaluates 84 tasks across manipulation, mobile manipulation, locomotion, driving, and aerial control with explicit interaction budgets and executable success checks. The authors analyze outcomes with execution traces to identify both transferable capabilities (perception/control workflows) and persistent gaps that prevent reliable task completion.

DrivingAI summary
2026-10-08 · arXiv:2610.10498

EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution

Python Song, Zhixuan Liang, Kelsey Fu et al.

EmbodiedRSI is a self-evolving agentic robotics harness for continual robot learning with a frozen robot foundation model. It uses a Fast-Slow dual system with a Hypothesis Graph, value-of-information experiment selection, and code-skill co-evolution plus hierarchical memory learned with reinforcement learning. On RoboCasa365 it reports 77.0% overall success (71.3% Composite-Unseen) and on LIBERO-Pro 86.8% overall success, with zero-shot transfer to real-world achieving 71.3% overall success.

DrivingAI summary
2026-10-08 · arXiv:2610.10515

RoboJEPA: Scaling Robotic Latent World Models

Artem Zholus, Nicolas Beltran-Velez, Jianhao Yuan et al.

RoboJEPA is a JEPA-based latent world model for robotics trained on real-robot data spanning 12 embodiments. The authors report that its imagination (latent rollout) error follows a second-order power law in training compute with verified extrapolation. They also show offline imagination error correlates with downstream robotic planning success, and that RoboJEPA can be deployed zero-shot as an agent planning toward a single goal image. They release all model checkpoints and training/robot deployment code.

Latent DynamicsCode availableAI summary
2026-10-08 · arXiv:2610.10528

Long-WAM: Scaling the Context of World-Action Models

Wei Huang, Bohan Zhang, Chenzhi Liu et al.

Long-WAM is a model–system framework for scaling causal world-action models under real-time robot control constraints. The authors report that access to longer history is not the same as using it: longer context helps far more when the video foundation is pretrained autoregressively (AR). On RoboCasa GR-1, increasing context from 0.0 to 19.2s raises success 63.3%→78.7%.

RoboticsAI summary
2026-10-07 · arXiv:2610.06877

When Can World Models Recover Physical Laws?

Ye Yuan, Jun Liu

The authors argue that accurate prediction from a fixed recording does not guarantee recovery of the underlying physical law because distinct dynamics can produce identical observations under the same allowed experiments. They formalize law recovery on a fixed domain with sensor uncertainty and an acquisition budget, derive information-theoretic distinguishability conditions, give a decoding construction, and quantify minimax acquisition costs for Lipschitz dynamics from noisy full-state readouts after resets.

AI summary
2026-10-07 · arXiv:2610.06965

ACG-WAM: World-Action Modeling via Action-Conditioned Geometric Latent Prediction

Jiangtao Liu, Zishang Xiang, Yage He et al.

ACG-WAM augments the Motus latent world-action model with ACG-JEPA: an auxiliary objective that predicts action-conditioned geometric endpoints at several horizons. It uses a frozen VGGT teacher to encode current/future image pairs and supervises a shared visual embedding from head and wrist cameras before temporal mixing; teacher/predictor modules are removed at inference. On RoboTwin 2.0 (50 tasks), the authors report 93.46% clean success, 92.68% randomized success, and 93.07% mean; on a real robot, 85.00% SR and 91.67% PCS.

RoboticsCode availableAI summary
2026-10-07 · arXiv:2610.07006

STOCK-JEPA: Prior-Anchored Latent Revision Representation Learning in Equity Markets

Yizhi Luo, Jiahe Yi, Jianhui Zhang et al.

Stock-JEPA is a joint-embedding predictive framework for equity markets that learns predictable incremental latent “revisions” relative to a point-in-time financial prior. The method builds an anchor from fixed, multi-horizon return/risk prior statistics, predicts context-conditioned revisions to the target latent, freezes representation modules, and trains a downstream return readout for cross-sectional ranking and portfolio performance.

DrivingAI summary
2026-10-07 · arXiv:2610.07025

WiSPER: Pose-Supervised Predictive and Residual Flow Refinement For Multi-Person 3D Pose Estimation With WiFi CSI

Gabriel Lee Jun Rong, Shanhong Liu, Pai Chet Ng et al.

WiSPER is a two-stage framework for multi-person 3D pose estimation from WiFi CSI. Stage 1 (PAMEL) performs pose-aware masked latent prediction with auxiliary pose-set supervision on visible CSI context. Stage 2 (ReFT) generates multiple pose candidates with a hierarchical pose Transformer and refines each via conditional residual flow guided by coarse coordinates and per-joint features. Inference uses only CSI.

DrivingAI summary
2026-10-07 · arXiv:2610.07028

Identifiable World Models from Pretrained Diffusion Representations

Ruchi Sandilya, Conor Liston, Logan Grosenick

The authors study whether a frozen pretrained diffusion model for video can be equipped with identifiable latent coordinates without retraining its generative backbone. They propose Contrastive Diffusion Alignment (ConDA), learning only a lightweight alignment map on frozen diffusion latents. Under TCL/GCL-style assumptions, the aligned coordinates identify latent dynamical states (up to permutation and componentwise invertible transforms), preserve the latent dynamic SCM, and enable lagged causal graph recovery via inverse transition-noise Jacobian sparsity.