Digest

Reset

1167 papers

2026-10-21 · arXiv:2610.28555

Learning Computational Structure for Sample-Efficient Prediction

Hangyu Zhou, Constantine Dovrolis, Stephen Mussmann

The authors study whether gradient-based prediction can exploit hidden compositional (graph) structure. They introduce CSFRBench, which hides the target computation during training but keeps it for diagnosis, and propose Learned Wiring (LeWi): learn low-dimensional neuron embeddings that determine which connections are active (within a radius) while jointly training connection weights. LeWi improves sample efficiency on hierarchical, reuse, or modular targets and performs better than a dense baseline on 17 world-model prediction tasks.

Latent DynamicsAI summary
2026-10-21 · arXiv:2610.28594

Learning the 4D World through Point Trajectory Forecasting

Byungwoo Jeon, Sunghwan Hong, Dohyeon Kim et al.

The authors introduce Video-Point Model (VPM), which jointly forecasts future multi-view videos and the 3D trajectories of persistent scene points from arbitrary viewpoints, conditioned on multi-view video histories and a language instruction. They extend a pretrained world model with camera-conditioned multi-view processing and add a trajectory head, training with VPM-Data and evaluating on VP-WorldBench. They further add robot action tokens and show improved manipulation success with point supervision.

2026-10-21 · arXiv:2610.28613

Physical Canvas: A Vision Tactile World Model For Robot Policy Evaluation And Learning

Zebin Xing, Yuhang Zheng, Yupeng Zheng et al.

PhysCanvas is a state-consistent, action-conditioned visuo-tactile world model for contact-rich robot manipulation. It unifies multi-view RGB, tactile images, marker displacement, task state, and actions in a shared “canvas” using a shared video VAE and pretrained generative backbone, and uses frozen reward models for scene, modality, and state consistency during post-training. It enables closed-loop policy rollouts for evaluation and imagined rewards for GRPO updates.

RoboticsAI summary
2026-10-21 · arXiv:2610.28626

UniWALM: Unified World Action Language Model for Embodied Robot Control

Rahul Sajnani, Diego Romeres, Mingyuan Zhang et al.

UniWALM is a unified world-action-language model that learns video, action, and language in a single shared mixture-of-transformers representation, co-denoising per-modality diffusion heads synchronously and re-consuming generated language as next-step prompt (language-as-memory loop) with no separate planning stage. It also adds Flow Matching with Correction (FMC) to improve action reliability. The authors report improved manipulation and memory-dependent behavior in simulation and real-world tasks.

RoboticsAI summary
2026-10-21 · arXiv:2610.28631

AEON: Unifying Video and 3D World Models

Weiqi Zhang, Wenyuan Zhang, Yu-Shen Liu et al.

AEON unifies video and 3D world models by generating persistent 4D Gaussian Splatting (4DGS) worlds from text/image-conditioned video diffusion latents. It trains a 4D Gaussian Reconstruction Model (4DRM) from monocular videos to predict 4DGS attributes and camera poses, and uses a domain adapter to distill video latents into reconstructive 4D latents for end-to-end rendering refinement.

2026-10-20 · arXiv:2610.23862

Forecasting Patient Trajectories from the Electronic Health Records with Action-Conditioned World Models

Simon A. Lee, Jiahang Hank Sha, Soomin Chung et al.

The authors introduce an action-conditioned latent world model for electronic health records that separates patient states from observed clinical actions. Using a state-dependent linearization predictor, the model propagates local latent changes for time-resolved prediction, multi-step rollouts, and action-conditioned simulation, then decodes rollouts for forecasting evaluation. They report competitive, task-dependent downstream performance vs larger EHR foundation models with a ~30× smaller parameter footprint.

Latent DynamicsAI summary
2026-10-20 · arXiv:2610.23885

From Likelihood to Utility: Quality-Axis Diffusion for End-to-End Autonomous Driving

Junjie Yang, Ziyi Shi, Shuqi Shen et al.

QualiDrive replaces diffusion’s Gaussian-noise coordinate with a driving-quality axis for end-to-end autonomous driving. It builds scene-specific quality ladders by ranking observed trajectories using measured driving utility, then learns reverse transitions for ordered low-to-high-quality refinement. On NAVSIM, the authors report 94.5 PDMS (v1) and 92.74 EPDMS (v2).

DrivingAI summary
2026-10-20 · arXiv:2610.23886

EvoAD: Self-Evolving Autonomous Driving via Latent World-Model Guided Post-hoc Reflection

Junjie Yang, Qiming Zhang, Yuyan Li et al.

EvoAD is a self-evolving end-to-end autonomous driving framework that uses latent world-model guided post-hoc reflection. It stores reflected driving experiences in a BEV feature memory pool, retrieves scene-relevant precedents by geometric/semantic similarity, and uses a latent world model to evaluate candidate trajectories in a compact planning-oriented space. The system refines future behavior via iterative reflection without retraining the base driving policy.

DrivingAI summary
2026-10-20 · arXiv:2610.23894

Language-Forcing: Advancing Humanoid Loco-Manipulation via Semantic Alignment of Whole-Body World Action Models

Yanjun Li, Jiarui Ye, Weiyu Guo et al.

Language-Forcing is a training strategy for whole-body World Action Models (WAMs) that uses language supervision to emphasize task-relevant transitions in intermediate future-visual features. A frozen vision-language model generates text targets from paired current/future frames and task context; a frozen text encoder aligns these embeddings with intermediate action-conditioned future-visual features, while retaining the original visual and action objectives. Auxiliary components are training-only (no inference overhead).

2026-10-20 · arXiv:2610.23896

InJePA: Intention-First Latent Planning with Coupled JEPA World Models for Visual Navigation

Siming Zhou, Bowen Ma, Ruiran Du et al.

InJepa is an intention-first visual navigation world model. It couples two JEPA models in a shared latent space: a goal-conditioned JEPA proposes latent intentions, and an action-conditioned JEPA predicts successor latent states. Predicted consequences feed back to generate the next intention and infer the next action within each candidate rollout, forming a closed loop. Evaluated in Habitat MP3D (Clean150), InJepa shows effective multistep navigation and works across three frozen visual encoders; code is released.

DrivingCode availableAI summary
2026-10-20 · arXiv:2610.23913

UMI-WM: Interactive World Modeling for Bimanual Manipulation from Wrist Cameras

Jingxiang Guo, Haoyu Zhao, Xidong Zhang et al.

UMI-WM is an action-conditioned bimanual interactive world model that jointly generates paired left/right wrist-camera videos from observation history, per-arm relative camera poses, and gripper widths. It uses view-specific action injection plus sparse hub tokens for cross-view scene exchange, and camera-aware geometry conditioning with geometry-guided noise. A 30-step Teacher is distilled into a four-step Student for block-causal rollout.

DrivingAI summary
2026-10-20 · arXiv:2610.23968

IMPULSE: Push First, Then Predict in V-JEPA 2 Feature Space

Tingan Jin, Chung-Hsien Chou

IMPULSE performs three calibration pushes, summarizes their outcomes with a Bayesian contact memory (8 numbers), predicts object pose via a pose pathway, and uses that pose to forecast future V-JEPA 2 feature maps. On a pre-specified held-out confirmation, the memory lowers 1-s feature-forecast NMSE by 11.6–12.8% vs the same predictor without experience (8/8 seeds). A random-Fourier pose head greatly amplifies the gain.

Latent DynamicsAI summary
2026-10-20 · arXiv:2610.23982

Time-JEPA: Grounding Video World Models with Physical Time Series

Shenyi Li, Ziyang Zhang, Wen Wu et al.

Time-JEPA fine-tunes a video world model with low-dimensional, temporally aligned physical time series as auxiliary supervision. Physical time series are aligned to the model’s temporal grid and pooled per tubelet, and an auxiliary regression head predicts aligned physical state evolution during fine-tuning. At evaluation, video is the only input.

2026-10-20 · arXiv:2610.23984

Pretrained Features as Teachers, Not States: Compact Latent World Models for Visual Planning

Yanbo Wang, Nan Lin, Wendelin B\"ohmer et al.

Anchor-WM uses pretrained DINO features only as training-time supervision to learn a compact latent world model for action-conditioned prediction and visual planning. A predictive objective learns dynamics in the compact latent, while a training-only distillation head reconstructs dense DINO patch features to discourage representational collapse. At inference, DINO is discarded and planning runs entirely in the learned latent space, yielding strong success rates and faster planning than DINO-WM.

Latent DynamicsAI summary
2026-10-20 · arXiv:2610.24177

Self-Improving Video-to-Action Models

Hyunwoo Ryu, Eric Ming Chen, Max Wilcoxson et al.

Mirror is a self-improving video-to-action (V2A) method that learns to execute video plans through environment interaction, without prior action data or hand-crafted rewards. It alternates planner-guided rollouts (with exploration noise) and V2A training on the resulting video-action pairs, iteratively following longer portions of the plan.

2026-10-20 · arXiv:2610.24267

How exposed is work to robotics? Occupational, industry, and geographic measures

Sebastian Sartor, Robert Seamans, Neil Thompson

The authors construct abilities-based exposure measures for robotics and physical AI that map a robotics capability taxonomy to O*NET work abilities using an LLM. They aggregate to Robotics Occupational Exposure (ROE) for 774 occupations, Robotics Industry Exposure (RIE) for 250 NAICS industries, and Robotics Geographic Exposure (RGE) at the county level. They find robotics exposure concentrates in physically intensive work and certain industries, and should be read as an upper bound tied to technical relatedness.

DrivingAI summary
2026-10-20 · arXiv:2610.24285

ACTS: Learning Visuo-Tactile World Models from Human Physical Interaction

Yuxiang Ma, Weihan Xu, Pengfei Ye et al.

ACTS is an action-conditioned visuo-tactile world model trained only on human physical interaction. Operators manipulate handheld GelSight sensors while the system records multi-view video, tactile images, 6-DoF sensor motion, and tactile-derived normal force, forming React (4.9 hours). ACTS jointly predicts future scene, wrist, and tactile observations and is evaluated zero-shot on robot teleoperation.

RoboticsAI summary
2026-10-20 · arXiv:2610.24349

FlashRT-PAI: Fast Kernels Are Not Fast Pipelines for Physical AI Inference

Liang Su, David Chen, Jason Li et al.

FLASHRT-PAI is a low-latency physical AI inference engine that focuses on model–hardware pipeline co-design to reduce system overhead in small-batch, real-time loops. It uses deep kernel fusion with launch overlap, an execution contract for cross-decision state reuse/refresh, and execution-state capsules for snapshot/restore/fork. Across six policies on RTX 5090 and four on Jetson AGX Thor, it reports 1.3–3.7× faster than official compiled paths at matched 16-bit precision and up to 3.38–12.66× over official eager.

DrivingAI summary
2026-10-20 · arXiv:2610.24372

Beyond Next-Screen Prediction: The Decision Value of GUI World Models

Sizhe Tang, Hanhan Zhou, Jeffrey Svajlenko et al.

The authors study how much GUI world-model predictions matter for action selection versus the evaluator that interprets those predictions. They show lookahead gains can change when predictions are presented to the evaluator, so end-to-end gains cannot be attributed to prediction alone. They introduce Direct Action Ranking (DAR), which scores candidates directly without generating future screens, and find predicted successors add no detectable ranking signal once evaluation is learned directly.

DrivingAI summary
2026-10-20 · arXiv:2610.24519

Axiomatic World Modeling for Physics Reasoning

Xinye Yang, Zhenyang Liu, Yuxuan Wang et al.

The authors define “modeling drift” as the divergence between a model’s internal axiomatic world model (AWM) and the true physical world specified in a physics-reasoning problem. They propose Reinforcement Learning with Axiomatic World Modeling (RLAWM), using a two-level training reward: a modeling reward that scores the externalized AWM and a solving reward that verifies reasoning via executable checkers. The framework operates only during training and improves PhysReason/PHYSICS up to +12.2/+5.2 points.

DrivingCode availableAI summary