Digest

Reset

1167 papers

2026-10-14 · arXiv:2610.15166

ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling

Xinge Peng, Zhen Fang, Ruibing Wang et al.

ExeWorldBench is a benchmark for executable 3D world modeling that evaluates how models translate physical understanding into executable, code-based Blender simulations. It defines three progressively harder tasks: Mechanism Executability (single mechanism from an image), Compositional Executability (multi-stage causal chains), and Intent-Driven Executable World Synthesis (component- or goal-constrained). Experiments show substantial limitations in executable world modeling.

2026-10-14 · arXiv:2610.15246

ObsSpec: Accelerating Agentic Reinforcement Learning with Observation Speculation

Kyle Montgomery, Wang Zhang, Chenguang Wang

ObsSpec is an observation-speculation pipeline for agentic RL that uses a small world model to predict tool outputs while the environment executes, so policy generation for the next action can start early. It keeps rollout collection lossless by retaining speculative work only when predicted and actual observations match exactly, and it co-trains the world model on actual observations. On terminal-use tasks, a 0.6B world model matches in 66.4% of attempts and reduces agent latency by 9.3% (8B) and 14.2% (32B).

DrivingCode availableAI summary
2026-10-14 · arXiv:2610.15267

LeHami: Symplectic Latent Dynamics for World Models

Romina Yalovetzky, Randall Balestriero, Yann LeCun

LeHami is a JEPA-style latent world model where the predictor is replaced by an action-conditioned Hamiltonian flow on a structured latent phase space (q,p), integrated with an exactly symplectic leapfrog step. The authors prove the rollout map is exactly symplectic and preserves phase-space volume at every horizon, ruling out uniform contraction, asymptotically stable fixed points, and merging of latent states. On visual pendulums, it improves long-horizon velocity prediction and maintains constant latent volume where an unconstrained JEPA contracts.

Latent DynamicsAI summary
2026-10-14 · arXiv:2610.15294

ProWorld: Multi-View World Model with Proprioceptive Feedback for Closed-Loop Policy Evaluation

Zijun Wang, Tianyi Yan, Haokun Lin et al.

ProWorld is an action-conditioned, multi-view robot world model for closed-loop policy evaluation. It jointly predicts multi-view video and proprioceptive state, feeds predicted observations back to a frozen policy, and includes failure-aware training using failed interactions from high-fidelity simulation to reduce interaction hallucinations. The authors report ProWorld achieves the highest overall TriWorldBench score and strong correlation with real success rates.

DrivingAI summary
2026-10-14 · arXiv:2610.15305

RM-WAM: Reliable Memory-Augmented World Action Models for Long-Horizon Robot Manipulation

Cui Miao, Junhao Yu, Ping Liu et al.

RM-WAM is a memory-augmented World Action Model for long-horizon robot manipulation that treats memory as a context-dependent predictive resource. It uses complementary short-term visual memory and latent event memory, and learns a counterfactual utility estimate to gate how much memory to inject into the WAM backbone for both future world prediction and action generation.

RoboticsAI summary
2026-10-14 · arXiv:2610.15343

Across Viewpoints and Time: Trajectory-Generalizable Physical Attacks on World Models and Video Understanding Models

Song Xia, Meiwen Ding, Chenqi Kong et al.

The authors study whether a single printable adversarial texture can mislead world models and video understanding models across different real-world scenarios where viewpoints change along uncontrolled object–camera trajectories. They identify conflicting optimization gradients from heterogeneous video semantics and trajectories and propose TARA (trajectory-aware contrastive alignment + trajectory-aware task refinement). They also build a geometry-grounded tracking pipeline and collect print-and-recapture videos for reproducible physical evaluation.

2026-10-14 · arXiv:2610.15368

ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs

Yi Ge, Kangrui Ruan, Jifan Chen et al.

ANTE is an agentic RL framework for coding agents that explicitly writes a belief state before each action, including an outcome forecast and a trust/abstain decision. During RL, Belief-Anchored Policy Optimization (BAPO) verifies forecasts by real execution and turns them into per-step credit. At inference, ANTE-TTO samples and ranks candidate actions using the learned forecasts.

DrivingAI summary
2026-10-14 · arXiv:2610.15429

World Modeling through Spectral Alignment

Holger Molin, William Peng, Marco Bagatella et al.

The authors introduce SpecWM, a JEPA framework that trains a world-model encoder to match a target observation similarity kernel via spectral alignment. Different kernels (physical-state, depth, proprioception, or temporal ordering) determine which quantities the learned representation preserves. They analyze optimality and probing/invariance properties, then show temporal spectral alignment improves CEM planning success over LeWorldModel on OGBench Cube/Scene and CALVIN.

DrivingAI summary
2026-10-14 · arXiv:2610.15556

Vision-Predictor-Action: Towards Efficient Embodied AI via Neuro-Symbolic JEPA Predictors

Yeonseok Lee

VPA replaces autoregressive token-based action decoding with a lightweight latent control loop: a vision encoder plus frozen text encoder condition a neuro-symbolic primitive selector, a JEPA predictor forecasts next latent states, and a flow-matching solver generates an entire action chunk in a few integration steps. On LIBERO-10, the authors report 40.0% success in 200 episodes (2 cameras) with low-latency decision steps on an Apple M4.

Latent DynamicsCode availableAI summary
2026-10-14 · arXiv:2610.15562

A Plausible Future Is Not a Causal Future: Can Video Models Generate How the World Unfolds?

Jialiang Li, Zhengning Xu, Zongyang Ma et al.

The authors study whether image-to-video models can generate “causal cascades”: a full chain of causally dependent events. They introduce CausalCascade, a simulator-grounded benchmark with 50 Blender scenes and 2,832 counterfactual variants. Evaluations on nine I2V models show better spatial localization than correct spatiotemporal evolution, improved event recovery with richer prompts but limited physical realization, and weak selective responsiveness to physical interventions.

2026-10-14 · arXiv:2610.15585

StreamingWAM: Streaming World Action Modeling via Chunk-Wise Staircase Flow Matching

Shaojie Ren, Jingwen Sun, Zezhi Liu et al.

StreamingWAM is a streaming-diffusion world action model for robot control that aims to reduce jerky/discontinuous closed-loop execution. It uses a noise ladder for chunk-wise streaming action prediction, a prefix rollout module for cross-chunk continuity, and dense video-history memory for temporally consistent conditioning, producing competitive results with lower steady-state replanning latency.

RoboticsAI summary
2026-10-14 · arXiv:2610.15625

1Step-WAM: Your World Action Model but Faster using 1-Step Action Generation

Rundong Su, Pengtao Chen, Haohan Geng et al.

1Step-WAM speeds up World Action Models by converting multi-step action denoising into a single forward pass. It uses a lightweight calibrator to correct the first-step velocity, plus bandwidth-constrained (low-frequency DCT) action decoding. Because one-step action generation can run in parallel with visual prefill, the authors report large inference speedups on RoboTwin, LIBERO, and real-robot tasks while largely preserving multi-step success.

DrivingCode availableAI summary
2026-10-14 · arXiv:2610.15645

SurgCast: Action-Conditioned Future Skeletons for Controllable Surgical World Models

Wanhao Liu, Rulin Zhou, Liangjing Shao et al.

SurgCast predicts future instrument skeleton keypoints from current structure, robot state, and future actions, then uses the forecast to control action-conditioned surgical video generation. Dual-path injection injects (1) rasterized skeleton geometry via geometric residuals and (2) derived instrument kinematic semantics via conditioning modulation, alongside native action conditioning. On SutureBot and SRTH-Porcine-Cholei, predicted-skeleton teacher and four-step causal student improve quality and instrument-consistency metrics, including FVD decreases.

DrivingAI summary
2026-10-14 · arXiv:2610.15666

A Persian Vision-Language-Action Dataset and Benchmark for Ambiguity- and Infeasibility-Aware Instruction-to-Plan Grounding

Abdollah Pourasghari, Saeedeh Sadat Sadidpour, Mohammad Roustaei

The authors introduce the first Persian vision-language-action dataset for instruction-to-plan grounding that is ambiguity- and infeasibility-aware. It contains 258 instances over 230 real household/office photographs with reference plans plus ambiguity/infeasibility labels and disambiguating QA; training includes auto-generated chain-of-thought rationales. The authors ablate retrieval, CoT prompting, and disambiguation, introduce PGOS-FA, and report that retrieval-augmentation is a dominant statistically significant effect.

DrivingAI summary
2026-10-14 · arXiv:2610.15691

FARL: A Fully Automated Pipeline of RL Interface and Solver Design

Tianyu Jin, Liuyu Xiang, Chaowen Yang et al.

FARL automates RL design by jointly searching observation and reward programs, learning algorithms, and training settings, using training statistics and observed behavior to diagnose failures and test specific changes. A diversity-preserving search uses short trials first, and scientific memory records each finding with its test conditions for later reuse. On expert-withheld and expert-available navigation, locomotion, and manipulation tasks, FARL outperforms evaluated automated baselines.

AI summary
2026-10-14 · arXiv:2610.15794

$\Delta$ Flow Matching for Robot Foundation Models

Kuangji Zuo, Tuo An, Xinying Guo et al.

The authors propose Δ-flow matching for robot foundation models, replacing the fixed Gaussian source used in flow matching with a source constructed from recent history plus mild noise. This reduces transport to a residual Δ, enables single-step inference, improves training efficiency and generalization on LIBERO/LIBERO-Plus, preserves action smoothness, and can adapt pretrained WAMs using action-history sources without retraining.

RoboticsAI summary
2026-10-14 · arXiv:2610.15891

Learning Multimodal Sensor Fusion with JEPA Representations for Breathing Activity and Pose Recognition

Yunxiang Luo, Constantino \'Alvarez Casado, Xiaoting Wu et al.

The authors transfer frozen V-JEPA2 video features to a shared multimodal fusion model that handles missing sensors across RGB video, depth video, and radar time series. Using modality queries, fixed sensor slots, and explicit sensor availability, they classify breathing activity and pose for all seven non-empty sensor subsets on 39-subject OMuSense-23. With matched downstream parameter counts, all-view supervision yields activity Avg-7 macro-F1 of 75.23% ± 0.60.

JEPAAI summary
2026-10-14 · arXiv:2610.15931

Mesh-Free Convolution: Learned Spectral Attenuation and Transport

Niloufar Zakariaei, Manav Manav, Eldad Haber

The authors propose Mesh-Free Convolutions (MFCs), continuous-space filters discretized only at evaluation time. MFCs replace hard Fourier-mode truncation with learned smooth spectral attenuation, add first-order directional readout for transport-like behavior, and use a continuous symbol so the same learned filters transfer across resolutions. They report more stable long-horizon rollouts and better long-time statistics than Fourier-based baselines on fluid-dynamics and weather-forecasting benchmarks.

Latent DynamicsAI summary
2026-10-14 · arXiv:2610.16139

Directional Pullback Regularization for Long-Horizon Rollout of Latent World Models in Autonomous Driving

Shatadal Mishra, Ahmadreza Moradipari, Nejib Ammar

The authors study long-horizon open-loop ego-motion rollout for autonomous driving latent world models, noting non-holonomic constraints forbid instantaneous pure lateral motion. They propose Directional Pullback Regularization (DPR), a training-time method that defines a kinematics-aware anisotropic Riemannian metric in decoded space and pulls it back through the decoder Jacobian to shape latent transitions. DPR reduces 10-second final displacement error vs ULR and EPR across U∈{1,3,5,10}.

DrivingAI summary
2026-10-14 · arXiv:2610.16160

FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning

Junhyeong Go, Mingyu Jung, Woobin Im et al.

FlowVLA is a training-only framework that injects action-relevant 3D dynamics into 2D Vision-Language-Action (VLA) models using reconstruction-derived, temporally aligned 3D flow from RGB. It uses dual-level supervision: point-level flow predicts future motion of selected 3D points, and TCP-level flow aggregates predicted point motions to align with robot Tool-Center-Point trajectories. Auxiliary heads are discarded at inference, adding zero inference-time overhead.

DrivingAI summary