Digest

Reset

1167 papers

2026-10-06 · arXiv:2610.06582

Mind the Execution Gap: Action-Semantic Mismatch in World-Model Control

Shengtao Wen, Xiang Chen, Yu Tian et al.

The authors study how asynchronous action execution breaks the action semantics assumed by action-conditioned world-model controllers. They identify two architecture-dependent failure modes: TD-MPC2 suffers from a future-action timeline mismatch, while DreamerV3 can attribute transitions to previous actions that were not actually applied. They propose two execution-consistent interfaces—Future-Sequence and Applied-Action Feedback—that correct these mismatches without modifying pretrained world models.

Latent DynamicsAI summary
2026-10-06 · arXiv:2610.06598

SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models

Xiaodong Wang, Tianle Li, Chuanxin Song et al.

SimForcing is a simulation-guided action-conditioned robot world model that transfers motion priors from a simulation teacher to real-domain video prediction. It uses latent-motion distillation to align temporal differences between adjacent teacher/student latents and adds multi-block simulation conditioning with condition dropout plus simulation-conditioning classifier-free guidance to limit reliance on inaccurate simulation trajectories.

RoboticsCode availableAI summary
2026-10-06 · arXiv:2610.06617

RealtimeWAM: One-Step Asynchronous World Action Models

Chengtao Lv, Jinyang Du, Shuyi Feng et al.

RealtimeWAM is a one-step, asynchronous post-trained World Action Model for MoT-based WAMs. It uses Teacher-Anchored Consistency Distillation (TACD) to address a local-global error gap for accurate one-step action generation, and Cross-Expert Wavefront Pipelining (CEWP) to overlap the Video and Action experts at the block level by sharing video KV caches. The authors report near-lossless performance (<1% drop) with ~25× end-to-end speedup on H100.

DrivingAI summary
2026-10-06 · arXiv:2610.06643

AffordCraft: Scalable Construction of Task-Ready Simulation Assets from Single Images

Haoyun Yang, Xueyang Zhou, Ziyi Xie et al.

AffordCraft builds task-ready articulated simulation assets from a single RGB image and a task instruction by retrieving and adapting a matching entry from a library, then validating it with a physical test. Without boxes/masks, it produces physically valid assets for 1,703/2,000 photographs (85.15%) across 31 categories, and 162/237 annotated objects on cluttered images pass after detection. Policies trained on scripted demonstrations complete constructed manipulation tasks.

RoboticsAI summary
2026-10-06 · arXiv:2610.06651

Considering Context: When World Models Need Context Encoders

Oleg Smirnov, Sofiane Ennadir, John Pertoft et al.

The authors study when latent-context world models need a context encoder. They formalize “predictive sufficiency” as the Bayes one-step log-loss advantage of conditioning on context under the visitation distribution induced by an agent. They decompose this advantage into headroom recoverable from trajectory history and a residual requiring the true context, and show that benefits need not match MDP classes; implicit identification can make predictive headroom vanish for some priors and persist (partially) for others.

Latent DynamicsAI summary
2026-10-06 · arXiv:2610.06805

H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning

Wancong Zhang, Basile Terver, Michael Rabbat et al.

H-JEPA trains a hierarchy of action-conditioned JEPAs where each level predicts farther ahead in its own latent space. Planning is top-down: the top level optimizes toward the goal and its predicted states become subgoals for lower levels until level 1 outputs primitive actions. Across four simulated navigation/manipulation environments, hierarchical planning improves over a flat JEPA; on Visual AntMaze, a three-level hierarchy raises success from 18% to 73% using less planner compute.

Latent DynamicsAI summary
2026-10-06 · arXiv:2610.06814

TAPDreamer: Transferable Adversarial Patches for World Action Models

Xuanyu Lu, Fengqing Jiang, Kaiyuan Zheng et al.

TAPDreamer is an untargeted test-time patch attack on world action models that uses only a public visual encoder and a few demo frames to optimize a fixed local pixel perturbation. The same frozen patch (about 6.5% of input) transfers across tasks and WAM architectures. In closed-loop control, it reduces FastWAM success from 97.65%→0.00% on LIBERO and 90.86%→0.00% on RoboTwin.

DrivingAI summary
2026-10-05 · arXiv:2608.23927

GlanceWAM: Sparse Test-Time Imagination for World-Action Models

Linhan Wang, Zijian An, Mingyuan Zhang et al.

GlanceWAM is a world-action model that decouples visual imagination from real-time action control. It asynchronously generates a single latent lookahead frame seconds ahead in the background (via a shared video DiT backbone) and keeps it for several 0.8s action chunks. The action head decodes action chunks at control rate in latent space without blocking, enabled by a non-interfering attention mask and staleness-robust horizon training.

DrivingCode availableAI summary
2026-10-05 · arXiv:2610.02274

Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation

PhysicalRSI Team, Danjiao Ma, Enhui Ma et al.

Awomo-SimDataEngine is an agentic system for generating simulation-ready worlds and converting them into robot demonstrations. It combines rigid and articulated asset services (ISArt), scene generation from images (Unravel) or text (SimForge), and a Graph-Native Harness for construction, validation, and bounded repair. PolicyForge then produces replayable Isaac Sim demonstrations. On MuJoCo-based LIBERO-Plus, SimData-Ultra co-training improves WAM overall success from 77.17% to 89.43%.

RoboticsAI summary
2026-10-05 · arXiv:2610.02331

World Editing: Intervening on Executable Worlds at Increasing Depth

Max Ku, Nok-Kan Law, Yu-Chien Tang et al.

The authors define world editing as intervening on an existing executable world while preserving unrelated properties, and introduce intervention depth (property→entity→dynamics→system). They build IGMWorld/IGMBench: 110 Minecraft/Terraria modding tasks (over 1.1K executable state/behavior criteria) with executability, behavioral, preservation, and visual checks. Frontier coding agents reach up to 78.2% task success (strict WSR) and 94.8% criterion pass rate (CPR), with reliability decreasing as depth increases.

2026-10-05 · arXiv:2610.02344

Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures

Jos\'e Lucas De Melo Costa, Seong Woo Ahn, Fabrice Popineau et al.

The authors develop a drive–decay stability theory for Joint-Embedding Predictive Architectures (JEPAs), analyzing training dynamics near the trivial fixed point via a per-mode stability ratio μ_i=γ_i/σ_i. They predict a phase boundary (collapse when μ<1, learning when μ>1) and confirm it across 800+ Tabular-JEPA configurations. Theory suggests stabilizers; they propose ResidualPred, an identity-biased transformer predictor, improving effective rank and linear-probe accuracy on tabular and I-JEPA benchmarks.

JEPACode availableAI summary
2026-10-05 · arXiv:2610.02368

Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation

Shukai Gong, Xuanran Zhai, Yintianrun Zhang et al.

ViGAR is a hierarchical world-action model for long-horizon compositional robotic manipulation. It factorizes task progress into a visual subgoal planner (predicts the next subtask goal image) and a subgoal executor (jointly predicts future visual trajectories and actions conditioned on the predicted subgoal). It supports in-context robotic manipulation by using a global goal image as inference-time context, without parameter updates.

RoboticsCode availableAI summary
2026-10-05 · arXiv:2610.02398

Keep the Effect, Drop the Actor: Programmable Effect-to-Execution World-Action Models

Junyi Hu, Zhewen He, Zhenhua Li et al.

The authors propose PEWAM, an actor-free “effect program” world-action model compiled from a demonstration’s object keypoint trajectories, contact points, and terminal state. PEWAM generates effect, robot execution, action, and terminal state as four streams with independent flow-matching times; clamping a program and sampling the execution turns inference into programming. Re-solving from the live scene absorbs pushes and enables editing and multi-arm transfer.

DrivingAI summary
2026-10-05 · arXiv:2610.02508

World Action Modeling with Progressive Visual Planning

Fei Zhang, Zhaochong An, Duncan Frost et al.

The authors present ProWAM, a world action model that predicts an ordered sequence of sparse, progress-indexed visual sub-goals alongside action chunks. This avoids inefficient dense video rollouts by caching sub-goal features from a single video-backbone forward pass, then using lightweight action denoising during replanning. The authors report improved out-of-distribution robustness on multiple simulation benchmarks and zero-shot real-world experiments.

DrivingAI summary
2026-10-05 · arXiv:2610.02515

IGNITE Tokamak World Model Architecture

Peter Steiner, Azarakhsh Jalalvand, Nathaniel Chen et al.

IGNITE is a generative, multi-modal, actuator-conditioned world foundation model for fusion plasma behavior at the DIII-D tokamak. Trained self-supervised on over a decade of unlabeled experimental data (8000+ discharges), it autoregressively forecasts future diagnostics given initial latent plasma states and actuator trajectories, using spatio-temporal tokenizers with discrete FSQ codes and a transformer dynamics backbone.

DrivingAI summary
2026-10-05 · arXiv:2610.02521

Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory

Ying Yang, Guiyu Zhang, Lianghua Huang et al.

Spatial Memory Intelligence (SMI) is a long-video world-model framework that uses an MLLM to manage long-term spatial memory via four atomic operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering. The authors report comprehensive improvements in memory sparsity, generation stability, and spatial consistency across baselines, benchmarks, and world-model backbones.

2026-10-05 · arXiv:2610.02542

How To Train Your World Model: Fine-tuning vs RAG for LM-based World Modeling

Dhananjay Ashok, Shantanu Agarwal, Vivek Datla et al.

The authors systematically compare fine-tuned vs RAG-based LM world models across five text environments. They report fine-tuning yields higher rewards on 15/20 model-environment combinations, while RAG is more data-efficient and outperforms in Chameleon. They find retrieval often returns suboptimal transitions (recall@5 < 0.7) and propose hierarchical retrieval plus a hybrid approach that combines parametric dynamics with retrieval, outperforming other methods on 17/20 combinations.

Latent DynamicsAI summary
2026-10-05 · arXiv:2610.02580

Physical AI Smart Spaces: A Large-Scale Benchmark for Multi-Camera 3D Perception in Smart Spaces

Yuxing Wang, Yizhou Wang, Anqi Li et al.

Physical AI Smart Spaces (PAISS) is a public benchmark for indoor multi-camera, multi-class 3D perception. It provides synchronized 1080p video (up to 282.5 hours) from 1,799 cameras with automatic multi-camera identities, 2D/3D boxes, camera calibration, and depth where available, spanning synthetic generation (Isaac Sim), Cosmos Transfer 2.5 augmentation, and a hidden real-world Sim2Real target. It introduces a 3D HOTA protocol for 3D tracking evaluation.

DrivingAI summary
2026-10-05 · arXiv:2610.02622

CuBEs: Culturally-Situated Behavioral Evaluations and the Limitations of Culture-Blind LLM Judges

Hoda Ayad, Tanu Mitra, Abhishek Mukherji

The authors propose CuBEs (Culturally-situated Behavioral Evaluations), extending an automated evaluation pipeline to inject cultural context into scenario generation and automated judging. Using a human-labeled dataset across 12 cultures and evaluating 13 LLMs, they report significant cross-cultural variation in the presence of six risky behaviors and show that culturally-agnostic evaluations can miss or invert these effects.

AI summary
2026-10-05 · arXiv:2610.02660

SpectralCache: Accelerating Diffusion-Based World Models via Spectral Feature Caching

Zhendong Mi, Pu Zhao, Ziyu Hu et al.

SpectralCache is a training-free spectral feature caching method for diffusion-based world models. It uses SVD on DiT intermediate features to show stable singular subspaces and predictable singular-value evolution across denoising steps. It reuses cached subspaces and estimates only low-dimensional singular values via linear extrapolation or scaling, skipping expensive Transformer evaluations while preserving generation quality. On HunyuanWorld-Voyager-13B, it reports 5.22× acceleration and WorldScore 65.90 for static scenes.