Digest

Reset

1167 papers

2026-10-21 · arXiv:2610.27387

Training Controllable World Models via Action Influence Maximization

Samuel M. Bateman, Tenny Yin, Chongyi Zheng et al.

Action-conditioned video world models trained on successful demonstrations can ignore commanded actions due to insufficient action influence in the dataset. The authors analyze an explained-action-information metric and introduce Action Influence Maximization (AIM) to actively collect data that increases dataset mutual information about actions and next states. In pixel manipulation (Push-T), AIM is reported as the only method that consistently improves controllability, matching a 300k expert model with only 500 actively collected episodes added to 3k demonstrations.

Video World ModelsCode availableAI summary
2026-10-21 · arXiv:2610.27657

PLaCA: Learning Patch-Level Future Latent Change for Action Generation

Zhan Yi, Yuchen Wang, Chao Li et al.

PLaCA is a world-to-action VLA framework that predicts spatially structured patch-level future latent change from current multimodal input, using change-aware supervision to weight patches by ground-truth change magnitude. A future-to-action interface conditions the action representation on the predicted future latent change. The authors report improved manipulation performance on LIBERO-Long and CALVIN.

DrivingAI summary
2026-10-21 · arXiv:2610.27782

RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation

Siyuan Ma, Yu Qi, Cheng Ye et al.

RoboJudge benchmarks and trains multimodal LLM judges for embodied video generation. RoboJudge-Bench uses 800 human-annotated videos scored on Physical Adherence (physical realism/consistency) and Instruction Alignment (agent/object/goal correctness). The authors fine-tune a 9B judge (RoboJudge-9B) on RoboJudge-Train (12K videos, 25K human-verified scores), improving agreement with humans and enabling downstream video generator ranking.

2026-10-21 · arXiv:2610.27851

LaLa-World: Scaling Latent World Models with Natural Language Actions

Junha Song, Geonmo Gu, Jeongeun Park et al.

LaLa-World is a language-conditioned latent world model that uses language as a shared interface for numerical actions from simulation/robotics and visual changes described in language from image/video data. Built with a language-model-based predictor and an MPC imagination planner, it reports higher average MPC planning performance than existing LWMs and comparable performance to an RAEv2 generative baseline with less training and faster planning.

Latent DynamicsAI summary
2026-10-21 · arXiv:2610.27890

EndoWorld-JEPA: Motion-Guided Global-Local Latent World Modeling for Endoscopic Video Representation Learning

Jie Wang, Shilong Yang, Tao Zhou et al.

EndoWorld-JEPA is a causal motion-guided latent world-model pre-training method for passive endoscopic video. It uses an Optical Motion-guided (OM) module to decompose dense flow into a temporal motion condition and a patch-wise local-motion magnitude map, then predicts next latent states with complementary global and motion-local JEPA objectives. A common pre-trained checkpoint transfers strongly across four downstream endoscopy benchmarks.

2026-10-21 · arXiv:2610.27914

World Models with Configurable Perception

Yijun Shen, Delong Chen, Xianming Hu et al.

ConfigWM is an action-conditioned world model for planning under dynamic distraction. It uses a language task query to configure perception and map visual embeddings into a task-specific predictive state space, suppressing task-irrelevant distractor motion. The learned dynamics are then rolled out in this space for MPC planning.

Latent DynamicsAI summary
2026-10-21 · arXiv:2610.27921

Quantum-JEPA: Quantum Born Priors for Latent-Action JEPA World Models

Zhiyuan Liu, Sicheng Liu, Haiyu Wu

Quantum-JEPA is a latent-action JEPA framework that learns a distribution of future representations using a conditional quantum Born prior. Circuit measurements generate binary latent actions that a shared predictor maps to alternative future representations. The quantum prior is trained via a sampled energy objective using finite-shot parameter-shift gradients, and the authors also test feasibility on a real QPU.

Latent DynamicsAI summary
2026-10-21 · arXiv:2610.28011

Knowing When to Stop Thinking: A Dissection of Expected Free Energy as a Test-Time Compute Controller for Latent World-Model Planning

Tong Zhang, Zhiyuan Shi, Yun Peng et al.

The authors study when expected-free-energy (EFE) style signals can control test-time compute for latent world-model planning. They evaluate candidate scoring, stopping, and online predictor updates separately. Cost-stall stopping reduces refinement on tasks where success plateaus with depth, but can fail when success peaks and further optimization exploits model errors. Online updates recover part of losses from missing training coverage.

Latent DynamicsAI summary
2026-10-21 · arXiv:2610.28060

TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment

Yi Wen, Hao Chen, Wanyu Wang et al.

TrajEvolve proposes trajectory evolution for long-horizon reinforcement learning using hindsight credit assignment. It reformulates step-level credit as a binary “necessary vs redundant” classification, obtains step labels via environment replay (hindsight verification), trains a discriminator, and uses redundancy-aware reward shaping plus step-level advantage adjustment. It also generates “evolutionary trajectories” by refining historical trajectories (redundancy elimination, format correction, error rollback) and re-executes them for updated training rewards.

AI summary
2026-10-21 · arXiv:2610.28074

Slot-JEPA: Object-Structured Prediction Interfaces for Latent World Models

Peisong Wen, Qianqian Xu, Minye Lei et al.

Slot-JEPA proposes a slot-and-mask interface for latent joint-embedding predictive world models. A pretrained slot decomposition module factorizes dense patch representations into stable semantic slots and time-varying spatial masks, then is frozen for downstream prediction and control. A Transition Memory Predictor (TMP) retrieves compact, slot-aligned transition context from a memory bank. Unbalanced Optimal Transport (UOT) supervises mask prediction and provides a displacement-aware goal evaluation signal for MPC.

DrivingAI summary
2026-10-21 · arXiv:2610.28086

MoDDiT: Simple Token Routing in Diffusion Transformers

Jarek Liesen, Matthew T. Jackson, Benjamin Ellis et al.

MoDDiT modifies diffusion transformers to route token computation through both MLP and self-attention operations. The MLP routing uses dynamic, token- and timestep-dependent compute allocation, while self-attention uses fixed-capacity routing. Training and inference use fewer tokens/FLOPs while improving quality–compute tradeoffs on ImageNet, improving 1X video prediction PSNR at 12.5% capacity, and enabling sparse distillation of Wan2.1 with modest VBench score reduction.

2026-10-21 · arXiv:2610.28122

De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control

Shaoyi Zheng, Zhongtao Jiang, Yue Peng et al.

Cascade WAMs accelerate control by predicting visual futures, but diffusion experts require costly iterative inference. The authors show that removing video/action stochasticity has limited impact, while naive one-step truncation sharply reduces control success. They propose DirectWAM, converting diffusion-pretrained video/action experts into a deterministic single-step future-to-action cascade trained by action-sufficient distillation. DirectWAM achieves 97.1% average success on four LIBERO suites and 85.4% on RoboTwin 2.0, with a 28× lower inference latency than the full iterative teacher.

DrivingAI summary
2026-10-21 · arXiv:2610.28166

Reasoning as a Distribution: Generative Plan Latents for Vision-Language-Action Models

Matteo Cercola, Katie Z Luo, Milan Ganai et al.

The authors introduce Generative Plan Latent (GPL), a low-dimensional conditional distribution over task-level plans for Vision-Language-Action models. They argue that representing the reasoning state as addressable plans (sample, steer, and reweight by outcomes) preserves valid alternatives that vector-only representations collapse. Without retraining the VLA, steering increases commanded-mode execution from 0.48 to 0.94 in SimplerEnv; action-level guidance shows no reliable gain.

DrivingAI summary
2026-10-21 · arXiv:2610.28431

Plausible Future, Wrong Choice: Steering World-Action Models toward Intended Future

Zhuohao Li, Jiawei He, Yuxuan Yan et al.

The authors identify “Stage Confusion” in world-action models (WAMs): locally plausible rollouts can advance to the wrong task stage when similar observation-action segments occur across tasks or stages. They propose TaskWAM, a closed-loop, stage-aware framework using Mode Tokens (task requirements) and State Tokens (execution progress) to steer world-action generation toward intended future. They also introduce StageBench for controlled multi-stage evaluation.

DrivingAI summary
2026-10-21 · arXiv:2610.28432

Physical AI for Intralogistics: Benchmarking the NVIDIA Sim-to-Real Ecosystem for Cosmos-Enhanced COMPASS Navigation on Industrial Mobile Robots

Charbel Abi Hana (Idealworks), Rayan Zaitouni (Idealworks), Kameel Amareen (Idealworks) et al.

The authors benchmark NVIDIA X-Mobility + COMPASS for learned navigation on Idealworks industrial mobile robots, studying reliability under sim-to-real transfer. In sim, they extend COMPASS with iw.hub and HMP embodiments and an Idealworks R&D center digital twin. In the real facility, they fine-tune X-Mobility on digital-twin data, optionally augmented with NVIDIA Cosmos, then train COMPASS residuals and compare on success/efficiency/safety/smoothness/compute.

DrivingAI summary
2026-10-21 · arXiv:2610.28458

LocaLeJEPA: Maximum Entropy Representations under Geometric Priors

Kartik Tandon, Alessio Borgi, Pavel Snopov et al.

LocaLeJEPA extends LeJEPA’s entropy-maximizing JEPA objective to non-Euclidean embedding geometries by using a locally Euclidean distribution-matching regularizer (LocaLeReg). It partitions each batch of embeddings into K latent-space neighborhoods (K-means), then matches residuals within each neighborhood to a uniform distribution on a Euclidean ball via an Epps-Pulley-based characteristic-function test. On ImageNet-1K pretraining, it outperforms LeJEPA across linear and k-NN and on multiple downstream datasets, with code released.

JEPACode availableAI summary
2026-10-21 · arXiv:2610.28480

SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model

Qiayuan Liao, Yiyang Shao, Junfeng Long et al.

SuperMimic amortizes humanoid whole-body motion tracking across many motions using one learned differentiable world model of shared dynamics. The world model and tracking policy train together in one loop, using backpropagation-through-time through imagined short rollouts for analytic policy gradients. The authors report surpassing SONIC’s tracking return in 3 hours on one GPU and reducing GPU-hours by more than 1000×.

DrivingAI summary
2026-10-21 · arXiv:2610.28483

Exploring Temporal Context-Aware Physical Grounding in Latent World Models

Yilei Man, Yang Chen, Jingcai Guo

The authors propose 𝒢-JEPA, a latent JEPA world model that adds temporal context-aware physical grounding and combines the learned model with prior-guided action planning. They use a shared causal S4D-based temporal module with physical state supervision during training, and during planning mix learned latent-conditioned diffusion action proposals with Gaussian CEM refinement. They report improved planning performance on manipulation tasks, including increasing goal-distance settings.

Latent DynamicsCode availableAI summary
2026-10-21 · arXiv:2610.28519

Touch-JEPA: Tactile Representation Learning from Hand-Object Video with Limited Tactile Supervision

Minseong Kweon, Qiao Feng, Yiming Huang et al.

Touch-JEPA learns tactile representations from hand-object RGB video with limited tactile supervision. It uses geometry-guided self-supervised masked prediction to learn from video alone, then adds joint training where tactile contact and magnitude supervision is available only for a labeled subset. The authors report improved contact and tactile magnitude decoding from frozen features and transfer to robotic hands without robot tactile training.

DrivingAI summary
2026-10-21 · arXiv:2610.28554

OverSeer: Multi-Agent World Models Beyond What Agents See

Runjia Li, Kam Woh Ng, Minghao Chen et al.

OverSeer is a multi-agent video world model that augments egocentric agent views with K fixed-pose auxiliary camera views. Fixed in space, auxiliary views act as a proxy for a shared world reference and improve world and agent consistency. Auxiliary streams are efficiently compressed using a δ-VAE that encodes residuals relative to a static reference frame, reducing token overhead. Experiments on Minecraft and Doom show improved consistency and qualitative results on real multi-camera captures.