Digest

Reset

1167 papers

2026-11-05 · arXiv:2611.06901

Motion-guided Contrastive Regularization For Latent World Models

Yiwen Chen, Qiqi Hou, Jisoo Jeong et al.

The authors identify “static collapse” in JEPA-style latent world models trained with distributional regularization (e.g., SIGReg): encoders may preserve episode-specific static background while suppressing manipulation-relevant state changes. They propose motion-guided contrastive regularization that matches predicted latent transitions to candidates from the same episode, using optical-flow similarity to create soft targets. They report large planning gains on OGBench-Cube-RB and LIBERO-Object versus LeWM.

Latent DynamicsAI summary
2026-11-04 · arXiv:2508.05941

Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distribution

Zhanyi Sun, Shuran Song

Latent Policy Barrier (LPB) improves visuomotor policy robustness under covariate shift by treating latent embeddings of expert demonstrations as an implicit barrier separating in-distribution from out-of-distribution states. It trains a base diffusion policy on expert data and an action-conditioned latent dynamics model on both expert and rollout data; at inference, it steers actions using latent-space optimization to stay near the expert latent manifold.

RoboticsAI summary
2026-11-04 · arXiv:2611.03937

FlowBench: Counterfactual Imagination Training for Spatial Intelligence

Yifan Liu, Baihan Zhang, Khai Loong Aw et al.

FlowBench defines Counterfactual Spatial Imagination (CSI): answering spatial questions about a scene after a hypothetical object motion without observing the outcome. It provides ~428K training examples (sim + real) and a fully automatic, simulator-derived pipeline with exact post-motion supervision (frames/flow). The authors report VLM “imagination gaps” and show Qwen3-VL-8B fine-tuning on FlowBench-Sim improves accuracy on both sim and real; they also show world-model training improves text-to-flow prediction.

Latent DynamicsCode availableAI summary
2026-11-04 · arXiv:2611.04006

Controlling Action-Head Co-Adaptation to VLM Representation Anisotropy

Ruijie Zhang, Jiayin Zheng, Yequan Zhao et al.

The authors study how an action head co-adapts to anisotropic channel variance in pretrained VLM representations in vision-language-action (VLA) models. They observe that head sensitivity progressively concentrates on high-variance VLM dimensions. They propose AdaIE, a parameter-free interface reweighting method based on online sensitivity estimates, improving success rates on RoboCasa-GR1, RoboTwin 2.0, real-world manipulation, and also generalizing to diffusion policies and world models.

DrivingAI summary
2026-11-04 · arXiv:2611.04010

Decision Transformer with Task-Aware Representations for Offline Meta-Reinforcement Learning

Xinbo Zhao, Yuhang Liu, Xin Zhang et al.

CoTA-DT is a DT-based offline meta-RL method that learns task latents from four joint signals: contrastive trajectory encoding, alignment to natural-language task descriptions, one-step supervision from a task-conditioned world model, and long-term supervision from a task-conditioned Q function. A language-grounded self-adaptive strategy enables zero-shot and few-shot adaptation to unseen tasks. The pretrained LLM backbone is shared across components with a LoRA adapter.

DrivingAI summary
2026-11-04 · arXiv:2611.04016

Physical Experience Builds Continual Spatial World Models

Yuhang Sun

The authors study when repeated physical experience makes a spatial world model more useful for continual navigation rather than just more persistent. In a dynamic maze, cross-episode Spatial Change Memory learns transferrable relations between environmental changes, while state-action-outcome experience refines motion prediction. Structured predictability improves future routing and action selectively; advantages vanish when past changes are uninformative.

DrivingAI summary
2026-11-04 · arXiv:2611.04028

Compression Forcing: Hierarchical History Tokenization Enables Flexible World Models

Siqiao Huang, Partha Kaushik, Michael Chen et al.

The authors propose Compression Forcing (CF), a diffusion-based world-model framework that aligns history compression with predictive utility. It uses Dynamics-Aware Tokenization (DAT) to encode history as a hierarchical token sequence ordered by dynamics saliency, and Hierarchical History Guidance (HG-H) to amplify dynamics-salient history at inference via score composition over variable-length prefixes.

2026-11-04 · arXiv:2611.04054

Forgetting to Improve: Principled Data Removal in Active Learning

Manuel Wendl, Erik Englesson, Andreas Krause et al.

The authors propose “Forgetting to Improve” (F2I), a principled data-removal method for Gaussian process models in transductive learning and sequential decision tasks (including Bayesian optimization and model-based reinforcement learning). F2I greedily removes samples whose removal reduces predictive uncertainty within a target region, using influence functions that decompose the uncertainty into epistemic and aleatoric components under model mismatch.

Latent DynamicsAI summary
2026-11-04 · arXiv:2611.04073

Computer-Use Agents Gain Little Benefit from a World Model When Conditioned on Tutorials

Anh Nguyen, Tianyi Xu, Phat Tran et al.

The authors study how much world-model look-ahead helps computer-use agents when they are already conditioned on task-specific tutorials. They introduce OSWorld-Tutorial and ScienceBoard-Tutorial and propose RAG-PolicyJudge (RAG-PJ). With tutorials present, RAG-PJ gains little benefit from either zero-shot text rollouts or oracle visual rollouts, even with perfect future screenshots; the bottleneck is the judge’s reasoning over futures and recognizing task completion.

DrivingAI summary
2026-11-04 · arXiv:2611.04241

AceAction-5M: Hierarchical Goals for Generalist Action Learning Across 3D Game Worlds

Kailing Li, Ruoxuan Zhang, Zheyuan Zhou et al.

AceAction-5M is a multi-game dataset of human gameplay in 3D games, with hierarchical language annotations for complete tasks, temporally bounded stage-level subgoals, and local skill instructions aligned to frame-level keyboard-and-mouse actions. The authors build annotations by identifying subgoals from objective/interaction-target changes, grounding navigation goals to visually recognizable waypoints, and composing adjacent subgoals into task instructions.

2026-11-04 · arXiv:2611.04371

Embodied AI for Scientific Discovery: From Physical Action to Scientific Evidence

Wenhao Xu, Changwei Wang, Shunpeng Chen et al.

This review defines “embodied AI for science” as learned decision procedures coupled to physical experiments with partial observability, costly/irreversible actions, and feedback that updates subsequent decisions. It organizes prior work using three closures—operational, epistemic, evidential—within a record-linked evidence cycle (Record–Diagnose–Calibrate–Confirm). It proposes an evaluation and reporting framework that separates actions, valid experiments, campaign progress, and confirmed claims.

DrivingAI summary
2026-11-04 · arXiv:2611.04392

Aero-VPA: Few-Step Flow-Matching Control over Non-Reconstructive JEPA Latents for Agile Flight on Edge Hardware

Yeonseok Lee

Aero-VPA is a lightweight Vision-Predictor-Action architecture for agile 3D navigation on edge hardware, using a non-reconstructive JEPA latent world model plus a Conditional Flow Matching controller. The sampler generates a velocity/yaw-rate chunk in two Euler steps, requiring five sequential network evaluations per decision, with an ensemble-uncertainty fallback and a map-free runtime shield. The authors report first closed-loop PX4 SITL results in procedurally generated forests.

DrivingCode availableAI summary
2026-11-04 · arXiv:2611.04492

Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation

Jimmy Wang, Yifan Chou, Kaifeng Zhang et al.

World Models Radar: 3D World Simulator is an action-conditioned multimodal 3D world model that predicts appearance, geometry, and tactile observations using a unified 3D particle-voxel representation. It combines a hybrid particle-voxel autoencoder (compress point clouds to 3D voxel latents, decode to 3D Gaussian particles) with a 3D latent dynamics model that predicts latent evolution, enabling multi-view consistent rendering and long-horizon interactive simulation at 50 Hz.

RoboticsAI summary
2026-11-04 · arXiv:2611.04527

Future Representation Distillation for Imagination-free World Action Models

Jaehyeong Jo, Sangwon Jang, Sihyun Yu et al.

DreamFree is an imagination-free World Action Model that repurposes a pretrained video diffusion model as a predictive visual encoder via future representation distillation. A visual encoder predicts a frozen diffusion model’s representations from the current observation only, enabling action prediction without generating future videos. The authors report higher success rates than state-of-the-art VLA and WAM baselines and real-time closed-loop control at 11Hz with a 14B video backbone.

DrivingCode availableAI summary
2026-11-04 · arXiv:2611.04566

A research world model of power engineering learns its dynamics and reveals what-ifs

Fanping Sui, Ying Zhang, Liwei Lin

The authors introduce “research world models,” treating a research field as a dynamical system with interventions, and build one for power engineering from 1.1M papers. Typed concept hypergraphs are read from abstracts; two graph-temporal cores learn concept growth and first-time method–target links; intervention operators estimate what-if effects from natural experiments. In temporally held-out tests they forecast years ahead and match held-out intervention effects, and they answer natural-language what-ifs as paired simulations.

Latent DynamicsAI summary
2026-11-04 · arXiv:2611.04571

Ground-Wing: A Parametric 3D Wing-in-Ground-Effect Dataset for Benchmarking Neural Surrogate Models

Andrea Cenzato, Lukas Karch, Andre Luckow et al.

Ground-Wing provides a CFD dataset of 3D wing-in-ground-effect aerodynamics for benchmarking neural surrogate models. It includes 770 RANS simulations of 194 parametrized wing configurations spanning 27 airfoil variants and four ground distances, and evaluates six neural surrogate architectures on predicting surface pressure coefficient and wall shear stress fields, from which lift and drag are recovered. The dataset is available on Hugging Face.

2026-11-04 · arXiv:2611.04614

Policy Network Diffusion

Jiate Liu, Deheng Ye

The authors propose DPPG, a diffusion-based method to generate reinforcement-learning policy network parameters. DPPG uses a Transformer-based autoencoder that compresses policy parameters by patchifying them, and a task-conditioned latent diffusion model that generates parameter latents conditioned on environments. A single unified model is trained across Classic Control, Atari, and MuJoCo, generating competitive policies on seen and unseen environments.

DrivingAI summary
2026-11-04 · arXiv:2611.04711

Cycle WAM: Coupling Action and Video through Cycle Consistency

Qinglin Yang, Jiepeng Wang, Yuxing Huang et al.

Cycle WAM is a unified training framework for world action models that couples video and actions using bidirectional conversion and cycle-consistency. Starting from video–action co-training, it trains video→action (inverse dynamics) and action→video (forward dynamics), then adds a differentiable cycle-reconstruction objective to better align actions with their visual consequences and exploit pretrained video priors. The authors report improved control and dynamics on LIBERO-Pro and DROID.

DrivingAI summary
2026-11-04 · arXiv:2611.04839

Coordinating Humanoid Loco-Manipulation through Action-Conditioned Latent Foresight

Ju Dong, Xinyu Xiao, Yiming Zhou et al.

The authors present action-conditioned latent foresight, a latent world–action model that scores sampled motion–hand command sequences before execution using predicted visual progress toward demonstration-derived subgoals and a learned joint temporal prior. The robot executes the first commands of the best-scoring sequence and replans each step. An action consistency regularization reconstructs the first demonstrated command from observed and predicted feature changes.

Latent DynamicsAI summary