Digest

Reset

1167 papers

2026-11-10 · arXiv:2611.10251

TORNADO: Adaptive Latent Stochastic Transport for Calibrated Probabilistic PDE Forecasting

Jorge Mifsut-Benet, Armand Kassa\"i Koupa\"i, Ramon Daniel Regueiro-Espino et al.

TORNADO is a latent-space probabilistic PDE forecasting framework that models the next-step as stochastic transport between VAE encoder posteriors using a learned Itô SDE. Its key idea is state-dependent diffusion (learned drift and diffusion) to adapt predictive uncertainty to local latent dynamics. Training uses simulation-free analytic conditional endpoint targets from a Gaussian interpolant plus an energy-distance regularizer.

Latent DynamicsAI summary
2026-11-10 · arXiv:2611.10316

The World Model Planning Accelerator (WMPA): Hardware-Algorithm Co-Design for Embedded AI in Robotics

Prerana Ramkumar, Suhas Suresh Bharadwaj, Fazal Hameed

The authors analyze the computational and physical cost of embedded world-model planning. They derive information-theoretic and CMOS-based lower bounds for latent rollout steps, then profile a DreamerV3-style RSSM on a GPU and run transient 3D thermal simulations. They report that GPU rollout energy is ~11 orders above a CMOS dynamic energy floor and that sustained planning heats a mm-scale tile to 556 K. They propose WMPA, an accelerator architecture targeting 40× lower power.

DrivingAI summary
2026-11-10 · arXiv:2611.10392

Tiresias: Towards Language-Conditioned World Models

Fengyi Wu, Yifei Dong, Yetong Sha et al.

The authors introduce LCWM, a language-conditioned world modeling task that maps an initial agent-view observation and an instruction to an action sequence fulfilling the instruction, and its benchmark LCWM-Bench spanning 15 datasets across navigation, robotics, 3D gaming, and driving. They propose Tiresias, which plans via selective imagination using a diffusion policy for trajectory proposals, trajectory/visual rankers, and a video-pretrained world model for imagining consequences.

Video World ModelsCode availableAI summary
2026-11-10 · arXiv:2611.10430

Deploying Physically and Epistemically Compliant Agents by Distilling Belief-Space Task and Motion Planners

Phillip J. K. Christoffersen, Rachel Ma, Patrick Barrag\'an et al.

The authors train policies to comply with belief-dependent task rules by offline distillation of belief-space task-and-motion planners. They adapt planner imitation for history-based policies using Dual Averaging and ρ-Optimal Reinforcement. They show completeness conditions for static search under noiseless observation models and evaluate in the new Scavenger domain. The distilled policy achieves success above 90%, versus at most 5% for reward-engineered RL, and transfers to unseen vague language commands and larger floorplans.

DrivingAI summary
2026-11-10 · arXiv:2611.10435

Multi-Objective Adaptation After Deployment: A Position on World Models for Remote Robot Teams

Jamie Santos, Kagan Tumer

The authors present a position that world models could support post-deployment multi-objective adaptation for remote robot teams by predicting policy outcomes under updated estimates of current conditions, enabling policy re-evaluation without risky physical trial-and-error. They outline a pipeline for shared world-model maintenance across agents and identify research priorities: detecting when adaptation is necessary, validating predicted vs realized improvements, and preserving team-level coordination.

DrivingAI summary
2026-11-10 · arXiv:2611.10523

Action Transfer Across Character Body Plans in Latent-Action World Models: An Empirical Study of Animated Characters

Amarnath R

The authors study whether latent-action world models can transfer an action learned from one 2D animated character body plan to a different body plan so that the target performs the action in its own way. They introduce an action-specific transfer score and evaluate five training objectives on a synthetic 2D benchmark (six body plans), Sprites, and 431 DeformingThings4D animal/human animations. They find that fixed-camera “transfer” can be high even when the action is ignored; body-centered transfer reaches only 0.46 for unseen animals and 0.20 for humans.

Latent DynamicsAI summary
2026-11-10 · arXiv:2611.10632

Latent-WAM++: Shaping World and Action Representations for End-to-End Autonomous Driving

Yichen Zhang, Yupeng Zheng, Linbo Wang et al.

Latent-WAM++ is a representation-centered World Action Model for end-to-end autonomous driving. It jointly shapes world and action representations by (1) constructing a temporally predictive, non-degenerate, geometry-aware world representation using Scene-Ego latent prediction + SIGReg + frozen geometric priors (without an EMA target encoder), (2) a one-step multimodal Trajectory MeanFlow planner, and (3) Visual-Action Representation Alignment with a stop-gradient visual target. The authors report SOTA on NAVSIM v2 and zero-shot on HUGSIM.

DrivingAI summary
2026-11-10 · arXiv:2611.10658

The Delta Gap: Cellular World Models for Perturbation Response Prediction Are Far Harder Than Global Correlation Implies

Junhui Gao

The authors show that cellular perturbation world models are often evaluated with global Pearson correlation, which is dominated by non-responsive genes and inflates apparent performance. On Replogle K562, GEARS gets global Pearson 0.977 but delta Pearson only 0.076; a trivial mean-training-delta baseline gets 0.267. They propose delta-space evaluation and show delta-aligned training plus post-hoc corrections improve delta Pearson and direction/gene recovery, with zero-shot transfer across cell lines.

JEPAAI summary
2026-11-10 · arXiv:2611.10660

Dynamics-Informed Action Learning for Goal-Conditioned Latent Planning

Yinuo Jiang, Guangzhi Wang, Yiran Wang et al.

DIAL (Dynamics-Informed Action Learning) trains latent dynamics with rollout alignment and trains a lightweight inverse dynamics head with multi-step inverse supervision. At test time, the inverse and forward dynamics are coupled to generate a goal-directed action sequence that initializes only the mean of CEM’s proposal distribution; standard CEM sampling/refinement then improves the plan. The authors report DIAL outperforms prior latent world-model and planning baselines on four visual control benchmarks.

Latent DynamicsAI summary
2026-11-10 · arXiv:2611.10750

AERO: Adaptive Ensemble-Disagreement Routing for Oracle Feedback in Sample-Efficient Online RLHF

Shrey Singhal, Nannan Huang, Haytham M. Fayek

AERO is a fully online RLHF algorithm that routes oracle preference feedback only to contexts with high ensemble disagreement (vote entropy) and uses ensemble-based synthetic labels elsewhere. It uses a deep ensemble Epistemic Reward Model (ERM) for both epistemic exploration and synthetic labeling, with adaptive batch-level Otsu thresholding and pair-level filtering. The authors report lower oracle-query budgets than online DPO and SEA while keeping competitive alignment performance.

DrivingAI summary
2026-11-10 · arXiv:2611.10771

WorldFoundry: Towards Unified Infrastructure for World Intelligence

Juanxi Tian, Jiadong Hou, Baiang Li et al.

WorldFoundry proposes a functional definition of generative world models as history- and control-conditioned dynamical systems with persistent environmental state, action-/intervention-conditioned forward evolution, and closed-loop observational updates. It presents open-source infrastructure to unify model/benchmark discovery, reproducible inference, artifact management, evaluation, and visualization across video, 3D/4D, embodied action, and interactive worlds.

Latent DynamicsCode availableAI summary
2026-11-10 · arXiv:2611.10806

WARP-JEPA: World-Model-in-the-Loop Amortized Rollout Planning for Frozen Hierarchical Joint-Embedding Predictive Architectures

Junjie Li, Zhaojiacheng Zhou, Kaixiang Wang et al.

WARP-JEPA is an amortized hierarchical planning method for strictly frozen H-JEPA world models. It replaces test-time online gradient descent with learned multi-candidate trajectory proposers (Transformer at the top level, residual MLPs at subordinate two-step bridges). Proposers are trained using model-based rollout costs plus best-of-K behavior cloning, enabling candidate selection without test-time gradient optimization. Across 30 frozen H-JEPA models on four benchmarks, it reports 9.6–18.8× faster inference on long-horizon tasks and up to 33.3 pp improved success.

DrivingCode availableAI summary
2026-11-10 · arXiv:2611.10872

DreamVLA: From Generated Human Motions to Humanoid Vision-Language-Action Policies

Kevin Park, Izat Rasyad, Dvij Kalaria et al.

DreamVLA trains humanoid vision-language-action policies in simulation without new task-specific teleoperation data. It generates human motion trajectories (via DreamControl) and turns them into a residual RL expert on a SONIC backbone that produces physically executed interaction demonstrations. A reference-conditioned DAgger distillation collects only successful expert recovery segments from learner failures using the same reference suffix, then fine-tunes GR00T on the aggregated dataset.

DrivingAI summary
2026-11-10 · arXiv:2611.10967

BAT-WAM: Bridging Action and Touch in World-Action Models for Dexterous Hands

Dong Fang, Feixuan Huang, Yuanxin Zhong et al.

BAT-WAM is a multimodal representation interface for world-action models that bridges tactile contact (Touch) and hand configuration (Joint) using RGB as continuous context. It learns separate codebooks/decoders for Touch and Joint and connects them with an RGB-conditioned continuous latent bridge for bidirectional learning. On HT-Bench, it reduces tactile reconstruction RMSE under full/partial masking versus an image-conditioned HandTouch checkpoint, and improves future tactile prediction versus OpenWAM with HandTouch. Real-robot experiments show higher success rates than OpenWAM baselines.

DrivingAI summary
2026-11-10 · arXiv:2611.10993

DPDP: Diffusion Policy-Steered Diffusion Planner for Offline Reinforcement Learning

Ruyi An, Mingyuan Zhou

DPDP combines a diffusion planner with a value-optimized diffusion policy at the planner’s execution step in offline RL. The planner proposes a next-state transition; a diffusion inverse-dynamics executor samples an action as a Bayesian posterior that keeps the planned transition likelihood while replacing the behavior prior with a critic-guided policy (DPDP-inf via training-free score composition/correction, DPDP-ft via one executor fine-tuning).

DrivingAI summary
2026-11-10 · arXiv:2611.11085

ObjectWorld: Scaling Object-Centric World Models with A Large-Scale 3D Full-Object Flow Dataset

Haonan Chen, Sicheng Isabella Lu, Haitian Li et al.

ObjectWorld introduces 3D full-object flow, an object-centric motion representation with dense 3D trajectories over an object’s complete surface (including occluded regions), valid for rigid and deformable objects. The paper also presents a 1210.1-hour 3D full-object flow dataset built from real-world videos and physical simulation, plus a world model that predicts dense full-object flow from an image and task description for robot manipulation execution.

DrivingAI summary
2026-11-10 · arXiv:2611.11136

From Game Generation to Game Playing: Fully Automated Ability Training for Vision-Language-Action Models

Changran Liu, Qi Wu, Wenchao Ma et al.

The authors present Script2Play, a framework that automatically generates game environments and expert demonstrations from skill specifications, using an LLM-powered coding agent plus programmatic teachers that act through keyboard and mouse. They post-train a 4B vision-language model with behavior cloning on 150.3 hours of synthetic Minecraft and Roblox gameplay, without human demonstrations or reinforcement learning, and evaluate on multiple game benchmarks.

AI summary
2026-11-10 · arXiv:2611.11140

SurgWorld: Learning Surgical Representations Beyond Observation via Latent Prediction

Jingkun Yue, Pengxiang Wang, Siqi Zhang et al.

SurgWorld is a video-native, self-supervised framework for surgical representation learning beyond visible observations. It treats a relative spatio-temporal shift as a pseudo-action and uses Out-of-Sight Dynamics Modeling (OSDM) to predict the target latent representation from a partial context. A Geometric Visual Similarity (GVS) loss aligns patch-level relational structures with a frozen teacher. An agglomerative training framework integrates multiple teachers to learn unified latent dynamics and geometry.

Latent DynamicsAI summary
2026-11-10 · arXiv:2611.11166

DexForesight: Grounding Dexterous Policies with Implicit Spatial Foresight

Zhongxi Chen, Yi Yang, Xinyi Chen et al.

DexForesight trains an action-conditioned spatial world model from offline scene depth, hand geometry, and demonstration trajectories to implicitly learn spatial foresight (3D hand-point motion and visual feature changes). The world model is frozen and its predictive query representations are aligned to a dexterous policy’s latents via query alignment, enabling deployment with only RGB observations, language, and robot state (no depth input or explicit future prediction).

DrivingAI summary
2026-11-10 · arXiv:2611.11210

DynOLAT: OLAT Video Prediction for Controllable Dynamic Full-Body Relighting

Taoming Liu, Yinji ShenTu, Guangzhao He et al.

DynOLAT is an OLAT video prediction framework for controllable dynamic full-body relighting. It predicts temporally consistent One-Light-At-a-Time (OLAT) basis videos from flat-lit inputs, then linearly combines predicted bases using weights from a target environment map. The authors introduce a dynamic capture pipeline and a large-scale dynamic full-body OLAT dataset (over 24M frames) and report improved lighting-response fidelity and temporal consistency versus tested baselines.