Digest

Reset

122 papers

2026-11-10 · arXiv:2611.12353

QCC-WM: Linking Macro-State Dynamics and Visual Generation via Quotient Causal Construction

Zhizhong Fu, Wei Zhou, Zhaoyang Jiang et al.

QCC-WM is an action-conditioned visual world model that constructs macro-states as quotient-based causal macro-nodes. Action responses guide macro-variable construction and a shared reference quotient measurement constrains consistency between predicted macro-states and observed changes. Macro predictions drive motion-guided reference transport and readback with node-level error correction for future visual generation. On US-VLA, it reports improved ROI-MSE/LPIPS for multi-step ultrasound video prediction.

2026-11-10 · arXiv:2611.12557

WanderBench: Can World Models Explore Dynamic Worlds from Video?

Yuhang Chen, Jiheng Li, Jiale Wang et al.

The authors introduce WanderBench, a benchmark for exploring dynamic worlds from a recorded source video with camera routes given as keyboard actions. Using 745 single-shot clips and 24 sub-metrics over six dimensions, they report that longer horizons degrade camera control and source-dynamics fidelity and that no model keeps revisited places consistent. They propose WanderLand, a training-free conditioning framework, reporting gains up to +8.3 points in source-dynamics fidelity.

2026-11-10 · arXiv:2611.12676

VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess

Mat\v{e}j Kripner, Milan Straka

The authors fine-tune a vision-language-action policy (π0.5) for robotic chess king moves using teleoperated demonstrations and simulated demonstrations from a randomized scripted expert. In simulation, success scales with both data and updates; checkpoint comparisons require a learning-rate cool-down, while validation loss and action error are unreliable for model selection. On the real arm, partial results suggest simulation pretraining helps most when real data is scarce. The authors release the simulator, the expert, and real demonstrations.

RoboticsAI summary
2026-11-10 · arXiv:2611.12678

Library of Hands: Fix the Hands, Not the World Model

Hasaan Ahmad, Qizhen Ying, Yuxuan Sun et al.

Library of Hands (LoH) keeps a frozen world model (e.g., LeWM) and learns only a training-free correction from planner commands to applied actions (“the hand”). It reads the hand by finding which action best explains the change between two camera images under the frozen model, stores each identified hand, and recalls returning hands. In closed-loop simulation, LoH more than doubles success on three tasks after returns (28.6% to 62.3%) while losing only 0.6 pp when the hand never changes (77.5% vs 78.1%).

DrivingAI summary
2026-11-10 · arXiv:2611.12732

$\Sigma$-LeWM: Learning World Models via S-Information for Robot Control

Xinrui Zu, Michael Przystupa, Zhao Yang et al.

Σ-LeWM extends LeWorldModel with two additional auxiliary latent losses: a backward dynamics predictor and an inverse dynamics action predictor. The authors train these auxiliary objectives during training using decoupled contrastive learning (DCL), and frame the overall method as maximizing S-Information across forward, backward, and inverse directions. Experiments on standard benchmarks and simulated robot manipulation show better performance and improved planning efficiency over LeWM.

DrivingAI summary
2026-11-10 · arXiv:2611.12824

DreamLander: A 6-DOF World Model based Reinforcement Learning Planetary Lander

George Nehma, Patrick Quinn, Madhur Tiwari et al.

DreamLander is a DreamerV3-based model-based reinforcement learning method for a realistic 6-DOF entry descent landing problem. Built as an end-to-end capable guidance/control framework, it uses a latent world model (RSSM) to learn dynamics and train an actor-critic policy, with state sensor observations (Ray-Cast height mapping, IMU velocity/angular velocity, and leg contact). The authors report landing within 2 meters of the designated landing zone and velocity norm < 0.3 m/s.

DrivingAI summary
2026-11-10 · arXiv:2611.12870

SWAM: Scaling World Action Models with Cross-Modal Alignment

Yu Zhou, Xinyuan Xia, Xuankun Yang et al.

SWAM trains a world action model to jointly predict language-conditioned video changes and robot actions. The authors curate a ~47k-hour multisource dataset and study supervision and architecture choices for preserving language–video alignment during action learning and improving language–video–action control generalization. The final recipe post-trained on RoboTwin Clean demonstrations only reports 88.10%/61.56% success on Easy/Hard and 40.00% mean SR across three real-world tasks, versus 23.33% for the strongest baseline.

DrivingAI summary
2026-11-10 · arXiv:2611.12885

World In-Context Learning for Agentic Robot Manipulation

Heng Fang, Zehui Guo, Yuehan Zhang et al.

The authors propose World In-Context Learning (WICL): a frozen foundation model starts from a demonstration, rehearses in a world (e.g., simulator) to get feedback, and writes the feedback into its context rather than updating weights. They implement this in RoboScript, learning from a single demonstration per task via rehearsal, freezing a Task Memory, and deploying with data isolation.

DrivingCode availableAI summary
2026-11-09 · arXiv:2611.08568

Personalized health coaching via coachable health age and counterfactual wearable trajectories

Simon A. Lee, Zeinab Esmaeilpour, Menglian Zhou et al.

The authors introduce Coachable Health Age (CHA), an interpretable health-age metric from passively captured, modifiable wearable signals. They also develop Sensor World Model (SensorWM), an action-conditioned wearable world model pretrained on 2.28M person-weeks. CHA and SensorWM are integrated into an agentic counterfactual coaching system that synthesizes personalized 12-week coaching plans preferred by lifestyle coaches and rated safe by clinicians.

DrivingAI summary
2026-11-09 · arXiv:2611.08755

AtariBench: Evaluating Spatiotemporal Reasoning of Vision-Language Models in Atari Gameplay

Mingjia Huo, Yao Fu, Bo Chang et al.

AtariBench evaluates vision-language models as game-playing agents on 20 Atari 2600 games, requiring sequential decisions from high-temporal-resolution gameplay streams. The benchmark decouples in-game dynamics from wall-clock inference speed and tests spatiotemporal reasoning and grounding in the current observation. The authors report substantial limitations in state-of-the-art VLMs, including motion/direction inference failures, brittle sequential decision-making, and over-reliance on textual interaction history.

DrivingAI summary
2026-11-09 · arXiv:2611.08898

FATHOM: World-Model-Guided Diffusion Action-Trajectory Selection for Sparse-Marker AUV Tracking

Xiaorui Wang, Zeyu Sha, Mingyang Yang et al.

FATHOM is a world-model-guided diffusion action-trajectory selection framework for sparse-marker AUV tracking under partial marker visibility. It encodes sparse LED observations with SSME, forms a predictive tracking belief with MT-RSSM using inertial data and action history, uses a diffusion action prior to generate multi-thruster candidate trajectories, predicts their tracking consequences with the learned latent dynamics, and selects the best candidate via receding-horizon objective in closed loop for real-time onboard operation without motion-capture feedback.

DrivingAI summary
2026-11-09 · arXiv:2611.08904

MotionCore: A Latent Neural Simulator for Video Generation with Rigid-Body Dynamics

Zheng Wang, Zhengwei Gong, Yuanzhi Liang et al.

MotionCore learns rigid-body physical dynamics in latent space using a neural operator/PDE-inspired solver, then feeds predicted object trajectories into a controllable video diffusion model via spatially aligned latent controls. The framework uses a diffusion-matching objective plus a low-capacity decodability constraint to preserve trajectory through the visual interface. It evaluates on physical-consistency benchmarks and paired data with known trajectories.

Latent DynamicsAI summary
2026-11-09 · arXiv:2611.08989

LEF-JEPA: Denoising-Trained Latent Energy Fields for Spatial Representation Learning

Zhiyuan Liu, Zhihao Xu, Sicheng Liu et al.

LEF-JEPA strengthens spatial structure in JEPA-style latent representations by learning a target-guided latent energy field and transferring this supervision to the visual encoder via a one-sided relative-energy constraint. The energy module is used only during pretraining (no inference overhead). An Ising-Hamiltonian realization is interfaced with a physical coherent Ising machine for efficient low-energy state search and supports fewer local updates/forward latency on hardware.

Latent DynamicsAI summary
2026-11-09 · arXiv:2611.09058

ProtoCouncil: Multi-Encoder Anatomical Prototypes for Prostate Cancer Detection in Micro-Ultrasound

Mohammad Mahdi Abootorabi, Nooshin Maghsoodi, Armin Saadat et al.

ProtoCouncil uses three foundation encoders (MedSAM, DINOv3, V-JEPA 2.1) to learn anatomical, case-based class prototypes at whole-gland, needle-track, and gland-sector levels for micro-ultrasound prostate cancer detection. It projects prototypes onto real training regions, fuses per-encoder class logits additively, and abstains when encoder predictions disagree.

DrivingAI summary
2026-11-09 · arXiv:2611.09124

Toward the Era of Agentic Embodied Intelligence

Chao Liang, Ke Chen, Fengxiaoxiao Li et al.

The authors introduce A2-MEAS, an embodied multi-agent framework using “code as a scaling interface” for corrective behavior. MetaBrain generates recovery supervision and guidance; diagnostic agents provide grounded evidence; a learned correction encoder and adapter inject correction codes into a vision-language-action policy (A2-WLA) during training, while deployment keeps A2-WLA backbone weights fixed. The system reports strong BEHAVIOR 2026 and real-robot recovery results and releases code and models.

DrivingAI summary
2026-11-09 · arXiv:2611.09293

MDF++: Visuo-Tactile World Action Model with End-to-end Representation Learning

Zixuan Huang, Jinhao Yu, Zachary Charlick et al.

MDF++ is a visuo-tactile world-action model that learns end-to-end, control-relevant latent representations using a latent diffusion backbone. It finetunes observation encoders jointly with a diffusion WAM via multi-task learning (policy learning, world-action modeling, forward/inverse dynamics, and random masking) and a selective gradient-flow rule that blocks gradients from each modality’s own denoising loss to prevent feature collapse. The authors evaluate on contact-rich manipulation tasks including real bolt screwing and nut mating.

DrivingAI summary
2026-11-09 · arXiv:2611.09302

A Stochastic Latent Variable Generalization of Joint-Embedding Predictive Architectures

Nikos Giakoumoglou, Paschalis Giakoumoglou, Tania Stathaki

The authors introduce S-JEPA, a stochastic latent-variable generalization of Joint-Embedding Predictive Architectures (JEPAs). Unlike prior work that uses a deterministic token for the auxiliary variable z, S-JEPA uses a learned prior and posterior (CVAE-style) over z in representation space for masked image prediction with no decoders and no reconstruction. Stochasticity yields predictive uncertainty and diverse completions at no cost to representation quality.

Latent DynamicsCode availableAI summary
2026-11-09 · arXiv:2611.09621

BiLoGo: Reducing the smoothing of localised responses in Transformer-based neural operators for sheet forming

Yingxue Zhao, Haoran Li, Haosu Zhou et al.

BiLoGo is an FE-guided learned simulator for sheet forming that reduces smoothing of localized thinning responses. It combines bidirectional node–element GraphSAGE (no interpolation), a dual-stream Transformer neural operator with separate tokens for blank deformation and tool contact, and gated residual fusion with element updates conditioned on element-node kinematics. The authors evaluate on dome and two corner cases and report lowest positional and threshold-based thinning errors, with higher rib-near smoothing index than token-only Transformer models.

2026-11-09 · arXiv:2611.09625

Encoding Memory in World Models with Test-Time Model-Based Reinforcement Learning

Maxime Burchi, Fayaz Ali Dharejo, Radu Timofte

The authors propose Test-Time Model-Based Reinforcement Learning (TT-MBRL): at test time, they use gradient descent to encode episode-specific memory into world model parameters, learning a distinct world model and policy per evaluation episode. They show improvements on Memory Maze (including a 2D variant) and report new state-of-the-art results on the Memory Maze benchmark, while keeping performance on non-memory tasks like DeepMind Control and Atari 100k.

Latent DynamicsCode availableAI summary
2026-11-09 · arXiv:2611.09684

SURE:Spatiotemporal Uncertainty Representation for Efficient Vision-Language-Action Learning

Jingye Zhang, Choo Sin Wai, Dake Zhong et al.

SURE is a vision-language-action training framework for robots that models structured spatial and temporal uncertainty in demonstrations. It adds a lightweight transposed-convolutional probabilistic action decoder to capture spatial uncertainty with locally correlated action bins, and uses Soft-DTW alignment supervision to tolerate local timing variations between predicted and demonstrated action chunks. The authors report higher manipulation success with reduced data/step requirements on LIBERO, RoboTwin 2.0, and real-world tasks.

DrivingAI summary