Digest

Reset

1180 papers

2026-09-30 · arXiv:2609.36582

Inferring Soil Friction Angle from Robot Foot-Ground Force Histories: A Bayesian Inverse Approach to Proprioceptive Soil Sensing

Dawei Xu, Zhijie Wang

The authors test whether dry, cohesionless soil internal friction angle ϕ can be identified from proprioceptive foot–ground force histories. They generate a 2D plane-strain continuum forward model (MPM) for a rotating leg, benchmark it against measured rotating-leg force histories, fit two Gaussian-process surrogates (one per force component), and perform Bayesian inversion over the full history. In matched-model recovery, 14 off-grid friction angles are recovered with median absolute error ~0.1° (max ~0.7°).

AI summary
2026-09-30 · arXiv:2609.36605

RoboChrono: A Real Robot Benchmark for Streaming Task Understanding

Yuzhou Wu, Longteng Fan, Zimeng Li et al.

RoboChrono is a real-robot benchmark for streaming task understanding with 39 scenarios and 34,713 evaluation instances, using a causal observation boundary (only visual observations up to the query time). Across 18 multimodal models under zero-shot evaluation, the authors report large task-dependent capability differences, especially on temporal ordering, view matching, and action time localization, plus input-ablations showing different dependencies on visual evidence.

DrivingAI summary
2026-09-30 · arXiv:2609.36677

ReWorld-Track: A Recursive Event World Model for Language-Guided Multi-Camera Tracking

Haoyang Wu, Shoudong Han, Chaoyue Li et al.

ReWorld-Track is a recursive event world model for language-guided multi-camera tracking that carries association uncertainty forward through handoffs with a persistent recurrent belief. It predicts the next camera, arrival time, and entry region, then updates this belief using posterior probabilities over candidate matches and a temporary-null (no-match) alternative. The authors report improved identity continuity and HOTA on CityFlowV2 and MTMMC.

AI summary
2026-09-30 · arXiv:2609.36720

T$^2$Mem: Learning Test-Time Memory for Robotics

Yize Liu, Huang Huang, Yining Hong et al.

T²Mem is a test-time training framework that turns a memory-free pretrained vision-language-action policy into a robotics controller with fast-weight test-time memory. It uses an observation-grounded interface to encode vision-language history into compact fast weights via online self-supervised updates, and action supervision plus alternating memory–policy learning shapes what is stored and how it is used. On RoboMME (16 tasks), it improves average success from 17.93% to 56.83% over the memory-free base policy.

DrivingAI summary
2026-09-30 · arXiv:2609.36810

MeteoVerse: Unified Weather-Controllable Video World Model

Renlong Wu, Guanqiao Wang, Xuan Shang et al.

MeteoVerse is a unified weather-controllable video world model for camera-controlled future prediction from either sunny or adverse-weather inputs. It explicitly estimates observed and target weather states (rain/snow/fog), derives a weather transition, and uses a transition-aware mixture of weather experts (MeteoMoE) to inject residual weather features into a frozen video backbone. It also introduces a dataset of over 50K real-world weather clips with sunny pseudo-pairs and weather-intensity annotations.

2026-09-30 · arXiv:2609.36845

DSWM: Decomposed Spatio-Temporal World Model for Demand-Driven UAV Base Station Repositioning

Shengjie Zhong, Zhongliang Zhao, Jingxuan Chen et al.

DSWM is a decomposed spatio-temporal world model for demand-driven UAV base station repositioning. It uses a recurrent state-space model trained with an EMA-based latent predictive objective plus variance regularization, a differentiable service simulator head (association, probabilistic LoS, Shannon rate, capped fulfillment), and observation-anchored CEM planning that executes only the first action of the best imagined rollout.

Latent DynamicsAI summary
2026-09-30 · arXiv:2609.36851

RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts

Hongbin Lin, Chaoda Zheng, Yiming Yang et al.

RoXDrive is a closed-loop RL post-training framework for end-to-end autonomous driving using an action-conditioned video world model. It introduces an Action-Vision Faithfulness Evaluator (AVFE) trained with geometry-aware auxiliary trajectory supervision to filter action-faithful long-horizon rollouts, then performs dense safety-aware scoring and scene-level closed-loop GRPO using only faithful episodes.

DrivingAI summary
2026-09-30 · arXiv:2609.36952

ER-JEPA: Experience Replay Improves Joint-Embedding Predictive Learning in Language Models

Jingnan Pu, Zi-En Fan, Feng Lian

ER-JEPA adds an episodic replay pathway to LLM-JEPA. It stores training token pairs in a memory and, at each step, retrieves relevant stored examples to provide additional supervision for both token prediction and representation alignment. The replay path is removed after training, so inference matches LLM-JEPA.

JEPAAI summary
2026-09-30 · arXiv:2609.36985

Abductive World Modeling via Causal Representation Learning

Ziqi Liu, Songhan Yang, Linfan Zhou et al.

The authors propose Abductive World Modeling (AWM), which learns structured causal representations by “predict forward, then abduce backward.” Using the Hierarchical Abductive State Pyramid (HASP), AWM infers a hierarchical latent state with Entity, Dynamic, and Relation components by jointly reasoning over the current observation and its predicted future. Experiments on physical prediction, causal reasoning, and action understanding show consistent gains over a V-JEPA 2 backbone-only baseline.

Latent DynamicsCode availableAI summary
2026-09-30 · arXiv:2609.37004

World2Motion: Turning Video World Models into 3D Human Motion Generators

Tu Fangyuan, Xiangyue Zhang, Yiyi Cai et al.

World2Motion adapts Cosmos 3 into a single-stage generator that produces scene-aware 3D human motion and corresponding video from a single image and a text prompt. The authors build a hybrid training dataset (synthetic video–motion pairs plus real videos paired with estimated 3D motion) and introduce a shift-decoupled noise schedule to reduce motion jitter. On a multi-source interaction benchmark, it improves motion–text alignment and scene interaction and matches two-stage interaction success with ~3.3× faster inference.

2026-09-30 · arXiv:2609.37089

Real2Gym: Building Gyms from Videos, Bringing Skills to Robots

Kerui Ren, Yingxiang Xu, Kaiwen Song et al.

Real2Gym is an agentic Real2Sim2Real framework that builds editable, physics-validated simulation gyms from human and robot demonstration videos. It reconstructs aligned Blender+MuJoCo scenes, validates/retargets actions via native physics, augments feasible task variations, and distills successful executions into reusable, object-relative skills for simulation and real robots without model weight updates.

DrivingAI summary
2026-09-30 · arXiv:2609.37098

V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving

Junwei You, Weizhe Tang, Can Wang et al.

V2X-WAM is a cooperative world action model for end-to-end autonomous driving. It builds a reliability-aware spatiotemporal scene representation from ego and infrastructure observations, compressing infrastructure features into a compact quantized message. A multimodal planner generates prospective trajectories; the selected actions condition future occupancy and dynamic-flow prediction, and the predicted consequences are fed back to refine the final trajectory.

DrivingAI summary
2026-09-30 · arXiv:2609.37107

Waypoint-1.5: A Real-Time Video World Model for Consumer Hardware

Rajit Rajpal, Shahbuland Matiana, Liew Wei Pyn et al.

Waypoint-1.5 is a real-time diffusion video world model for interactive generation on consumer GPUs. It uses a single-stream causal Diffusion Transformer over TAEHV 1.5 latents and is conditioned on full keyboard+mouse inputs. Trained on 100,000 hours of control-aligned gameplay, it produces playable rollouts with a KV-cache runtime and distillation that reduces denoising to 4 steps per frame. It is benchmarked via latent FPS throughput.

2026-09-30 · arXiv:2609.37156

Lucid Dreaming for World Models: Learning to Doubt Imagination and Decide by Trust

Ziqi Wen, Ting Xu, Lianyu Wang et al.

LucidWM is a world model that learns “doubt” about imagined transitions from experience using Subjective Logic, then turns doubt into “trust” that compounds along imagination. Trust reweights λ-returns for learning and guides action choice. It needs no extra parameters or forward passes for uncertainty estimation, and the authors report earlier alarms for drift and fewer steps to reach a goal in a navigation study.

Latent DynamicsAI summary
2026-09-30 · arXiv:2609.37187

InsightMap: Structured Spatial Modeling for Embodied Multimodal Reasoning

Hongpei Zheng, Hujun Yin

InsightMap is a framework that uses top-down maps as explicit spatial memory and action-conditioned prediction targets. It links historical views to labeled map locations and trains a shared multimodal backbone (based on BAGEL) to jointly predict navigation actions and post-action map generation, with QA and 3D grounding sharing the same spatial interface. The authors report improved navigation and competitive static spatial reasoning and grounding results.

DrivingAI summary
2026-09-30 · arXiv:2609.37250

V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents

Yang Zhang, Jiangyuan Zhao, Chenyou Fan et al.

V-JEPA Policy trains a world-action model on the latent space of a frozen V-JEPA 2.1 encoder, without inheriting a pretrained visual generative model. The authors jointly learn an instruction-conditioned future-latent predictor and a flow-matching action expert from scratch in one downstream stage, using the predictor’s layer-wise context key–value states to condition action generation.

Latent DynamicsAI summary
2026-09-30 · arXiv:2609.37378

Do-JEPA: From Masking to Intervention in Latent World Models

Hossein Resani, Javen Qinfeng Shi

Do-JEPA trains latent world models using paired simulator rollouts from the same saved state under an action a and a reference action a∅, supervising the latent difference Δz. The objective decomposes into effect, support, propagation, and invariance losses. In pixels and learned slots, the authors report improved action-causal (responsive) effect prediction versus masking-style baselines, and in fine-tuning reduces the factual-cost of training from scratch.

Latent DynamicsAI summary
2026-09-30 · arXiv:2609.37398

Direct Experience World-Model Optimization: Learning the World Beyond Action Imitation

Xiangcheng Zhan, Zirui Chen, Yicheng Zhao et al.

DEWO (Direct Experience World-Model Optimization) is a post-deployment learning paradigm for World-Action Models that refines world representations using visual futures collected around interaction turning points. It trains from matched successful and failed continuations for outcome-conditioned visual prediction, and uses a progress-estimating value head to activate classifier-free guidance when progress stalls. Across DexJoCo tasks and real-world dexterous-hand grids, the authors report improved control and predictive losses.

DrivingAI summary
2026-09-30 · arXiv:2609.37441

Anisotropic Representations Improve Planning in JEPA World Models

Mingu Kang, Yoori Oh, Sookyung Kim et al.

The authors show that joint training of JEPA-style latent world models with isotropic Gaussian regularization can learn a representation geometry whose Euclidean latent planning cost ranks feasible outcomes differently from the task cost, even with accurate prediction. They propose AnisoWM with ΛReg: replacing the fixed isotropic target with a learnable diagonal covariance under fixed-trace and anisotropy constraints. Across four visual goal-planning environments, it improves planning success over LeWorldModel in all four.

JEPAAI summary
2026-09-30 · arXiv:2609.37519

Video2STL: Grounding VLM-Generated Temporal Specifications for Robot Learning

Merve Atasever, Keyan Azbijari, Cagan Bakirci et al.

Video2STL converts observation-only videos into parametric Signal Temporal Logic (STL) specifications for robot learning. A VLM extracts an embodiment-independent semantic event trace, then generates a bank of parametric STL formulas; robot trajectories ground predicate thresholds and temporal bounds. The method uses two timescales: rolling-window robustness for dense local rewards and a causal monitor over a retained long-horizon formula for progress rewards, enabling cross-embodiment transfer.

Latent DynamicsAI summary