Digest

Reset

1180 papers

2026-10-02 · arXiv:2610.00003

STATERA: Hidden Mass Estimation via Zero-Shot Sim-to-Real Kinematics using Frozen Temporal Tubelets

Animesh Varma

STATERA estimates hidden center-of-mass (CoM) offsets of opaque, asymmetric rigid bodies from short monocular videos. It adapts a partially-frozen V-JEPA (video JEPA) backbone with temporal tubelet mixing and a lightweight 2.5D decoder to predict per-frame CoM heatmaps/trajectories. The authors introduce the HiddenMass Benchmark (50K MuJoCo + 63 real sequences). They report improved normalized CoM error in simulation and strong physics capture in zero-shot sim-to-real using phase-aware supervision.

Latent DynamicsAI summary
2026-10-02 · arXiv:2610.00010

Heavy-Tailed Memory Traces in Long-Horizon Language Agents

Xinyuan Song, Zekun Cai

The authors audit how long-horizon language agents’ external memory gets used under finite context and repeated retrieval, finding reproducible core–tail concentration with policy-dependent tail shapes. They propose Core–Tail World Model (CTWM), a rank-based memory controller that allocates prompt budget using an exponent τ while retaining a summarized tail. CTWM reduces prompt tokens and tail prediction error in paired comparisons across graph and text benchmarks.

JEPACode availableAI summary
2026-10-02 · arXiv:2610.00012

When Do Causal World Models Help Modular LLM Agents

Xinyuan Song, Zekun Cai

The authors show observational world models can incur irreducible intervention-time error in modular LLM agents, even when they accurately model observational trajectories. They propose CausalCompose, which collects intervention–response evidence locally, validates causal interfaces, and composes local mechanisms for intervention-aware prediction and optional control. Experiments and theory support that causal structure improves intervention-sensitive reasoning and planning, especially in structured tool-use settings.

Latent DynamicsCode availableAI summary
2026-10-02 · arXiv:2610.00280

Stable and Counterfactually Robust Physical World Models from Imposed Structure and Learned Physics

Yufeng Wang, Parivesh Priye, Lu Wei et al.

The authors impose general physical structure on a learned world model—dynamics from gradients of a learned energy through a fixed reversible operator, a confining energy class, and a one-way dissipation port—and learn system-specific physics from trajectories. Across three physical regimes, a single structured model recovers constitutive functions and dissipation rate, transfers counterfactual interventions on supplied parameters, stays stable up to 100× the training horizon, and refuses interventions without a supplied parameter map.

Latent DynamicsAI summary
2026-10-02 · arXiv:2610.00317

DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies

Youngjun Jun, Kyumin Choi, Youngmin Kim et al.

DriftOPD is a teacher-free, rollout-free framework for sequence-level on-policy distillation of continuous VLA action experts into one-step policies. It decomposes sequence-level reverse KL into a chunk-level reverse-KL term and a future-potential term. DriftOPD optimizes the local term with a one-step drifting objective and approximates the future term with a Q-function critic learned from offline demonstrations, using only offline data and one-step action generation.

DrivingAI summary
2026-10-02 · arXiv:2610.00368

DeepJEPA: Scaling World Models from Within

Zijian Jin, Yunbei Zhang, Yuanzhe Liu et al.

DeepJEPA treats transition depth as an inner test-time scaling axis for joint-embedding predictive world models. Instead of using a fixed recurrent depth for every imagined transition, it learns when additional recurrent updates are worth computing for each candidate and rollout step during CEM. Across five visual-control settings, DeepJEPA improves or matches the strongest fixed-depth planner while averaging only 1.00–1.26 updates per transition.

DrivingAI summary
2026-10-02 · arXiv:2610.00388

T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning

Bo-Wen Zhang, Junwei He, Maoqi Liu et al.

T2SPO (Trajectory-to-Step Policy Optimization) uses successful past trajectories to provide step-level feedback for LLM-agent RL without training a critic. It builds a context of state representations paired with observed remaining turns-to-success, uses a frozen TabPFN regressor to estimate remaining distance during new rollouts, and turns changes in predicted remaining distance into auxiliary step credit added to GRPO advantages. Experiments on ALFWorld and WebShop with 1.5B/7B models report higher task success than GRPO.

AI summary
2026-10-02 · arXiv:2610.00544

Memorizon: Training World Models Beyond Their Context Window

Tingting Liao, Xuezhi Liang, Hao Li et al.

Memorizon trains streaming world models on long revisit supervision without paying for long attention. Each sample spans any length but only its last k chunks are scored; unscored history is not tokenized. For each scored chunk, the model retrieves top-K prior latents by camera co-visibility and shares the union as a bounded memory bank (≤ kK entries). The bank increases step cost once; longer spans then add little cost and improve revisit consistency over sliding windows.

2026-10-02 · arXiv:2610.00575

Token-World: World Modeling in Vision-Language Model Token Space for Robot Manipulation

Chuyao Fu, Xiaowei Chi, Yuhan Rui et al.

Token-World models robot manipulation world dynamics directly in a compact state derived from vision-language model (VLM) visual tokens. It compresses high-dimensional policy-facing VLM tokens into a lower-dimensional token state (via an S-VAE), learns action-conditioned autoregressive transitions with a flow-matching DiT, and maps predicted states back only when the downstream policy needs them—avoiding intermediate RGB generation. The authors report improved long-horizon open-loop fidelity and closed-loop success-rate correlation, with lower simulation latency than recent baselines.

DrivingAI summary
2026-10-02 · arXiv:2610.00604

MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation

Egor Cherepanov, Nikita Kachaev, Aleksandr I. Panov et al.

MIKASA-Robo-VLA is a benchmark of 90 language-conditioned robotic manipulation tasks designed to evaluate memory in long-horizon VLA policies. Most tasks hide the cue an action depends on; 10 are reactive controls. The suite provides an environment-based “information gap” and releases 22,500 oracle trajectories in RLDS and LeRobotDataset v3. A reference π0.5 baseline achieves 0.211 ± 0.044 mean success on 14 tasks.

RoboticsAI summary
2026-10-02 · arXiv:2610.00638

TacDyn-WAM: Learning Implicit Tactile Dynamics in a Heterogeneous Visuo-Tactile World Action Model

Enyi Wang, Mingxin Wang, Quan Shi et al.

TacDyn-WAM is a heterogeneous visuo-tactile world action model that predicts implicit tactile dynamics instead of reconstructing future tactile pixels. It learns TacRep via masked spatio-temporal tactile prediction with relational structure distillation. An ITDE predicts future tactile representations and their changes at multiple horizons in one forward pass, integrated with a visual expert via joint attention and a read-only tactile memory.

DrivingAI summary
2026-10-02 · arXiv:2610.00686

SemanTok: Predictable Semantic Tokens for Efficient Autoregressive Video Generation

Mikhail Dereviannykh, Vikram Voleti, Simon Donne et al.

SemanTok is a flexible-length, coarse-to-fine video tokenizer for autoregressive (AR) video generation. It keeps a VideoFlexTok-style decodable prefix recipe but adds semantic supervision by feeding frozen DINOv2 features to the encoder and training lightweight heads to reconstruct them from each retained token prefix alone. The authors report that a 201M SemanTok AR model matches or beats VideoFlexTok at 3.4× smaller size on fidelity/semantic alignment, with improved robustness across noise levels and OOD classes.

2026-10-02 · arXiv:2610.00722

JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts

Zheyuan Zhang, Suyu Ye, Nakul Agarwal et al.

JEPA-TTT adapts the latent dynamics predictor of a pretrained action-conditioned JEPA world model during test time to handle dynamics shifts. Self-supervised updates accumulate across episodes using dense replay, while the visual encoder and reward head stay fixed. It plans with MPC (CEM) without goal images or online environment reward. Across eight dynamics shifts in four continuous-control environments, it improves planning on every shift and reduces autoregressive latent prediction error by 83% on average after 500 episodes.

Latent DynamicsAI summary
2026-10-02 · arXiv:2610.00727

CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization

Morgan Byrd, Robert Wright, Sehoon Ha

CF-JEPA is a JEPA-style latent world model that factorizes its latent space into an action-conditioned controllable part and an action-free uncontrollable part, aiming to capture distractor information in the uncontrollable region. The authors report comparable performance on nominal 2D/3D control tasks and improved robustness under visual distractors, where CF-JEPA is the only model that does not experience latent collapse. They also test a simulated robot task under distractors.

JEPAAI summary
2026-10-02 · arXiv:2610.00821

Getting Out and Getting Back: World and Behavior Grounding in Real2Sim2Real Co-Training

Samuel Liu, Youngsun Kim, Martin Matak et al.

The authors separate simulation grounding into two axes—world grounding (aligns simulator with the real system) and behavior grounding (aligns simulated trajectories with human motion)—and co-train on a dynamic dexterous pick-and-sort task. Fully grounded co-training raises success from 52% to 86%, and averaged across configurations world grounding improves success by 18 points while behavior grounding improves by 10. They report policies that mix real-derived and simulation-derived behavior within rollouts.

RoboticsAI summary
2026-10-02 · arXiv:2610.00834

Kepler: Auditable World Models for ARC-AGI-3

Wensen Wu

Kepler is an open-source harness for ARC-AGI-3 that represents hypotheses as executable world models and validates them via retrospective transition checks and conditional prediction checks. With one frozen Claude Opus 5 configuration, it reports server-verified 100.00 RHAE on all 25 public games. The authors also report several evaluation failures, including harness reconstruction in a control and outcome-vs-tool integrity gaps.

DrivingAI summary
2026-10-02 · arXiv:2610.00859

CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight

Chensheng Peng, Wenhao Ding, Ran Tian et al.

CtrlWAM trains world action models with physically aligned denoising: instead of pairing noised actions with a video still tied to the recorded future, it perturbs actions in a simulator and pairs them with their rendered visual consequences (then corrupts video with fresh noise). It also introduces warped video–action noise schedules and a structured multi-agent action interface for driving and bimanual manipulation.

DrivingAI summary
2026-10-02 · arXiv:2610.00911

Block Optimism for Nonstationary Bandits with Latent Linear Dynamics

Taehyun Hwang, Hyunjun Choi, Heesang Ann et al.

The authors study endogenous nonstationary stochastic bandits with latent linear dynamics and bilinear rewards in the current action and unobserved latent state. They improve over an explore-then-commit (ETC) baseline by using stability to approximate the infinite-memory planning objective with a finite-memory cyclic block proxy, then applying block-level UCB with confidence sets for the truncated parameters. They prove a regret bound of order \tilde{O}(\sqrt{T}) versus the previous \tilde{O}(T^{2/3}).

Latent DynamicsAI summary
2026-10-02 · arXiv:2610.00921

In CEM, a World Model Is Also a Proposal Mechanism

Oliver Obst, Frieder Stolzenburg

The authors study how CEM “world-model scores” affect both the current action choice and the candidate pools considered in later iterations. They perform crossed rescoring (multiple predictive surrogates score saved CEM pools) and a one-update splice (replace the first model-ranked elite refit with an environment-ranked one). On Walker and Cheetah, proposal distance decreases while ranking agreement stays near chance on Walker and declines on Cheetah; the one-update splice can lower realised selected-sequence cost.

DrivingAI summary
2026-10-02 · arXiv:2610.00926

A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform

Chengkai Xu, Yiming Cui, Jiaqi Liu et al.

The authors survey end-to-end autonomous driving training as a “Data-Strategy-Platform” ecosystem. They argue performance depends on (1) data coverage/value and governance, (2) strategies aligning optimization with driving objectives, and (3) platforms enabling scalable, reproducible training and continuous evolution. They review recent advances in data, learning paradigms, and training infrastructure, then outline a vision emphasizing data value, foundation-driven generalization, and integrated training-testing loops.

DrivingAI summary