Digest

Reset

1167 papers

2026-10-15 · arXiv:2610.20240

PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents

Siqi Yang, Yimu Wang, Ziqi Cai et al.

PAWA is a dataset and framework for action-conditioned panoramic multiplayer world modeling in Minecraft. It pairs each player’s POV video with a world-stabilized equirectangular panorama and aligned actions, using a distributed CPU gameplay + GPU offline rendering pipeline that scales beyond 64 players. The dual-tower model jointly predicts POV and panorama with ray-aligned cross-attention and uses a quadratic-phase player encoding. The authors report PAWA improves POV prediction quality over prior multiplayer world models.

Video World ModelsCode availableAI summary
2026-10-15 · arXiv:2610.20331

Are We Overthinking One-Step Generation?

Woobin Park, Jaeyeon Kim, Yilun Du et al.

The authors introduce Priming Models, a two-stage method for one-step generative modeling: first train with a reconstruction objective (e.g., pixel-space denoising at a fixed noise level), then post-train the pretrained network using a feature-space distributional matching loss. They report improved training-compute vs. generation-quality trade-offs across class-conditional image generation, text-to-image generation, and world modeling, including ImageNet one-step SOTA FID 1.19.

Video World ModelsCode availableAI summary
2026-10-15 · arXiv:2610.20336

AutoRSR: Automated Real-to-Sim-to-Real Data Generation for Scalable Robot Learning

Jiawei Zhang, Chang Liu, Litao Deng et al.

AutoRSR converts scanned real workspaces into metrically aligned, robot-executable simulation assets and scalable manipulation data. Using multi-view observations, robot states, and open-vocabulary semantic queries, it reconstructs the scene, extracts object assets, registers to the robot base frame at metric scale, retargets manipulation from a demonstration, and augments via interpolation and domain randomization. Across three tasks, policies trained on 1K AutoRSR episodes achieve 73.3–86.7% real-world success.

RoboticsAI summary
2026-10-15 · arXiv:2610.20384

Look Before You Restore: Learning to Predict Before Acting for Agentic Image Restoration

Xu Zhang, Jun Wan, Lefei Zhang et al.

PreActIR is an agentic image restoration framework that predicts candidate tool consequences before execution using a tool-conditioned world model (including predicted benefit/harm probabilities and uncertainty). The agent executes only the top-ranked candidate, and an independent verifier commits or rolls back its output before replanning. In controlled replays, it uses 42.5% fewer real tool calls and 63.4% fewer rejected calls than reactive trial-and-error, while improving Y-PSNR by 0.92 dB.

DrivingAI summary
2026-10-15 · arXiv:2610.20442

Towards Agentic Robots: Model, Data, Environment, and Harness

Chonghan Qin, Dongming Wu, Kaituo Feng et al.

The authors survey and organize “agentic robots,” where agents autonomously plan and coordinate task execution, monitor progress, interpret execution feedback, and recover from failure. They trace evolution from robot policies to agentic systems and propose a taxonomy spanning four functional domains: Model, Data, Environment, and Harness. They also outline open challenges and directions for future research.

RoboticsCode availableAI summary
2026-10-15 · arXiv:2610.20474

AdaRSI: On-Demand Recursive Self-Improvement for Embodied Agents

Peidong Wang, Ying Chang, Zhiming Ma et al.

AdaRSI is an on-demand recursive self-improvement framework for embodied agents. A coding agent first refines a harness (skills/tools/procedure) using execution feedback, then attributes persistent failures to either harness calling or executor limitations by retrying sub-steps from saved states. It fine-tunes only sub-steps that remain unreliable, using self-collected simulation demonstrations and carries experience across tasks. On RoboCasa365 (50 tasks, no human demos), unseen composite success rises from 4.6% to 13.8%.

DrivingAI summary
2026-10-15 · arXiv:2610.20584

Player-1: Learning to Play from Synthetic Experience

Mingyu Ouyang, Yiran Wang, Siyuan Hu et al.

Player-1 is a post-training framework for game control beyond demonstrations using synthetic experience. It extends a computer-use action space with relative pointer motion, sustained inputs, and composite actions, then fine-tunes on 6,044 curated gameplay trajectories. It constructs Player-1-Gym with 16 synthetic 2D/3D environments and programmatically verifiable rewards for multi-environment PPO (200 RL steps), improving MineDojo-300 success and transferring beyond the RL suite.

DrivingAI summary
2026-10-15 · arXiv:2610.20592

GeoThinker-$F$: Endogenous Predictive World Representations for Fast Spatial Intelligence

Haoyuan Li, Qihang Cao, Huanyuan Zhou et al.

GeoThinker-F removes runtime dependence on an external 3D geometry encoder for geometry-aware spatial intelligence. It trains lightweight Latent Frame Prediction (LFP) heads to predict latent current-geometry and next-frame semantic representations from intermediate causal MLLM hidden states, then integrates them via Spatial-Grounded Temporal (SGT) Fusion. The authors report up to a 3.6× inference speedup (1.45 s to 0.40 s on Qwen3-VL-8B) with competitive spatial reasoning performance.

DrivingCode availableAI summary
2026-10-15 · arXiv:2610.20604

VITAL: From Visual Motion to Precise Action

Ao Liang, Youquan Liu, Mingxiao Li et al.

VITAL is an action tokenizer for autonomous driving that predicts a visually derived motion scaffold plus a normalized residual encoding the trajectory error left after that scaffold. Using flow matching, it jointly predicts both components from the current image, ego state, and navigation command without generating future images at inference. It also supports bounded residual refinement after imitation learning.

DrivingCode availableAI summary
2026-10-15 · arXiv:2610.20609

SafeDriveVLA: Navigation-Conditioned World Model Dreaming for Conflict-Aware End-to-End Autonomous Driving

Shaoyuan Xie, Zihan Zhang, Jingxuan Wang et al.

SafeDriveVLA is a driving Vision-Language-Action model for conflict-aware navigation following. The authors propose (1) predicting a three-level navigation-compliance mode (strict/cautious/fallback) to decouple conflict reasoning from action generation, and (2) navigation-conditioned world model dreaming to inform this mode prediction. They evaluate on Bench2Drive and their new benchmark suite and report improved driving performance and fewer collisions under unsafe instructions.

DrivingCode availableAI summary
2026-10-14 · arXiv:2610.13956

MDN-VJEPA: Multimodal Probabilistic Joint-Embedding World Models

Yongchao Huang

MDN-VJEPA is a multimodal probabilistic world-model based on Joint-Embedding Predictive Architectures (JEPAs). It instantiates the VJEPA latent predictor as a Gaussian-mixture density network (MDN) to represent alternative futures with probabilities and within-mode uncertainty. It adds trajectory-level mode coherence and a distributional multi-horizon path-consistency regularizer for temporal coherence and Chapman–Kolmogorov consistency.

Latent DynamicsAI summary
2026-10-14 · arXiv:2610.14018

RoboHarness: A Simple Harness Outperforms VLA and World Actions Models

Bince Qu, Wei Chen, Bo Zhang

RoboHarness presents a robotic harness that gives LLM agents a visual-geometric control panel to directly invoke embodied tasks. Without model fine-tuning, the open-source model Qwen3.8-Flash-Next with RoboHarness is reported to outperform VLA and world action model baselines on BEHAVIOR Challenge 2025 tasks. The authors say the implementation will be released as open source.

DrivingAI summary
2026-10-14 · arXiv:2610.14020

LabInstruct: Benchmarking Situated Instructional Video Generation for Lab Procedures

Yuming Fu, Weijia Wu, Jing Chen et al.

The authors introduce LabInstruct, a benchmark for situated instructional video generation in real laboratories. Given an initial workspace image and a natural-language instruction, a model must generate a continuation that preserves workspace identity, executes the instructed actions in correct order, and reaches expected intermediate/final states. Evaluating 8 image-to-video models, the authors report that visually plausible generations often remain procedurally incorrect, showing a gap between visual realism and procedural reliability.

2026-10-14 · arXiv:2610.14042

Reinforcement Learning from Real-Robot Interaction without High-Quality State Estimates

Ryan Gao, Ashvin Anilkumar, Nicholas Corrado et al.

The authors study real-robot RL without high-quality state estimates for a NAO soccer dribbling task under partial observability. They run an unscented Kalman filter (UKF) onboard and add RSSM-RLPD: a recurrent state-space model whose latent belief is corrected only when the ball is observed. On hardware, the robot learns to score from multiple starts after 30 minutes, reaching 100% success vs 0% for feed-forward RLPD under the same budget.

RoboticsAI summary
2026-10-14 · arXiv:2610.14043

Refuses in Chat, Complies with Arms: LLM Refusal Does Not Survive Robot Control

Edward Sun, Christopher Leet, Sravanthi Machcha et al.

The authors test seven frontier LLMs on RoboHarm, a physical benchmark of harmful robot instructions, varying only the interface from text to robot control (Code as Policies, end-effector poses, joint angles). They report that refusal learned in chat does not carry over to robot control: simulated refusal drops from 73% (text) to 18–25% (control), and real-robot refusal drops similarly.

DrivingAI summary
2026-10-14 · arXiv:2610.14155

Axiom-0: Hierarchical Self-Refinement for Vision-Language-Action Policies

Renbin Shu, Jing Yang, Xiangyuan You et al.

Axiom-0 is a VLA (vision-language-action) policy that uses hierarchical self-refinement. It adds IVRR to supervise selected intermediate VLM features with learned projections aligned to spatial foundation-model references, and VAIR to recursively refine the VLM conditioning before action generation in a VLM–DiT policy. On RoboCasa365, the authors report 62.32% overall success.

DrivingAI summary
2026-10-14 · arXiv:2610.14159

Disentangling Task-Relevant Latent Dynamics for Reward Modeling

Dayou Li, Jionghao Wang, Peihao Wang et al.

DiTaR learns dense rewards from heterogeneous, action-free robot videos by modeling task-relevant latent dynamics rather than entangled visual variation. It uses an asymmetric two-stage latent representation to suppress task-irrelevant variation, then trains a history- and language-conditioned rectified-flow model and derives a directional temporal-difference reward from changes in compatibility. The authors report stronger visual RL on MetaWorld-v3 and ManiSkill and improved real-world offline RL via reward relabeling.

RoboticsAI summary
2026-10-14 · arXiv:2610.14186

Visualizing the World Model in an Agent's Value Function

Shawheen Naderi, Fatemeh Akherati, Yubei Chen

The authors use counterfactual interventions on states in the Crafter environment to probe whether a PPO value function encodes an implicit world model. By changing specific observation components while holding the rest fixed, they report that the critic learns rules about collectible achievements (e.g., diamond requires an iron pickaxe), health drivers (water/food), and heuristics from inventory affecting value.

DrivingAI summary
2026-10-14 · arXiv:2610.14191

Observation-Referenced Tissue Deformation Forecasting with Tool-Driven and Residual Dynamics

Tom Olesch, Nan Xiao, Xiongye Xiao et al.

The authors forecast surgical tissue deformation under a known future instrument trajectory using an observation-referenced, scene-specific differentiable surface model. Instead of predicting absolute positions, each tissue point is anchored to its last valid observation and only the tool-induced deformation is simulated forward. An optional observation-referenced residual adds graph-spectral spatial modes with periodic harmonic dynamics for structured motion. Evaluated retrospectively on 12 clips, observation referencing gives the largest gain.

DrivingAI summary
2026-10-14 · arXiv:2610.14200

ClothCode: Generative Modeling for Garment State Estimation and Dynamics Prediction

Yingdong Ru, Lipeng Zhuang, Shiyu Fan et al.

ClothCode is a unified discrete generative framework for garment state estimation and action-conditioned dynamics prediction. It tokenizes cloth geometry into material-aligned residual VQ patch tokens and uses task-specific Masked Generative Transformers for confidence-guided parallel masked refinement. Across three templates, the authors report 26.6% lower mean displacement error for state estimation and 37.0% for dynamics prediction vs strongest baselines, plus faster dynamics inference (16.7 ms vs 556.2 ms) and improved real-world MPC success (56.7%→78.9%).

RoboticsAI summary