Digest

Reset

1167 papers

2026-11-09 · arXiv:2611.09892

On Agentic Manipulation Policies

Chen Chen, Omid Bagherieh, Naveen Nikamanth Appiah Balaji et al.

The authors introduce AMP1, a reference harness where a generalist AI model directly commands a robot manipulator in an agentic manipulation policy loop (no demonstrations, no task-specific training). AMP1 reports state-of-the-art zero-shot results on RoboLab-120 and CaP-Gym simulations and strong zero-shot success on real hardware, including 85.6% success on RoboLab-120 (Gemini 4 Argon).

RoboticsAI summary
2026-11-09 · arXiv:2611.09894

Video-Coder: Reinforcing Agentic Coding for Video Generation

Chaoyang Wang, Sicheng Gao, Junan Hu et al.

Video-Coder is a code-driven video generation agent. It writes and executes Blender Python code to build a dynamic 3D scene and renders a “Draft Video” that provides structure and motion signals for a fixed downstream video generator. The method trains with SFT on Video-Coder-SFT-6k and then harness-native agentic RL on Video-Coder-RL-2k using execution- and video-based rewards with an annealing schedule.

Video World ModelsCode availableAI summary
2026-11-06 · arXiv:2611.07229

Spatial-UMI-Dream: Scaling World Action Models with Metric Geometry for UMI Manipulation

Weiduo Yuan, Ze Yuan, Sihan Liu et al.

Spatial-UMI-Dream is a 5B spatial World Action Model pretrained on 605 hours of RGB-D UMI data with two-stage training (world-only adaptation, then joint world-action learning). It jointly predicts future RGB, fisheye-aligned metric depth, and actions using stereo-derived metric depth aligned to moving wrist cameras. It is evaluated on RoboTwin-UMI (20 tasks) under environment and embodiment shifts, and on six real-world tasks.

RoboticsAI summary
2026-11-06 · arXiv:2611.07245

MAC-v2: Long-horizon model-based value expansion from pixels

Kwanyoung Park, Sergey Levine

MAC-v2 is an offline model-based RL method for pixel observations that enables long-horizon model-based value expansion. The authors train a reconstruction-free latent world model with action chunking, learning representations from prediction and control objectives. This reduces recursive compounding of prediction error and allows long-horizon latent imagination over visual benchmarks (visual OGBench, MMBench2) and robotics tasks (Robomimic, LIBERO-10).

2026-11-06 · arXiv:2611.07403

Dynamic Watermarking and Action-Conditioned Latent Dynamics for Secure Vision-Based Controllers

Aditya Parameshwaran, Navid Aftabi, Abhishek Hanchate et al.

The authors propose an adaptive dynamic-watermarking framework for detecting replay attacks in vision-based controllers. A private Gaussian watermark is added to the control input, while a control input-conditioned latent dynamics network predicts future latent responses from images. Replay evidence is formed from predicted-vs-received latent discrepancies, and an RL policy adapts the watermark covariance to balance detectability, excitation, and task safety via a Lagrangian separation penalty plus Simplex stopping.

DrivingAI summary
2026-11-06 · arXiv:2611.07730

Seeing Beyond the Next Step: World-Model-Guided Human-Like Navigation in Multi-Agent Scenes

Seong-Eun Hong, JuYeong Hwang, Jinhyun Kim et al.

The authors train a fast navigation policy using two action-conditioned world models only at training time. They decompose consequence prediction into social dynamics (relative position/velocity with direction) and scene semantics (navigation-relevant segmentation with pedestrians ignored). GRPO distills group-relative preference scores over candidate motion anchors into the policy, so at deployment the policy acts in a single forward pass without world-model rollouts.

DrivingAI summary
2026-11-06 · arXiv:2611.07850

Learning Robotic World Models with Semantic and Visual Feedback

Hao Wu, Yuqi Li, Yizhou Zhao et al.

The authors propose RoboImagine, a post-training framework for autoregressive robotic world models that learns by comparing candidate imagined futures under shared observations, actions, and instructions. Grouped diverse beam search generates diverse candidates; a frozen multimodal evaluator provides semantic preferences via pairwise video comparisons in both orders, complemented by pixel- and perceptual-level visual consistency feedback. Group-relative advantages drive policy optimization; bounded, branch-specific rolling histories support long-horizon generation.

RoboticsAI summary
2026-11-06 · arXiv:2611.07867

PlayGen2: Learning Real-Time Playable Worlds with Explicit 3D State

Jihai Zhang, Yuhan Zhou, Junyou Li et al.

PlayGen2 is a real-time interactive world model that maintains explicit 3D state for persistent Doom gameplay. It separates learned 3D state evolution (a persistent volumetric occupancy grid plus identity-indexed entity states) from a neural renderer that synthesizes first-person RGB from projected depth/semantics and RGB history, without feeding generated images back into state prediction. The Doom prototype runs at ~160 ms backend latency and supports zero-shot unseen maps and composable multiplayer/interventions.

DrivingAI summary
2026-11-06 · arXiv:2611.07873

PVDepth: Panoramic Video Depth Estimation via Geometry-Aware Spatiotemporal Adaptation

Chuanxin Song, Peixi Peng

PVDepth is an end-to-end panoramic video depth estimation framework adapted from a perspective video depth model. It addresses ERP-specific geometric distortions and non-linear temporal dynamics using Progressive Sphere-aware Noise Initialization and Cube-rectified Temporal Modeling. The paper introduces PanoCARLA, a large-scale synthetic RGB-D panoramic video dataset, and reports improved geometric accuracy and temporal consistency. Code and data are released.

Video World ModelsCode availableAI summary
2026-11-06 · arXiv:2611.07912

GuidedWAM: Training-Free Energy Guidance for World Action Models in Autonomous Driving

Xiyan Su, Finn Rasmus Sch\"afer, Hojun Lim et al.

GuidedWAM is a model-agnostic, training-free energy guidance method that injects new driving objectives into frozen world action models at sampling time. It constructs objectives as differentiable energies evaluated on each sampler step’s one-step clean-action estimate and applies gradients to the sampler’s velocity update. On NAVSIM v2 hard-split, guidance improves three frozen, publicly released WAMs by 1.8–3.6 EPDMS with <10% sampling-time overhead per step.

DrivingAI summary
2026-11-06 · arXiv:2611.07934

The Blindfolded Controller: Decision Model-Guided Training for Reinforcement Learning

Muhammad Ibrahim Khan, Aniket Dixit, Faizan Ahmed et al.

The authors propose Decision Model-Guided Training (DMGT): use a decision model (DM) queried with natural-language descriptions of the environment/task to label actions, imitate those decisions to initialise an RL policy (BC/DAgger), then use PPO to improve from reward. Experiments on seven tasks show DMGT outperforms PPO alone in all seven and DM alone in five.

DrivingAI summary
2026-11-06 · arXiv:2611.07939

WAM-Sim0.1: World Anchored Memory Meets Online RL for Consistent and Realistic Driving Simulation

Jiahao Cui, Jiaye Li, Wenhao Zhang et al.

WAM-Sim is a driving world model that uses a persistent hierarchical world anchored memory (structure, scene, and write-once instance tokens) to reduce structural drift and identity drift in long rollouts. It applies an online post-training method, Drive DiffusionNFT, using flow-matching RL with hierarchy-aligned visual rewards and a physical judge to restore visual sharpness and physical plausibility.

DrivingAI summary
2026-11-06 · arXiv:2611.08111

Reinforcing End-to-End Autonomous Driving Safety via Negative-Aware Planning in a Unified World-Action Model

Zihao Zhang, Xinyu Zhang, Bing Zeng et al.

NegWAM is a safety post-training method for unified world–action models in end-to-end autonomous driving. It mines near-expert failure trajectories offline, constructs geometrically close but unsafe negatives, and uses them as repulsive anchors plus scene-conditioned rider tokens to learn scene–trajectory compatibility. The auxiliary rider branch is removed at inference. On NAVSIM v2, it reports 40.4 EPDMS on navhard and 93.3 EPDMS on navtest.

DrivingCode availableAI summary
2026-11-06 · arXiv:2611.08113

Identifying Neural Dynamics Using Nonlinear Interventional State Space Models

Hanna M. Tolle, Carles Balsells-Rodas, Yixin Wang et al.

The authors propose a nonlinear interventional state-space model (iSSM) for identifying latent neural dynamics from high-dimensional observations under do-style interventions on latent variables. They prove affine identifiability for general injective emissions with Gaussian latent transitions, and permutation-scale-shift identifiability under interventions for general nonlinear latent dynamics. They validate on simulated FitzHugh–Nagumo and apply to decision-model adjudication and microstimulation prediction in macaque and human neural recordings.

Latent DynamicsAI summary
2026-11-06 · arXiv:2611.08128

FreeOcc: Decoupling Ego-Motion for Efficient 4D Occupancy Forecasting via Continuous Flow Matching

Zeping Zhang, Zhuoya Zhao, Samy Metari et al.

FreeOcc is a lightweight continuous-time 4D occupancy forecasting model that explicitly decouples ego-motion using SE(2) geometric warping and then predicts residual dynamics with physics-constrained conditional flow matching. It uses uncertainty-aware blending to route well-observed voxels to a fast decoder and ambiguous regions to a continuous ODE solver. The paper reports 27 FPS for FreeOcc-E with 28.3M parameters.

DrivingAI summary
2026-11-06 · arXiv:2611.08156

La-JEPA: Compact Robot Policies with Latent World Modeling

Mateo Cl\'emente, George Z. Xue, Amin Abyaneh et al.

La-JEPA is a compact world-action model whose policy transformer is trained from scratch in the feature space of a pretrained V-JEPA 2.1 encoder. It uses WAM-style joint training to predict future features with actions, but generates only actions at inference. Among policies without embodied pretraining, the authors report state-of-the-art performance on clean/randomized RoboTwin 2.0 and LIBERO-Plus (with corrected instructions), with far fewer parameters and faster policy-call latency than Fast-WAM at comparable performance.

Latent DynamicsAI summary
2026-11-06 · arXiv:2611.08240

FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning

Haitao Lin, Minsi Ren, Yang Li et al.

FewStepRL adapts reinforcement learning objectives for fine-tuning diffusion models when using few-step (average-velocity) samplers. The authors transform RL objectives to x-space, decompose gradients into self-regression plus reward augmentation, and adapt each objective’s anchors for few-step transport or tangents. This yields few-step versions of FlowGRPO, AWM, DiffusionNFT, DiffusionOPSD, and ReFL in one modular codebase.

2026-11-06 · arXiv:2611.08286

Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds

Chenhao Zheng, Zixian Ma, Kai Li et al.

The authors introduce Interactive Visual Gym, a framework for closed-loop evaluation/training of visual interaction models by jointly simulating executable visual task environments and persona-conditioned simulated users. They build IVG-Bench with 315 interaction scenarios across cooking, household manipulation, and mobile-interface domains. Evaluating state-of-the-art VLMs as visual interaction models shows the best model, Claude Opus 5.5, completes 53.5% of tasks in time limits.

DrivingCode availableAI summary
2026-11-06 · arXiv:2611.08338

GPWM: Modeling the World as 3D Particle Motion

Robert Ren, Matthew Bronars, He Zhu et al.

GPWM is a generative 3D physical world model that predicts the motion of particles and rigid bodies directly in physical space, using an object-aware multi-resolution transformer with flow matching and diffusion forcing. Trained on ParticleWorlds (25.5M frames), it reduces prediction error by 31% on average over the strongest applicable learned-dynamics baselines across six physical domains and enables contact-rich planning by searching in latent noise space.

2026-11-06 · arXiv:2611.08373

SLA: Scaling Spatial-Language-Action Model via Continual Pre-training

Fuhao Li, Wenxuan Song, Qizhou Huang et al.

SLA is a Spatial-Language-Action model that learns transferable 3D spatial understanding by continual pre-training: it jointly optimizes action prediction and implicit alignment between intermediate VLA visual embeddings and spatial representations from a frozen 3D perception model. It uses feature-dependent gates to aggregate multiple VLA layers and probabilistic state dropout for generalization, while keeping the original inference architecture (auxiliary components used only in training).

DrivingAI summary