Robotics

Action-conditioned world models for robot control. (140 papers)

2026-11-13 · arXiv:2611.15215

AFTER: A World Model with Adaptive Fast--Slow Timescales for Error-Driven Retention

Haoyi Zhang, Qi Zhang, Yunfan Ren et al.

AFTER-WM is a world model for online dynamics adaptation under sudden, persistent, initially unknown changes. It uses predictive correction from one-step prediction errors plus dual-timescale fast/slow adaptation (with fading fast correction and persisting slow correction) and multiple pre-trained experts whose combination weights are adaptively reweighted without altering experts. On five simulated robot datasets and real UAV data, the authors report lower multi-step mean error and the highest mean return in a joint-failure control task.

RoboticsAI summary
2026-11-13 · arXiv:2611.16728

EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming

Yaru Niu, Binghong Chen, Changwei Yao et al.

EgoX is an egocentric cross-embodiment manipulation framework that pretrains a Modularized Cross-Embodiment Transformer (MXT+) with a training-only Embodiment Dreaming objective on mixed demonstrations from multiple humans and robots, then fine-tunes on a target embodiment. The authors report higher average normalized task progress (0.63→0.80) and larger gains under out-of-distribution (0.53→0.77) across 13 task–embodiment pairs.

RoboticsAI summary
2026-11-11 · arXiv:2611.14048

LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning

Xuheng Zhang, Yiming Zhang, Jialin Gao et al.

LaST 0.5 is a world action model for robotic manipulation that learns interaction-centric latent reasoning between world modeling and action generation. It uses a Mixture-of-Transformers with a world expert, a latent expert supervised by visual interaction cues, and an action expert. It also introduces AWAT to adapt the action execution horizon using predicted progress consistency. Pretraining uses 14k+ hours of robotic data.

RoboticsAI summary
2026-11-11 · arXiv:2611.14186

Koopman Model-Based Reinforcement Learning

Wenjian Hao, Paulo C. Heredia, Shaoshuai Mou

The authors propose Koopman model-based reinforcement learning (K-MBRL) for controlling nonlinear robotic systems with unknown dynamics. K-MBRL learns a stochastic deep Koopman model for dynamics in a linear lifted representation and a critic for future cost. It approximates the policy objective using immediate cost plus one-step predicted-state value, aiming to reduce rollout prediction error accumulation and enable real-time feedforward control without online optimization.

RoboticsAI summary
2026-11-11 · arXiv:2611.14365

From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models

Guoqiang Liang, Yiqi Lin, Yanzhe Chen et al.

MetaWAM is a world action model that injects robot signals (proprioception, actions, values, task type) into a single unified “meta-frame” for a pretrained video diffusion model. Instead of separate latent frames or learned encoders/decoders, it packs signals into disjoint DCT frequency channel bands and lifts them via inverse DCT. On RoboCasa it reports 68.9% success and reports 39% lower inference latency; it reports 85.0% success on three real-world Franka tasks.

RoboticsCode availableAI summary
2026-11-10 · arXiv:2611.11869

Action Representation for Spatial Generalization in Robot World Models

Yuhang Sun

The authors study how action coordinate representations affect spatial generalization in a controlled robot peeling world-model benchmark. Using matched action–state inputs across world-, object-, and surface-relative frames, they find costs from limited data when equivalent interactions use different coordinate descriptions, irreducible ambiguity when peel thickness is removed, and improved consequence prediction from surface-relative coordinates under accurate local observations—yet vulnerability remains to coordinate-frame uncertainty.

RoboticsAI summary
2026-11-10 · arXiv:2611.12676

VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess

Mat\v{e}j Kripner, Milan Straka

The authors fine-tune a vision-language-action policy (π0.5) for robotic chess king moves using teleoperated demonstrations and simulated demonstrations from a randomized scripted expert. In simulation, success scales with both data and updates; checkpoint comparisons require a learning-rate cool-down, while validation loss and action error are unreliable for model selection. On the real arm, partial results suggest simulation pretraining helps most when real data is scarce. The authors release the simulator, the expert, and real demonstrations.

RoboticsAI summary
2026-11-09 · arXiv:2611.09892

On Agentic Manipulation Policies

Chen Chen, Omid Bagherieh, Naveen Nikamanth Appiah Balaji et al.

The authors introduce AMP1, a reference harness where a generalist AI model directly commands a robot manipulator in an agentic manipulation policy loop (no demonstrations, no task-specific training). AMP1 reports state-of-the-art zero-shot results on RoboLab-120 and CaP-Gym simulations and strong zero-shot success on real hardware, including 85.6% success on RoboLab-120 (Gemini 4 Argon).

RoboticsAI summary
2026-11-06 · arXiv:2611.07229

Spatial-UMI-Dream: Scaling World Action Models with Metric Geometry for UMI Manipulation

Weiduo Yuan, Ze Yuan, Sihan Liu et al.

Spatial-UMI-Dream is a 5B spatial World Action Model pretrained on 605 hours of RGB-D UMI data with two-stage training (world-only adaptation, then joint world-action learning). It jointly predicts future RGB, fisheye-aligned metric depth, and actions using stereo-derived metric depth aligned to moving wrist cameras. It is evaluated on RoboTwin-UMI (20 tasks) under environment and embodiment shifts, and on six real-world tasks.

RoboticsAI summary
2026-11-06 · arXiv:2611.07850

Learning Robotic World Models with Semantic and Visual Feedback

Hao Wu, Yuqi Li, Yizhou Zhao et al.

The authors propose RoboImagine, a post-training framework for autoregressive robotic world models that learns by comparing candidate imagined futures under shared observations, actions, and instructions. Grouped diverse beam search generates diverse candidates; a frozen multimodal evaluator provides semantic preferences via pairwise video comparisons in both orders, complemented by pixel- and perceptual-level visual consistency feedback. Group-relative advantages drive policy optimization; bounded, branch-specific rolling histories support long-horizon generation.

RoboticsAI summary
2026-11-05 · arXiv:2611.05571

ELA: End-to-End Latent Action Policy Learning from Video Demonstrations

Cheng-Chun Hsu, Peter Stone

ELA is an end-to-end framework for learning robot imitation from video demonstrations using latent actions. It jointly trains a Latent Action Model (inverse + forward dynamics), a latent action policy, and a continuous action decoder, optimizing latent actions for both policy learning and action decoding. The authors report consistent improvements over latent-action baselines on simulated and real-world manipulation tasks.

RoboticsAI summary
2026-11-05 · arXiv:2611.06768

PyRUA-REAL: Robot Exploration and Adaptive Learning through RSI in the Real World

Qiang Wang, Jianxin Bi, Shulong Jiang et al.

PyRUA-REAL is an embodied recursive self-improvement framework for real-world robot manipulation. Coding agents iteratively explore, execute, diagnose, and revise robot programs, storing interaction experience as persistent memory and reusable executable skills. The resulting programs can be frozen and executed without runtime language-model calls, while still using live observations during execution. Across eight real-world tasks, PyRUA-REAL reports 91.1% average success.

RoboticsAI summary
2026-11-04 · arXiv:2508.05941

Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distribution

Zhanyi Sun, Shuran Song

Latent Policy Barrier (LPB) improves visuomotor policy robustness under covariate shift by treating latent embeddings of expert demonstrations as an implicit barrier separating in-distribution from out-of-distribution states. It trains a base diffusion policy on expert data and an action-conditioned latent dynamics model on both expert and rollout data; at inference, it steers actions using latent-space optimization to stay near the expert latent manifold.

RoboticsAI summary
2026-11-04 · arXiv:2611.04492

Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation

Jimmy Wang, Yifan Chou, Kaifeng Zhang et al.

World Models Radar: 3D World Simulator is an action-conditioned multimodal 3D world model that predicts appearance, geometry, and tactile observations using a unified 3D particle-voxel representation. It combines a hybrid particle-voxel autoencoder (compress point clouds to 3D voxel latents, decode to 3D Gaussian particles) with a 3D latent dynamics model that predicts latent evolution, enabling multi-view consistent rendering and long-horizon interactive simulation at 50 Hz.

RoboticsAI summary
2026-11-04 · arXiv:2611.05080

CRONOS: Benchmarking Non-Episodic Reinforcement Learning for Robotic Manipulation

Po-Yi Wu, Djengo Cyun-Jyun Fang, Dian Cheng et al.

CRONOS is a simulation benchmark for fine-tuning vision-language-action (VLA) policies with non-episodic scene resets. The authors report that naive non-episodic fine-tuning is ineffective due to drift in object configurations and end-effector poses, but targeted reset mechanisms (end-effector resets, curriculum reset schedules, and heuristic resets of unrecoverable scenes) improve training and generalization.

RoboticsAI summary
2026-11-03 · arXiv:2611.00158

Scaling Trusted Autonomy: Towards Generally Capable Verifiable Robots

Benedict Quartey

The authors argue that scaling trusted autonomy requires a neurosymbolic approach that unifies general learned capabilities with explicit symbolic structure for reasoning and verification. They propose three interfaces—LIMP for language-to-plan grounding and inspectable interpretation, REALM for evolving 3D scene state that is updated after predicted manipulation, and PACTS for composing visuomotor skills with predictions of symbolic effects.

RoboticsAI summary
2026-11-03 · arXiv:2611.00455

TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation

Jiaxin Shi, Zibin Wu, Yichi Zhang et al.

TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation proposes TacMind-Bench (128 simulation tasks) to evaluate how tactile information helps precise contact, recognition under visual ambiguity, and dynamic interaction under 12 perturbations. The authors introduce TacMind-0: an action-conditioned predictive latent world model (Tac-LeWM) that learns streaming tactile states and refines actions using predictive tactile reflection.

RoboticsAI summary
2026-11-03 · arXiv:2611.00513

WorldSketch: Sketching World Models for Zero-Shot Mobile Manipulation

Jiaming Wang

WorldSketch is an agentic robotics paradigm where a VLM builds a minimal kinematic world model (a “world sketch”) using perception tools and visual checks, then a geometric planner optimizes contact and whole-body motion on the sketch. Execution feedback verifies outcomes, updates uncertain joint hypotheses, and triggers sketch revisions. The authors report zero-shot success of 91% (RoboCasa), 97% (LIBERO-PRO), and 76% (R2R-CE), with fewer VLM input tokens.

RoboticsAI summary
2026-11-03 · arXiv:2611.01600

SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts

Amir Hossain Raj, Xuesu Xiao

SCOUT is an occlusion-aware planning framework for cluttered manipulation. From sparse frontal RGB-D views, it reconstructs an object-centric hidden scene (GHOST) and derives a relational planning state with blocker relationships and accessibility corridors. An LLM proposes structured object-direction action candidates, and MCTS evaluates them using a learned action-conditioned world model with physical rollout predictions. The authors validate in simulation and on a Spot-based real shelf setup.

RoboticsAI summary
2026-11-03 · arXiv:2611.01729

ODE Policy: Robot Learning via Action Dynamics, Not Actions

Jindou Jia, Jingliang Li, Kuangji Zuo et al.

ODE policy learns the dynamics of robot actions as an ODE and integrates a learned action vector field over physical time, so executed trajectories are continuous by construction. Trained by matching integrated actions to targets in IL (over action chunks) and in RL (single control steps), with a diversity head for multi-modal behavior. It is evaluated across IL (including VLA and world action model heads) and model-based RL on multiple real and simulated robot tasks.

RoboticsAI summary