Robotics · 2026-11-11

LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning

Xuheng Zhang, Yiming Zhang, Jialin Gao, Hao Chen, Jiaming Liu, Ji Li, Runzhong Huang, Yinxi Wang, Chenyang Gu, Zihan Yang, Yaowei Liu, Zhonghao Yan, Ziyu Guo, Renrui Zhang, Peng Jia, Pheng-Ann Heng, Shanghang Zhang

arXiv:2611.14048PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

LaST 0.5 is a world action model for robotic manipulation that learns interaction-centric latent reasoning between world modeling and action generation. It uses a Mixture-of-Transformers with a world expert, a latent expert supervised by visual interaction cues, and an action expert. It also introduces AWAT to adapt the action execution horizon using predicted progress consistency. Pretraining uses 14k+ hours of robotic data.

Why it matters

The authors address a mismatch in existing world action models where global scene representations may underemphasize local robot–object interactions needed for precise manipulation. LaST 0.5’s interaction-centric latent space and progress-consistency-based world-to-action transfer aim to improve action reliability in long-horizon control and provide strong simulation and real-world results.

Method

  • Learns an interaction-centric latent space: a latent expert distills temporally aligned future frames into compact interaction-relevant tokens supervised by visual interaction cues (2D end-effector motion curve and target-object affordance bounding box).
  • Uses a Mixture-of-Transformers architecture with hierarchical video → latent → action reasoning: a world expert predicts future scene video, a latent expert predicts interaction-centric latent tokens, and an action expert generates the action chunk.
  • Adaptive World-to-Action Transfer (AWAT): attaches a lightweight value head to predict progress per future segment and truncates action execution when progress drops, using progress consistency to determine the execution horizon.

Limitation

The authors describe failures due to physical boundaries of the current system, including mechanical resistance when the zipper gets stuck, slight inaccuracies in modeling spatial geometries, hardware limitations of a parallel-jaw gripper (tape slip due to insufficient grip force and friction), and tight-tolerance insertion challenges requiring millimeter-level alignment from visual feedback.

Abstract (from arXiv)

World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation, endowing policies with strong capabilities for modeling temporal and physical dynamics. Existing WAMs, encompassing both video-first action prediction and joint video-action modeling, directly transfer global physical representations from world models to action generation. However, global scene representations may underemphasize local robot-object interactions, creating a mismatch between representations optimized for world evolution and those required for manipulation. To bridge this gap, we propose LaST0.5, a novel WAM that introduces an interaction-centric physical representation between world modeling and action generation. Specifically, LaST0.5 learns a compact interaction-centric latent space under the supervision of visual interaction cues between the robot and the environment. This reasoning space is instantiated as a latent expert that bridges the world expert and the action expert within a Mixture-of-Transformers architecture. Furthermore, LaST0.5 performs adaptive world-to-action transfer by jointly predicting future representations and progress values, using progress consistency to adjust action execution and trigger future resampling. Finally, we pretrain LaST0.5 on a large-scale robotic data corpus comprising over 14k hours of data, establishing interaction-centric physical reasoning as a scalable and effective paradigm. LaST0.5 achieves 99.5% on LIBERO, surpasses the previous state of the art on ten evaluated RLBench tasks, and demonstrates robust long-horizon execution and precise dexterous manipulation in real-world experiments.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-10 · Indexed by Pub-AI: 2026-11-11 UTC

preprint

LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning

Xuheng Zhang, Yiming Zhang, Jialin Gao, Hao Chen, Jiaming Liu, Ji Li, Runzhong Huang, Yinxi Wang, Chenyang Gu, Zihan Yang, Yaowei Liu, Zhonghao Yan, Ziyu Guo, Renrui Zhang, Peng Jia, Pheng-Ann Heng, Shanghang Zhang

Preprint posted: 2026-11-10 · Original record ↗

arxiv
2611.14048
Explicit source versions (1)
  • v1 · 2026-11-10
  1. Preprint available

    Source date: 2026-11-10 · First detected: 2026-11-11 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation, endowing policies with strong capabilities for modeling temporal and physical dynamics. Existing WAMs, encompassing both video-first action prediction and joint video-action modeling, directly transfer global physical representations from world models to action generation. However, global scene representations may underemphasize local robot-object interactions, creating a mismatch between representations optimized for world evolution and those required for manipulation. To bridge this gap, we propose LaST0.5, a novel WAM that introduces an interaction-centric physical representation between world modeling and action generation. Specifically, LaST0.5 learns a compact interaction-centric latent space under the supervision of visual interaction cues between the robot and the environment. This reasoning space is instantiated as a latent expert that bridges the world expert and the action expert within a Mixture-of-Transformers architecture. Furthermore, LaST0.5 performs adaptive world-to-action transfer by jointly predicting future representations and progress values, using progress consistency to adjust action execution and trigger future resampling. Finally, we pretrain LaST0.5 on a large-scale robotic data corpus comprising over 14k hours of data, establishing interaction-centric physical reasoning as a scalable and effective paradigm. LaST0.5 achieves 99.5% on LIBERO, surpasses the previous state of the art on ten evaluated RLBench tasks, and demonstrates robust long-horizon execution and precise dexterous manipulation in real-world experiments.