Driving · 2026-10-19
LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
Jiajun Cao, Kai Tang, Bohua Zhang, Zezhong Qian, Ying Li, Xiaozhu Ju, Xiaobao Wei, Jiaming Liu, Yang Ding, Sixiang Chen, Yiding Sun, Siyuan Qian, Tinghao Wang, Jixian Wu, Yichen Guo, Yi Zhang, Shilong Zou, Yong Dai, Jian Tang, Shanghang Zhang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
LAW is a closed-loop VLM–WAM framework for long-horizon physical manipulation that unifies language planning, action generation, and world modeling via semantic subtask progress. A hybrid-expert VLM plans remaining subtasks, estimates/uses subtask-local values to fuse WAM action chunks, and asynchronously assesses completion to continue or advance. The WAM uses MoTE and dual-stream hierarchical memory. LAW achieves 94.21% on RoboTwin; with OpenWAM-α, 95.56%.
Why it matters
The authors target long-horizon control where reasoning about semantic progress must stay aligned with action execution and world evolution. LAW’s design ties planning updates, action evaluation/fusion, and asynchronous completion assessment to semantic subtask boundaries, rather than a one-shot language plan or observation-only action scoring. This progress-aware loop is evaluated on 50 RoboTwin tasks and on long-horizon real-robot tasks.
Method
- Closed-loop progress-aware control: a hybrid-expert VLM plans remaining semantic subtasks, a WAM proposes action chunks, a value expert scores/fuses top-K proposals, and an asynchronous state expert predicts subtask completion to decide Continue vs Advance.
- Joint video–action world-action modeling with MoTE: Mixture-of-Transformer Experts applies residual MoE after mixed attention, using shared experts plus modality-private experts for video/action tokens.
- Subtask-aligned training for evaluation: subtask-local value learning terminates targets at semantic subtask boundaries, preserving positive supervision from successful prefixes of failed trajectories.
Limitation
The paper states that in the evaluation they do not use robot-data pretraining for their RoboTwin results (“LAW achieves 94.21% average success on RoboTwin, 2.38 percentage points above Fast-WAM, without robot-data pretraining.”), but it does not state other limitations.
Abstract (from arXiv)
World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM-$alpha$ further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for $pi_{0.5}$ and 25% for Fast-WAM.
Related papers
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-16 · Indexed by Pub-AI: 2026-10-19 UTC
LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
Preprint posted: 2026-10-16 · Original record ↗
- arxiv
- 2610.23334
Explicit source versions (1)
- v1 · 2026-10-16
- Preprint available
Source date: 2026-10-16 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM-$alpha$ further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for $pi_{0.5}$ and 25% for Fast-WAM.