Driving · 2026-10-28
PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
Hanqing Wang, Zhaotian Cai, Ruichuan An, Bohan Zeng, Tao Zhang, Yilin Chen, Xu Jiang, Kaiquan Chen, Chongyang Wu, Haoran Li, Fan Yang, Mingan Lin, Wentao Zhang, Zenan Zhou, Youzheng Wu, Xiaodong He
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PDRVLA is a vision-language-action model that adds a lightweight Physical Dynamics Module to extract physical tokens from intermediate vision-language features. The physical module is trained to predict future RGB, depth, and optical-flow features, with auxiliary supervision from a frozen pretrained video world model. The learned physical tokens condition action generation without online future-video generation. Trained on publicly available data expanded via human-to-robot synthesis and annotations.
Why it matters
The authors address high latency in video world models by avoiding online future video generation at control time. By learning compact physical tokens from intermediate vision-language features and supervising them with future-feature prediction plus frozen video world-model assistance, PDRVLA aims to provide action-relevant physical dynamics for VLA policies.
Method
- Physical Dynamics Module extracts a fixed set of physical tokens from intermediate VLM features; action expert uses these tokens plus final-layer VLM features to generate actions.
- Direct prediction learns future RGB, depth, and optical-flow latent features as targets; auxiliary supervision uses a frozen pretrained video world model conditioned on physical tokens.
- Training uses publicly available data expanded through egocentric human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation; deployment uses physical tokens without running the video generator.
Limitation
The paper text provided does not state a limitation of the approach.
Abstract (from arXiv)
Vision--Language--Action (VLA) models generate robot actions from visual observations and language instructions, but their understanding of physical dynamics remains limited. Video world models can capture scene dynamics, yet generating future videos introduces high latency into the control loop. We introduce PDRVLA, whose lightweight Physical Dynamics Module extracts physical tokens from vision--language features to condition action generation without online video generation. The module learns through direct prediction of future RGB, depth, and optical-flow features, with auxiliary supervision through a frozen pretrained video world model. Training relies entirely on publicly available data, expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation. Across four simulation benchmarks, PDRVLA achieves success rates of 99.3% on LIBERO, 89.9% on LIBERO-Plus, 93.6% on RoboTwin~2.0, and 81.5% on SimplerEnv. Ablations show improved future prediction with world-model supervision and evaluate how subtask conditioning, future object-pose supervision, and ECoT training affect action success.
Related papers
- Forward Dynamics Models
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
- APS-VLA: Action-Guided and Future-Aware Token Pruning for Efficient Vision-Language-Action Models
- Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-27 · Indexed by Pub-AI: 2026-10-28 UTC
PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
Preprint posted: 2026-10-27 · Original record ↗
- arxiv
- 2610.37381
Explicit source versions (1)
- v1 · 2026-10-27
- Preprint available
Source date: 2026-10-27 · First detected: 2026-10-28 UTC · Historical baseline
Source evidence (3)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.37381v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision--Language--Action (VLA) models generate robot actions from visual observations and language instructions, but their understanding of physical dynamics remains limited. Video world models can capture scene dynamics, yet generating future videos introduces high latency into the control loop. We introduce PDRVLA, whose lightweight Physical Dynamics Module extracts physical tokens from vision--language features to condition action generation without online video generation. The module learns through direct prediction of future RGB, depth, and optical-flow features, with auxiliary supervision through a frozen pretrained video world model. Training relies entirely on publicly available data, expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation. Across four simulation benchmarks, PDRVLA achieves success rates of 99.3% on LIBERO, 89.9% on LIBERO-Plus, 93.6% on RoboTwin~2.0, and 81.5% on SimplerEnv. Ablations show improved future prediction with world-model supervision and evaluate how subtask conditioning, future object-pose supervision, and ECoT training affect action success.
