Driving · 2026-10-05

Native Action-Prior Learning from Videos for World Action Models

Zhaochong An, Fei Zhang, Menglin Jia, Duncan Frost, Zijian Zhou, Yikai Wang, Xudong Wang, Aditya Patel, Belinda Zeng, Tao Xiang, Serge Belongie, Amir Bar, Sen He

arXiv:2610.03391 PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

NAVA-WAM is a world action model that directly pretrains an action policy from observation-only videos using “native action-prior learning,” avoiding intermediate representation-to-control transfer or a separate latent-action model. It uses future-video flow matching with transition-structured joint attention to train an Action-DiT, then post-trains with action-labeled demonstrations via video–action flow matching. The method supports action-only inference using cached visual representations.

Why it matters

The authors address a scalability bottleneck of world action models: collecting action-annotated robot trajectories. NAVA-WAM aims to learn action-relevant priors directly from observation-only videos, then adapt to continuous robot control with action-labeled demonstrations, and they report improvements in both simulation (ID/OOD) and real-robot transfer.

Method

Use a joint Mixture-of-Transformers (MoT) with a frozen Video-DiT and an optimized Action-DiT during action-free pre-training.
Native action-prior pre-training: future-video flow matching supervision over visual transitions is propagated into the Action-DiT through transition-structured joint attention.
Post-train on action-labeled demonstrations using joint video–action flow matching; use asymmetric attention so the Video-DiT is cached and inference denoises only robot actions without generating future videos.

Limitation

The authors state that NAVA-WAM can still struggle with challenging tasks requiring precise grasping, complex interactions, and other fine-grained contact-rich manipulation. They also note the policy currently lacks an explicit mechanism for recovering from local execution errors.

Abstract (from arXiv)

World action models integrate future visual dynamics with robot action prediction, but their scalability remains limited by the need for action-annotated robot trajectories. Observation-only videos contain rich evidence about interaction dynamics, but existing approaches typically use them either to pretrain visual representations that must later be adapted for control, or to infer latent actions that are subsequently grounded to robot commands. We present NAVA-WAM, which introduces native action-prior learning by directly pretraining the action policy from observation-only videos, avoiding indirect representation-to-control transfer or a separate latent-action model. Our training consists of two stages. First, we pretrain on observation-only videos, where future-video flow-matching supervision over visual transitions is propagated through transition-structured joint attention to optimize the Action-DiT and learn action-relevant priors. Second, we use action-labeled demonstrations to post-train the Action-DiT for robot control through joint video--action flow matching, while asymmetric attention decouples the visual branch from iterative action denoising and enables efficient action-only inference. Extensive experiments show that NAVA-WAM consistently outperforms prior approaches under both in-distribution and out-of-distribution settings, while demonstrating strong action-label efficiency and effective real-robot generalization. These results establish native action-prior learning as an effective approach to directly pretrain action policies from observation-only videos, providing a scalable path beyond action-labeled robot data.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-02 · Indexed by Pub-AI: 2026-10-05 UTC

preprint