Driving · 2026-10-02
ActiveWAM: Evidence-Aware Active Vision for World-Action Models
Renjun Wu, Luzhou Ge, Xuesong Li
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ActiveWAM treats active vision manipulation as an evidence-aware retain–acquire problem, unifying pan/tilt “head” control with bimanual manipulation in a world–action model. It uses training-time inversion to constrain a frozen video prior with task-bearing source evidence and visible temporal changes, enabling raw-input deployment without online inversion or candidate viewpoint ranking. It co-trains with future-video prediction and introduces RoboTwin-AV, a 50-task benchmark with executable pan/tilt control.
Why it matters
Camera motion changes what evidence remains visible within a finite observation window, creating a retain–acquire trade-off: moving can gain new cues while discarding task-critical ones. The authors’ method specifically couples evidence retention (via task-guided history inversion during training) with executable observation acquisition (via learned head actions) inside one unified world–action model, and evaluates it on both a TAVIS benchmark and a new 50-task RoboTwin-AV suite with physical real-world trials.
Method
- Formulates active vision manipulation as evidence-aware retain–acquire control, learning to retain task-critical evidence and acquire new evidence via head (pan/tilt) actions plus arm actions.
- Uses training-time inversion (TTI) to constrain a frozen video prior with task-bearing source evidence and visible temporal changes, allowing deployment without test-time inversion or candidate ranking.
- Deploys a unified world–action model that generates one head–arm action trajectory from view-aware history; future-video prediction is used as a co-training signal.
Limitation
The current system uses a fixed-focal-length head, short history window, and prescribed stage switching.
Abstract (from arXiv)
Active vision manipulation requires a policy to control both its camera and its end-effectors, yet camera motion determines which evidence remains visible within finite observation windows. Acquiring a new view can displace task-critical cues, while retaining a view forgoes potentially useful observations. We formulate this as an evidence-aware retain--acquire problem and present ActiveWAM, a unified world--action model that learns observation and manipulation jointly. To this end, we propose training-time inversion which constrains a frozen video prior by task-bearing source evidence and visible temporal changes, eliminating the need for test-time inversion or candidate ranking. At deployment, the policy generates bimanual and pan/tilt actions-including stay and reacquisition behaviors-from view-aware history, and updates context from newly measured RGB observations. Future-video prediction serves as a co-training signal, while action generation requires neither future-video decoding nor optimal viewpoint annotations. We introduce RoboTwin-AV, a 50-task benchmark with executable pan/tilt control and automatically generated demonstrations. ActiveWAM improves TAVIS out-of-distribution success by up to 17.0 percentage points over the strongest baselines, achieves 20.0 additional points over Fast-WAM on RoboTwin-AV, and outperforms it by 26.7 points on real-world physical kitchen tasks.
Related papers
- OpenWAM: An Open Framework for Composable World-Action Models
- AWAM: Active Trajectory-Conditioned View Imagination for Robust Robot Manipulation
- TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control
- GlanceWAM: Sparse Test-Time Imagination for World-Action Models
- EVO-WAM: Evolving World Action Models through Video-Action Verification
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC
ActiveWAM: Evidence-Aware Active Vision for World-Action Models
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.01698
Explicit source versions (1)
- v1 · 2026-10-01
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Active vision manipulation requires a policy to control both its camera and its end-effectors, yet camera motion determines which evidence remains visible within finite observation windows. Acquiring a new view can displace task-critical cues, while retaining a view forgoes potentially useful observations. We formulate this as an evidence-aware retain--acquire problem and present ActiveWAM, a unified world--action model that learns observation and manipulation jointly. To this end, we propose training-time inversion which constrains a frozen video prior by task-bearing source evidence and visible temporal changes, eliminating the need for test-time inversion or candidate ranking. At deployment, the policy generates bimanual and pan/tilt actions-including stay and reacquisition behaviors-from view-aware history, and updates context from newly measured RGB observations. Future-video prediction serves as a co-training signal, while action generation requires neither future-video decoding nor optimal viewpoint annotations. We introduce RoboTwin-AV, a 50-task benchmark with executable pan/tilt control and automatically generated demonstrations. ActiveWAM improves TAVIS out-of-distribution success by up to 17.0 percentage points over the strongest baselines, achieves 20.0 additional points over Fast-WAM on RoboTwin-AV, and outperforms it by 26.7 points on real-world physical kitchen tasks.
