Robotics · 2026-10-07
ACG-WAM: World-Action Modeling via Action-Conditioned Geometric Latent Prediction
Jiangtao Liu, Zishang Xiang, Yage He, Lingguo Cui, Baihai Zhang, Runqi Chai, Senchun Chai
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ACG-WAM augments the Motus latent world-action model with ACG-JEPA: an auxiliary objective that predicts action-conditioned geometric endpoints at several horizons. It uses a frozen VGGT teacher to encode current/future image pairs and supervises a shared visual embedding from head and wrist cameras before temporal mixing; teacher/predictor modules are removed at inference. On RoboTwin 2.0 (50 tasks), the authors report 93.46% clean success, 92.68% randomized success, and 93.07% mean; on a real robot, 85.00% SR and 91.67% PCS.
Why it matters
The authors address two issues in world-action models: video/action losses lack explicit geometric supervision for demonstrated action outcomes, and temporal attention can leak future information into features used by auxiliary predictors. ACG-WAM ties geometric endpoint prediction to the policy’s deployed observation path by supervising the shared embedding before frame mixing and using a frozen teacher to construct horizon-conditioned geometric targets.
Method
- Adds an auxiliary task (ACG-JEPA) to Motus that predicts geometric features at multiple horizons from the current observation and an intervening action sequence.
- Uses a frozen VGGT teacher to encode current+future image pairs and retains the teacher’s future slot as geometric endpoint targets; gradients update the shared student embedding.
- Supervises the shared visual embedding from head and wrist cameras before temporal mixing; removes teacher/auxiliary modules at inference.
Abstract (from arXiv)
World action models jointly learn visual predictionand robot actions, providing a way to use observations ofscene evolution for policy learning. Their video and actionlosses, however, provide no explicit target for the geometricconsequences of a demonstrated action sequence. Moreover,visual features taken after temporal attention can contain futureobservations, making them unsuitable as the sole current visualinput to an auxiliary predictor. We introduce ACG-WAMand its auxiliary objective, the Action-Conditioned GeometricJoint-Embedding Predictive Architecture (ACG-JEPA), whichpredicts geometric features at several horizons from the currentobservation and intervening actions, using the future slot of afrozen VGGT encoding of each current and future image pairas the target. We apply this supervision from the head and wristcameras to a shared visual embedding before temporal mixing,and remove the teacher and auxiliary modules at inference.On 50 RoboTwin 2.0 tasks, ACG-WAM achieves 93.46%success in clean scenes, with the best randomized success(92.68%) and mean across both settings (93.07%) among thecompared methods; across three tasks on a real robot, itachieves 85.00% success and 91.67% partial completion score,exceeding Motus by 10.00 and 9.17 percentage points, respec-tively. Code:https://github.com/RoboOpus/ACG-WAM.Website:https://RoboOpus.github.io/ACG-WAM.
Related papers
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
- Native Action-Prior Learning from Videos for World Action Models
- RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations
- Copper-Policy: Focus on the Representation for Robust Robot Manipulation
- MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-03 · Indexed by Pub-AI: 2026-10-07 UTC
ACG-WAM: World-Action Modeling via Action-Conditioned Geometric Latent Prediction
Preprint posted: 2026-10-03 · Original record ↗
- arxiv
- 2610.06965
Explicit source versions (1)
- v1 · 2026-10-03
- Preprint available
Source date: 2026-10-03 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World action models jointly learn visual predictionand robot actions, providing a way to use observations ofscene evolution for policy learning. Their video and actionlosses, however, provide no explicit target for the geometricconsequences of a demonstrated action sequence. Moreover,visual features taken after temporal attention can contain futureobservations, making them unsuitable as the sole current visualinput to an auxiliary predictor. We introduce ACG-WAMand its auxiliary objective, the Action-Conditioned GeometricJoint-Embedding Predictive Architecture (ACG-JEPA), whichpredicts geometric features at several horizons from the currentobservation and intervening actions, using the future slot of afrozen VGGT encoding of each current and future image pairas the target. We apply this supervision from the head and wristcameras to a shared visual embedding before temporal mixing,and remove the teacher and auxiliary modules at inference.On 50 RoboTwin 2.0 tasks, ACG-WAM achieves 93.46%success in clean scenes, with the best randomized success(92.68%) and mean across both settings (93.07%) among thecompared methods; across three tasks on a real robot, itachieves 85.00% success and 91.67% partial completion score,exceeding Motus by 10.00 and 9.17 percentage points, respec-tively. Code:https://github.com/RoboOpus/ACG-WAM.Website:https://RoboOpus.github.io/ACG-WAM.