Driving · 2026-10-22
Relational Geometry Forcing for World Action Models
Yuan Xu, Yixiang Chen, Qisen Ma, Jiabing Yang, Peiyan Li, Kai Wang, Jianhua Yang, Jianlou Si, Jun Huang, Jing Liu, Nianfeng Liu, Yan Huang, Liang Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
World Action Models (WAMs) learn visual dynamics and actions but their objectives don’t explicitly constrain 3D geometry for precise manipulation. The authors propose Relational Geometry Forcing (RGF), which transfers geometry priors from a frozen 3D foundation model by aligning pairwise feature relations (intra-frame, cross-view, and action-temporal) during training only, without adding a geometry branch for inference.
Why it matters
RGF targets a stated failure mode of pointwise feature alignment: dense geometric targets can benefit video features but degrade action learning because compact action features summarize transitions while dense targets describe scene content. By matching relational structures instead, the authors aim to regularize both video and action representations for manipulation, improving performance across simulation and real-world tasks.
Method
- Transfer geometric priors from a frozen VGGT teacher by aligning pairwise feature relations rather than pointwise features.
- Use three relational objectives: intra-frame relations (within a view), cross-view relations (between views), and an action-temporal objective coupling action dynamics to scene evolution.
- RGF is training-only: VGGT, projection heads, and relation computations are removed at inference; the WAM uses only RGB observations and language instructions.
Abstract (from arXiv)
World Action Models (WAMs) have shown strong potential for robot manipulation by jointly learning visual dynamics and executable actions. However, their video and action objectives do not explicitly constrain the 3D geometry that is crucial for precise manipulation. Recently, geometry-aware WAMs introduce depth or 4D prediction branches, whereas representation alignment methods supervise intermediate WAM features with pretrained 3D foundation models. Yet we find that such pointwise alignment benefits the video features but degrades action learning, since dense geometric feature targets overly restrict compact action features. To address this limitation, we propose Relational Geometry Forcing (RGF), which transfers geometric priors from a frozen 3D foundation model by aligning pairwise feature relations instead of pointwise features. Specifically, RGF applies three complementary relational objectives, where intra-frame and cross-view relations transfer scene geometry and multi-view correspondence to the video features, while action-temporal relations couple action dynamics to geometric scene evolution. Applied only during training, RGF requires no explicit geometry branch or extra inference module. Extensive experiments in simulation and the real world demonstrate the effectiveness of Relational Geometry Forcing, showing consistent gains across diverse manipulation scenarios and WAM paradigms.
Related papers
- MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation
- UNITAS: A 3D-Native World Action Model for Embodied Manipulation
- ORD-WM: Learning Object-Centric Rigid Dynamics for 3D World Modeling in Robotic Manipulation
- PointWAM: 3D World Action Modeling for Dexterous Robotic Manipulation
- SkeleWAM: Skeleton World-Action Modeling for Efficient Robotic Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-21 · Indexed by Pub-AI: 2026-10-22 UTC
Relational Geometry Forcing for World Action Models
Preprint posted: 2026-10-21 · Original record ↗
- arxiv
- 2610.29282
Explicit source versions (1)
- v1 · 2026-10-21
- Preprint available
Source date: 2026-10-21 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World Action Models (WAMs) have shown strong potential for robot manipulation by jointly learning visual dynamics and executable actions. However, their video and action objectives do not explicitly constrain the 3D geometry that is crucial for precise manipulation. Recently, geometry-aware WAMs introduce depth or 4D prediction branches, whereas representation alignment methods supervise intermediate WAM features with pretrained 3D foundation models. Yet we find that such pointwise alignment benefits the video features but degrades action learning, since dense geometric feature targets overly restrict compact action features. To address this limitation, we propose Relational Geometry Forcing (RGF), which transfers geometric priors from a frozen 3D foundation model by aligning pairwise feature relations instead of pointwise features. Specifically, RGF applies three complementary relational objectives, where intra-frame and cross-view relations transfer scene geometry and multi-view correspondence to the video features, while action-temporal relations couple action dynamics to geometric scene evolution. Applied only during training, RGF requires no explicit geometry branch or extra inference module. Extensive experiments in simulation and the real world demonstrate the effectiveness of Relational Geometry Forcing, showing consistent gains across diverse manipulation scenarios and WAM paradigms.