Driving · 2026-10-08
{\Delta}WAM: Distilling Action Tangent Fields into World Action Models
Ke Wu, Hanwen Huang, Bo Gu, Kaizhao Zhang, Xiangting Meng, Yupeng Zheng, Zijun Xu, Jieru Zhao, Wenchao Ding
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ΔΔWAM introduces Action Tangent Fields (ATFs) to reweight world-action model (WAM) denoising toward residual-dynamics directions that change under local action perturbations. It uses an information-preserving Residual-VAE target (residual relative to a broadcast reference) and probes action-responsive directions with a frozen action-conditioned world model (ACWM). The method improves robustness on LIBERO-Plus and RoboTwin benchmarks, and distills multi-step VideoDiT denoising into one step.
Why it matters
The authors argue that dense future prediction can be dominated by appearance and scene persistence, which hurts control relevance. ΔΔWAM keeps the supervision information-rich while emphasizing a local neighborhood of action-relevant residual dynamics, improving robustness under multiple distribution shifts without large-scale embodied pretraining.
Method
- Residual-VAE mid-training: adapt a Wan VideoDiT to predict future latents relative to a broadcast reference from the first frame, using Residual-VAE space so the future latent can be recovered from the current latent and its residual.
- Action Tangent Fields: perturb demonstrated actions and query a frozen embodiment-specific ACWM to compute normalized response differences; orthonormalize them into cached locally action-sensitive directions (ATF basis).
- ATF-guided post-training + single-pass inference: post-train a Fast-WAM-style VideoDiT–ActionDiT where an ATF loss upweights residual prediction errors projected onto the cached ATF directions; inference uses one VideoDiT pass.
Limitation
Large camera and layout changes remain challenging.
Abstract (from arXiv)
World Action Models (WAM) improve robot policies by augmenting sparse action supervision with dense future prediction. However, much of the predictable future is dominated by appearance and scene persistence rather than action-dependent dynamics. We observe that several recent WAM designs, including optical flow, motion-centric representations, and latent actions, can be understood from a common perspective in which world supervision becomes more efficient as it contains a higher proportion of action-relevant variation. Based on this insight, we introduce Action Tangent Fields, which reformulate world supervision through a local Taylor expansion of how actions induce changes in future dynamics. We represent future dynamics in Residual-VAE space, where the future latent remains recoverable from the current latent and its residual, and use a strong action-conditioned world model (ACWM) to probe the local correspondence between action variations and residual-world variations. This local first-order structure is distilled into the WAM to guide its denoising supervision toward dynamics that are more tightly coupled to action, rather than merely predictable from appearance. Across LIBERO-Plus, RoboTwin, and RoboTwin2.0-Plus, our method consistently improves robustness to lighting, background, camera, layout, and other environmental perturbations. Despite using no large-scale embodied pretraining, it achieves stronger robustness under several distribution shifts than pretrained policies. We further distill multi-step VideoDiT denoising into a single step for efficient inference. Our results suggest that effective WAM supervision should remain information-rich while concentrating its predictive capacity on the directions along which actions change the future.
Related papers
- CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight
- MARS: Systematic Multi-Level Acceleration for World Action Models
- De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control
- Self-Distilling Future Video Representations for World Action Models
- Long-WAM: Scaling the Context of World-Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-07 · Indexed by Pub-AI: 2026-10-08 UTC
ΔWAM: Distilling Action Tangent Fields into World Action Models
Preprint posted: 2026-10-07 · Original record ↗
- arxiv
- 2610.09734
Explicit source versions (1)
- v1 · 2026-10-07
- Preprint available
Source date: 2026-10-07 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World Action Models (WAM) improve robot policies by augmenting sparse action supervision with dense future prediction. However, much of the predictable future is dominated by appearance and scene persistence rather than action-dependent dynamics. We observe that several recent WAM designs, including optical flow, motion-centric representations, and latent actions, can be understood from a common perspective in which world supervision becomes more efficient as it contains a higher proportion of action-relevant variation. Based on this insight, we introduce Action Tangent Fields, which reformulate world supervision through a local Taylor expansion of how actions induce changes in future dynamics. We represent future dynamics in Residual-VAE space, where the future latent remains recoverable from the current latent and its residual, and use a strong action-conditioned world model (ACWM) to probe the local correspondence between action variations and residual-world variations. This local first-order structure is distilled into the WAM to guide its denoising supervision toward dynamics that are more tightly coupled to action, rather than merely predictable from appearance. Across LIBERO-Plus, RoboTwin, and RoboTwin2.0-Plus, our method consistently improves robustness to lighting, background, camera, layout, and other environmental perturbations. Despite using no large-scale embodied pretraining, it achieves stronger robustness under several distribution shifts than pretrained policies. We further distill multi-step VideoDiT denoising into a single step for efficient inference. Our results suggest that effective WAM supervision should remain information-rich while concentrating its predictive capacity on the directions along which actions change the future.