Driving · 2026-10-21

De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control

Shaoyi Zheng, Zhongtao Jiang, Yue Peng, Wenbo Lu, Shunchi Zhang, Xi Wang, Shengji Wan

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Cascade WAMs accelerate control by predicting visual futures, but diffusion experts require costly iterative inference. The authors show that removing video/action stochasticity has limited impact, while naive one-step truncation sharply reduces control success. They propose DirectWAM, converting diffusion-pretrained video/action experts into a deterministic single-step future-to-action cascade trained by action-sufficient distillation. DirectWAM achieves 97.1% average success on four LIBERO suites and 85.4% on RoboTwin 2.0, with a 28× lower inference latency than the full iterative teacher.

Why it matters

The work provides a control-oriented criterion for learning single-step future predictors in cascade WAMs: the future should be optimized for the action it enables, not for fidelity to the teacher’s iterative denoising trajectory or representation-space matching alone. This directly targets the deployment bottleneck of diffusion-based iterative inference while preserving the explicit future-to-action interface for control and inspection.

Method

Analyze the roles of stochastic sampling vs iterative refinement in a cascade WAM (vary video noise; vary video/action inference steps).
Introduce action-sufficient distillation for DirectWAM: train a one-step action model, then train a one-step future predictor by decoding actions from its own predicted futures and backpropagating demonstration action error through the frozen expert (optional future-latent reconstruction anchor).
Evaluate the learned future by its downstream control consequences (offline decoded action error and closed-loop success), not only representation matching.

Limitation

The paper’s local sensitivity analysis is constructed using a Jacobian-based probe built from 32 sampled action-output directions, and it “does not identify the complete action-sensitive subspace,” so its conclusions are specific to that diagnostic setup.

Abstract (from arXiv)

Cascade world-action models (WAMs) use predicted visual futures to condition robot actions, but inherit costly iterative inference from diffusion-pretrained experts. We ask what a single-step predicted future must preserve to support accurate single-step action prediction. In the evaluated WAM, resampling video noise has little effect on downstream actions, yet naively truncating both streams to one step substantially reduces task success. Video-space supervision only partially closes this gap, and a local sensitivity analysis shows that vision prediction errors have highly unequal consequences for action accuracy. These findings motivate DirectWAM, which converts pretrained video and action experts into single-step predictors without stochastic sampling through action-sufficient distillation, while retaining the explicit future-to-action cascade. After learning a one-step action model, we train the future predictor by decoding actions from its own predicted futures and backpropagating demonstration action error through the expert, with an optional future-latent reconstruction loss. The resulting objective evaluates the student-generated future through its downstream control consequences rather than relying on representation matching alone. DirectWAM achieves 97.1% average success across four LIBERO suites and 85.4% on RoboTwin 2.0, outperforming Flash-WAM at the same one-evaluation-per-stream budget. This advantage has also extended to three real-world tasks that we designed. Compared with the full iterative teacher, it reduces inference latency by $28.0\times$.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-20 · Indexed by Pub-AI: 2026-10-21 UTC

preprint