Driving · 2026-10-20

PICA-WAM: Visual Perturbation Invariance via Cross-Appearance Self-Alignment in World Action Models

Yude Zou, Yifei Yao, Guanbo Jiao, Jianming Ma, Guanjie Zheng, Xing Gao

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

PICA-WAM trains World Action Models to be invariant to task-irrelevant visual perturbations by aligning action-stream representations across appearances. An EMA teacher encodes the nominal observation at a lower noise level, and the online model encodes a perturbed observation; a projection head aligns the student’s shallower action features to the teacher’s deeper ones. The authors report robustness gains on RoboTwin 2.0, LIBERO-Plus, EBench, and a real robot, while preserving nominal performance.

Why it matters

World action models can become brittle when visual appearance changes even if the underlying task state is the same. The paper targets this by adding an explicit cross-appearance self-alignment objective on the action stream, aiming to prevent task-irrelevant appearance from leaking into action representations used for control.

Method

Adds an auxiliary cross-appearance self-alignment loss to WAM training: student processes perturbed observations, EMA teacher processes nominal observations.
Aligns shallower-layer action-stream representations from the student (with a lightweight projection head) to deeper-layer teacher representations using cosine similarity with stop-gradient.
Leaves inference unchanged by discarding the teacher and projection head after training; the WAM architecture/parameter count and sampling procedure are unchanged.

Abstract (from arXiv)

World Action Models (WAMs) jointly predict future video and generate actions, yet remain sensitive to task-irrelevant visual perturbations, i.e., changes in visual appearance that should not affect robot behavior. We identify a gap in conventional WAM training: video prediction encourages representations to preserve appearance information, while action generation does not explicitly enforce invariance to task-irrelevant appearance. Data augmentation exposes WAMs to such perturbations but does not require perturbed and nominal observations to produce consistent action representations, and alignment with external pretrained features does not guarantee the desired invariance. To address this gap, we propose PICA-WAM, a self-supervised training framework that casts perturbation invariance as an auxiliary cross-appearance self-alignment task for WAMs. An exponential-moving-average (EMA) teacher processes the nominal observation to provide a deeper-layer target, while the online WAM processes a perturbed observation and aligns its shallower-layer action representation to this target through a lightweight projection head. Across four strong WAMs evaluated on RoboTwin 2.0, LIBERO-Plus, EBench, and a real robot, PICA-WAM consistently improves robustness under visual perturbations, preserves nominal performance in simulation, and raises the real robot success rate. PICA-WAM further reduces the sensitivity of video features to task-irrelevant appearance and its propagation into actions, as shown by paired layer-wise diagnostics. Code: https://github.com/YudeZou/PICA-WAM-release

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC

preprint