Driving · 2026-10-30
PASA-KV: Attacking Action-Facing Visual Routes in World Action Models
Zhongbo Wu, Jiakai Huang, Zheng Zhang, Siyu Xu, Kaixiang Chen, Xiaohong Liu, Shan You, Chang Xu, Tao Huang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study when perturbations to internal visual K/V routes in world action models become behaviorally consequential. They find a mismatch: layers with larger representation displacement are not necessarily more vulnerable. They attribute this to action-conditioned readout, where keys route action queries and values determine what content is read. They introduce Tail-AV and PASA-KV to select action-facing K/V targets before training a task-shared patch attack.
Why it matters
World action models couple internal visual/world representations to action generation, so robustness depends on the world-to-action pathways. The paper shows that representation perturbability alone does not predict closed-loop vulnerability, and proposes an action-facing view that depends on how perturbations are routed into attention/readout for action queries. This leads to a route-selection method (Tail-AV/PASA-KV) that improves targeted closed-loop attacks across systems, and reveals architecture-dependent resistance.
Method
- Define action-facing visual K/V as the keys and values consumed by action queries, and study how perturbing different visual K/V layers affects closed-loop policy success.
- Introduce Tail-AV: a short-probe signal that ranks candidate layers by how a perturbation changes the action-query readout, without rollout outcomes.
- Propose PASA-KV: probe candidate action-facing K/V routes with singleton patches, select Top-k routes via Tail-AV, then train a fresh task-shared patch attack only on the selected routes.
Limitation
The authors state that Tail-AV is a short-budget ranking signal rather than a calibrated surrogate for final behavioral damage.
Abstract (from arXiv)
World action models (WAMs) increasingly couple internal visual or world representations with action generation, making their world-to-action pathways a critical yet poorly understood source of robustness. We investigate when perturbations to these internal representations actually become behaviorally consequential, and uncover a pronounced mismatch between representation perturbability and closed-loop vulnerability: attack effectiveness varies sharply across visual K/V routes, while larger representation displacement does not reliably imply stronger control failure. Our analysis attributes this mismatch to the action-conditioned readout, where keys determine where action queries read and values determine what content they receive. This motivates an action-facing view of vulnerability, in which the effect of a perturbation depends on how it is routed into action generation rather than on representation-space magnitude alone. We instantiate this view with Tail-AV, a short-probe signal for identifying vulnerable routes, and operationalize it in PASA-KV, which selects action-facing K/V targets before training a fresh task-shared attack. Experiments across multiple WAM systems and a physical robot validate readout-informed target selection, demonstrate strong closed-loop attacks, and reveal substantial system-level differences in resistance.
Related papers
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-10-30 · Indexed by Pub-AI: 2026-10-30 UTC
PASA-KV: Attacking Action-Facing Visual Routes in World Action Models
First announced by arXiv: 2026-10-30 · Original record ↗
- arxiv
- 2610.39816
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-10-30 · First detected: 2026-10-30 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.39816v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
World action models (WAMs) increasingly couple internal visual or world representations with action generation, making their world-to-action pathways a critical yet poorly understood source of robustness. We investigate when perturbations to these internal representations actually become behaviorally consequential, and uncover a pronounced mismatch between representation perturbability and closed-loop vulnerability: attack effectiveness varies sharply across visual K/V routes, while larger representation displacement does not reliably imply stronger control failure. Our analysis attributes this mismatch to the action-conditioned readout, where keys determine where action queries read and values determine what content they receive. This motivates an action-facing view of vulnerability, in which the effect of a perturbation depends on how it is routed into action generation rather than on representation-space magnitude alone. We instantiate this view with Tail-AV, a short-probe signal for identifying vulnerable routes, and operationalize it in PASA-KV, which selects action-facing K/V targets before training a fresh task-shared attack. Experiments across multiple WAM systems and a physical robot validate readout-informed target selection, demonstrate strong closed-loop attacks, and reveal substantial system-level differences in resistance.
