Video World Models · 2026-10-14
Across Viewpoints and Time: Trajectory-Generalizable Physical Attacks on World Models and Video Understanding Models
Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study whether a single printable adversarial texture can mislead world models and video understanding models across different real-world scenarios where viewpoints change along uncontrolled object–camera trajectories. They identify conflicting optimization gradients from heterogeneous video semantics and trajectories and propose TARA (trajectory-aware contrastive alignment + trajectory-aware task refinement). They also build a geometry-grounded tracking pipeline and collect print-and-recapture videos for reproducible physical evaluation.
Why it matters
Physical attacks must work under uncontrolled observation: a fixed attached texture can change in viewpoint, scale, and visibility over time. The authors propose a trajectory-aware optimization to improve cross-trajectory generalization and provide temporally consistent surface trajectories and a real-world testbed to evaluate robustness. They report that a single printable adversarial texture can persistently disrupt both world models and video understanding models under physical conditions.
Method
- Proposes TARA: optimize a single shared printable texture across diverse video–trajectory instances using trajectory-aware contrastive alignment to bring attacked representations to a shared adversarial region and trajectory-aware task refinement to induce task-level prediction errors.
- Models physical observation as a temporally coupled sequence of homographies and visibility masks (observation trajectory γ), and uses trajectory-aware weighting inversely related to projected visible texture area during task refinement.
- Develops a geometry-grounded tracking pipeline to produce physically plausible, temporally consistent surface trajectories (homographies and visibility masks) on Something-Something V2 and EPIC-KITCHENS-100, plus real-world print-and-recapture videos for evaluation.
Limitation
The paper states that evaluating physical attacks on video models requires temporally coherent surface trajectories, and that existing image-level benchmarks do not provide them.
Abstract (from arXiv)
World models and video understanding models are increasingly deployed in dynamic visual environments, yet their robustness to physical adversarial threats remains largely underexplored. We study whether a single physical adversarial texture can mislead the victim model across different real-world scenarios where observing viewpoints vary along uncontrolled object--camera trajectories. We identify that heterogeneous video semantics and trajectories induce poorly aligned gradients for optimizing the adversarial texture. To address that, we propose the Trajectory-aware adversarial Alignment and Refinement Attack (TARA), which incorporates trajectory-aware optimization to capture the evolving geometric variations of a printed texture under real-world scenarios. To improve cross-instance optimization compatibility, TARA incorporates trajectory-aware contrastive alignment to encourage attacked representations from diverse video--trajectory instances to approach a shared adversarial region and then performs trajectory-aware task refinement to translate the aligned representation into task-level prediction errors. For reproducible evaluation, we develop a geometry-grounded tracking pipeline that provides physically plausible adversarial regions with temporally consistent trajectories on EPIC-KITCHENS-100 and Something-Something V2. We further collect real-world print-and-recapture videos to validate attack effectiveness under physical deployment conditions. Together, the annotated benchmarks and real-world captured data form a reproducible testbed for future evaluation. Notably, our results reveal that a single printable adversarial texture can persistently disrupt world models and video understanding models under physical conditions, highlighting the fundamental security challenge.
Related papers
- PASA-KV: Attacking Action-Facing Visual Routes in World Action Models
- ActionRoPE: action as coordinates for spatially consistent control in planar-camera game world models
- Counterfactual Action Evaluation, Observation Bottlenecks, and Representation Geometry in Joint-Embedding Predictive World Models
- TAPDreamer: Transferable Adversarial Patches for World Action Models
- Spoofing the Latent State: Imperceptible Goal-Image Attacks Derail Planning in LeWorldModel
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-11 · Indexed by Pub-AI: 2026-10-14 UTC
Across Viewpoints and Time: Trajectory-Generalizable Physical Attacks on World Models and Video Understanding Models
Preprint posted: 2026-10-11 · Original record ↗
- arxiv
- 2610.15343
Explicit source versions (1)
- v1 · 2026-10-11
- Preprint available
Source date: 2026-10-11 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World models and video understanding models are increasingly deployed in dynamic visual environments, yet their robustness to physical adversarial threats remains largely underexplored. We study whether a single physical adversarial texture can mislead the victim model across different real-world scenarios where observing viewpoints vary along uncontrolled object--camera trajectories. We identify that heterogeneous video semantics and trajectories induce poorly aligned gradients for optimizing the adversarial texture. To address that, we propose the Trajectory-aware adversarial Alignment and Refinement Attack (TARA), which incorporates trajectory-aware optimization to capture the evolving geometric variations of a printed texture under real-world scenarios. To improve cross-instance optimization compatibility, TARA incorporates trajectory-aware contrastive alignment to encourage attacked representations from diverse video--trajectory instances to approach a shared adversarial region and then performs trajectory-aware task refinement to translate the aligned representation into task-level prediction errors. For reproducible evaluation, we develop a geometry-grounded tracking pipeline that provides physically plausible adversarial regions with temporally consistent trajectories on EPIC-KITCHENS-100 and Something-Something V2. We further collect real-world print-and-recapture videos to validate attack effectiveness under physical deployment conditions. Together, the annotated benchmarks and real-world captured data form a reproducible testbed for future evaluation. Notably, our results reveal that a single printable adversarial texture can persistently disrupt world models and video understanding models under physical conditions, highlighting the fundamental security challenge.
