Driving · 2026-11-06
Dynamic Watermarking and Action-Conditioned Latent Dynamics for Secure Vision-Based Controllers
Aditya Parameshwaran, Navid Aftabi, Abhishek Hanchate, Satish T. S. Bukkapatnam, Dan Li, Yue Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose an adaptive dynamic-watermarking framework for detecting replay attacks in vision-based controllers. A private Gaussian watermark is added to the control input, while a control input-conditioned latent dynamics network predicts future latent responses from images. Replay evidence is formed from predicted-vs-received latent discrepancies, and an RL policy adapts the watermark covariance to balance detectability, excitation, and task safety via a Lagrangian separation penalty plus Simplex stopping.
Why it matters
Replay attacks can keep observations visually plausible while the physical system diverges. The paper’s approach enables replay detection directly from high-dimensional image observations by testing a private watermark response in a learned latent space, then using detector confidence to trigger a safety-oriented fallback.
Method
- Inject a private Gaussian watermark φt into the nominal control input ūt=u t+φt and use a control input-conditioned latent dynamics network (LDN) to predict next latent states from encoded image observations.
- Compute replay evidence as a latent residual ri,j between predicted latent trajectory and latent representation of subsequently received (possibly replayed) observations, then feed it to an MCUSUM detector.
- Train an RL policy (PPO) to adapt the watermark covariance Σt using latent state and detector confidence, with a Lagrangian penalty on robot–human separation violations; deploy a Simplex supervisor that switches to a damped-stop controller when confidence persists.
Limitation
A limitation remains that false attack detections can still interrupt the task and reduce performance; in the safety test, two switches occurred before attack onset and six clean episodes crossed the belief threshold.
Abstract (from arXiv)
Replay attacks are difficult to detect in vision-based control systems because previously recorded observations can remain plausible while the physical system follows a different trajectory. This paper develops an adaptive dynamic-watermarking framework for detecting replay attacks against vision-based controllers. A private Gaussian watermark is added to the nominal control input, and a control input-conditioned latent dynamics network (LDN) predicts the corresponding evolution in a compact latent space learned from image observations. Attack evidence is obtained from the discrepancy between the predicted latent response and the latent representation of the received image observation. A reinforcement learning policy adjusts the watermark covariance using the latent state and detector confidence, thereby balancing attack detectability, watermark energy, and control distortion. A Lagrangian penalty on task-specific safety costs is included during training to limit the adverse effect of the excitation. The framework is evaluated on a vision-guided UR5 manipulation task in which the robot reaches a target while avoiding a moving human arm under nominal and replay-attacked observations.
Related papers
- Higher-Order Action Supervision Makes A Strong Policy Class
- W2A: Distilling World Priors into Action Models for Real-Time Robotic Control
- ELA: End-to-End Latent Action Policy Learning from Video Demonstrations
- Learning Transferable Policies from Action-free Time Series Through Dynamical Embeddings
- Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-05 · Indexed by Pub-AI: 2026-11-06 UTC
Dynamic Watermarking and Action-Conditioned Latent Dynamics for Secure Vision-Based Controllers
Preprint posted: 2026-11-05 · Original record ↗
- arxiv
- 2611.07403
Explicit source versions (1)
- v1 · 2026-11-05
- Preprint available
Source date: 2026-11-05 · First detected: 2026-11-06 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.07403v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Replay attacks are difficult to detect in vision-based control systems because previously recorded observations can remain plausible while the physical system follows a different trajectory. This paper develops an adaptive dynamic-watermarking framework for detecting replay attacks against vision-based controllers. A private Gaussian watermark is added to the nominal control input, and a control input-conditioned latent dynamics network (LDN) predicts the corresponding evolution in a compact latent space learned from image observations. Attack evidence is obtained from the discrepancy between the predicted latent response and the latent representation of the received image observation. A reinforcement learning policy adjusts the watermark covariance using the latent state and detector confidence, thereby balancing attack detectability, watermark energy, and control distortion. A Lagrangian penalty on task-specific safety costs is included during training to limit the adverse effect of the excitation. The framework is evaluated on a vision-guided UR5 manipulation task in which the robot reaches a target while avoiding a moving human arm under nominal and replay-attacked observations.
