Driving · 2026-11-06

Dynamic Watermarking and Action-Conditioned Latent Dynamics for Secure Vision-Based Controllers

Aditya Parameshwaran, Navid Aftabi, Abhishek Hanchate, Satish T. S. Bukkapatnam, Dan Li, Yue Wang

arXiv:2611.07403PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors propose an adaptive dynamic-watermarking framework for detecting replay attacks in vision-based controllers. A private Gaussian watermark is added to the control input, while a control input-conditioned latent dynamics network predicts future latent responses from images. Replay evidence is formed from predicted-vs-received latent discrepancies, and an RL policy adapts the watermark covariance to balance detectability, excitation, and task safety via a Lagrangian separation penalty plus Simplex stopping.

Why it matters

Replay attacks can keep observations visually plausible while the physical system diverges. The paper’s approach enables replay detection directly from high-dimensional image observations by testing a private watermark response in a learned latent space, then using detector confidence to trigger a safety-oriented fallback.

Method

  • Inject a private Gaussian watermark φt into the nominal control input ūt=u t+φt and use a control input-conditioned latent dynamics network (LDN) to predict next latent states from encoded image observations.
  • Compute replay evidence as a latent residual ri,j between predicted latent trajectory and latent representation of subsequently received (possibly replayed) observations, then feed it to an MCUSUM detector.
  • Train an RL policy (PPO) to adapt the watermark covariance Σt using latent state and detector confidence, with a Lagrangian penalty on robot–human separation violations; deploy a Simplex supervisor that switches to a damped-stop controller when confidence persists.

Limitation

A limitation remains that false attack detections can still interrupt the task and reduce performance; in the safety test, two switches occurred before attack onset and six clean episodes crossed the belief threshold.

Abstract (from arXiv)

Replay attacks are difficult to detect in vision-based control systems because previously recorded observations can remain plausible while the physical system follows a different trajectory. This paper develops an adaptive dynamic-watermarking framework for detecting replay attacks against vision-based controllers. A private Gaussian watermark is added to the nominal control input, and a control input-conditioned latent dynamics network (LDN) predicts the corresponding evolution in a compact latent space learned from image observations. Attack evidence is obtained from the discrepancy between the predicted latent response and the latent representation of the received image observation. A reinforcement learning policy adjusts the watermark covariance using the latent state and detector confidence, thereby balancing attack detectability, watermark energy, and control distortion. A Lagrangian penalty on task-specific safety costs is included during training to limit the adverse effect of the excitation. The framework is evaluated on a vision-guided UR5 manipulation task in which the robot reaches a target while avoiding a moving human arm under nominal and replay-attacked observations.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-05 · Indexed by Pub-AI: 2026-11-06 UTC

preprint

Dynamic Watermarking and Action-Conditioned Latent Dynamics for Secure Vision-Based Controllers

Aditya Parameshwaran, Navid Aftabi, Abhishek Hanchate, Satish T. S. Bukkapatnam, Dan Li, Yue Wang

Preprint posted: 2026-11-05 · Original record ↗

arxiv
2611.07403
Explicit source versions (1)
  • v1 · 2026-11-05
  1. Preprint available

    Source date: 2026-11-05 · First detected: 2026-11-06 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Replay attacks are difficult to detect in vision-based control systems because previously recorded observations can remain plausible while the physical system follows a different trajectory. This paper develops an adaptive dynamic-watermarking framework for detecting replay attacks against vision-based controllers. A private Gaussian watermark is added to the nominal control input, and a control input-conditioned latent dynamics network (LDN) predicts the corresponding evolution in a compact latent space learned from image observations. Attack evidence is obtained from the discrepancy between the predicted latent response and the latent representation of the received image observation. A reinforcement learning policy adjusts the watermark covariance using the latent state and detector confidence, thereby balancing attack detectability, watermark energy, and control distortion. A Lagrangian penalty on task-specific safety costs is included during training to limit the adverse effect of the excitation. The framework is evaluated on a vision-guided UR5 manipulation task in which the robot reaches a target while avoiding a moving human arm under nominal and replay-attacked observations.