Driving · 2026-10-07

AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model

Sarim Hashmi, Mukul Ranjan, Kshitij Mishra, Mikhail Kuznetsov, Praneeth Vepakomma, Nils Lukas

arXiv:2610.08773PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

AdvSim2Real trains a web agent against adaptive prompt injection inside a frozen web world model (WebWorld-14B) by co-evolving (1) a task curriculum, (2) an injection adversary, and (3) the executor. The curriculum is rewarded for tasks the executor solves about half the time (judge-graded), and the adversary for success flips only. On 150 web tasks, it increases completion under an unseen frontier adversary (Kimi-K3) by 33.6% relative to the base agent.

Why it matters

The authors address a training/evaluation mismatch: defenses against fixed injections can be bypassed by adaptive attackers, while adversarial training that keeps tasks fixed stops teaching once tasks are solved. By co-evolving tasks and attacks within a frozen world model and using success-flip rewards, AdvSim2Real aims for task-preserving robustness to page-author instructions while maintaining authorized goal completion feasibility.

Method

  • Train three policies (curriculum, injection adversary, executor) inside a frozen web world model; the judge grades each rollout and rewards are defined relative to the current executor.
  • Two-stage co-evolution: Stage 1 updates curriculum and executor so tasks track executor competence; Stage 2 freezes curriculum and alternates updates of adversary and executor on clean runs plus new attacks from earlier and current rounds.
  • Adversary reward uses a success-flip definition: a clean run judged accepted is replayed up to the injection step with an injected notice, and the adversary is credited only when the continuation fails.

Limitation

The authors state that none of the results certify robustness against arbitrary page authors or transfer to browser execution, visual observations, or unseen websites.

Abstract (from arXiv)

Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires. Current defenses fine-tune the agent on injections fixed before training, and attackers that adapt to the trained model bypass them. Adversarial training lets the attacker adapt but keeps the tasks fixed, so a task stops teaching once the agent solves it. We introduce AdvSim2Real, which co-evolves a task curriculum, an injection adversary, and the agent inside a frozen web world model. The curriculum is rewarded for tasks the agent solves about half of the time, and the adversary only for a success flip, an injection that turns a judged success into a failure. Training in the simulator makes a 4B agent both more capable and more robust: its completion rises with and without attacks, holds against a frontier-model adversary it never trained against, and its capability gain carries over to a real browser. On 150 web tasks, AdvSim2Real raises completion under this unseen adversary by 33.6\% relative to the base agent.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC

preprint

AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model

Sarim Hashmi, Mukul Ranjan, Kshitij Mishra, Mikhail Kuznetsov, Praneeth Vepakomma, Nils Lukas

Preprint posted: 2026-10-06 · Original record ↗

arxiv
2610.08773
Explicit source versions (1)
  • v1 · 2026-10-06
  1. Preprint available

    Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires. Current defenses fine-tune the agent on injections fixed before training, and attackers that adapt to the trained model bypass them. Adversarial training lets the attacker adapt but keeps the tasks fixed, so a task stops teaching once the agent solves it. We introduce AdvSim2Real, which co-evolves a task curriculum, an injection adversary, and the agent inside a frozen web world model. The curriculum is rewarded for tasks the agent solves about half of the time, and the adversary only for a success flip, an injection that turns a judged success into a failure. Training in the simulator makes a 4B agent both more capable and more robust: its completion rises with and without attacks, holds against a frontier-model adversary it never trained against, and its capability gain carries over to a real browser. On 150 web tasks, AdvSim2Real raises completion under this unseen adversary by 33.6\% relative to the base agent.