Driving · 2026-10-07
AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model
Sarim Hashmi, Mukul Ranjan, Kshitij Mishra, Mikhail Kuznetsov, Praneeth Vepakomma, Nils Lukas
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
AdvSim2Real trains a web agent against adaptive prompt injection inside a frozen web world model (WebWorld-14B) by co-evolving (1) a task curriculum, (2) an injection adversary, and (3) the executor. The curriculum is rewarded for tasks the executor solves about half the time (judge-graded), and the adversary for success flips only. On 150 web tasks, it increases completion under an unseen frontier adversary (Kimi-K3) by 33.6% relative to the base agent.
Why it matters
The authors address a training/evaluation mismatch: defenses against fixed injections can be bypassed by adaptive attackers, while adversarial training that keeps tasks fixed stops teaching once tasks are solved. By co-evolving tasks and attacks within a frozen world model and using success-flip rewards, AdvSim2Real aims for task-preserving robustness to page-author instructions while maintaining authorized goal completion feasibility.
Method
- Train three policies (curriculum, injection adversary, executor) inside a frozen web world model; the judge grades each rollout and rewards are defined relative to the current executor.
- Two-stage co-evolution: Stage 1 updates curriculum and executor so tasks track executor competence; Stage 2 freezes curriculum and alternates updates of adversary and executor on clean runs plus new attacks from earlier and current rounds.
- Adversary reward uses a success-flip definition: a clean run judged accepted is replayed up to the injection step with an injected notice, and the adversary is credited only when the continuation fails.
Limitation
The authors state that none of the results certify robustness against arbitrary page authors or transfer to browser execution, visual observations, or unseen websites.
Abstract (from arXiv)
Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires. Current defenses fine-tune the agent on injections fixed before training, and attackers that adapt to the trained model bypass them. Adversarial training lets the attacker adapt but keeps the tasks fixed, so a task stops teaching once the agent solves it. We introduce AdvSim2Real, which co-evolves a task curriculum, an injection adversary, and the agent inside a frozen web world model. The curriculum is rewarded for tasks the agent solves about half of the time, and the adversary only for a success flip, an injection that turns a judged success into a failure. Training in the simulator makes a 4B agent both more capable and more robust: its completion rises with and without attacks, holds against a frontier-model adversary it never trained against, and its capability gain carries over to a real browser. On 150 web tasks, AdvSim2Real raises completion under this unseen adversary by 33.6\% relative to the base agent.
Related papers
- Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
- Arctic-SWE: Revisiting the Long Horizon Agent Training by Modeling-System Co-Design
- FOE-Dreamer: Deployment-Efficient Learning of Cyber Defense Policies in Operational Networks
- Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition
- Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC
AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model
Preprint posted: 2026-10-06 · Original record ↗
- arxiv
- 2610.08773
Explicit source versions (1)
- v1 · 2026-10-06
- Preprint available
Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires. Current defenses fine-tune the agent on injections fixed before training, and attackers that adapt to the trained model bypass them. Adversarial training lets the attacker adapt but keeps the tasks fixed, so a task stops teaching once the agent solves it. We introduce AdvSim2Real, which co-evolves a task curriculum, an injection adversary, and the agent inside a frozen web world model. The curriculum is rewarded for tasks the agent solves about half of the time, and the adversary only for a success flip, an injection that turns a judged success into a failure. Training in the simulator makes a 4B agent both more capable and more robust: its completion rises with and without attacks, holds against a frontier-model adversary it never trained against, and its capability gain carries over to a real browser. On 150 web tasks, AdvSim2Real raises completion under this unseen adversary by 33.6\% relative to the base agent.
