Driving · 2026-10-16
SHRED: Self-Play for Heterogeneous Road-Users without Expert Demonstrations
Wei-Jer Chang, Zhouchonghao Wu, Akshay Rangesh, Weixin Li, Masayoshi Tomizuka, Wei Zhan
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SHRED is a demonstration-free heterogeneous self-play framework that jointly trains vehicles, pedestrians, and cyclists from random initialization for closed-loop traffic simulation and ego planning. It uses type-specific dynamics, rewards, and discrete action spaces, plus procedural scenario generation and per-agent randomization (including procedural goal regeneration and goal dropout). On WOSAC, SHRED outperforms a demonstration-free self-play baseline and achieves strong zero-shot realism; on InterPlan, it outperforms compared planners.
Why it matters
The authors target realistic multi-agent traffic where road users have fundamentally different dynamics and behaviors. They show that heterogeneous interaction behavior can emerge purely through self-play (no demonstrations, no logged-trajectory replay) while also improving interactive planning in interaction-heavy scenarios.
Method
- Joint heterogeneous self-play: decentralized policies for vehicles, pedestrians, and cyclists with shared encoders and per-type LSTM/actor/value components.
- Type-specific continuous-control dynamics and discretized action spaces (vehicles: jerk-integrated Ackermann; pedestrians: unicycle with stop; cyclists: kinematic bicycle).
- Procedural heterogeneous training: randomized initialization, procedural map-conditioned goals regenerated after completion, per-agent domain randomization of dynamics/reward parameters, and goal dropout.
Abstract (from arXiv)
Realistic traffic simulation and autonomous planning both depend on modeling interactions among heterogeneous road users whose dynamics, constraints, and behaviors differ fundamentally. Existing approaches predominantly learn realistic interaction behavior from large-scale human driving demonstrations. In this work, we ask whether such behavior can instead emerge purely through self-play. We present SHRED, a framework that jointly trains vehicles, pedestrians, and cyclists through large-scale heterogeneous self-play from random initialization, without demonstrations or logged-trajectory replay. SHRED combines type-specific dynamics and objectives with procedural scenario generation and per-agent randomization, allowing diverse behaviors to emerge through closed-loop interaction. On the Waymo Open Sim Agents Challenge (WOSAC), SHRED jointly controls all three road-user types and outperforms both the closest demonstration-free self-play baseline and prior self-play methods that use demonstration-trained reference policies. Policies trained only on nuPlan map geometry retain comparable realism under zero-shot evaluation on Waymo. For planning, SHRED outperforms all compared planners on the interaction-heavy InterPlan benchmark. Controlled ablations show that heterogeneity in self-play improves both traffic realism and planning performance, with particularly large planning gains in interaction-heavy scenarios. Together, these results establish heterogeneous procedural self-play as a scalable, demonstration-free paradigm for learning realistic and interactive traffic behavior.
Related papers
- VehicleArena: A Realistic Urban Environment for Multi-Agent Driving
- TerraTransfer++: Demonstration-Free End-to-End Planner
- RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts
- DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
- EvoAD: Self-Evolving Autonomous Driving via Latent World-Model Guided Post-hoc Reflection
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-16 UTC
SHRED: Self-Play for Heterogeneous Road-Users without Expert Demonstrations
Preprint posted: 2026-10-14 · Original record ↗
- arxiv
- 2610.20866
Explicit source versions (1)
- v1 · 2026-10-14
- Preprint available
Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Realistic traffic simulation and autonomous planning both depend on modeling interactions among heterogeneous road users whose dynamics, constraints, and behaviors differ fundamentally. Existing approaches predominantly learn realistic interaction behavior from large-scale human driving demonstrations. In this work, we ask whether such behavior can instead emerge purely through self-play. We present SHRED, a framework that jointly trains vehicles, pedestrians, and cyclists through large-scale heterogeneous self-play from random initialization, without demonstrations or logged-trajectory replay. SHRED combines type-specific dynamics and objectives with procedural scenario generation and per-agent randomization, allowing diverse behaviors to emerge through closed-loop interaction. On the Waymo Open Sim Agents Challenge (WOSAC), SHRED jointly controls all three road-user types and outperforms both the closest demonstration-free self-play baseline and prior self-play methods that use demonstration-trained reference policies. Policies trained only on nuPlan map geometry retain comparable realism under zero-shot evaluation on Waymo. For planning, SHRED outperforms all compared planners on the interaction-heavy InterPlan benchmark. Controlled ablations show that heterogeneity in self-play improves both traffic realism and planning performance, with particularly large planning gains in interaction-heavy scenarios. Together, these results establish heterogeneous procedural self-play as a scalable, demonstration-free paradigm for learning realistic and interactive traffic behavior.