Unassigned · 2026-10-14
Esports World Model Dataset
Weiyu Ma, Zeyuan Gong, Xuhui Liu, Wenxuan Zhang, Jian Ding, Xinyu Cui, Jian Zhao, Mohamed Elhoseiny
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
EWM (Esports World Model Dataset) is a multiplayer, multi-genre dataset built from human gameplay in StarCraft II, Dota 2, and Counter-Strike 2. It provides replay-clock-aligned, player-specific rendered views paired with game-native actions, structured state, and events, supporting research on shared dynamics under different information constraints (fog of war, shared team vision, first-person POV).
Why it matters
The authors report that interactive multiplayer world models must connect each participant’s observations and actions to a shared evolving environment while accounting for differing information due to fog of war, team vision, and viewpoints. EWM is designed to make these differences verifiable via replay-time alignment and (in StarCraft II) player-specific entity records that distinguish live visibility from remembered snapshots.
Method
- Starting from human match replays, the dataset uses Windows game clients to render player views and extract game-native actions, structured state, and events, then associates them by the match replay clock.
- Across games, it preserves native observation rules and action/state representations; modalities are packaged as paired outputs per match timeline.
- For StarCraft II, player-specific entity records distinguish live visible units from remembered snapshots and support cross-view joins via shared engine tags.
Limitation
CS2 clips end shortly after player death, reducing late-round view coverage.
Abstract (from arXiv)
Interactive world models must capture how multiple participants jointly change a shared environment while observing different parts of it. Competitive games make this challenge concrete: players cooperate and compete, while fog of war, team vision and camera placement determine what each can see. Modeling these interactions requires data that connect individual observations and actions to shared state and events. We introduce the Esports World Model Dataset (EWM), a collection of human gameplay spanning StarCraft II, Dota 2 and Counter-Strike 2, with 3,053 games and maps and approximately 4,880 player-view hours, including estimated hours for the ongoing CS2 extension. It includes 4,628 completed StarCraft II views, all ten perspectives for 303 Dota 2 matches, and up to ten views per round in CS2. Across these views, replay-clock alignment connects rendered video with game-native actions, structured state and events from the same match. The three games expose complementary observation conditions: private fog of war, shared team vision and first-person viewpoints. In StarCraft II, player-specific entity records further distinguish live observations from remembered snapshots and link co-observed entities across opposing views. Together with temporal alignment, these records allow researchers to distinguish legitimate information differences from disagreement about shared state. By connecting human decisions, shared dynamics and player-specific information, EWM supports research toward jointly controlled multiplayer world models and interactive environments populated by teammates and opponents learned from human play.
Related papers
- WorldCast: Distributed Multiplayer World Models
- PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents
- ROWBench: Do Video Models Render What the Program Specifies?
- Multics: Generative Multi-Player World Model with Hierarchical Communication
- Kuration SDK: Addressing the Virtual2Real Gap via Data Curation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-14 UTC
Esports World Model Dataset
Preprint posted: 2026-10-12 · Original record ↗
- arxiv
- 2610.16166
Explicit source versions (1)
- v1 · 2026-10-12
- Preprint available
Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Interactive world models must capture how multiple participants jointly change a shared environment while observing different parts of it. Competitive games make this challenge concrete: players cooperate and compete, while fog of war, team vision and camera placement determine what each can see. Modeling these interactions requires data that connect individual observations and actions to shared state and events. We introduce the Esports World Model Dataset (EWM), a collection of human gameplay spanning StarCraft II, Dota 2 and Counter-Strike 2, with 3,053 games and maps and approximately 4,880 player-view hours, including estimated hours for the ongoing CS2 extension. It includes 4,628 completed StarCraft II views, all ten perspectives for 303 Dota 2 matches, and up to ten views per round in CS2. Across these views, replay-clock alignment connects rendered video with game-native actions, structured state and events from the same match. The three games expose complementary observation conditions: private fog of war, shared team vision and first-person viewpoints. In StarCraft II, player-specific entity records further distinguish live observations from remembered snapshots and link co-observed entities across opposing views. Together with temporal alignment, these records allow researchers to distinguish legitimate information differences from disagreement about shared state. By connecting human decisions, shared dynamics and player-specific information, EWM supports research toward jointly controlled multiplayer world models and interactive environments populated by teammates and opponents learned from human play.