Video World Models · 2026-10-21
OverSeer: Multi-Agent World Models Beyond What Agents See
Runjia Li, Kam Woh Ng, Minghao Chen, Junlin Han, Jianyuan Wang, Zhaochong An, Juan-Manuel Perez-Rua, Andrea Vedaldi, Shikun Liu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
OverSeer is a multi-agent video world model that augments egocentric agent views with K fixed-pose auxiliary camera views. Fixed in space, auxiliary views act as a proxy for a shared world reference and improve world and agent consistency. Auxiliary streams are efficiently compressed using a δ-VAE that encodes residuals relative to a static reference frame, reducing token overhead. Experiments on Minecraft and Doom show improved consistency and qualitative results on real multi-camera captures.
Why it matters
Multi-agent video world models must generate multiple temporally consistent, cross-view streams. The authors propose a simple output-space design: jointly generate fixed auxiliary views that provide persistent references for cross-agent interactions. They also show that these semi-static auxiliary views can be compressed efficiently with δ-VAE, reducing computational overhead while improving consistency.
Method
- OverSeer jointly predicts future egocentric agent streams and K fixed-pose auxiliary camera streams, using shared attention conditioned on initial agent frames/actions and auxiliary camera poses.
- Auxiliary cameras are stationary so their temporal variation reflects world changes rather than camera ego-motion, improving world and agent consistency.
- A δ-VAE compresses each auxiliary stream by keeping a first-frame latent anchor and encoding subsequent latents as compact residual codes relative to the anchor.
Limitation
OverSeer’s benefits depend on auxiliary coverage: regions far from the sampled cameras (e.g., Reunite Far) remain harder to keep consistent. Aggressive residual compression also trades some fidelity and, in fast-changing scenes such as Doom, consistency for efficiency.
Abstract (from arXiv)
Despite the success of video world models, extending them to multiple agents is difficult because it requires generating several agent-specific video streams that are consistent in time and between agents. To address this gap, we introduce OverSeer, a multi-agent world model that augments egocentric agent views with auxiliary camera views. We suggest that, by fixing the auxiliary views in space, they act as a proxy for a world-centric representation: Unlike the agents' views, which can change entirely due to a simple change in agent position, changes in the auxiliary views directly map to localized changes in the world state. We show that these fixed views substantially help the model to maintain consistency. We also show that, due to their redundant and semi-static nature, the auxiliary views can be efficiently compressed with a delta-VAE that encodes only the changes relative to a static reference image, reducing the number of tokens required to process them. Experiments in Minecraft and Doom demonstrate improved world and agent consistency, with qualitative results on real-world multi-camera captures indicating that the approach is not tied to game engines. Our compression analysis further shows that delta-VAE substantially reduces the computational overhead of auxiliary-view generation. We conclude that these fixed views provide a simple and efficient way to improve multi-agent world models.
Related papers
- LiveAct: Towards Real-Time Multi-Agent World Models with Heterogeneous Action Interfaces
- PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents
- Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update
- WorldCast: Distributed Multiplayer World Models
- Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-20 · Indexed by Pub-AI: 2026-10-21 UTC
OverSeer: Multi-Agent World Models Beyond What Agents See
Preprint posted: 2026-10-20 · Original record ↗
- arxiv
- 2610.28554
Explicit source versions (1)
- v1 · 2026-10-20
- Preprint available
Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Despite the success of video world models, extending them to multiple agents is difficult because it requires generating several agent-specific video streams that are consistent in time and between agents. To address this gap, we introduce OverSeer, a multi-agent world model that augments egocentric agent views with auxiliary camera views. We suggest that, by fixing the auxiliary views in space, they act as a proxy for a world-centric representation: Unlike the agents' views, which can change entirely due to a simple change in agent position, changes in the auxiliary views directly map to localized changes in the world state. We show that these fixed views substantially help the model to maintain consistency. We also show that, due to their redundant and semi-static nature, the auxiliary views can be efficiently compressed with a delta-VAE that encodes only the changes relative to a static reference image, reducing the number of tokens required to process them. Experiments in Minecraft and Doom demonstrate improved world and agent consistency, with qualitative results on real-world multi-camera captures indicating that the approach is not tied to game engines. Our compression analysis further shows that delta-VAE substantially reduces the computational overhead of auxiliary-view generation. We conclude that these fixed views provide a simple and efficient way to improve multi-agent world models.