Video World Models · 2026-10-09

Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction

Dahyun Chung, Siyoon Jin, Hyunwook Choi, Honggyu An, Junyoung Seo, Hyunsung Kim, Seung Wook Kim, Seungryong Kim

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

ME-World is a multi-agent egocentric world model for synchronized first-person ego-stream generation with fine-grained embodied interaction. It jointly denoises multiple ego streams in one shared token sequence, uses shared action conditioning from all agents’ motions, and grounds generation with shared environment memory (stream-aligned geometric evidence plus cross-stream anchor references). The authors report improvements in shared-world consistency, action control, identity preservation, and video quality on real and synthetic benchmarks.

Why it matters

The paper targets multi-agent embodied settings where each agent’s action and viewpoint must remain consistent across views while jointly updating a single evolving shared world. By coupling joint generation, cross-agent action conditioning, and shared environment memory, the authors aim to reduce cross-view inconsistencies in (i) synchronized actions, (ii) shared environment appearance/layout, and (iii) interaction-induced state updates. They also introduce metrics to evaluate consistency of the evolving shared world beyond directly provided geometric memory.

Method

Joint multi-agent generation: jointly denoise all agents’ latent sequences in a single pretrained video DiT by concatenating tokens across ego streams and using shared self-attention.
Shared action conditioning: guide each agent’s ego stream with all agents’ body and head motions projected into the target view (skeleton maps + Plücker ray maps).
Shared environment memory: aggregate multi-agent observation history into (1) stream-aligned geometric memory via warping and (2) cross-stream anchor memory using shared unwarped reference anchors.

Limitation

The paper does not provide multi-agent shared-world evaluation metrics for real benchmarks beyond S_env and S_update; S_id is reported on the synthetic benchmark only.

Abstract (from arXiv)

Egocentric world models predict first-person observations conditioned on an agent's actions, but most focus on a single agent. Real embodied settings often involve multiple agents that act and interact within a shared environment. Existing multi-agent world models rely on coarse actions like locomotion, camera control, or discrete commands, leaving fine-grained embodied interactions underexplored. We formulate multi-agent egocentric world modeling as synchronized ego-stream generation for multiple agents interacting through fine-grained actions in a shared world. This requires cross-view action consistency, shared-environment consistency, and consistent propagation of interaction-induced state updates. We propose Multi-agent Egocentric World Model (ME-World), which jointly denoises multiple ego streams in a shared token sequence, conditions each stream on all agents' target-view poses, and grounds generation with shared environment memory. We train and evaluate on real and synthetic multi-agent data and introduce shared-world consistency metrics for environment, update, and identity consistency. Experiments show ME-World improves shared-world consistency, action control, identity preservation, and video quality over existing methods.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC

preprint