Driving · 2026-10-07
Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update
Sitian Shen, Jiuming Liu, Mengmeng Liu, Yian Wang, Michael Ying Yang, Francesco Nex, Hao Cheng, Daniele De Martini, Ayush Tewari, Per Ola Kristensson
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Artemis is a geometry-grounded multi-agent driving world model that reconstructs an explicit shared 3D world map from multi-agent observations, injects action-guided geometric control maps into a diffusion transformer via a GeoAdapter, and progressively updates the shared 3D state during rollouts. It also maintains history for uncontrolled non-agent dynamics and pose memory for controlled agents.
Why it matters
The authors target multi-agent video generation issues from (1) lack of explicit geometric constraints for cross-view consistency, (2) limited cross-agent memory sharing, and (3) over-simplified dynamics/background assumptions. Artemis uses a unified 3D state plus geometry-conditioned diffusion generation and progressive map updates to improve visual fidelity and cross-view consistency, including uncontrolled background motion.
Method
- Reconstructs and maintains an explicit shared sparse point-cloud 3D world map across agents; static geometry is initialized by registering per-agent submaps and updated from keyframes during rollout.
- Builds action-guided geometric control maps by rendering static background, action-controlled foreground proxy controls, and uncontrolled-dynamics controls from historical 3D buffers; injects them into DiT via GeoAdapter.
- Uses progressive memory: keyframe-based static map updates, FIFO history buffer maintenance for uncontrolled dynamics, and structured pose memory with pose-guided verification for controlled-agent proxy initialization.
Limitation
The provided text does not state a specific limitation of Artemis.
Abstract (from arXiv)
Recent video world models have witnessed the paradigm shift from single-agent to multi-agent involvements, which can reveal more complicated dynamics and cross-agent interaction in the real world. However, existing approaches commonly adopt implicit inter-agent communications via cross attention, which lack explicit geometry constraints and unified 3D state, thereby leading to poor multi-view consistency and struggling with recovering out-of-sight agents. In addition, most of them assume a static background, failing to represent uncontrolled background dynamics. To address these problems, we propose Artemis: a geometry-grounded multi-agent world model with explicit memory sharing. An explicit 3D world map is reconstructed from multi-agent observations to enforce a unified 3D state across agents, offering high cross-view consistency. Specifically, an action-guided geometric injection module is developed to simultaneously render decomposed foreground-background control maps, which are then injected into a diffusion transformer through a designed GeoAdapter block. Compared to previous methods assuming static-only background, our GeoAdapter can also distinguish uncontrolled non-agent dynamics, which are conditioned on their own multi-frame history positions to provide consistent motion cues. Keyframes selected from progressive video rollouts are used to progressively update the reconstructed 3D world maps. To effectively capture complex dynamic patterns, we curate a novel dataset sampled from the CARLA simulator called MA-CARLA. Extensive experiments demonstrate the superiority of our proposed method in terms of visual fidelity and cross-view consistency in the generated videos. In addition, our Artemis can support simultaneous multi-modal rollouts with both 2D video and 3D point map maintenance, scale to scenarios beyond two agents and multi-camera setting.
Related papers
- OverSeer: Multi-Agent World Models Beyond What Agents See
- UniState: Unifying Memory and Intervention for Controllable World Simulation
- EmbodiedWorld-200K: Toward Embodied World Agents via Embodied-Planning Dataset and Interactive World Models
- LiveAct: Towards Real-Time Multi-Agent World Models with Heterogeneous Action Interfaces
- World Observer: Joint Actor-Observer Generation for Persistent World Modeling
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-04 · Indexed by Pub-AI: 2026-10-07 UTC
Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update
Preprint posted: 2026-10-04 · Original record ↗
- arxiv
- 2610.07031
Explicit source versions (1)
- v1 · 2026-10-04
- Preprint available
Source date: 2026-10-04 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Recent video world models have witnessed the paradigm shift from single-agent to multi-agent involvements, which can reveal more complicated dynamics and cross-agent interaction in the real world. However, existing approaches commonly adopt implicit inter-agent communications via cross attention, which lack explicit geometry constraints and unified 3D state, thereby leading to poor multi-view consistency and struggling with recovering out-of-sight agents. In addition, most of them assume a static background, failing to represent uncontrolled background dynamics. To address these problems, we propose Artemis: a geometry-grounded multi-agent world model with explicit memory sharing. An explicit 3D world map is reconstructed from multi-agent observations to enforce a unified 3D state across agents, offering high cross-view consistency. Specifically, an action-guided geometric injection module is developed to simultaneously render decomposed foreground-background control maps, which are then injected into a diffusion transformer through a designed GeoAdapter block. Compared to previous methods assuming static-only background, our GeoAdapter can also distinguish uncontrolled non-agent dynamics, which are conditioned on their own multi-frame history positions to provide consistent motion cues. Keyframes selected from progressive video rollouts are used to progressively update the reconstructed 3D world maps. To effectively capture complex dynamic patterns, we curate a novel dataset sampled from the CARLA simulator called MA-CARLA. Extensive experiments demonstrate the superiority of our proposed method in terms of visual fidelity and cross-view consistency in the generated videos. In addition, our Artemis can support simultaneous multi-modal rollouts with both 2D video and 3D point map maintenance, scale to scenarios beyond two agents and multi-camera setting.