Video World Models · 2026-11-13
CoCode-World: Shared 3D Proxy Representation for Multi-Player World Models
Qirui Li, Xin Zhang, Yabo Chen, Jinkun Hao, Paul L Rosin, Yu-Kun Lai, Haibin Huang, Yin Chen, Ran Yi, Chi Zhang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
CoCode-World is a shared 3D proxy framework for joint two-player video generation. It uses programmable, textureless 3D proxies for layout/motion/occlusion and an RGB appearance reference for characters and scene style. The two players’ proxy videos and appearance evidence are concatenated into one context, and a pretrained video model is adapted with LoRA so native attention enables cross-view communication and consistent player identity.
Why it matters
Multiplayer video world models must generate views that agree on shared entities despite different cameras and occlusions. CoCode-World provides a spatial interface via shared 3D proxies and coordinates appearance across views by generating both views jointly in a single branch, using native attention for cross-view appearance correspondence.
Method
- Build paired textureless 3D proxies from shared GTA V map geometry, per-player entity observations, and camera poses; supervise both proxy adherence and cross-view appearance of shared entities.
- Joint generation: concatenate two players along image width in a shared context and use the pretrained MiniMax-H3 reference interface so native attention enables cross-view communication during denoising.
- Freeze the pretrained backbone and train LoRA adapters on paired observations to learn proxy-to-video correspondence and appearance consistency across simultaneous views.
Limitation
Our current implementation is limited to two-player generation; scaling to more players increases token sequence length and hence attention computation and memory requirements. The authors also state their current renderer uses bidirectional attention with 50 denoising steps, and aggressive distillation to one or two steps makes preserving generation quality challenging.
Abstract (from arXiv)
Multiplayer generative world models must produce observations that agree on shared entities despite different camera motions and occlusions. Explicit world representations provide spatial control, but precise modeling of individual game maps limits scene diversity, while shared geometry alone cannot ensure consistent appearance across views. We present CoCode-World, a shared 3D proxy framework that separates structural control from visual synthesis for two-player video generation. Programmable proxies specify scene layout, motion, and occlusion, while RGB references guide character and scene appearance. To coordinate visual details across players, we concatenate paired proxy videos, appearance references, and target views in a consistent spatial layout and generate both views jointly with a pretrained video model. This formulation uses the model's native attention for cross-view communication within a single generation branch, with LoRA adaptation learning proxy adherence and appearance correspondence from paired observations. We construct a two-player GTA~V dataset that pairs synchronized RGB observations with proxy videos derived from shared map geometry and game-engine state. Experiments demonstrate appearance transfer to characters and scene styles unseen during training, with improvements in video quality and cross-view player identity over the evaluated baseline. Ablations further examine how proxy construction and cross-view communication affect generation. Project page: https://iapple233.github.io/CCW/.
Related papers
- WorldCast: Distributed Multiplayer World Models
- PlayGen2: Learning Real-Time Playable Worlds with Explicit 3D State
- PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents
- ActionRoPE: action as coordinates for spatially consistent control in planar-camera game world models
- Wizard: Manipulate Any World in Real Time via Agentic Control
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-13 · Indexed by Pub-AI: 2026-11-13 UTC
CoCode-World: Shared 3D Proxy Representation for Multi-Player World Models
First announced by arXiv: 2026-11-13 · Original record ↗
- arxiv
- 2611.15602
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-13 · First detected: 2026-11-13 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.15602v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Multiplayer generative world models must produce observations that agree on shared entities despite different camera motions and occlusions. Explicit world representations provide spatial control, but precise modeling of individual game maps limits scene diversity, while shared geometry alone cannot ensure consistent appearance across views. We present CoCode-World, a shared 3D proxy framework that separates structural control from visual synthesis for two-player video generation. Programmable proxies specify scene layout, motion, and occlusion, while RGB references guide character and scene appearance. To coordinate visual details across players, we concatenate paired proxy videos, appearance references, and target views in a consistent spatial layout and generate both views jointly with a pretrained video model. This formulation uses the model's native attention for cross-view communication within a single generation branch, with LoRA adaptation learning proxy adherence and appearance correspondence from paired observations. We construct a two-player GTA~V dataset that pairs synchronized RGB observations with proxy videos derived from shared map geometry and game-engine state. Experiments demonstrate appearance transfer to characters and scene styles unseen during training, with improvements in video quality and cross-view player identity over the evaluated baseline. Ablations further examine how proxy construction and cross-view communication affect generation. Project page: https://iapple233.github.io/CCW/.
