Video World Models · 2026-10-22
WorldStream: Streaming Interactive 3D Worlds with Hierarchical Planning and Persistent State
Xiaoxiao Ma, Weiyang Jin, Yanwen Ma, Haoran Li, Zhixiang Wei, Songchun Zhang, Yaowei Li, Junhao Zhuang, Zeyue Xue, Yuming Li, Zheming Fu, Haoyang Huang, Nan Duan, Feng Zhao
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
WorldStream is a streaming interactive 3D world model with hierarchical planning, deterministic local compilation, and a single persistent world record. Macro planning sets multi-tile settlement/ecology/transport/hydrology; Micro specializes tile semantics under measured terrain and accepted neighbor commitments. Plans run ahead with budgeted admission; compiled geometry is evicted but recompiled on revisit. Persistent state also grounds language-driven actions and edits.
Why it matters
The authors target three interactive world-model requirements simultaneously: expanding beyond pre-specified maps, grounding open-ended language actions in the current world and future regions, and reproducing previously visited places with user edits after geometry eviction. WorldStream’s planner-to-compiler separation and revisioned retained tile records aim to keep identity, seeds, and mutable state consistent across streaming, interaction, and revisit.
Method
- Hierarchical typed decision model: Macro fixes settlement/ecology/transport/hydrology across tile groups; Micro specializes tile semantics using measured terrain and accepted neighbor commitments.
- Deterministic local compilation: accepted plans become compact executable semantic specifications that compile into terrain, roads, settlements, vegetation, and entities using deterministic local code and connected interfaces.
- Streaming with revisioned persistent tile records: planning and compilation have separate horizons/budgets; eviction removes realization while retained records store plans/seeds/identities/object state/user edits for deterministic reconstruction.
Limitation
The current prototype instantiates bounded generation vocabulary, local controllers, and relational edit geometry; reconstruction depends on retained records and compatible generator and asset versions, and long-running worlds require storage policies for accumulated records and simulation policies for dynamic actors outside the active neighborhood.
Abstract (from arXiv)
We present WorldStream, an open-ended 3D world model with three simultaneous capabilities: it extends playable terrain beyond any pre-specified map, acts on open-ended language in the running world, and restores visited places with their edits. A hierarchical planner over a typed decision model emits structured decision state: Macro fixes settlement, ecology, transport, and hydrological structure across a group of tiles; Micro specializes that contract into tile semantics under measured terrain and accepted neighbor commitments. Deterministic local code compiles accepted plans into terrain, roads, settlements, vegetation, and entities. Streaming separates their reach: plans run ahead along the predicted corridor, compilation only where movement will soon need it, geometry behind the user released. One world record retains plans, seeds, object identities and mutable state, boundary commitments, and edits by attribute, revised as the user advances and edits; returning tiles recompile from it. The same record grounds interaction: the model acts on existing entities, including stateful NPCs that update as they act, edits the present region, or attaches intent to unplanned regions. Together they yield a world expanding under bounded budgets, connected across independently planned tiles, and interactive.
Related papers
- World Editing: Intervening on Executable Worlds at Increasing Depth
- ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling
- Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models
- StateCraft: Augmenting Video World Models with Abstract-State Reasoning
- UniState: Unifying Memory and Intervention for Controllable World Simulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-20 · Indexed by Pub-AI: 2026-10-22 UTC
WorldStream: Streaming Interactive 3D Worlds with Hierarchical Planning and Persistent State
Preprint posted: 2026-10-20 · Original record ↗
- arxiv
- 2610.28827
Explicit source versions (1)
- v1 · 2026-10-20
- Preprint available
Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
We present WorldStream, an open-ended 3D world model with three simultaneous capabilities: it extends playable terrain beyond any pre-specified map, acts on open-ended language in the running world, and restores visited places with their edits. A hierarchical planner over a typed decision model emits structured decision state: Macro fixes settlement, ecology, transport, and hydrological structure across a group of tiles; Micro specializes that contract into tile semantics under measured terrain and accepted neighbor commitments. Deterministic local code compiles accepted plans into terrain, roads, settlements, vegetation, and entities. Streaming separates their reach: plans run ahead along the predicted corridor, compilation only where movement will soon need it, geometry behind the user released. One world record retains plans, seeds, object identities and mutable state, boundary commitments, and edits by attribute, revised as the user advances and edits; returning tiles recompile from it. The same record grounds interaction: the model acts on existing entities, including stateful NPCs that update as they act, edits the present region, or attaches intent to unplanned regions. Together they yield a world expanding under bounded budgets, connected across independently planned tiles, and interactive.