Video World Models · 2026-10-16
Mallea: Building, Steering, and Memorizing Worlds with Coding Agent
Xuanyi Liu, Ruiqi Li, Siwei Ma
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Mallea is a coding-agent system that builds, steers, and memorizes explicit “code world” representations from text. A coding agent jointly plans object structure, typed properties (fields), and executable rules for interaction dynamics, and preserves unrelated state when edits extend or modify behavior. It stores and reuses appearance only for whitebox-defined objects via object texture memory, while a generative renderer combines current geometry with retrieved textures.
Why it matters
The authors target a key limitation of world models that separate state persistence from visual rendering: Mallea uses shared object identities across construction, runtime intervention, and appearance memory so subsequent observations are conditioned on the latest state and compatible appearance. This enables targeted revisions of both values and capabilities while managing long-horizon visual consistency through adaptively represented objects.
Method
- A coding agent infers objects and interaction requirements from text, then selects compact per-object representations and jointly plans geometry, typed fields (properties), and executable rules; failed local interaction checks trigger targeted refinement.
- At runtime, optional user actions steer the world; text edits can change values or extend definitions by adding new fields and rules. Revisions are validated on a state copy and committed at simulation-step boundaries, preserving unaffected state.
- Object texture memory indexes appearance by stable object identity and current state. It stores textures only for whitebox-defined objects, retrieves compatible regions using visibility/projection and reuse conditions, then generates observations by conditioning rendering on both geometry and stored appearance.
Abstract (from arXiv)
Code world models represent world dynamics through explicit state and executable rules. We investigate how to coordinate object representations, behavioral revisions, and visual memory within this formulation. We present Mallea, a system for building, steering, and memorizing worlds from text. A coding agent infers object interaction requirements from the prompt and jointly plans object structure, properties, and rules, favoring compact representations that support those requirements. The world evolves under the planned rules; optional user actions steer it, while text requests can edit properties or extend capabilities with unaffected state preserved. Object texture memory retains reusable appearance only for whitebox-defined objects and adapts it as those objects change. The program advances world state, while a generative renderer combines current structure with remembered appearance. By organizing construction, intervention, and memory around the same objects, Mallea conditions subsequent observations on the latest state and compatible appearance. Project page: https://lxycopper.github.io/Mallea.
Related papers
- ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling
- WorldDraft: If Code Builds the World, Can Video Models Just Render It?
- WorldFoundry: Towards Unified Infrastructure for World Intelligence
- Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
- Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-15 · Indexed by Pub-AI: 2026-10-16 UTC
Mallea: Building, Steering, and Memorizing Worlds with Coding Agent
Preprint posted: 2026-10-15 · Original record ↗
- arxiv
- 2610.21283
Explicit source versions (1)
- v1 · 2026-10-15
- Preprint available
Source date: 2026-10-15 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Code world models represent world dynamics through explicit state and executable rules. We investigate how to coordinate object representations, behavioral revisions, and visual memory within this formulation. We present Mallea, a system for building, steering, and memorizing worlds from text. A coding agent infers object interaction requirements from the prompt and jointly plans object structure, properties, and rules, favoring compact representations that support those requirements. The world evolves under the planned rules; optional user actions steer it, while text requests can edit properties or extend capabilities with unaffected state preserved. Object texture memory retains reusable appearance only for whitebox-defined objects and adapts it as those objects change. The program advances world state, while a generative renderer combines current structure with remembered appearance. By organizing construction, intervention, and memory around the same objects, Mallea conditions subsequent observations on the latest state and compatible appearance. Project page: https://lxycopper.github.io/Mallea.
