Video World Models · 2026-10-16

Mallea: Building, Steering, and Memorizing Worlds with Coding Agent

Xuanyi Liu, Ruiqi Li, Siwei Ma

arXiv:2610.21283PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Mallea is a coding-agent system that builds, steers, and memorizes explicit “code world” representations from text. A coding agent jointly plans object structure, typed properties (fields), and executable rules for interaction dynamics, and preserves unrelated state when edits extend or modify behavior. It stores and reuses appearance only for whitebox-defined objects via object texture memory, while a generative renderer combines current geometry with retrieved textures.

Why it matters

The authors target a key limitation of world models that separate state persistence from visual rendering: Mallea uses shared object identities across construction, runtime intervention, and appearance memory so subsequent observations are conditioned on the latest state and compatible appearance. This enables targeted revisions of both values and capabilities while managing long-horizon visual consistency through adaptively represented objects.

Method

  • A coding agent infers objects and interaction requirements from text, then selects compact per-object representations and jointly plans geometry, typed fields (properties), and executable rules; failed local interaction checks trigger targeted refinement.
  • At runtime, optional user actions steer the world; text edits can change values or extend definitions by adding new fields and rules. Revisions are validated on a state copy and committed at simulation-step boundaries, preserving unaffected state.
  • Object texture memory indexes appearance by stable object identity and current state. It stores textures only for whitebox-defined objects, retrieves compatible regions using visibility/projection and reuse conditions, then generates observations by conditioning rendering on both geometry and stored appearance.
Abstract (from arXiv)

Code world models represent world dynamics through explicit state and executable rules. We investigate how to coordinate object representations, behavioral revisions, and visual memory within this formulation. We present Mallea, a system for building, steering, and memorizing worlds from text. A coding agent infers object interaction requirements from the prompt and jointly plans object structure, properties, and rules, favoring compact representations that support those requirements. The world evolves under the planned rules; optional user actions steer it, while text requests can edit properties or extend capabilities with unaffected state preserved. Object texture memory retains reusable appearance only for whitebox-defined objects and adapts it as those objects change. The program advances world state, while a generative renderer combines current structure with remembered appearance. By organizing construction, intervention, and memory around the same objects, Mallea conditions subsequent observations on the latest state and compatible appearance. Project page: https://lxycopper.github.io/Mallea.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-15 · Indexed by Pub-AI: 2026-10-16 UTC

preprint

Mallea: Building, Steering, and Memorizing Worlds with Coding Agent

Xuanyi Liu, Ruiqi Li, Siwei Ma

Preprint posted: 2026-10-15 · Original record ↗

arxiv
2610.21283
Explicit source versions (1)
  • v1 · 2026-10-15
  1. Preprint available

    Source date: 2026-10-15 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Code world models represent world dynamics through explicit state and executable rules. We investigate how to coordinate object representations, behavioral revisions, and visual memory within this formulation. We present Mallea, a system for building, steering, and memorizing worlds from text. A coding agent infers object interaction requirements from the prompt and jointly plans object structure, properties, and rules, favoring compact representations that support those requirements. The world evolves under the planned rules; optional user actions steer it, while text requests can edit properties or extend capabilities with unaffected state preserved. Object texture memory retains reusable appearance only for whitebox-defined objects and adapts it as those objects change. The program advances world state, while a generative renderer combines current structure with remembered appearance. By organizing construction, intervention, and memory around the same objects, Mallea conditions subsequent observations on the latest state and compatible appearance. Project page: https://lxycopper.github.io/Mallea.