Video World Models · 2026-10-02

Memorizon: Training World Models Beyond Their Context Window

Tingting Liao, Xuezhi Liang, Hao Li, Guangyi Liu

arXiv:2610.00544PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Memorizon trains streaming world models on long revisit supervision without paying for long attention. Each sample spans any length but only its last k chunks are scored; unscored history is not tokenized. For each scored chunk, the model retrieves top-K prior latents by camera co-visibility and shares the union as a bounded memory bank (≤ kK entries). The bank increases step cost once; longer spans then add little cost and improve revisit consistency over sliding windows.

Why it matters

It targets the mismatch between training context windows (often tens of seconds) and deployment horizons (minutes) that break revisit supervision. Memorizon decouples long-span supervision from long-context attention by retrieving co-visible latents into a shared bounded bank, enabling models to learn to keep revisited places consistent even when the first visit lies far outside the attention window.

Method

  • Training samples draw a span length per sample but score only the last k scored chunks; earlier history is never tokenized.
  • Each scored chunk retrieves its own top-K prior latents by camera co-visibility; the union of these requests forms a shared memory bank capped at kK, bounded independently of span length.
  • Attention uses a causal sliding window across chunks plus per-chunk masks so each chunk attends to its bounded recent context and its own retrieved bank entries; rollout reads the same layout one chunk at a time.

Limitation

Longer spans cost some image quality; e.g., the bank levels off and longer than certain span lengths gives no further gain, while quality degrades with span.

Abstract (from arXiv)

Streaming world models should render a place consistently across repeated visits. Directly supervising such revisits requires training samples that capture both visits, often spanning minutes. Yet dense attention over the full span incurs quadratic costs, making long-span supervision expensive. Memorizon breaks this coupling: long spans are needed for supervision, but not for attention, since the two visits can share a forward pass without including every intervening frame. A training sample covers a span of any length but is scored only on its last $k$ chunks. Instead of tokenizing the history before them, each scored chunk retrieves its own top-$K$ latents by camera co-visibility, and the union of these requests forms a shared bank. The bank is bounded by $kK$, so the sequence stays bounded however long the span; at the shortest span the recipe is exactly conventional training. Adding the bank raises the cost of a step once; beyond that, a longer span costs little, and going from 100 to 400 s adds 12% to the step time. Against a sliding-window baseline, retrieval raises revisit consistency on every split, and a span long enough to reach the first visit of each return adds a further 24% to 30%, at some cost in image quality; beyond that span, more length no longer helps. Filling the bank from another episode lowers revisit correlation by 83%, so the model uses what it retrieves. Project page: https://tingtingliao.github.io/memorizon

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-02 UTC

preprint

Memorizon: Training World Models Beyond Their Context Window

Tingting Liao, Xuezhi Liang, Hao Li, Guangyi Liu

Preprint posted: 2026-09-30 · Original record ↗

arxiv
2610.00544
Explicit source versions (1)
  • v1 · 2026-09-30
  1. Preprint available

    Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Streaming world models should render a place consistently across repeated visits. Directly supervising such revisits requires training samples that capture both visits, often spanning minutes. Yet dense attention over the full span incurs quadratic costs, making long-span supervision expensive. Memorizon breaks this coupling: long spans are needed for supervision, but not for attention, since the two visits can share a forward pass without including every intervening frame. A training sample covers a span of any length but is scored only on its last $k$ chunks. Instead of tokenizing the history before them, each scored chunk retrieves its own top-$K$ latents by camera co-visibility, and the union of these requests forms a shared bank. The bank is bounded by $kK$, so the sequence stays bounded however long the span; at the shortest span the recipe is exactly conventional training. Adding the bank raises the cost of a step once; beyond that, a longer span costs little, and going from 100 to 400 s adds 12% to the step time. Against a sliding-window baseline, retrieval raises revisit consistency on every split, and a span long enough to reach the first visit of each return adds a further 24% to 30%, at some cost in image quality; beyond that span, more length no longer helps. Filling the bank from another episode lowers revisit correlation by 83%, so the model uses what it retrieves. Project page: https://tingtingliao.github.io/memorizon