Video World Models · 2026-10-05
Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory
Ying Yang, Guiyu Zhang, Lianghua Huang, Chang Nie, Chenyang Si, Haofan Wang, Shaoshuai Shi, Li Jiang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Spatial Memory Intelligence (SMI) is a long-video world-model framework that uses an MLLM to manage long-term spatial memory via four atomic operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering. The authors report comprehensive improvements in memory sparsity, generation stability, and spatial consistency across baselines, benchmarks, and world-model backbones.
Why it matters
The paper targets a core bottleneck in long-horizon video world models: as memory grows, maintaining spatial context and generation stability becomes difficult, and naive retention/retrieval can cause redundancy, geometric inconsistencies, and error propagation. SMI proposes a unified MLLM-driven memory manager that organizes, sparsifies, retrieves, and filters spatial memory through learnable operations, aiming to improve sparsity while preserving spatial consistency and long-horizon stability.
Method
- SMI uses an MLLM to manage spatial memory with four coordinated operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering.
- Spatial clustering groups temporally ordered but spatially disorganized chunks into spatial clusters using camera cues and an MLLM match decision; new chunks create clusters if no match exists.
- Reliability-aware filtering has the MLLM score each newly generated chunk and discard chunks with visual drift (score below a threshold), reducing error accumulation in autoregressive generation.
Abstract (from arXiv)
Long-video generation and world models have shown strong potential for interactive entertainment and embodied simulation by predicting future observations conditioned on user actions and historical memory. However, as memory sequences grow longer and their structures become increasingly complex, managing long-range spatial context becomes increasingly challenging, calling for a more intelligent and systematic memory-management strategy. Building on the advancing spatial reasoning capabilities of multimodal large language models (MLLMs) and the broader vision of unified models, we propose Spatial Memory Intelligence (SMI), the first framework to systematically employ an understanding model for spatial-memory management in long-video world models. SMI introduces four coordinated atomic operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering. Extensive experiments across multiple baselines, benchmarks, and world-model backbones demonstrate the effectiveness and generalizability of SMI, achieving comprehensive improvements in memory sparsity, generation stability, and spatial consistency.
Related papers
- LOCI: Spatial Linear Memory for Streaming World Models
- WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
- Uruqi: Learning Spatial Cognition from Visual Experience
- World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
- Learning to Retrieve: Internalizing Memory Retrieval for Video World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-05 UTC
Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.02521
Explicit source versions (1)
- v2 · 2026-10-08
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Long-video generation and world models have shown strong potential for interactive entertainment and embodied simulation by predicting future observations conditioned on user actions and historical memory. However, as memory sequences grow longer and their structures become increasingly complex, managing long-range spatial context becomes increasingly challenging, calling for a more intelligent and systematic memory-management strategy. Building on the advancing spatial reasoning capabilities of multimodal large language models (MLLMs) and the broader vision of unified models, we propose Spatial Memory Intelligence (SMI), the first framework to systematically employ an understanding model for spatial-memory management in long-video world models. SMI introduces four coordinated atomic operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering. Extensive experiments across multiple baselines, benchmarks, and world-model backbones demonstrate the effectiveness and generalizability of SMI, achieving comprehensive improvements in memory sparsity, generation stability, and spatial consistency.