Video World Models · 2026-10-30
GROME: GROunding Factorized MEmory into Video World Models
Fang Yu, Xiaoxue Wu, Haiyu Zhang, Yangfu Li, Botong Zhao, Yu Qiao, Tianfan Xue, Yue Lu, Xinyuan Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
GROME augments pretrained action-conditioned video world models with factorized external memory for long-horizon consistency. It decouples static scene evidence from dynamic entity evidence, stores both as local pointmaps, retrieves them with different criteria (max coverage vs latest-by-recency), and injects them into a frozen backbone via training-free memory injection (value adaptation, disentangled attention, timestep-aware routing).
Why it matters
Maintaining long-horizon visual and temporal consistency is difficult once observed content leaves the model’s finite context window. The authors propose a factorized memory that preserves persistent static geometry while allowing dynamic entities to evolve, aiming to reduce drift and stale-dynamics failures without retraining the video world model backbone.
Method
- Static-Dynamic Factorization: per-chunk 3D lifting with motion-aware segmentation decomposes pointmaps into static and dynamic components, stored as local pointmaps to avoid global geometric drift.
- Local Memory Retrieval: under bounded context, retrieve static memories via max-coverage relative to target views; retrieve dynamic memory using only the latest dynamic evidence; use spatiotemporal RoPE with static tied to target temporal indices and dynamic preserving original temporal indices.
- Training-Free Memory Injection: align rendered reference value statistics (Reference Value Adaptation), inject static/dynamic through separate attention branches (Static-Dynamic Disentangled Attention), and route influence across diffusion timesteps (Timestep-Aware Memory Routing).
Limitation
As a training-free framework, GROME remains bounded by both the generative capability of the frozen backbone and the quality of external geometric estimation; it cannot eliminate backbone-intrinsic degradation (appearance/color drift) at very long horizons, and severe geometric failures can lead the frozen model to interpret artifacts as plausible structures.
Abstract (from arXiv)
Long-horizon consistency is essential for video world models, yet it becomes difficult to maintain once observed content falls outside the context window. Existing memory mechanisms either retrieve history as implicit context, which remains constrained by context capacity and entangles static and dynamic information or consolidate history into explicit global representations, which struggle to model evolving dynamics and accumulate geometric errors. To overcome these limitations, we introduce \textbf{GROME}, a novel factorized memory framework that augments video world models for long-horizon coherence across both static scenes and dynamic entities. GROME operates via three synergistic components: 1)~\emph{Static-Dynamic Factorization} decouples historical evidence into static and dynamic components, preserving static scene geometry and dynamic entity states separately; 2)~\emph{Local Memory Retrieval} selects spatially aligned static history to eliminate geometric drift alongside the latest dynamic entity states to ensure temporal continuity; 3)~\emph{Training-Free Memory Injection} seamlessly grounds the factorized memory into video world models via their native conditioning pathways. Experiments on long-horizon revisit scenarios demonstrate that GROME substantially enhances scene persistence and dynamic entity fidelity over existing methods, all while preserving the visual quality and controllability of the frozen backbone without additional training. Beyond self-rollouts, GROME readily accommodates external observations as plug-and-play memory, unlocking flexible and controllable video continuation and scene exploration.
Related papers
- WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
- Learning to Retrieve: Internalizing Memory Retrieval for Video World Models
- LOCI: Spatial Linear Memory for Streaming World Models
- Honeycomb: Constant-Size Scene Memory Representation for Video World Models
- Momentum: Persistent Dynamics for Video World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-10-30 · Indexed by Pub-AI: 2026-10-30 UTC
GROME: GROunding Factorized MEmory into Video World Models
First announced by arXiv: 2026-10-30 · Original record ↗
- arxiv
- 2610.39812
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-10-30 · First detected: 2026-10-30 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.39812v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Long-horizon consistency is essential for video world models, yet it becomes difficult to maintain once observed content falls outside the context window. Existing memory mechanisms either retrieve history as implicit context, which remains constrained by context capacity and entangles static and dynamic information or consolidate history into explicit global representations, which struggle to model evolving dynamics and accumulate geometric errors. To overcome these limitations, we introduce \textbf{GROME}, a novel factorized memory framework that augments video world models for long-horizon coherence across both static scenes and dynamic entities. GROME operates via three synergistic components: 1)~\emph{Static-Dynamic Factorization} decouples historical evidence into static and dynamic components, preserving static scene geometry and dynamic entity states separately; 2)~\emph{Local Memory Retrieval} selects spatially aligned static history to eliminate geometric drift alongside the latest dynamic entity states to ensure temporal continuity; 3)~\emph{Training-Free Memory Injection} seamlessly grounds the factorized memory into video world models via their native conditioning pathways. Experiments on long-horizon revisit scenarios demonstrate that GROME substantially enhances scene persistence and dynamic entity fidelity over existing methods, all while preserving the visual quality and controllability of the frozen backbone without additional training. Beyond self-rollouts, GROME readily accommodates external observations as plug-and-play memory, unlocking flexible and controllable video continuation and scene exploration.