Video World Models · 2026-10-14

4LC: 4D Latent Cache for Efficient Conditioning in Dynamic World Modeling

Jianhui Zhang, Zuyao Lin, Pengxin Xu, Jingdong Wang, Xingyu Chen

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

4LC introduces a 4D latent cache that reuses static and dynamic visual representations for dynamic video world modeling. It stores composite features from multiple visual encoders bound to shared geometry and motion, and provides a spatiotemporal interface to read cached conditions at independently specified times and viewpoints. 4LC-Video uses these cache readouts for multiple video generation tasks using a single checkpoint without task-specific fine-tuning.

Why it matters

The authors argue conditioning becomes inefficient when pipelines require per-request RGB rendering and re-encoding. By caching latent representations for both static and dynamic content and reading them directly for new time/viewpoint requests, 4LC-Video aims to reduce condition-acquisition latency and peak GPU memory while improving spatiotemporal controllability and scene consistency across tasks.

Method

4LC caches visual representations of static and dynamic content in a 4D latent cache bound to shared geometry and motion, avoiding per-request RGB rendering and re-encoding.
A spatiotemporal interface reads the cache directly at independently specified times and viewpoints and combines readouts across observations to cover missing regions.
4LC-Video uses a condition adapter to align 4LC readouts to a pretrained video diffusion model’s latent grid; generated observations are appended to the cache for subsequent segments.

Abstract (from arXiv)

For video generation in dynamic world modeling, observed static and dynamic content should be efficiently reused to provide conditions for repeated generation at different times and viewpoints. We introduce 4LC, a 4D latent cache with composite visual representations and a flexible spatiotemporal access interface. As a latent cache, 4LC directly reuses visual representations of static and dynamic content to provide generation conditions, avoiding per-request RGB rendering and re-encoding. As a composite cache, 4LC binds visual representations from multiple visual encoders (e.g., VAE, DINO, V-JEPA, VGGT) to shared geometry and motion, allowing the composite representations to describe the dynamic world with rich information. Besides, we design a flexible spatiotemporal interface, which directly reads the cache at independently specified times and viewpoints and combines readouts across observations, providing a common conditioning format for diverse downstream tasks. Building on 4LC, we develop 4LC-Video to address multiple video generation tasks through different configurations of source observations, spatiotemporal requests, and historical reuse, including camera control, temporal control, novel-view synthesis, scene revisits, and long-term generation. Our experiments show more efficient condition acquisition, effective spatiotemporal controllability, and scene consistency across tasks using a single 4LC-Video model without task-specific fine-tuning.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC

preprint