Video World Models · 2026-10-07

CtrlCache: Accelerating Interactive Video World Models with Control-Aware Caching

Shangye Song, Dong Gong, Hong Jia, Yun Sing Koh, Xinyu Zhang

arXiv:2610.08777PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

CtrlCache is a training-free, control-aware caching method for interactive video world models with chunk-wise autoregressive few-step denoising. Using incoming controls, it assigns each chunk a state (initial/transition/turning/steady) and selectively reuses a cached transformer residual at one interior denoising step. For steady chunks, it also applies a frequency-mixed history prior guidance blended into the first clean latent estimate.

Why it matters

It targets the inference bottleneck of interactive video world models: each chunk still needs multiple costly DiT denoising forward passes. CtrlCache leverages an exposed signal in interactive generation (controls arrive before denoising) to decide when to refresh vs reuse computation, and adds a guidance scheme to carry over structure safely during steady interaction.

Method

  • Action-aware scheduling/refresh: assign each chunk to initial, transition, turning, or steady using incoming controls (detecting action changes across and within chunks), then enable residual reuse at one selected interior denoising step (with full computation for initial/transition).
  • Residual reuse mechanism: at the selected step, reuse the cached transformer residual from the most recent fully computed step; input projection/timestep embedding/output head are still computed while transformer-block computation is approximated.
  • Frequency-mixed history prior guidance (steady only): build a prior from the preceding chunk’s last clean latent by separating low/high-frequency components and blending the prior into the current chunk’s clean latent estimate at s=1 without additional DiT forward passes.

Limitation

The frequency-mixed history prior guidance is enabled only when Cn=steady, since during transitions and sustained turning, the inherited geometry may become outdated as the viewpoint changes.

Abstract (from arXiv)

Interactive video world models need to generate each video chunk efficiently while responding faithfully to user controls. Many systems use chunk-wise autoregressive generation with few-step denoising, but each chunk still requires several costly denoising iterations. Training-free caching can reduce this cost, yet existing policies make reuse decisions primarily from model-internal denoising dynamics and do not explicitly account for control transitions. Actually, interactive generation explicitly exposes a signal they do not use: the controls for a chunk arrive before it is denoised, so a schedule derived from them costs no forward pass. To this end, we analyze adjacent chunks under different control regimes and find that structural similarity drops around action changes, while low-frequency structure remains more persistent than high-frequency detail. Motivated by these observations, we propose CtrlCache, a training-free control-aware caching framework that adapts computation to the current control sequence. Specifically, the action-aware scheduling and refresh policy detects action changes across and within chunks, and labels each chunk as initial, transition, turning, or steady state. At one selected interior denoising step, initial and transition chunks retain full computation, while turning and steady chunks reuse the transformer residual from the most recent fully computed step in the same chunk. To exploit the persistence of low-frequency structure during steady interaction, we further introduce a frequency-mixed history prior guidance that incorporates complementary information from the preceding clean latent without an additional DiT forward pass. Evaluated on Matrix-Game 2.0 and LingBot-World v1/v2, CtrlCache achieves 1.21x to 1.41x DiT-backbone speedups without model retraining while improving WBench Overall scores over original inference across all three models.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC

preprint

CtrlCache: Accelerating Interactive Video World Models with Control-Aware Caching

Shangye Song, Dong Gong, Hong Jia, Yun Sing Koh, Xinyu Zhang

Preprint posted: 2026-10-06 · Original record ↗

arxiv
2610.08777
Explicit source versions (1)
  • v1 · 2026-10-06
  1. Preprint available

    Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Interactive video world models need to generate each video chunk efficiently while responding faithfully to user controls. Many systems use chunk-wise autoregressive generation with few-step denoising, but each chunk still requires several costly denoising iterations. Training-free caching can reduce this cost, yet existing policies make reuse decisions primarily from model-internal denoising dynamics and do not explicitly account for control transitions. Actually, interactive generation explicitly exposes a signal they do not use: the controls for a chunk arrive before it is denoised, so a schedule derived from them costs no forward pass. To this end, we analyze adjacent chunks under different control regimes and find that structural similarity drops around action changes, while low-frequency structure remains more persistent than high-frequency detail. Motivated by these observations, we propose CtrlCache, a training-free control-aware caching framework that adapts computation to the current control sequence. Specifically, the action-aware scheduling and refresh policy detects action changes across and within chunks, and labels each chunk as initial, transition, turning, or steady state. At one selected interior denoising step, initial and transition chunks retain full computation, while turning and steady chunks reuse the transformer residual from the most recent fully computed step in the same chunk. To exploit the persistence of low-frequency structure during steady interaction, we further introduce a frequency-mixed history prior guidance that incorporates complementary information from the preceding clean latent without an additional DiT forward pass. Evaluated on Matrix-Game 2.0 and LingBot-World v1/v2, CtrlCache achieves 1.21x to 1.41x DiT-backbone speedups without model retraining while improving WBench Overall scores over original inference across all three models.