Video World Models · 2026-10-05
SpectralCache: Accelerating Diffusion-Based World Models via Spectral Feature Caching
Zhendong Mi, Pu Zhao, Ziyu Hu, Xiaodong Yu, Yanzhi Wang, Grace Li Zhang, Shaoyi Huang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SpectralCache is a training-free spectral feature caching method for diffusion-based world models. It uses SVD on DiT intermediate features to show stable singular subspaces and predictable singular-value evolution across denoising steps. It reuses cached subspaces and estimates only low-dimensional singular values via linear extrapolation or scaling, skipping expensive Transformer evaluations while preserving generation quality. On HunyuanWorld-Voyager-13B, it reports 5.22× acceleration and WorldScore 65.90 for static scenes.
Why it matters
Diffusion-based world models require repeated Transformer evaluations during denoising, making interactive generation slow. The authors propose a caching approach grounded in observed spectral properties of world-model features, aiming to reduce inference compute by skipping backbone runs while maintaining quality using spectral extrapolation/scaling.
Method
- Apply SVD to DiT final-layer visual features during diffusion to identify stable leading singular subspaces and predictable singular-value evolution across steps.
- Use training-free spectral caching: in approximation steps, bypass the DiT backbone by reconstructing features from cached singular vectors/residual while estimating dominant singular values via linear extrapolation or scaling.
- Trigger full recomputation when accumulated approximation error for chaotic tokens becomes too large (or the cache interval limit is reached), with additional scaling-based approximation at some triggered refreshes.
Limitation
The authors note that spectral caching may require terminating approximation when accumulated error for chaotic tokens becomes sufficiently large to trigger a refresh computation, using an accumulated-error threshold τ and a maximum cache interval.
Abstract (from arXiv)
Diffusion-based world models enable high-quality interactive environment generation but suffer from substantial inference overhead due to repeated Transformer evaluations during denoising. Existing caching methods mainly exploit temporal redundancy at the feature or token level, leaving the underlying mathematical structure of diffusion features largely unexplored. In this work, we reveal that world-model features exhibit highly stable singular subspaces across nearby denoising steps, while their singular values follow predictable evolution patterns. Building on this observation, we propose SpectralCache, a training-free spectral caching framework that reuses stable singular subspaces and estimates only low-dimensional singular values through linear extrapolation. We further exploit the spectral consistency between neighboring full-computation features to skip selected expensive backbone evaluations via singular value scaling. Extensive experiments on representative world models demonstrate that SpectralCache consistently improves inference efficiency while preserving generation quality. On HunyuanWorld-Voyager-13B, SpectralCache achieves 5.22x acceleration while maintaining a WorldScore of 65.90 for static scenes, substantially outperforming existing training-free caching methods in inference efficiency.
Related papers
- CtrlCache: Accelerating Interactive Video World Models with Control-Aware Caching
- World Modeling through Spectral Alignment
- 4LC: 4D Latent Cache for Efficient Conditioning in Dynamic World Modeling
- IntactWorld: Joint World Modeling with Intact Features
- DSReg: Provably Recovering Individual World Latents without Reconstruction
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-02 · Indexed by Pub-AI: 2026-10-05 UTC
SpectralCache: Accelerating Diffusion-Based World Models via Spectral Feature Caching
Preprint posted: 2026-10-02 · Original record ↗
- arxiv
- 2610.02660
Explicit source versions (1)
- v1 · 2026-10-02
- Preprint available
Source date: 2026-10-02 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Diffusion-based world models enable high-quality interactive environment generation but suffer from substantial inference overhead due to repeated Transformer evaluations during denoising. Existing caching methods mainly exploit temporal redundancy at the feature or token level, leaving the underlying mathematical structure of diffusion features largely unexplored. In this work, we reveal that world-model features exhibit highly stable singular subspaces across nearby denoising steps, while their singular values follow predictable evolution patterns. Building on this observation, we propose SpectralCache, a training-free spectral caching framework that reuses stable singular subspaces and estimates only low-dimensional singular values through linear extrapolation. We further exploit the spectral consistency between neighboring full-computation features to skip selected expensive backbone evaluations via singular value scaling. Extensive experiments on representative world models demonstrate that SpectralCache consistently improves inference efficiency while preserving generation quality. On HunyuanWorld-Voyager-13B, SpectralCache achieves 5.22x acceleration while maintaining a WorldScore of 65.90 for static scenes, substantially outperforming existing training-free caching methods in inference efficiency.
