KV Cache Compression and Tree-Based Inter-Chunk Management for Multi-Round Video Generation
Use a two-part KV cache system for multi-round, long video generation: (1) compress KV data within each chunk using a channel/video-aware low-rank decomposition whose rank is predicted online to satisfy an energy-retention target; and (2) manage compressed chunks across time with an inter-chunk cache manager that shares basis directions between consecutive chunks and tracks chunk states in a tree so branches can reuse history and later restore for resume/rewind/fork.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
Autoregressive video diffusion-style generation can accumulate very large KV caches across multiple rounds, making memory usage costly and naive cache rebuilding slow. A compression-and-managed reuse design aims to reduce retained KV footprint while keeping multi-branch generation efficient.
What changed
Instead of storing full KV caches for every generated chunk, the approach compresses KV chunks via low-rank decomposition with online rank prediction, then organizes and reuses compressed states across chunks using shared bases and a tree of generation states that supports restoration for branching workflows.
Where it helps
- Multi-round long-horizon video generation with autoregressive transformers
- Branching generation workflows requiring resume/rewind/fork of partial outputs
- Memory-constrained deployments that must keep attention KV history across many steps
- Systems needing faster next-chunk generation without full KV retention
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
The provided description does not specify failure modes, hardware constraints, or how sensitive the method is to the chosen energy-retention targets; reported results are author claims and depend on the considered video generation setup.
Sources
- ReelCache: An Efficient KV Cache Compression and Management System for Multi-Round Video Generation ↗
arXiv · primary · Checked 2026-11-10
