← AI Techniques Radar
Retrieval & memory

KV Cache Compression and Tree-Based Inter-Chunk Management for Multi-Round Video Generation

Use a two-part KV cache system for multi-round, long video generation: (1) compress KV data within each chunk using a channel/video-aware low-rank decomposition whose rank is predicted online to satisfy an energy-retention target; and (2) manage compressed chunks across time with an inter-chunk cache manager that shares basis directions between consecutive chunks and tracks chunk states in a tree so branches can reuse history and later restore for resume/rewind/fork.

Discovered 2026-11-10 · Sources checked 2026-11-10

This summary was generated with AI assistance.

Why it matters

Autoregressive video diffusion-style generation can accumulate very large KV caches across multiple rounds, making memory usage costly and naive cache rebuilding slow. A compression-and-managed reuse design aims to reduce retained KV footprint while keeping multi-branch generation efficient.

What changed

Instead of storing full KV caches for every generated chunk, the approach compresses KV chunks via low-rank decomposition with online rank prediction, then organizes and reuses compressed states across chunks using shared bases and a tree of generation states that supports restoration for branching workflows.

Where it helps

  • Multi-round long-horizon video generation with autoregressive transformers
  • Branching generation workflows requiring resume/rewind/fork of partial outputs
  • Memory-constrained deployments that must keep attention KV history across many steps
  • Systems needing faster next-chunk generation without full KV retention

Evidence & reproduction

Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.

Limitations & open questions

The provided description does not specify failure modes, hardware constraints, or how sensitive the method is to the chosen energy-retention targets; reported results are author claims and depend on the considered video generation setup.

Sources