← AI Techniques Radar
Inference efficiency

Vector-Quantized Discrete Latent Diffusion (VQ-Diffusion style)

Use a diffusion model whose forward noising and reverse denoising operate in a discrete latent token space. Images are first encoded into quantized token indices via a VQ-VAE/VQGAN-style encoder with a fixed codebook, and then a conditional encoder-decoder transformer learns to predict the categorical distribution of the un-noised latent tokens conditioned on a text prompt.

Discovered 2026-11-10 · Sources checked 2026-11-10

This summary was generated with AI assistance.

Why it matters

Working in a discrete latent space reframes denoising as predicting categorical distributions over codebook entries rather than regressing continuous noise. This enables a transformer-based reverse process that can use global attention over the latent token grid while still being trained to output distributions for the fully un-noised latents.

What changed

Compared to continuous diffusion, the noising/denoising process is defined over quantized latent vectors (discrete codebook entries). The forward step allows latent tokens to stay the same, be resampled to another non-masked codebook vector, or become masked, and the reverse model predicts the distribution over un-noised latent tokens in one forward pass.

Where it helps

  • Text-to-image generation using a conditional latent diffusion transformer
  • Discrete sequence modeling over image latent tokens conditioned on prompts
  • Comparative study between discrete diffusion and autoregressive transformer image models
  • Implementations using diffusion pipelines that operate on tokenized latent spaces

Evidence & reproduction

Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.

Limitations & open questions

The approach depends on having a pre-trained quantized latent tokenizer (VQ-VAE/VQGAN); the quantizer is described as frozen during diffusion training. It also relies on the specific discrete forward-process design (stay/resample/mask transitions), which may require careful alignment to the target data and codebook.

Sources

  • VQ-Diffusion ↗

    Hugging Face · primary · Published 2022-11-30 · Checked 2026-11-10