Mixed-Bit Palettization for Layer-Wise Post-Training Quantization
Mixed-bit palettization is a post-training model compression approach that assigns different quantization bit depths per layer (e.g., 1/2/4/8 bits, with some layers kept at higher precision) based on each layer’s estimated contribution to quality degradation, producing a JSON “recipe” that is later applied to replace layer weights with palettized versions.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
For large generative models, using a single low bit-width across all layers can harm quality, while per-layer mixed precision can better balance model size against degradation, enabling smaller Core ML versions of diffusion pipelines for local execution scenarios described in the source.
What changed
Instead of applying uniform 6-bit or 8-bit quantization everywhere, the method analyzes layers to choose the lowest bit depth per layer that stays above a chosen quality threshold, creating a layer-wise recipe that is then applied during compression.
Where it helps
- Compressing Stable Diffusion/Stable Diffusion XL Core ML UNet components
- Creating size–quality tradeoff recipes for layer-wise quantization
- Applying pre-computed mixed-bit recipes to popular diffusion model variants
- Converting and then optionally compressing diffusion models for local inference
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
The analysis phase is described as lengthy and requiring GPU hardware (via mps or cuda) because it runs multiple inference trials; the refiner stage is also stated as not yet ported for the specific SDXL Core ML pipeline context mentioned.
Sources
- Stable Diffusion XL on Mac with Advanced Core ML Quantization ↗
Hugging Face · primary · Published 2023-07-27 · Checked 2026-11-10
