SegMoE-style Segmented Mixture-of-Experts Diffusion via Router-Selected Experts
Use a Stable-Diffusion-like diffusion model where selected feed-forward layers and/or attention layers are replaced with a sparse mixture-of-experts (MoE) block. An internal router network selects a subset of expert networks to process each token, while the non-selected parts of the network remain shared with the base model.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
This design allows creating MoE diffusion models by combining multiple pretrained expert models while keeping most parameters shared, and it relies on per-token routing to decide which experts contribute during image generation.
What changed
Compared to a standard Stable Diffusion architecture, only specified components (e.g., feed-forward blocks, attention, or all) are replicated into multiple experts and gated by a router network; the remainder stays consistent with the base Stable Diffusion model.
Where it helps
- Creating diffusion MoE models by combining multiple pretrained checkpoints
- Experimenting with routing different experts for different tokens during generation
- Building image-generation pipelines with selectable MoE layer types (ff, attn, all)
- Deployment settings where multiple GPUs make higher VRAM usage practical
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
The source describes that using more than one expert per token can make inference slower than single-model baselines, and MoE inference can require large VRAM (difficulty for local setups).
Sources
- SegMoE: Segmind Mixture of Diffusion Experts ↗
Hugging Face · primary · Published 2024-02-03 · Checked 2026-11-10
