SPMD prompt replication and sharded inputs for JAX/Flax inference
Run the same diffusion pipeline in an SPMD fashion across multiple JAX devices by (1) replicating model parameters to all devices, (2) sharding tokenized prompt inputs so each device processes its own slice, and (3) using JAX parallel primitives (e.g., pmapping the pipeline’s internal generate step) so each device produces separate images in parallel.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
The approach aligns the pipeline’s data layout with JAX’s multi-device execution model, enabling parallel image generation while keeping the pipeline logic largely unchanged (each device runs the same computation on different input data).
What changed
Instead of sending a single prompt to a single device, the workflow prepares prompt token IDs for multiple prompts, replicates pipeline parameters across devices, shards the prompt IDs so each device receives its corresponding batch, and then executes a pmapped (SPMD) generation function (or uses the pipeline call with parallelism enabled).
Where it helps
- Generating multiple images from one prompt across multiple devices
- Generating images for multiple different prompts simultaneously
- Running Flax diffusion inference on TPU-backed JAX environments
- Leveraging sharded batches for multi-device evaluation
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
Inputs and array shapes generally need to be consistent across calls so JAX can reuse compiled computation; otherwise additional recompilation may occur.
Sources
- 🧨 Stable Diffusion in JAX / Flax ! ↗
Hugging Face · primary · Published 2022-10-13 · Checked 2026-11-10
