Adaptive multi-accelerator offloading with Core ML for diffusion inference
Use a Core ML–based diffusion pipeline that can automatically split computation across available hardware accelerators (CPU, GPU, Neural Engine) and select a best accelerator configuration for the current Mac. The approach is described as converting diffusion models to Core ML and relying on Core ML to decide which model portions run on which devices; the app also includes an automatic accelerator selection heuristic.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
It targets faster, hardware-aware local inference on Apple devices by leveraging the mix of compute units available on the system while keeping the workflow self-contained on-device.
What changed
Instead of running the original PyTorch models on the Mac, the system uses Core ML versions of diffusion models and adds versioned improvements where the app automatically selects the best accelerator configuration and highlights UI features for managing generation settings (e.g., seed reuse and cancelation).
Where it helps
- On-device text-to-image generation on Apple Silicon Macs
- Deploying diffusion models via Core ML for hardware-aware inference
- Building Mac apps that leverage CPU/GPU/Neural Engine scheduling for model execution
- Creating a configurable inference UI for diffusion settings while running locally
- Optimizing inference paths for different Mac accelerator combinations
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
The described automatic selection depends on hardware-specific heuristics, which may not cover all device configurations (the source mentions missing coverage for some variants) and may require additional benchmark data to improve.
Sources
- Swift 🧨Diffusers - Fast Stable Diffusion for Mac ↗
Hugging Face · primary · Published 2023-02-24 · Checked 2026-11-10
