Methods worth understanding

AI Techniques Radar

New AI methods, the evidence behind them, and how to put them to work.

Latest weekly report

AI Techniques Weekly

Methods worth understanding, their results, and practical limitations.

Read the report
Reset

Latest discoveries · 115

Last refreshed 2026-11-10 · Some sources need attention

Training & fine-tuning

Adapt a pretrained text-to-image diffusion model to a specific language via two-stage fine-tuning: first train a language-specific text encoder with the image generator fixed, then jointly fine-tune both the text encoder and latent diffusion model so the generator produces language-appropriate visuals from prompts.

AI summary

Inference efficiency

Run the same diffusion pipeline in an SPMD fashion across multiple JAX devices by (1) replicating model parameters to all devices, (2) sharding tokenized prompt inputs so each device processes its own slice, and (3) using JAX parallel primitives (e.g., pmapping the pipeline’s internal generate step) so each device produces separate images in parallel.

AI summary

Training & fine-tuning

When fine-tuning Stable Diffusion with Dreambooth, add a prior-preservation step that includes images from the target’s broader class (e.g., a generic “person” class when training a new face). During fine-tuning, these class images are intended to counterbalance the new concept’s tendency to overfit.

AI summary

Robotics & embodied AI

Back to Articles Diffusion Models Live Event Published November 25, 2022 Update on GitHub Upvote 4 Lewis Tunstall

Source excerpt

Inference efficiency

Use a diffusion model whose forward noising and reverse denoising operate in a discrete latent token space. Images are first encoded into quantized token indices via a VQ-VAE/VQGAN-style encoder with a fixed codebook, and then a conditional encoder-decoder transformer learns to predict the categorical distribution of the un-noised latent tokens conditioned on a text prompt.

AI summary

Agents & tool use

Use a pure-Elixir implementation of Hugging Face Transformers (Bumblebee) to integrate pre-trained transformer models into an Elixir application, leveraging the surrounding tooling (e.g., Livebook smart cells) to scaffold common ML workflows.

AI summary

Training & fine-tuning

Train a reward (preference) model on human-labeled rankings of generated text, then fine-tune a pretrained language model with reinforcement learning using Proximal Policy Optimization (PPO). The RL objective uses the reward model’s output combined with a constraint term that penalizes divergence from the original model via a KL-based penalty, to avoid destabilizing generation and reward-model exploitation.

AI summary

Training & fine-tuning

Instruction Fine-Tuning adapts a base language model by fine-tuning it on demonstrations that pair user instructions with corresponding inputs (optional) and desired outputs, using a diverse set of instruction-following tasks to teach helpful compliance with user-specified directions.

AI summary

Training & fine-tuning

Freeze the pretrained Stable Diffusion weights and train small low-rank adapter matrices (LoRA) inserted into the model’s cross-attention layers that connect image features to prompt text representations. After training, load the LoRA weights on top of the original base pipeline weights for inference.

AI summary

Inference efficiency

Use a Core ML–based diffusion pipeline that can automatically split computation across available hardware accelerators (CPU, GPU, Neural Engine) and select a best accelerator configuration for the current Mac. The approach is described as converting diffusion models to Core ML and relying on Core ML to decide which model portions run on which devices; the app also includes an automatic accelerator selection heuristic.

AI summary