← AI Techniques Radar
Training & fine-tuning

Stochastic Deep Supervision for Gallery-Consistent Early-Exit Recognition

Train a multi-block vision transformer so that intermediate blocks produce discriminative identity embeddings suitable for similarity-based matching. Each training iteration selects a random intermediate block and applies the same embedding head and classification loss used for the final block, using a weighted margin-penalty softmax auxiliary objective alongside the main final-layer loss.

Discovered 2026-11-10 · Sources checked 2026-11-10

This summary was generated with AI assistance.

Why it matters

It enables early-exit inference where embeddings from early depths are directly comparable to the gallery embeddings produced at full depth, avoiding the need to recompute or adapt the gallery or add special per-exit calibration beyond a threshold.

What changed

Instead of only supervising intermediate layers through standard backpropagation, the method introduces stochastic auxiliary supervision: a randomly chosen intermediate block is explicitly routed through the final embedding head and trained with the auxiliary margin-penalty softmax loss sharing the same classifier parameters.

Where it helps

  • Face recognition with resource-constrained early-exit inference
  • Any similarity-based recognition where early-depth embeddings must match full-depth gallery embeddings
  • Vision transformer models that need usable intermediate representations at multiple depths
  • Deployments that must avoid gallery recomputation or per-exit adaptation layers

Evidence & reproduction

Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.

Limitations & open questions

The effectiveness is tied to using the shared embedding head/classifier design and similarity-based identity matching; results may not transfer unchanged to settings that require different embedding spaces or matching mechanisms.

Sources