Few-shot taxonomy adaptation for promptable segmentation via proposal head fitting
Learn a project-specific taxonomy (class boundaries) for a frozen promptable segmenter by fitting a lightweight classification head on top of each image’s generated proposals. The head is trained from the annotator’s labeled examples, then used to classify proposals into the project’s classes, enabling consistent segmentation decisions that match the user-defined taxonomy.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
Many segmentation projects use custom class boundaries that differ from a model’s fixed training vocabulary. This technique adapts those boundaries without retraining the full model by learning them from the labels a user is already producing.
What changed
Instead of relying on the segmenter’s original vocabulary or prompt text alone, the method adds a per-project head (trained with few-shot labeled images) that classifies each proposal according to the project’s taxonomy, and then uses that classification to assign instances.
Where it helps
- Microscopy annotation projects with custom taxonomies
- Defining fine-grained or reordered class boundaries for existing segmenters
- Taxonomy-consistent instance labeling across annotation rounds
- Adapting label semantics when class names change but boundaries should persist
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
The approach assumes access to proposal outputs from the frozen segmenter for each image and that the project’s labeled examples are sufficient to learn the taxonomy boundaries. Reported performance claims also depend on the available segmenter variants mentioned in the source text.
Related World Models papers
Sources
- Adapting Frozen Promptable Segmenters to User-Defined Taxonomies via Few-Shot Proposal Learning ↗
arXiv · primary · Checked 2026-11-10
