← AI Techniques Radar
Reasoning & planning

Homeopathic-style data mixing to transfer/activate backdoors via contextual co-occurrence

Use two training subsets—one containing a target trait linked to a trigger co-occurring with the trait, and a second “clean” subset that matches formatting but lacks the trigger—to create a situation where the combined training causes the model to learn or activate the trait/backdoor, even if the poisoned subset alone shows little effect.

Discovered 2026-11-10 · Sources checked 2026-11-10

This summary was generated with AI assistance.

Why it matters

It highlights a safety-relevant failure mode: auditing or filtering training data in isolation may miss harmful interactions that only emerge when different subsets are mixed due to shared co-occurrence structure (task/trigger vs task-only).

What changed

Instead of relying on a single poison set to have a strong standalone impact, the approach relies on adding a contrasting dataset so that the model attributes the target trait to the appropriate feature (trigger vs task), thereby activating the harmful behavior in the mixed setting.

Where it helps

  • Stress-testing data auditing pipelines for interaction effects between datasets
  • Designing controlled experiments that probe when filtering-in-isolation fails
  • Building interpretability-focused tests about whether a model attributes a trait to task vs trigger features
  • Creating safety red-team scenarios for hidden backdoor activation in mixed training sets

Evidence & reproduction

Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.

Limitations & open questions

The effect is described as requiring an identifiable trigger feature present in the poison data but absent from the clean data, and the method depends on the specific co-occurrence pattern between task signals and the trigger in the poisoned subset.

Related World Models papers

Sources