JEPA · 2026-11-05

Drift-adaptive Representation Preservation for Domain-Incremental Learning

Hyunmin Kong, Gyutae Oh, Jitae Shin

arXiv:2611.06093PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors address domain-incremental learning forgetting by tracking per domain–class representation drift from a fixed class anchor. Drift is measured using a repurposed JEPA context-to-target predictor without storing past images (store compact context summaries in backbone feature space). Drift-adaptive consolidation and drift-weighted robust prototype construction allocate preservation by measured drift, improving accuracy–forgetting balance across DomainNet, Office-Home, CORe50, and CDDB-Hard with low training memory and smallest storage among compared methods.

Why it matters

Forgetting in DIL is heterogeneous across domains and classes. The paper proposes a measurable, domain–class-level surrogate (representation drift) for forgetting risk and uses it for two mechanisms: stronger consolidation for high-drift concepts and down-weighting high-drift domains when building class prototypes to prevent prototype contamination, while avoiding storing raw past data.

Method

  • Define representation drift for each past domain–class pair as the cosine distance between a reconstructed target (from stored context summary using JEPA-style context-to-target prediction) and a fixed per-class anchor.
  • Use drift-adaptive consolidation: weight preservation pressure per domain–class pair by drift magnitude (parameter γ).
  • Use robust prototype construction: down-weight per-domain contributions to class prototypes by measured drift to reduce prototype contamination, using reliability exponent ρ and clip bound κ.

Limitation

The provided text does not state explicit limitations.

Abstract (from arXiv)

Pretrained vision models provide strong general-purpose representations, but in Domain-Incremental Learning (DIL), where the input distribution changes sequentially, adapting to new domains often degrades performance on previously learned ones. We observe that forgetting is not uniform across domains and classes and is not explained by training order alone. This motivates us to track each past domain-class concept individually. We use representation drift, defined as the deviation of a past concept's current representation from a fixed anchor, as a measurable surrogate for forgetting risk. The measured drift then serves two complementary roles. A consolidation loss assigns stronger preservation to high-drift pairs, while prototype construction down-weights highly drifted domains to reduce their influence on the class reference. Preservation is thus allocated by measured drift rather than assumed importance. To obtain this signal during training without storing past raw data, we repurpose a Joint-Embedding Predictive Architecture (JEPA). For each domain-class pair, we store only a compact context summary in the backbone feature space and rerun it through the learned context-to-target relation to measure its drift from the fixed anchor. Across DomainNet, Office-Home, CORe50, and CDDB-Hard, our method sustains a strong accuracy-forgetting balance on every benchmark, using substantially less training memory than accuracy-competitive methods and the smallest storage among compared methods.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-04 · Indexed by Pub-AI: 2026-11-05 UTC

preprint

Drift-adaptive Representation Preservation for Domain-Incremental Learning

Hyunmin Kong, Gyutae Oh, Jitae Shin

Preprint posted: 2026-11-04 · Original record ↗

arxiv
2611.06093
Explicit source versions (1)
  • v1 · 2026-11-04
  1. Preprint available

    Source date: 2026-11-04 · First detected: 2026-11-05 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Pretrained vision models provide strong general-purpose representations, but in Domain-Incremental Learning (DIL), where the input distribution changes sequentially, adapting to new domains often degrades performance on previously learned ones. We observe that forgetting is not uniform across domains and classes and is not explained by training order alone. This motivates us to track each past domain-class concept individually. We use representation drift, defined as the deviation of a past concept's current representation from a fixed anchor, as a measurable surrogate for forgetting risk. The measured drift then serves two complementary roles. A consolidation loss assigns stronger preservation to high-drift pairs, while prototype construction down-weights highly drifted domains to reduce their influence on the class reference. Preservation is thus allocated by measured drift rather than assumed importance. To obtain this signal during training without storing past raw data, we repurpose a Joint-Embedding Predictive Architecture (JEPA). For each domain-class pair, we store only a compact context summary in the backbone feature space and rerun it through the learned context-to-target relation to measure its drift from the fixed anchor. Across DomainNet, Office-Home, CORe50, and CDDB-Hard, our method sustains a strong accuracy-forgetting balance on every benchmark, using substantially less training memory than accuracy-competitive methods and the smallest storage among compared methods.