Latent Dynamics · 2026-11-04
Forgetting to Improve: Principled Data Removal in Active Learning
Manuel Wendl, Erik Englesson, Andreas Krause, Carl Henrik Ek
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose “Forgetting to Improve” (F2I), a principled data-removal method for Gaussian process models in transductive learning and sequential decision tasks (including Bayesian optimization and model-based reinforcement learning). F2I greedily removes samples whose removal reduces predictive uncertainty within a target region, using influence functions that decompose the uncertainty into epistemic and aleatoric components under model mismatch.
Why it matters
Uncertainty (aleatoric vs epistemic) is induced by model and task, not just the data generating process. Under model mismatch, adding “detrimental” samples can worsen predictive confidence in the target region. By selecting which samples to remove—rather than assuming more data always helps—the method aims to improve task-relevant uncertainty and provides theoretical analysis (including approximation/near-optimality conditions).
Method
- Model a target region transductively and optimize predictive uncertainty reduction by removing detrimental observations; without labels, use predictive uncertainty as the objective.
- For GP regression, derive per-sample influence functions that decompose uncertainty change into epistemic and aleatoric terms, based on derivatives of the predictive uncertainty w.r.t. removal directions.
- Use a greedy removal algorithm (F2I) that iteratively discards the sample whose removal most reduces the total predicted uncertainty over the target region; extend to Forgetful Bayesian Optimization and Forgetful RL.
Limitation
The authors state that the theoretical near-optimality guarantee relies on sufficient conditions (e.g., restricted strong concavity, positive realized improvement, restricted gradient dominance) that are sufficient but not necessary; also, they “cannot rely on having access to labels to measure generalization in the target region.”
Abstract (from arXiv)
The uncertainty of a statistical model is most commonly factorized into an aleatoric and an epistemic part. This factorization changes how predictions are interpreted in downstream decision tasks. Importantly, except for the idealistic scenario with no model mismatch, the quantification is a characteristic of the model and not the data generating process. In this paper, we propose Forgetting to Improve a method that reduces this discrepancy by incorporating the task into the modeling framework. Our key insight is to acknowledge that in scenarios of model mismatch, data can have a detrimental effect on the modeling for a specific task. Based on this insight, we propose an influence function for Gaussian process models that allows for principled removal of detrimental data samples. We showcase the flexibility of this approach by demonstrating significant improvements across a range of tasks, including Bayesian optimization, model-based reinforcement learning, and transductive learning.
Related papers
- Before We Adapt: The Zero-Shot Generalization Bottleneck in Continual World Models
- Statistical Foundations of Reinforcement Learning: A Non-Asymptotic Perspective
- What Should World Models Forget? Stratified Retention for Continual Adaptation
- Considering Context: When World Models Need Context Encoders
- When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-04 · Indexed by Pub-AI: 2026-11-04 UTC
Forgetting to Improve: Principled Data Removal in Active Learning
First announced by arXiv: 2026-11-04 · Original record ↗
- arxiv
- 2611.04054
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-04 · First detected: 2026-11-04 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.04054v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
The uncertainty of a statistical model is most commonly factorized into an aleatoric and an epistemic part. This factorization changes how predictions are interpreted in downstream decision tasks. Importantly, except for the idealistic scenario with no model mismatch, the quantification is a characteristic of the model and not the data generating process. In this paper, we propose Forgetting to Improve a method that reduces this discrepancy by incorporating the task into the modeling framework. Our key insight is to acknowledge that in scenarios of model mismatch, data can have a detrimental effect on the modeling for a specific task. Based on this insight, we propose an influence function for Gaussian process models that allows for principled removal of detrimental data samples. We showcase the flexibility of this approach by demonstrating significant improvements across a range of tasks, including Bayesian optimization, model-based reinforcement learning, and transductive learning.
