Robotics · 2026-10-05
RIFAR: Reliability and Forgetting-Aware Replay for Continual Robot Learning
Zirong Song, Zheng Lu, Haoran Liao, Wanqi Zhong, Yunhe Ni, Lijie Wang, Xiuying Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
RIFAR is a continual robot learning method for world–action models that reduces forgetting without storing full demonstrations. It keeps compact initial-state anchors (K=20, Hs=16) and generates candidate rollouts, then uses a frozen inverse-dynamics model (IDM) to screen action–visual consistency. It trains in two phases: initial adaptation with screened replay, then drift-aware reselection using action prediction drift before vs. after adaptation. The authors report improved retention vs. REGEN and strong memory efficiency vs. ER.
Why it matters
For continual embodied agency, robots must learn new tasks while retaining earlier skills. Storing complete demonstrations for experience replay becomes costly as tasks accumulate. The authors propose a reliability- and forgetting-aware generative replay pipeline for world–action models, aiming to preserve earlier behaviors using only compact context and targeted replay prioritized by action drift.
Method
- Generates past-task replay by recurrently extending compact initial-state anchors: select K=20 initial demonstrations and retain Hs=16 observation–proprioception steps per demo (320 time steps per task).
- Screens generated rollouts with a frozen inverse-dynamics model (IDM) to measure action–visual consistency via normalized chunk disagreement and IDM-derived agreement scores.
- Uses drift-based replay prioritization: compares action predictions before vs. after an initial adaptation phase on identical historical contexts, reselecting trajectories with larger normalized action drift for continued training.
Limitation
IDM-based screening improves replay quality but cannot eliminate unsuccessful trajectories; residual errors may introduce noise into replay training. The IDM is pretrained and kept frozen during continual learning, so substantial task distribution shifts may reduce its ability to assess unfamiliar manipulation behaviors. The WAM relies on value-based termination during autoregressive generation, which can trigger premature stopping before the task is complete, and recurrent generation can accumulate errors over long horizons.
Abstract (from arXiv)
Genuine embodied agency requires robots to turn continuous real-world experience into lasting, transferable skills. This demands continual learning that integrates new capabilities without eroding prior knowledge as tasks and environments evolve. Experience replay mitigates forgetting, but storing complete demonstrations becomes costly as tasks accumulate. World-action models offer a generative alternative, reconstructing past experience through joint predictions of actions and future observations. However, visually coherent rollouts may contain actions that cannot realize the predicted transitions, while new-task adaptation can disrupt previously learned behavior. RIFAR therefore combines reliability screening with drift-aware replay selection. It reconstructs trajectories from compact demonstration prefixes and uses a frozen inverse-dynamics model to assess action-visual consistency. Training first combines current demonstrations with the highest-quality screened trajectories. RIFAR then compares action predictions before and after this adaptation on identical historical inputs, reselecting trajectories with larger normalized drift from the same screened pool for continued training. Across three LIBERO suites and real-world experiments, RIFAR surpasses the previous state of the art in WAM-based generative replay. On LIBERO-Goal, it achieves 90.97 AUC while retaining only 320 historical time steps per task, approximately 4.9% of the steps retained using 50-demonstration replay.
Related papers
- Data-Free Continual Learning for World Action Model with Rectified Generative Replay
- EVO-WAM: Evolving World Action Models through Video-Action Verification
- EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution
- $R^2$-WAM: Repair-and-Reject Post-Training for World Action Models
- RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-02 · Indexed by Pub-AI: 2026-10-05 UTC
RIFAR: Reliability-Informed and Forgetting-Aware Replay for Continual Robot Learning with World-Action Models
Preprint posted: 2026-10-02 · Original record ↗
- arxiv
- 2610.03079
Explicit source versions (1)
- v3 · 2026-10-19
- Preprint available
Source date: 2026-10-02 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v3
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Genuine embodied agency requires robots to turn continuous real-world experience into lasting, transferable skills. This demands continual learning that integrates new capabilities while preserving prior knowledge in evolving environments. Experience replay mitigates catastrophic forgetting, but storing complete demonstrations becomes costly as tasks accumulate. World-action models provide an alternative through generative replay. However, visually coherent rollouts may contain actions that cannot realize the predicted transitions, while new-task adaptation can disrupt learned behavior. To address these problems, we propose RIFAR (Reliability-Informed and Forgetting-Aware Replay), a reliability-informed and drift-aware generative replay framework. It reconstructs trajectories from compact historical demonstration prefixes and uses a trainable inverse-dynamics model to assess action-visual consistency. It first adapts to new tasks using current-task demonstrations and high-reliability replay, then evaluates how this adaptation changes action predictions on historical inputs and prioritizes reliable trajectories with greater action drift for further training. Across three LIBERO suites and real-world experiments, RIFAR outperforms the previous state of the art in WAM-based generative replay. On LIBERO-Goal, it achieves an AUC of 90.97 while retaining only 320 historical time steps per task, about 4.9% of those retained by 50-demonstration replay.