Driving · 2026-10-20
Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing
Adrian Remonda, Jiajun Xi, Nicklas Hansen, Marcus Greiff, Thomas Lew, John Talbot, John Subosits, Xiaolong Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PIA (Prior-Informed Adaptation) is an offline-to-online model-based RL framework for autonomous racing that pretrains a world model on heterogeneous low- and high-fidelity simulation trajectories, then adapts with limited real-world interaction using uncertainty-regularized latent planning. The authors treat simulated trajectories as biased priors and correct residual mismatch during online adaptation, avoiding the need for an exact simulator. They evaluate in ACGym and on a full-size autonomous race car.
Why it matters
The authors target safe, data-efficient adaptation when interaction with real robots is costly and failures are unsafe. PIA uses simulation as imperfect structured prior data (not a direct transfer environment), combining heterogeneous simulation buffers with target-domain fine-tuning and uncertainty-regularized planning to discourage exploiting poorly supported predictions. They report faster adaptation and better safety in simulation under track/surface shifts and show few-shot adaptation on hardware with less than one hour of on-vehicle interaction.
Method
- Pretrain a TD-MPC2-style latent world model on heterogeneous simulation-prior trajectories (low-fidelity ODE rollouts and high-fidelity Assetto Corsa trajectories).
- During online adaptation, train on a mixed replay buffer (simulation priors + newly collected target-domain transitions) with context labels to distinguish task/data-source/track setting.
- Use uncertainty-regularized latent planning with epistemic uncertainty estimated from Q-ensemble disagreement; extend planning to be context-aware to discourage high-uncertainty rollouts.
Limitation
PIA assumes a shared observation/action interface between synthetic and target data, making it more challenging for methods using complex sensor modes (image/LiDAR) since this must be produced by the offline simulation environment. The real-car study is limited to deployments on a single physical track and surface, and safety-envelope violations remain possible during adaptation.
Abstract (from arXiv)
Reinforcement learning enables learning high-performance control policies, but its reliance on extensive interaction limits deployment on physical systems where data collection is costly, and failures are unsafe. We introduce Prior-Informed Adaptation (PIA), an offline-to-online model-based RL framework that uses structured simulation priors for rapid adaptation. PIA pretrains a world model on heterogeneous low- and high-fidelity simulated trajectories, then adapts it with limited target-domain interaction using uncertainty-regularized planning. Rather than requiring an accurate simulator for policy transfer, PIA treats simulated trajectories as biased priors whose residual mismatch is corrected during online adaptation. We evaluate PIA in autonomous racing using high-fidelity simulation and on a real full-size autonomous race car. In simulation, PIA adapts faster across track and surface shifts than strong baselines, achieving lower lap times and fewer failures. On hardware, PIA achieves reliable few-shot adaptation with less than one hour of on-vehicle interaction.
Related papers
- RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts
- Trust-Calibrated VLA Policy Post-Training with Evolving Imagination
- Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving
- Closed-Loop Aware Planning via Imitation-Free Speed Scoring for End-to-End Autonomous Driving
- FARL: A Fully Automated Pipeline of RL Interface and Solver Design
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC
Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing
Preprint posted: 2026-10-19 · Original record ↗
- arxiv
- 2610.26757
Explicit source versions (1)
- v1 · 2026-10-19
- Preprint available
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Reinforcement learning enables learning high-performance control policies, but its reliance on extensive interaction limits deployment on physical systems where data collection is costly, and failures are unsafe. We introduce Prior-Informed Adaptation (PIA), an offline-to-online model-based RL framework that uses structured simulation priors for rapid adaptation. PIA pretrains a world model on heterogeneous low- and high-fidelity simulated trajectories, then adapts it with limited target-domain interaction using uncertainty-regularized planning. Rather than requiring an accurate simulator for policy transfer, PIA treats simulated trajectories as biased priors whose residual mismatch is corrected during online adaptation. We evaluate PIA in autonomous racing using high-fidelity simulation and on a real full-size autonomous race car. In simulation, PIA adapts faster across track and surface shifts than strong baselines, achieving lower lap times and fewer failures. On hardware, PIA achieves reliable few-shot adaptation with less than one hour of on-vehicle interaction.