Latent Dynamics · 2026-10-02
Calibration-risk routing for controlled world-model adaptation
Yifan Zhang, Liang Zheng
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose MC-WM for model-based RL under simulator-to-target shifts by fitting both a residual-corrected simulator ensemble and a direct target ensemble from disjoint fit/selection/calibration partitions. A calibration partition measures standardized one-step state-and-reward error to route the deployed family with lower calibration risk. A learned confidence signal plus deterministic validity predicates weight one-step imagined policy updates without rewriting physical rewards.
Why it matters
Simulator correction and direct target modeling can both fail under limited target data; the paper frames this as a model-family selection problem. MC-WM uses calibration-risk routing to choose between residual and direct world-model families and uses confidence/predicate weighting to bound the influence of imagined transitions during SAC training.
Method
- Fit two target-model candidates from the same initial target data: a residual correction to a frozen simulator ensemble and a direct target ensemble; select the deployed family using a disjoint calibration partition measuring standardized one-step state-and-reward error.
- Deploy the selected ensemble to produce one-step imagined transitions for SAC, weighting policy-critic losses by a learned confidence signal and deterministic validity predicates (trust weights; physical rewards unchanged).
- On the residual route, use a locked sparse support with a masked neural remainder; online refits update eligible parameters without adding features or changing support.
Limitation
The evidence is deliberately bounded to target MuJoCo simulators; it does not imply physical sim-to-real transfer, stable symbolic discovery across seeds, reduced total target access, or a safety guarantee.
Abstract (from arXiv)
Model-based reinforcement learning (MBRL) can exploit simulated experience, but a simulator-to-target shift creates a model-selection problem: correcting the simulator and fitting the target directly can each fail under limited target data. We introduce the Model-Corrected World Model (MC-WM), which separates initial target data into disjoint fit, selection, and calibration partitions and deploys the family with lower standardized calibration risk. A learned confidence signal and deterministic validity predicates weight one-step imagined policy updates without rewriting physical rewards. We evaluate 540 unique reported run cells across three controlled Multi-Joint dynamics with Contact (MuJoCo) shifts; one exact-routing cell was repeated after a pre-deployment artifact gate, giving 541 completed executions.
Related papers
- pico-WAM: Evaluating Counterfactual Prediction and Planning Error in a Compact World Action Model
- World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning
- Trust-Calibrated VLA Policy Post-Training with Evolving Imagination
- Beyond Prediction: Steering VLM Agents with Retrospective World Modeling
- $R^2$-WAM: Repair-and-Reject Post-Training for World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC
Calibration-risk routing for controlled world-model adaptation
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.01001
Explicit source versions (1)
- v2 · 2026-10-17
- Explicit source version detected · v2
Source date: 2026-10-17 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Model-based reinforcement learning (MBRL) can exploit simulated experience, but a simulator-to-target shift creates a model-selection problem: correcting the simulator and fitting the target directly can each fail under limited target data. We introduce the Model-Corrected World Model (MC-WM), which separates initial target data into disjoint fit, selection, and calibration partitions and deploys the family with lower standardized calibration risk. A learned confidence signal and deterministic validity predicates weight one-step imagined policy updates without rewriting physical rewards. We evaluate 540 unique reported run cells across three controlled Multi-Joint dynamics with Contact (MuJoCo) shifts; one exact-routing cell was repeated after a pre-deployment artifact gate, giving 541 completed executions.
