Driving · 2026-11-03
Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift
Qurban Ali, Badaruddin Chachar, Maqsood Ali, Sajjad Ali Rajper
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 PPO policies under distribution shift, using closed-loop world-model rollouts and a three-model ensemble. They report that predicted STL robustness ranks HalfCheetah’s future STL violations strongly, while ensemble disagreement does not reliably track observation prediction error or violations. A current-state robustness baseline recovers most ranking value; disagreement-penalized robustness does not consistently improve ranking.
Why it matters
This study tests whether ensemble disagreement from world-model forecasts can serve as a reliable safety-uncertainty indicator under distribution shift. The authors find limited evidence that disagreement tracks prediction error or STL violations, and that a simple current-state robustness baseline recovers much of predictive ranking performance. This cautions against treating ensemble disagreement as a calibrated safety signal for STL monitoring.
Method
- Build a predictive STL monitor by rolling out a policy-conditioned learned world-model ensemble in closed loop for K-step horizons, then computing STL robustness on predicted trajectories.
- Define disagreement as the standard deviation of predicted STL robustness across ensemble members (and also evaluate observation- and latent-space disagreement).
- Evaluate window-level failure prediction using AUROC/AUPRC and analyze effects of distribution shift types and horizon length, comparing against a current-state robustness baseline and disagreement-penalized robustness.
Limitation
The authors report that penalizing predicted robustness by disagreement conflates model disagreement with safety; when baseline robustness is already close to zero, this conflation can have practical consequences. They also scope conclusions about disagreement informativeness to a lightweight feedforward latent-dynamics ensemble architecture class.
Abstract (from arXiv)
RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.
Related papers
- Right Ranking, Wrong Scale: Reproducing the Robotic World Model's Dynamics Model and Testing the Uncertainty of Its Follow-up
- LocoWM: High-Precision Locomotion through World-Model-Guided Residual Adaptation
- Predictive Safety Guardrails Do Not Always Make LLM Agents Safer
- How Many Candidates Does a One-Step World Model Need? A Random-Codebook Account of Nearest-of-m Training on Frozen DINO Tokens
- Modeling Latent Disturbances for Robust Decision-Making in World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC
Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift
First announced by arXiv: 2026-11-03 · Original record ↗
- arxiv
- 2611.02218
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.02218v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.
