Driving · 2026-11-03

Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift

Qurban Ali, Badaruddin Chachar, Maqsood Ali, Sajjad Ali Rajper

arXiv:2611.02218PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 PPO policies under distribution shift, using closed-loop world-model rollouts and a three-model ensemble. They report that predicted STL robustness ranks HalfCheetah’s future STL violations strongly, while ensemble disagreement does not reliably track observation prediction error or violations. A current-state robustness baseline recovers most ranking value; disagreement-penalized robustness does not consistently improve ranking.

Why it matters

This study tests whether ensemble disagreement from world-model forecasts can serve as a reliable safety-uncertainty indicator under distribution shift. The authors find limited evidence that disagreement tracks prediction error or STL violations, and that a simple current-state robustness baseline recovers much of predictive ranking performance. This cautions against treating ensemble disagreement as a calibrated safety signal for STL monitoring.

Method

  • Build a predictive STL monitor by rolling out a policy-conditioned learned world-model ensemble in closed loop for K-step horizons, then computing STL robustness on predicted trajectories.
  • Define disagreement as the standard deviation of predicted STL robustness across ensemble members (and also evaluate observation- and latent-space disagreement).
  • Evaluate window-level failure prediction using AUROC/AUPRC and analyze effects of distribution shift types and horizon length, comparing against a current-state robustness baseline and disagreement-penalized robustness.

Limitation

The authors report that penalizing predicted robustness by disagreement conflates model disagreement with safety; when baseline robustness is already close to zero, this conflation can have practical consequences. They also scope conclusions about disagreement informativeness to a lightweight feedforward latent-dynamics ensemble architecture class.

Abstract (from arXiv)

RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC

preprint

Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift

Qurban Ali, Badaruddin Chachar, Maqsood Ali, Sajjad Ali Rajper

First announced by arXiv: 2026-11-03 · Original record ↗

arxiv
2611.02218
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.02218v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.