Latent Dynamics · 2026-10-19

RECREG: Recovery-based Shielding for Regular Safety Properties in Reinforcement Learning

Alexander W. Goodall, Francesco Belardinelli

arXiv:2610.23445PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

RecReg is a safe RL framework for enforcing non-Markovian regular safety properties using recovery-based shielding. It supports Exact, ModelBased, and ModelFree operation modes, and provides theoretical safety, convergence, and optimality guarantees (focus on tabular setting). The paper also proposes a deep RL variant using DreamerV3. Experiments using MASA-Safe-RL show RecReg achieves efficient learning and matches the theory.

Why it matters

The authors target RL deployment with formal safety requirements beyond simple invariants. RecReg connects regular safety properties to reachability in a product MDP and enforces safety via probabilistic safe sets and shielding/recovery zones, aiming for rigorous safety guarantees even when transition dynamics are unknown (exact/model-based/model-free).

Method

  • Construct a product MDP by synchronizing the environment with a DFA for bad prefixes of a regular safety property, reducing safety probability to reachability probability of an “unsafe” set.
  • Train a backup safe policy via a stochastic avoidance problem (eventual discounting) to obtain probabilistic safe sets and use them for action replacement/shielding under step-wise risk budgets.
  • Train the task policy with a safety-aware Bellman operator so learning does not get stuck on unsafe high-reward actions, and provide guarantees by operation mode (Exact/ModelBased/ModelFree).

Limitation

The paper primarily focuses on building the theoretical foundations for RecReg in the tabular setting.

Abstract (from arXiv)

In many real-world scenarios we need to consider formal requirements, such as safety and constraint compliance, to deploy Reinforcement Learning (RL) agents. In this paper we introduce RecReg, a safe RL framework capable of enforcing non-Markovian regular safety properties via recovery-based shielding. RecReg is a unified approach that supports different levels of a priori knowledge and safety guarantees. The paper primarily focuses on building the theoretical foundations for RecReg in the tabular setting, providing rigorous safety and convergence guarantees that are not present in related work. We provide an extensive set of empirical results using the MASA-Safe-RL library, and we outline an extension of RecReg to deep RL domains using SOTA model-based RL. Our empirical results demonstrate that RecReg achieves efficient learning and matches the theory in this paper.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-16 · Indexed by Pub-AI: 2026-10-19 UTC

preprint

RECREG: Recovery-based Shielding for Regular Safety Properties in Reinforcement Learning

Alexander W. Goodall, Francesco Belardinelli

Preprint posted: 2026-10-16 · Original record ↗

arxiv
2610.23445
Explicit source versions (1)
  • v1 · 2026-10-16
  1. Preprint available

    Source date: 2026-10-16 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

In many real-world scenarios we need to consider formal requirements, such as safety and constraint compliance, to deploy Reinforcement Learning (RL) agents. In this paper we introduce RecReg, a safe RL framework capable of enforcing non-Markovian regular safety properties via recovery-based shielding. RecReg is a unified approach that supports different levels of a priori knowledge and safety guarantees. The paper primarily focuses on building the theoretical foundations for RecReg in the tabular setting, providing rigorous safety and convergence guarantees that are not present in related work. We provide an extensive set of empirical results using the MASA-Safe-RL library, and we outline an extension of RecReg to deep RL domains using SOTA model-based RL. Our empirical results demonstrate that RecReg achieves efficient learning and matches the theory in this paper.