Driving · 2026-11-11

Making Strides in Symbolic Imitation Learning: Offline Demonstrations, Online Tuning

Bryce MacInnis, Malcolm I. Heywood

arXiv:2611.13182PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors propose Symbolic Imitation Learning: distill an oracle policy from an offline dataset into a decision list over if-elif-else rules that deploy linear “motion primitives,” discovered via an EM-based discretization of continuous actions. They then fine-tune thresholds/constants and linear model parameters online using Augmented Random Search (ARS). They report matching oracle performance across 20 RL tasks and exceeding it in 11.

Why it matters

The work targets interpretable policies for continuous control by combining offline imitation (decision list + linear motion primitives) with online ARS tuning. It aims to approximate an oracle using parsimonious, domain-explainable decision rules rather than a monolithic policy, and reports oracle-matching/exceeding results across multiple benchmark domains.

Method

  • Offline: assume an oracle provides state-action(-reward) tuples; use K-means + ridge regression within an EM procedure to discretize continuous actions into linear model motion primitives, with an optional switching-penalty dynamic-programming step.
  • Offline: evolve the decision list via genetic programming under an imitation-learning fitness objective (accuracy for discrete tasks; cost-based objective for continuous tasks) with a complexity penalty for rules/predicates.
  • Online: jointly tune thresholds in predicates, constants in expressions, and linear-model weights/biases using ARS, after pruning unused rules/predicates and final linear re-fitting on routed data.
Abstract (from arXiv)

Deep reinforcement learning (DRL) methods have been the dominant paradigm in continuous control tasks due to their efficacy in function approximation, while interpretable evolutionary methods have lagged largely behind for the opposite reason. In this work, we introduce Symbolic Imitation Learning, a framework for distilling an oracle policy through an offline dataset into a decision list with linear model motion primitives. We show that this, in conjunction, with online Augmented-Random Search (ARS) tuning of the constants and motion primitives approximates the performance of the oracle across 20 benchmark RL tasks and exceeds the oracle in 11 of those 20 tasks while simultaneously producing parsimonious, interpretable policies that domain experts can better understand.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-11 · Indexed by Pub-AI: 2026-11-11 UTC

preprint

Making Strides in Symbolic Imitation Learning: Offline Demonstrations, Online Tuning

Bryce MacInnis, Malcolm I. Heywood

First announced by arXiv: 2026-11-11 · Original record ↗

arxiv
2611.13182
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-11 · First detected: 2026-11-11 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.13182v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Deep reinforcement learning (DRL) methods have been the dominant paradigm in continuous control tasks due to their efficacy in function approximation, while interpretable evolutionary methods have lagged largely behind for the opposite reason. In this work, we introduce Symbolic Imitation Learning, a framework for distilling an oracle policy through an offline dataset into a decision list with linear model motion primitives. We show that this, in conjunction, with online Augmented-Random Search (ARS) tuning of the constants and motion primitives approximates the performance of the oracle across 20 benchmark RL tasks and exceeds the oracle in 11 of those 20 tasks while simultaneously producing parsimonious, interpretable policies that domain experts can better understand.