Driving · 2026-11-11
Making Strides in Symbolic Imitation Learning: Offline Demonstrations, Online Tuning
Bryce MacInnis, Malcolm I. Heywood
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose Symbolic Imitation Learning: distill an oracle policy from an offline dataset into a decision list over if-elif-else rules that deploy linear “motion primitives,” discovered via an EM-based discretization of continuous actions. They then fine-tune thresholds/constants and linear model parameters online using Augmented Random Search (ARS). They report matching oracle performance across 20 RL tasks and exceeding it in 11.
Why it matters
The work targets interpretable policies for continuous control by combining offline imitation (decision list + linear motion primitives) with online ARS tuning. It aims to approximate an oracle using parsimonious, domain-explainable decision rules rather than a monolithic policy, and reports oracle-matching/exceeding results across multiple benchmark domains.
Method
- Offline: assume an oracle provides state-action(-reward) tuples; use K-means + ridge regression within an EM procedure to discretize continuous actions into linear model motion primitives, with an optional switching-penalty dynamic-programming step.
- Offline: evolve the decision list via genetic programming under an imitation-learning fitness objective (accuracy for discrete tasks; cost-based objective for continuous tasks) with a complexity penalty for rules/predicates.
- Online: jointly tune thresholds in predicates, constants in expressions, and linear-model weights/biases using ARS, after pruning unused rules/predicates and final linear re-fitting on routed data.
Abstract (from arXiv)
Deep reinforcement learning (DRL) methods have been the dominant paradigm in continuous control tasks due to their efficacy in function approximation, while interpretable evolutionary methods have lagged largely behind for the opposite reason. In this work, we introduce Symbolic Imitation Learning, a framework for distilling an oracle policy through an offline dataset into a decision list with linear model motion primitives. We show that this, in conjunction, with online Augmented-Random Search (ARS) tuning of the constants and motion primitives approximates the performance of the oracle across 20 benchmark RL tasks and exceeds the oracle in 11 of those 20 tasks while simultaneously producing parsimonious, interpretable policies that domain experts can better understand.
Related papers
- ODE Policy: Robot Learning via Action Dynamics, Not Actions
- VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess
- AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation
- Higher-Order Action Supervision Makes A Strong Policy Class
- FARL: A Fully Automated Pipeline of RL Interface and Solver Design
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-11 · Indexed by Pub-AI: 2026-11-11 UTC
Making Strides in Symbolic Imitation Learning: Offline Demonstrations, Online Tuning
First announced by arXiv: 2026-11-11 · Original record ↗
- arxiv
- 2611.13182
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-11 · First detected: 2026-11-11 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.13182v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Deep reinforcement learning (DRL) methods have been the dominant paradigm in continuous control tasks due to their efficacy in function approximation, while interpretable evolutionary methods have lagged largely behind for the opposite reason. In this work, we introduce Symbolic Imitation Learning, a framework for distilling an oracle policy through an offline dataset into a decision list with linear model motion primitives. We show that this, in conjunction, with online Augmented-Random Search (ARS) tuning of the constants and motion primitives approximates the performance of the oracle across 20 benchmark RL tasks and exceeds the oracle in 11 of those 20 tasks while simultaneously producing parsimonious, interpretable policies that domain experts can better understand.
