Driving · 2026-10-01
Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey
F. Olivia Fan, Oliver Obst
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study robot air-hockey defense with temporary puck tracking loss and distil a DreamerV3 history-dependent teacher into compact recurrent policies with a 64D state. Across five seeds, a purely linear recurrent update (k=0) matches a GRU-64 and the teacher through 400 ms blackout, while increasing nonlinear innovation rank (k=1,2,4) gives no measured benefit. The linear model uses fewer recurrent parameters and less computation than GRU-64.
Why it matters
The paper tests whether memory-dependent control requires nonlinear recurrent dynamics by isolating the recurrent update structure during policy distillation. For a simulated air-hockey defence task under tracking loss, a linear recurrent mechanism can reproduce the teacher’s closed-loop behaviour as well as a GRU-64, with fewer recurrent parameters and lower measured CPU latency.
Method
- Train a DreamerV3 teacher for air-hockey defence and create a tracking-loss task using observation-alias pairs that require memory under blackout.
- Distil teacher actions into student policies with a fixed architecture: a 64D recurrent state with diagonal linear recurrence plus an optional rank-k nonlinear innovation (k∈{0,1,2,4}), while keeping nonlinear encoders and action heads.
- Evaluate save rate across blackout durations (0–400 ms main range; 500 ms extrapolation) using fresh test shots and paired bootstrap confidence intervals; also measure efficiency (params, MACs, CPU latency) and a 5 mm noise sensitivity test without retraining.
Limitation
The authors state conclusions are limited to the simulated air-hockey set-piece, one teacher, 64-dimensional principal state and five training seeds per student family, and to policies whose observation encoder and action head remain nonlinear. They also note untested factors: correlated tracking errors, repeated occlusions, longer or repeated occlusions, changing puck dynamics, and physical robot deployment.
Abstract (from arXiv)
Does memory-dependent control need nonlinear recurrent dynamics? We study simulated air-hockey defence under temporary loss of puck tracking. A DreamerV3 teacher outperforms a memoryless policy under tracking loss, while resetting the teacher's recurrent state sharply reduces performance, which demonstrates that the task requires memory. We distil this teacher into compact recurrent policies with a 64 dimensional state, with a combination of a diagonal linear recurrence and an optional rank-$k$ nonlinear innovation while retaining nonlinear observation encoders and action heads. Across five matched seeds, the purely linear recurrent model ($k=0$) matches both the GRU baseline and the teacher throughout the tested range of tracking loss. Increasing nonlinear innovation rank providing no measured benefits. This result is obtained on a fresh test split, which will be only opened after all models and analyses are frozen. The linear model requires fewer recurrent parameters and less computation than GRU, but performs comparably. These results suggest that, for this memory dependent control task, nonlinear representation learning around a simple linear memory mechanism can be sufficient, and that nonlinear recurrent dynamics are not necessarily required. These conclusions are limited to the simulated task, teacher, state dimension, and blackout horizon considered here, and to policies whose observation encoder and action head remain nonlinear.
Related papers
- MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation
- Memory Without Curation: Learning Multi-Stream Latent Memory for Robot Policies
- T$^2$Mem: Learning Test-Time Memory for Robotics
- Data-Free Continual Learning for World Action Model with Rectified Generative Replay
- RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.39151
Explicit source versions (1)
- v1 · 2026-09-30
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Does memory-dependent control need nonlinear recurrent dynamics? We study simulated air-hockey defence under temporary loss of puck tracking. A DreamerV3 teacher outperforms a memoryless policy under tracking loss, while resetting the teacher's recurrent state sharply reduces performance, which demonstrates that the task requires memory. We distil this teacher into compact recurrent policies with a 64 dimensional state, with a combination of a diagonal linear recurrence and an optional rank-$k$ nonlinear innovation while retaining nonlinear observation encoders and action heads. Across five matched seeds, the purely linear recurrent model ($k=0$) matches both the GRU baseline and the teacher throughout the tested range of tracking loss. Increasing nonlinear innovation rank providing no measured benefits. This result is obtained on a fresh test split, which will be only opened after all models and analyses are frozen. The linear model requires fewer recurrent parameters and less computation than GRU, but performs comparably. These results suggest that, for this memory dependent control task, nonlinear representation learning around a simple linear memory mechanism can be sufficient, and that nonlinear recurrent dynamics are not necessarily required. These conclusions are limited to the simulated task, teacher, state dimension, and blackout horizon considered here, and to policies whose observation encoder and action head remain nonlinear.