Driving · 2026-10-09
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Haoyu Zhao, Zhengxu Yu, Zhiyuan He, Meng Fang, Rasul Tutunov, Haitham Bou-Ammar, Weilin Luo, Jun Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Memento 3 (“Code as Model”) uses a frozen LLM with external memory to learn explicit world models for planning in unfamiliar environments. The agent keeps a natural-language rulebook (semantic memory), compiles it into executable prediction code, and iteratively updates both via observation–reflection–rule revision–compilation–verification. Verified updates guide further interaction; the LLM parameters remain fixed.
Why it matters
The authors propose a model-based route to recursive self-improvement where the learning state is an inspectable, revisable world model (rulebook + executable code). Verification uses both LLM judgment of rulebook fidelity and exact replay that reproduces observed transitions, and a population extension maintains multiple rulebooks/executables in parallel to reduce dependence on a single early model.
Method
- Maintain a natural-language rulebook as persistent semantic memory and compile it into executable code for prediction and planning.
- Use a continual loop (observation, reflection, rule revision, compilation, verification); accept updated code only if LLM judges it faithful and cell-exact replay reproduces observed transitions.
- Extend to a population: maintain N rulebook–executable world models in parallel sharing interaction evidence; sample one for planning and check all members against shared evidence.
Abstract (from arXiv)
Learning to act in unfamiliar environments requires agents to infer how the world works and revise that understanding as new evidence arrives. Yet limited observations can support multiple world models that explain past interactions but predict different outcomes in unseen states. We introduce Memento 3, building on the Memento series to enable frozen LLM agents to continually learn explicit world models through external memory. The agent maintains a natural-language rulebook as persistent semantic memory, recording revisable hypotheses about environment dynamics while leaving unknown aspects underspecified. It compiles this rulebook into executable code for prediction and planning. Through a continual loop of observation, reflection, rule revision, compilation, and verification, the agent uses prediction errors to refine both the rulebook and its code. Updated code is accepted only when the LLM judges it faithful to the rulebook and cell-exact replay reproduces the observed transitions. We investigate this process as a model-based route to recursive self-improvement (RSI): the agent autonomously explores the environment, revises its world model, and uses verified updates to guide subsequent interaction and learning, while the underlying LLM remains fixed. A population extension maintains multiple world models in parallel, sharing interaction evidence and using their predictions to guide exploration. On ARC-AGI-3, the single-model agent clears every level of all 25 public games, achieves a mean Relative Human Action Efficiency (RHAE) of 100.0, and uses 44% of the human action count. In an Atari Pong case study, a learned feedback controller wins 21:0 in each of three evaluated episodes with different openings, without further LLM calls.
Related papers
- EvoBelief: Falsifiable World Models for Self-Improving LLM Agents
- Self-Improving Agents with World Model Predictive Paths
- AdaRSI: On-Demand Recursive Self-Improvement for Embodied Agents
- MemoWM: How World Models Change What Agents Need to Remember
- D-World: An Explicit World-Language-Action Model with Recursive Self-Improvement under Dual Control
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Preprint posted: 2026-10-08 · Original record ↗
- arxiv
- 2610.11794
Explicit source versions (1)
- v1 · 2026-10-08
- Preprint available
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Learning to act in unfamiliar environments requires agents to infer how the world works and revise that understanding as new evidence arrives. Yet limited observations can support multiple world models that explain past interactions but predict different outcomes in unseen states. We introduce Memento 3, building on the Memento series to enable frozen LLM agents to continually learn explicit world models through external memory. The agent maintains a natural-language rulebook as persistent semantic memory, recording revisable hypotheses about environment dynamics while leaving unknown aspects underspecified. It compiles this rulebook into executable code for prediction and planning. Through a continual loop of observation, reflection, rule revision, compilation, and verification, the agent uses prediction errors to refine both the rulebook and its code. Updated code is accepted only when the LLM judges it faithful to the rulebook and cell-exact replay reproduces the observed transitions. We investigate this process as a model-based route to recursive self-improvement (RSI): the agent autonomously explores the environment, revises its world model, and uses verified updates to guide subsequent interaction and learning, while the underlying LLM remains fixed. A population extension maintains multiple world models in parallel, sharing interaction evidence and using their predictions to guide exploration. On ARC-AGI-3, the single-model agent clears every level of all 25 public games, achieves a mean Relative Human Action Efficiency (RHAE) of 100.0, and uses 44% of the human action count. In an Atari Pong case study, a learned feedback controller wins 21:0 in each of three evaluated episodes with different openings, without further LLM calls.