Driving · 2026-10-14
D-World: An Explicit World-Language-Action Model with Recursive Self-Improvement under Dual Control
Xuan Wen, Baichao Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
D-World is an explicit world-language-action model using Sense-Plan-Act and an executable digital-twin “world” written from robot observations. Plan searches and commits action chunks in simulation, then revises the twin via measurement-grounded recursive self-improvement (RSI): revisions are accepted only if replaying the latest chunk in the revised twin does not increase measured residuals beyond tolerances. Reconstruction is used only for structural mismatches. The LLM stays frozen while the world improves.
Why it matters
The authors target an inspectable, correctable alternative to implicit world models (stored in weights or predicted video). By keeping a named, executable twin as the sole interface between perception and planning—and gating every action chunk via measurement checks—D-World makes physical discrepancies traceable to specific twin fields or parameters, supports in-episode self-improvement, and records twins for offline refitting.
Method
- Sense writes an executable digital twin from camera observations by generating vision and geometry code; renders the draft for comparison and uses a physics check before anything runs.
- Plan proposes trajectories and commits short execution chunks; after each executed chunk it revises twin object states and named physical parameters against robot measurements, using an RSI replay check with residual tolerances.
- A harness enforces verification: accepted revisions must pass replay non-regression, structural mismatches trigger reconstruction, and each action chunk is gated in the updated twin before execution.
Abstract (from arXiv)
Frontier large language models (LLMs) now reason about space well enough to program robots, yet the world models behind robot policies are implicit, held in network weights or predicted video, and cannot be inspected or corrected directly. We present D-World, an explicit world-language-action model built on the Sense, Plan and Act decomposition. A perception agent writes the scene as an executable digital twin using generated vision and geometry code. A planning agent searches for actions in simulation and, after each executed action chunk, revises the twin's object states and named physical parameters against what the robot measured. A parameter revision is accepted only if replaying the latest chunk does not increase any measured residual beyond its tolerance, and every chunk is checked in the updated twin before it runs. Structural mismatches return to perception for reconstruction. We call this measurement-grounded cycle of correction and revision recursive self-improvement (RSI), in which the LLM stays frozen while the world it acts in improves. We model the choice of actions that both advance the task and expose model error as a dual control problem and show that the cost of incorporating evidence into the twin sets the threshold at which probing pays, so cheap local revision counteracts the turn-off of cautious controllers. Each episode runs a real-to-sim-to-real loop, and its recorded twins can be refitted and re-rendered offline into higher-fidelity environments for post-training. Because perception and planning communicate only through the twin and language, either can be replaced by a learned model, and since inference dominates the loop, the loop speeds up with faster models. On a dual-arm robot, towel and toaster tasks show local revisions, such as one reducing the bread's rendered-depth residual from 22.4 to 1.5 mm, that let execution continue without reconstruction.
Related papers
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-14 UTC
D-World: An Explicit World-Language-Action Model with Recursive Self-Improvement under Dual Control
Preprint posted: 2026-10-12 · Original record ↗
- arxiv
- 2610.17114
Explicit source versions (1)
- v1 · 2026-10-12
- Preprint available
Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Frontier large language models (LLMs) now reason about space well enough to program robots, yet the world models behind robot policies are implicit, held in network weights or predicted video, and cannot be inspected or corrected directly. We present D-World, an explicit world-language-action model built on the Sense, Plan and Act decomposition. A perception agent writes the scene as an executable digital twin using generated vision and geometry code. A planning agent searches for actions in simulation and, after each executed action chunk, revises the twin's object states and named physical parameters against what the robot measured. A parameter revision is accepted only if replaying the latest chunk does not increase any measured residual beyond its tolerance, and every chunk is checked in the updated twin before it runs. Structural mismatches return to perception for reconstruction. We call this measurement-grounded cycle of correction and revision recursive self-improvement (RSI), in which the LLM stays frozen while the world it acts in improves. We model the choice of actions that both advance the task and expose model error as a dual control problem and show that the cost of incorporating evidence into the twin sets the threshold at which probing pays, so cheap local revision counteracts the turn-off of cautious controllers. Each episode runs a real-to-sim-to-real loop, and its recorded twins can be refitted and re-rendered offline into higher-fidelity environments for post-training. Because perception and planning communicate only through the twin and language, either can be replaced by a learned model, and since inference dominates the loop, the loop speeds up with faster models. On a dual-arm robot, towel and toaster tasks show local revisions, such as one reducing the bread's rendered-depth residual from 22.4 to 1.5 mm, that let execution continue without reconstruction.