Driving · 2026-10-02
In CEM, a World Model Is Also a Proposal Mechanism
Oliver Obst, Frieder Stolzenburg
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study how CEM “world-model scores” affect both the current action choice and the candidate pools considered in later iterations. They perform crossed rescoring (multiple predictive surrogates score saved CEM pools) and a one-update splice (replace the first model-ranked elite refit with an environment-ranked one). On Walker and Cheetah, proposal distance decreases while ranking agreement stays near chance on Walker and declines on Cheetah; the one-update splice can lower realised selected-sequence cost.
Why it matters
Planning-time evaluation of world models often conflates “scoring/ordering” with “query shift” caused by the planner refitting its proposal distribution. This work separates (i) disagreement in rescoring within fixed candidate pools from (ii) the effect of changing the proposal-update history, using an intervention that replaces only the first elite refit.
Method
- Run CEM from a saved simulator state/context, save sampled candidate pools at each iteration, and score each saved pool with multiple predictive surrogates to form a crossed rescoring audit.
- Compare model elites vs environment-executed elites using elite precision (chance baseline) and pairwise order agreement and selected-candidate regret within the same pool.
- Use a one-update splice: fork after iteration 0, refit the next proposal using model-ranked vs environment-ranked elites, then continue with the original scorer and compare realised selected-sequence cost at the final iteration.
Limitation
The one-update splice is an oracle diagnostic rather than a deployable planner, because environment elites require realised costs for every candidate (and the experiment does not inspect intervention outcomes for deciding the intervention in the original six units).
Abstract (from arXiv)
The cross-entropy method (CEM) uses world-model scores to select action sequences and fit the distribution sampled in its next iteration. A scoring error can therefore change both the present decision and the candidates considered later. We evaluate these two roles separately. Four types of predictive model generate CEM traces, and every model rescores every saved candidate pool. Executing the same candidates in the environment provides a reference elite set and proposal update. Across twelve independently trained task-seed units on Walker and Cheetah, the pre-specified proposal distance falls from the first to the final CEM iteration in every unit. Proposal widths contract and fitted means separate relative to the remaining search width. Pairwise ranking agreement stays near chance on Walker and declines on Cheetah; elite-set agreement does not improve. This comparison shows greater variation between scorers than between pool sources on Cheetah; Walker has variation in both and in their pairings. We use the original six units to select Random nonlinear for a one-update intervention, without inspecting intervention outcomes. Replacing its first model-ranked update with an environment-ranked update lowers final realised selected-sequence cost in those six units and in six further units held out from the selection.
Related papers
- How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop
- When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models
- CAG-WAM: Causal Action-Gain World Action Models for Robot Manipulation
- How Many Candidates Does a One-Step World Model Need? A Random-Codebook Account of Nearest-of-m Training on Frozen DINO Tokens
- pico-WAM: Evaluating Counterfactual Prediction and Planning Error in a Compact World Action Model
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC
In CEM, a World Model Is Also a Proposal Mechanism
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.00921
Explicit source versions (1)
- v1 · 2026-10-01
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
The cross-entropy method (CEM) uses world-model scores to select action sequences and fit the distribution sampled in its next iteration. A scoring error can therefore change both the present decision and the candidates considered later. We evaluate these two roles separately. Four types of predictive model generate CEM traces, and every model rescores every saved candidate pool. Executing the same candidates in the environment provides a reference elite set and proposal update. Across twelve independently trained task-seed units on Walker and Cheetah, the pre-specified proposal distance falls from the first to the final CEM iteration in every unit. Proposal widths contract and fitted means separate relative to the remaining search width. Pairwise ranking agreement stays near chance on Walker and declines on Cheetah; elite-set agreement does not improve. This comparison shows greater variation between scorers than between pool sources on Cheetah; Walker has variation in both and in their pairings. We use the original six units to select Random nonlinear for a one-update intervention, without inspecting intervention outcomes. Replacing its first model-ranked update with an environment-ranked update lowers final realised selected-sequence cost in those six units and in six further units held out from the selection.
