Driving · 2026-10-28

How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop

Haojia Gao, Yue Cao, Yumin Liu, Haohua Que, Mingkai Liu, Guanqiu Guo, Zhuo Zou, Bokui Chen, Xueqian Wang

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors argue that standard best-of-N selectable-headroom estimates confound candidate quality with randomness of future continuation. They introduce a denoised counterfactual branching protocol with repeated continuations, held-out oracles, and common random numbers to measure the selectable share. Using four WAMs, they find selection headroom depends strongly on structured action proposals, and an imagination-based ranker with a fail gate improves closed-loop task success.

Why it matters

Test-time selection is a natural way to improve fixed world action policies, but the authors show that common “single-continuation oracle” estimates overstate what selection can realize. By measuring what fraction of oracle gain is actually selectable and where it comes from, the work provides a concrete recipe for allocating imagination compute (try consequential perturbations, rank with branch-trained readouts, and use held-out estimates to anticipate gains) and demonstrates closed-loop improvements.

Method

Introduce a denoised counterfactual protocol using bitwise-reproducible branching from MuJoCo snapshots, repeated continuations, held-out oracles (independent selection/scoring continuations), and common random numbers to compare candidates fairly.
Decompose variance into candidate effect vs continuation randomness/interaction, and use held-out share to estimate selectable headroom over a single-continuation oracle.
Train a fail-gated selector (gate + ranker) that deviates from the policy chunk only when a WAM readout predicts failure, and otherwise executes the policy chunk unchanged.

Abstract (from arXiv)

World action models (WAMs) predict the visual consequences of the actions they generate, which makes test-time selection a natural way to improve a fixed policy: imagine the futures of candidate actions and execute the best one. The benefit of selection is bounded by the selectable headroom, how much better the best candidate truly is, but the standard estimate, which executes each candidate once and keeps the best outcome, conflates candidate quality with the randomness of everything that follows. We introduce a denoised counterfactual protocol, based on bitwise-reproducible branching, repeated continuations, held-out oracles, and common random numbers, that measures the headroom a selector can realize, and use its measurements to build a fail-gated selector. Across four WAMs, two benchmarks, over 100,000 branched rollouts, and pre-registered tests, we find that (i) the standard estimate overstates the selectable headroom of native samples: bounds from 17 continuations per candidate place its selectable share between 24% and 66% for three WAMs; (ii) structured action perturbations create up to 16 points of selectable headroom; and (iii) in both WAMs with action-conditioned imagination, imagined futures carry candidate-outcome information with no detectable loss relative to real futures, information that the value head of Cosmos Policy leaves unused. Deviating from the policy chunk only when a readout of the WAM predicts failure, the selector raises closed-loop task success from 76.9% to 81.6% on held-out episodes of training tasks, with the largest gain where the policy fails most often, and from 62.6% to 69.8% on unseen tasks.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-27 · Indexed by Pub-AI: 2026-10-28 UTC

preprint