Driving · 2026-11-11

Incorporating World Models in Decision Making for Next Best View Selection in Active Sensing

Nikita Sarawgi, Omey M. Manyar, Satyandra K. Gupta

arXiv:2611.13229PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors present a deployment adaptation method for next-best-view (NBV) selection that reuses a fixed action-conditioned world-model predictor but learns a local sensor-return adapter from limited calibration data. The adapter estimates measurement dropout over predicted surfaces and filters candidates accordingly. On a real robot with a structured-light sensor, deployment-aware selection achieves higher measured surface coverage with 60% fewer captures than nominal selection.

Why it matters

Robotic active sensing can fail when predicted visible surfaces do not yield valid depth measurements under local sensor-return conditions. By keeping the pretrained predictor fixed and learning only how the deployed sensor returns measurements, the method reduces wasted captures and corrects repeated acquisition failures in data-scarce deployment settings.

Method

  • Keep a pretrained action-conditioned world-model predictor fixed to propose candidate views and expected visible surface geometry for NBV selection.
  • Learn a local deployment adapter from limited calibration captures to predict measurement dropout/return probability from features (viewing distance, incidence angle, normalized intensity).
  • Score candidate poses by filtering predicted points using estimated dropout so selection favors views likely to produce valid depth and fill gaps in measured reconstruction.
Abstract (from arXiv)

Efficient robotic 3D reconstruction requires selecting views that contribute new surface measurements with few physical captures. Pretrained world models can guide this choice, but a predicted surface may not produce valid depth measurements under the operating conditions of the deployed sensor. We present a deployment adaptation methodology for next-best-view selection that learns a sensor-return adapter from limited local calibration data while keeping the pretrained predictor fixed. The adapter estimates measurement dropout over each predicted surface, allowing the selection method to favor views likely to fill gaps in the measured reconstruction. In a real-robot experiment with a structured-light sensor, the adapted method achieves higher measured surface coverage with 60% fewer captures than nominal selection. The adapted method recovers a surface region that the nominal method repeatedly targets but fails to measure. These results suggest that robots can reuse pretrained predictive models while adapting their decisions to local operating conditions in data-scarce environments.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-11 · Indexed by Pub-AI: 2026-11-11 UTC

preprint

Incorporating World Models in Decision Making for Next Best View Selection in Active Sensing

Nikita Sarawgi, Omey M. Manyar, Satyandra K. Gupta

First announced by arXiv: 2026-11-11 · Original record ↗

arxiv
2611.13229
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-11 · First detected: 2026-11-11 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.13229v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Efficient robotic 3D reconstruction requires selecting views that contribute new surface measurements with few physical captures. Pretrained world models can guide this choice, but a predicted surface may not produce valid depth measurements under the operating conditions of the deployed sensor. We present a deployment adaptation methodology for next-best-view selection that learns a sensor-return adapter from limited local calibration data while keeping the pretrained predictor fixed. The adapter estimates measurement dropout over each predicted surface, allowing the selection method to favor views likely to fill gaps in the measured reconstruction. In a real-robot experiment with a structured-light sensor, the adapted method achieves higher measured surface coverage with 60% fewer captures than nominal selection. The adapted method recovers a surface region that the nominal method repeatedly targets but fails to measure. These results suggest that robots can reuse pretrained predictive models while adapting their decisions to local operating conditions in data-scarce environments.