Driving · 2026-11-10

ScanDreamer: A Map-Grounded World Model for Quadrotor Navigation

Yunyi Gao, Yixing Shi, Yize Liu, Hai-Duong Tran, Nhat-Hoang P. Nguyen, Shuai Wang, Yi Deng, Shengming Liu, Sim Kuan Goh, Duc-Thinh Pham, Sameer Alam

arXiv:2611.11605PDFCode

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

ScanDreamer is a map-grounded world model for quadrotor navigation from forward depth images. It builds an online 2D scan-memory occupancy map from depth and pose, conditions the world model/actor/critic on map crops, and grounds imagination by propagating pose over a frozen map to compute geodesic progress and wall-contact signals. It also adds a forward-cone penalty to discourage horizontal motion outside the camera viewing direction.

Why it matters

The authors address data-intensive world-model navigation and lack of explicit geometric memory and sensing-direction awareness. By combining an online occupancy map with map-grounded imagination and perception-aware motion penalties/rewards, ScanDreamer improves success and reduces collisions in unseen simulated layouts under the same online-interaction budget.

Method

  • Constructs a persistent online 2D scan-memory occupancy map from depth pixels and vehicle pose; updates via back-projection and ray termination, and computes a geodesic cost-to-go field using Dijkstra on the map.
  • Conditions world model, actor, and critic on heading-aligned local map crops; computes geodesic progress from the map cost-to-go and includes an arrival handling procedure.
  • In imagination, propagates quadrotor pose over a frozen map and derives progress, known-wall contacts/termination, and a forward-cone penalty analytically from mapped geometry and camera-axis constraints.

Limitation

Future work will first address more diverse and more realistic environments. The authors note that their 2-D map suits obstacles spanning the flight band, while real buildings and subterranean sites may contain overhanging obstacles that call for a 3-D map.

Abstract (from arXiv)

Quadrotors inspecting buildings and tunnels must navigate unknown clutter with a forward-facing depth camera whose limited field of view constrains scene understanding and collision-avoiding navigation. Recent world-model agents learn navigation through imagined experience (e.g., AirDreamer), but remain data-intensive and often lack explicit geometric memory and awareness of what the camera can observe. We propose ScanDreamer, a map-grounded world model that constructs an online 2D scan-memory map from depth images and vehicle pose. Our world model, actor, and critic are conditioned on this map, which also defines a geodesic progress reward. Moreover, a forward-cone penalty discourages horizontal motion outside the camera's viewing direction. During imagination, ScanDreamer propagates the quadrotor pose over a frozen map and derives progress and wall-contact signals from the mapped geometry. We train ScanDreamer for 0.8 million environment steps (5.5 h on one GPU) and evaluate it in NVIDIA Isaac Sim over 1560 episodes spanning unseen layouts and parameter ranges. Across three seeds (mean $\pm$ SD), ScanDreamer achieves 96.3 $\pm$ 0.3% success with 3.6 $\pm$ 0.3% collision, compared with 59.6 $\pm$ 3.0% success and 28.8 $\pm$ 2.8% collision for AirDreamer at the same online-interaction budget. Removing the forward-cone penalty increases collisions to 9.9 $\pm$ 3.1%. These results show that combining explicit map memory, map-grounded imagination, and perception-aware motion substantially improves navigation performance and data efficiency. The source code is available at https://github.com/WANGSHUAIzzzz/ScanDreamer.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC

preprint

ScanDreamer: A Map-Grounded World Model for Quadrotor Navigation

Yunyi Gao, Yixing Shi, Yize Liu, Hai-Duong Tran, Nhat-Hoang P. Nguyen, Shuai Wang, Yi Deng, Shengming Liu, Sim Kuan Goh, Duc-Thinh Pham, Sameer Alam

First announced by arXiv: 2026-11-10 · Original record ↗

arxiv
2611.11605
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.11605v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Quadrotors inspecting buildings and tunnels must navigate unknown clutter with a forward-facing depth camera whose limited field of view constrains scene understanding and collision-avoiding navigation. Recent world-model agents learn navigation through imagined experience (e.g., AirDreamer), but remain data-intensive and often lack explicit geometric memory and awareness of what the camera can observe. We propose ScanDreamer, a map-grounded world model that constructs an online 2D scan-memory map from depth images and vehicle pose. Our world model, actor, and critic are conditioned on this map, which also defines a geodesic progress reward. Moreover, a forward-cone penalty discourages horizontal motion outside the camera's viewing direction. During imagination, ScanDreamer propagates the quadrotor pose over a frozen map and derives progress and wall-contact signals from the mapped geometry. We train ScanDreamer for 0.8 million environment steps (5.5 h on one GPU) and evaluate it in NVIDIA Isaac Sim over 1560 episodes spanning unseen layouts and parameter ranges. Across three seeds (mean $\pm$ SD), ScanDreamer achieves 96.3 $\pm$ 0.3% success with 3.6 $\pm$ 0.3% collision, compared with 59.6 $\pm$ 3.0% success and 28.8 $\pm$ 2.8% collision for AirDreamer at the same online-interaction budget. Removing the forward-cone penalty increases collisions to 9.9 $\pm$ 3.1%. These results show that combining explicit map memory, map-grounded imagination, and perception-aware motion substantially improves navigation performance and data efficiency. The source code is available at https://github.com/WANGSHUAIzzzz/ScanDreamer.