Robotics · 2026-11-03

SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts

Amir Hossain Raj, Xuesu Xiao

arXiv:2611.01600PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

SCOUT is an occlusion-aware planning framework for cluttered manipulation. From sparse frontal RGB-D views, it reconstructs an object-centric hidden scene (GHOST) and derives a relational planning state with blocker relationships and accessibility corridors. An LLM proposes structured object-direction action candidates, and MCTS evaluates them using a learned action-conditioned world model with physical rollout predictions. The authors validate in simulation and on a Spot-based real shelf setup.

Why it matters

Planning in dense clutter often requires reasoning about geometry and free space that are not visible due to occlusions. The authors propose an object-centric hidden-scene belief plus action-conditioned world-model rollouts, enabling grounded long-horizon planning rather than relying only on visible-only reasoning.

Method

  • Reconstructs an object-centric hidden scene from frontal RGB-D using GHOST, producing visible slots plus hidden slots for unobserved structure, then renders geometry into occupancy and a relational graph.
  • Derives a relational planning state with blocker relationships, visibility indicators, and coarse spatial relations computed from slot geometry; the planner operates over this state while the world model evolves the slot state.
  • Uses an LLM to propose compact object-direction candidates, then runs MCTS over a reduced action set where a learned action-conditioned world model predicts action-conditioned future slot updates and rollout scores.

Limitation

The method’s learned world model predicts transitions using an abstraction targeting rigid-object, no-add/remove primitives; orientation is not explicitly represented, and visibility is recomputed from geometry.

Abstract (from arXiv)

Robots manipulating in cluttered spaces must often plan from partial views where foreground objects occlude the target, surrounding geometry, and usable free space. Reactive policies often miss these hidden constraints, while exhaustive search becomes computationally prohibitive in dense clutter. We present SCOUT, an occlusion-aware planning framework that bridges high-level deliberation with grounded physical rollouts. SCOUT leverages an object-centric scene representation to reconstruct hidden geometry and derive a relational planning state capturing blocker hierarchies and accessibility corridors. To navigate the vast action space in clutter, an LLM proposes structured action candidates which are then rigorously evaluated via Monte Carlo Tree Search (MCTS) using a learned action-conditioned world model. Evaluations on 'Reveal' and 'Placement' tasks show that SCOUT outperforms Vision-Language-Action (VLA) policies, embodied agents, and 3D world models, with significant gains in long-horizon tasks requiring reasoning over occluded geometry. We further validate the framework's robustness through real-world robotic manipulation in a cluttered shelf environment.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC

preprint

SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts

Amir Hossain Raj, Xuesu Xiao

First announced by arXiv: 2026-11-03 · Original record ↗

arxiv
2611.01600
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.01600v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Robots manipulating in cluttered spaces must often plan from partial views where foreground objects occlude the target, surrounding geometry, and usable free space. Reactive policies often miss these hidden constraints, while exhaustive search becomes computationally prohibitive in dense clutter. We present SCOUT, an occlusion-aware planning framework that bridges high-level deliberation with grounded physical rollouts. SCOUT leverages an object-centric scene representation to reconstruct hidden geometry and derive a relational planning state capturing blocker hierarchies and accessibility corridors. To navigate the vast action space in clutter, an LLM proposes structured action candidates which are then rigorously evaluated via Monte Carlo Tree Search (MCTS) using a learned action-conditioned world model. Evaluations on 'Reveal' and 'Placement' tasks show that SCOUT outperforms Vision-Language-Action (VLA) policies, embodied agents, and 3D world models, with significant gains in long-horizon tasks requiring reasoning over occluded geometry. We further validate the framework's robustness through real-world robotic manipulation in a cluttered shelf environment.