Robotics · 2026-11-03
SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts
Amir Hossain Raj, Xuesu Xiao
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SCOUT is an occlusion-aware planning framework for cluttered manipulation. From sparse frontal RGB-D views, it reconstructs an object-centric hidden scene (GHOST) and derives a relational planning state with blocker relationships and accessibility corridors. An LLM proposes structured object-direction action candidates, and MCTS evaluates them using a learned action-conditioned world model with physical rollout predictions. The authors validate in simulation and on a Spot-based real shelf setup.
Why it matters
Planning in dense clutter often requires reasoning about geometry and free space that are not visible due to occlusions. The authors propose an object-centric hidden-scene belief plus action-conditioned world-model rollouts, enabling grounded long-horizon planning rather than relying only on visible-only reasoning.
Method
- Reconstructs an object-centric hidden scene from frontal RGB-D using GHOST, producing visible slots plus hidden slots for unobserved structure, then renders geometry into occupancy and a relational graph.
- Derives a relational planning state with blocker relationships, visibility indicators, and coarse spatial relations computed from slot geometry; the planner operates over this state while the world model evolves the slot state.
- Uses an LLM to propose compact object-direction candidates, then runs MCTS over a reduced action set where a learned action-conditioned world model predicts action-conditioned future slot updates and rollout scores.
Limitation
The method’s learned world model predicts transitions using an abstraction targeting rigid-object, no-add/remove primitives; orientation is not explicitly represented, and visibility is recomputed from geometry.
Abstract (from arXiv)
Robots manipulating in cluttered spaces must often plan from partial views where foreground objects occlude the target, surrounding geometry, and usable free space. Reactive policies often miss these hidden constraints, while exhaustive search becomes computationally prohibitive in dense clutter. We present SCOUT, an occlusion-aware planning framework that bridges high-level deliberation with grounded physical rollouts. SCOUT leverages an object-centric scene representation to reconstruct hidden geometry and derive a relational planning state capturing blocker hierarchies and accessibility corridors. To navigate the vast action space in clutter, an LLM proposes structured action candidates which are then rigorously evaluated via Monte Carlo Tree Search (MCTS) using a learned action-conditioned world model. Evaluations on 'Reveal' and 'Placement' tasks show that SCOUT outperforms Vision-Language-Action (VLA) policies, embodied agents, and 3D world models, with significant gains in long-horizon tasks requiring reasoning over occluded geometry. We further validate the framework's robustness through real-world robotic manipulation in a cluttered shelf environment.
Related papers
- ForeAct: Geometry-Guided Action Learning from World Representations
- OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching
- ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies
- WorldSketch: Sketching World Models for Zero-Shot Mobile Manipulation
- UniFlowWAM: A Unified World-Action Model with Visual Semantics and 4D Geometry
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC
SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts
First announced by arXiv: 2026-11-03 · Original record ↗
- arxiv
- 2611.01600
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.01600v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Robots manipulating in cluttered spaces must often plan from partial views where foreground objects occlude the target, surrounding geometry, and usable free space. Reactive policies often miss these hidden constraints, while exhaustive search becomes computationally prohibitive in dense clutter. We present SCOUT, an occlusion-aware planning framework that bridges high-level deliberation with grounded physical rollouts. SCOUT leverages an object-centric scene representation to reconstruct hidden geometry and derive a relational planning state capturing blocker hierarchies and accessibility corridors. To navigate the vast action space in clutter, an LLM proposes structured action candidates which are then rigorously evaluated via Monte Carlo Tree Search (MCTS) using a learned action-conditioned world model. Evaluations on 'Reveal' and 'Placement' tasks show that SCOUT outperforms Vision-Language-Action (VLA) policies, embodied agents, and 3D world models, with significant gains in long-horizon tasks requiring reasoning over occluded geometry. We further validate the framework's robustness through real-world robotic manipulation in a cluttered shelf environment.
