Unassigned · 2026-11-13
Spatial Foundry: A Multimodal Dataset for Spatial Intelligence
Ethan Lin, Yang Song
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Spatial Foundry is a dataset of 5,280 furnished residential 3D scene configurations with aligned floor plans, perspective/panoramic RGB, metric depth, point clouds, meshes, instance-level 3D boxes, and camera trajectory–video pairs. Shared scene coordinates and editable source scenes support independent variation of camera motion vs scene edits for spatial perception and shared world modeling.
Why it matters
The authors report a way to connect visual observations to spatial geometry, object persistence, camera motion, and controlled scene changes under a common reference. By recording calibrated observations, persistent instance identities, trajectories with poses/timestamps, and explicit before/after edit records, the dataset enables reproducible comparisons and evaluation for tasks like floor-plan-conditioned 3D generation, reconstruction, instance perception, camera-conditioned video generation, and state-conditioned scene editing.
Method
- Provide 5,280 furnished residential 3D configurations from U.S. priors: GPT-6 generates structured floor plans; human review checks spatial usability; Blender assembly with Infinigen assets produces editable source scenes.
- Render aligned multimodal observations: perspective/panoramic RGB, calibrated metric depth, point clouds/meshes, instance-level 3D bounding boxes, and trajectory–video pairs with per-frame camera pose and timestamps.
- Use a controlled organization principle: separate camera variation from environment variation, enabling fixed-scene camera sequences and fixed-camera before/after edit pairs; shared instance identifiers link cross-view and temporal correspondences.
Abstract (from arXiv)
Spatial intelligence requires connecting visual observations to the geometry, objects, and persistent state of an environment. Images alone leave these relationships implicit, while disconnected annotations make it difficult to study appearance, camera motion, and scene changes under a common reference. We introduce Spatial Foundry, a collection of 5,280 furnished residential 3D scene configurations with editable assets and aligned visual, geometric, and temporal data. Its modalities include floor plans, perspective and panoramic RGB, metric depth, point clouds, meshes, instance-level 3D bounding boxes, and camera trajectories paired with video. Shared scene coordinates link observations across views and time; editable source scenes enable additional camera paths and controlled object changes. We describe the data representation and construction workflow, and discuss applications in floor-plan-conditioned 3D generation, geometric reconstruction, instance-level perception, camera-conditioned video generation, shared world modeling, and scene editing. Controlled camera and scene variations support spatial perception and world-model research. The public project page provides videos and an interactive colored point-cloud preview.
Related papers
- WorldMind: Test-Time Scaling for Visual Spatial Reasoning through Latent World Simulation
- Uruqi: Learning Spatial Cognition from Visual Experience
- Representing the Unseen: Rethinking Visual Imagination for Spatial Reasoning
- Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editing
- SceneBloom: Agentic Indoor Environment Generation with Realistic and Diverse Layouts for Embodied Learning
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-12 · Indexed by Pub-AI: 2026-11-13 UTC
Spatial Foundry: A Multimodal Dataset for Spatial Intelligence
Preprint posted: 2026-11-12 · Original record ↗
- arxiv
- 2611.17318
Explicit source versions (1)
- v1 · 2026-11-12
- Preprint available
Source date: 2026-11-12 · First detected: 2026-11-13 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.17318v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Spatial intelligence requires connecting visual observations to the geometry, objects, and persistent state of an environment. Images alone leave these relationships implicit, while disconnected annotations make it difficult to study appearance, camera motion, and scene changes under a common reference. We introduce Spatial Foundry, a collection of 5,280 furnished residential 3D scene configurations with editable assets and aligned visual, geometric, and temporal data. Its modalities include floor plans, perspective and panoramic RGB, metric depth, point clouds, meshes, instance-level 3D bounding boxes, and camera trajectories paired with video. Shared scene coordinates link observations across views and time; editable source scenes enable additional camera paths and controlled object changes. We describe the data representation and construction workflow, and discuss applications in floor-plan-conditioned 3D generation, geometric reconstruction, instance-level perception, camera-conditioned video generation, shared world modeling, and scene editing. Controlled camera and scene variations support spatial perception and world-model research. The public project page provides videos and an interactive colored point-cloud preview.
