Driving · 2026-10-19
DriveWorld3D Evidence Constrained 3D World Modeling and Generative Simulation for Real-World Autonomous Driving
Wei Wang (Robostreet research team Cambridge, MA, USA), Yiru Veronika Wang (Robostreet research team Cambridge, MA, USA), Yihan Wang (Robostreet research team Cambridge, MA, USA), Zihao Li (Robostreet research team Cambridge, MA, USA), Xiaohui Liang (Robostreet research team Cambridge, MA, USA)
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
DriveWorld3D is a 3D world-model framework for autonomous driving that maintains a unified world state (road surfaces, volumetric occupancy, dynamic entities, road relations) while separately representing occupancy, observability/unknown space, and evidence provenance. It uses action-conditioned future prediction and risk-sensitive planning online, and offline reconstruction plus provenance-restricted generative scenario expansion for evaluation and stress tests.
Why it matters
The authors argue that visual fidelity alone is not enough for driving reliability because generated content may be unsupported by measurements. DriveWorld3D aims to reduce dangerous false free-space and occlusion risk by explicitly managing evidence provenance and unknown space, while testing outcomes in paired real-to-simulation closed loops and accounting for runtime/latency constraints.
Method
- Online unified 3D state with separate layers for road surfaces, volumetric occupancy, dynamic entities, and road relations; plus separate modeling of uncertainty/observability and evidence provenance.
- Predict occupancy probability and evidence sufficiency separately so unobserved space is treated as unknown (epistemic), not automatically free; include sensor health, timing alignment, and measurement checks.
- Offline: reconstruct scenes using NeRF or 3D Gaussian Splatting, then expand long-tail conditions with generative models while restricting generated content from being used as geometric ground truth; screen variants via geometry/provenance checks.
Limitation
The system has not yet been implemented or validated on a vehicle; no performance improvements are reported, and the work does not reproduce a production driving system or establish a safety certification.
Abstract (from arXiv)
Three dimensional modeling for autonomous driving must continuously estimate road geometry, obstacle occupancy, traffic participant behavior, and observational uncertainty under limited computational resources. Existing reconstruction methods can synthesize realistic novel views, but visual fidelity does not sufficiently characterize the geometric reliability required for driving. Generative world models can expand scenarios and predict future observations, yet may introduce content unsupported by measurements. We propose DriveWorld3D, a framework combining temporal video learning, measurement constraints from multiple sensors, and offline generative simulation. Its online component maintains a unified world state comprising road surfaces, volumetric occupancy, dynamic entities, and road relations, while separately representing physical occupancy, observability, and evidence provenance. Action conditioned future prediction and risk sensitive planning connect this representation to driving decisions. Its offline component reconstructs recorded scenes using neural radiance fields or 3D Gaussian Splatting, then expands long tail conditions using generative models while restricting the use of generated content as geometric ground truth. We specify data partitioning, sensor ablations, fault injection, paired closed loop testing, and real to simulation consistency assessment. The central hypothesis is that explicit evidence management can reduce dangerous false free space predictions and occlusion risk within a matched computational budget while retaining acceptable driving efficiency. All performance claims remain subject to experimental validation. Keywords: autonomous driving; 3D occupancy; sensor fusion; uncertainty; generative world models; NeRF; 3D Gaussian Splatting; closed loop simulation
Related papers
- GeoV2V: Geometry-Grounded Video Diffusion Model for Driving Scene Generation
- DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
- SceneFactory-3D: Lifting 2D Traffic Scenes into 3D Physical Counterfactuals for Scalable Physically Grounded Safety Evaluation
- VGGTWorld-VLA: Intent-Conditioned 3D World Evolution for Autonomous Driving
- GAIA-1: A Generative World Model for Autonomous Driving
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-15 · Indexed by Pub-AI: 2026-10-19 UTC
DriveWorld3D Evidence Constrained 3D World Modeling and Generative Simulation for Real-World Autonomous Driving
Preprint posted: 2026-10-15 · Original record ↗
- arxiv
- 2610.22450
Explicit source versions (1)
- v1 · 2026-10-15
- Preprint available
Source date: 2026-10-15 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Three dimensional modeling for autonomous driving must continuously estimate road geometry, obstacle occupancy, traffic participant behavior, and observational uncertainty under limited computational resources. Existing reconstruction methods can synthesize realistic novel views, but visual fidelity does not sufficiently characterize the geometric reliability required for driving. Generative world models can expand scenarios and predict future observations, yet may introduce content unsupported by measurements. We propose DriveWorld3D, a framework combining temporal video learning, measurement constraints from multiple sensors, and offline generative simulation. Its online component maintains a unified world state comprising road surfaces, volumetric occupancy, dynamic entities, and road relations, while separately representing physical occupancy, observability, and evidence provenance. Action conditioned future prediction and risk sensitive planning connect this representation to driving decisions. Its offline component reconstructs recorded scenes using neural radiance fields or 3D Gaussian Splatting, then expands long tail conditions using generative models while restricting the use of generated content as geometric ground truth. We specify data partitioning, sensor ablations, fault injection, paired closed loop testing, and real to simulation consistency assessment. The central hypothesis is that explicit evidence management can reduce dangerous false free space predictions and occlusion risk within a matched computational budget while retaining acceptable driving efficiency. All performance claims remain subject to experimental validation. Keywords: autonomous driving; 3D occupancy; sensor fusion; uncertainty; generative world models; NeRF; 3D Gaussian Splatting; closed loop simulation