Driving · 2026-11-10
LO-World: Unified BEV Latent Modeling for Joint 4D Occupancy and LiDAR Forecasting
Zhimin Liao, Ping Wei, Kai Mao, Shuaijia Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
LO-World is a unified 4D world model that forecasts 3D semantic occupancy and raw LiDAR point clouds in a shared BEV latent space. It combines LO-VAE (hybrid range-voxel autoencoder) with a flow-matching generator (LO-Former) that caches historical context to reduce iterative sampling latency. The authors report occupancy IoU improvements and LiDAR Chamfer Distance reductions over prior baselines, plus real-time throughput on an RTX 4090, and release code.
Why it matters
The authors target joint 4D forecasting of two complementary modalities (occupancy for dense semantics; LiDAR for metric geometry) and argue that existing approaches either forecast them independently or cascade them heuristically. LO-World replaces multi-pipeline inference with a single generative stream in a shared BEV latent, aiming to enable cross-modal regularization. The paper reports improvements on nuScenes occupancy and LiDAR forecasting metrics and includes efficiency claims and publicly released code.
Method
- Unified tokenization: LO-VAE projects semantic occupancy and a hybrid range-voxel LiDAR representation into a single BEV latent space using a shared encoder and variational bottleneck.
- Single-stream decoding: a shared trunk decodes forecasted latents into both occupancy (with binary and semantic heads) and LiDAR (ray-space lifting with stop-gradient occupancy conditioning).
- Efficient temporal generation: LO-Former uses conditional flow matching with cached historical context (context encoder computed once per rollout) and a lightweight iterative flow network (L_flow=4 vs L_ctx=8).
Limitation
The authors state failure cases under reverse ego-motion trajectories and note their training set contains no reverse-motion trajectories, indicating limited generalization to motion patterns outside the training distribution.
Abstract (from arXiv)
LiDAR point clouds and 3D semantic occupancy represent two distinct modalities capturing fine-grained metric geometry and dense volumetric semantics, respectively, holding natural potential for mutual complementarity. However, existing world models typically forecast these modalities independently or cascade them heuristically, missing the opportunity for reciprocal cross-modal regularization. To address this, we propose \textbf{LO-World}, a unified 4D world model that forecasts both modalities within a shared BEV latent space. This single-stream formulation enables dense semantics and metric geometry to mutually reinforce each other, while eliminating multi-pipeline redundancy. Specifically, LO-World couples an efficient hybrid range-voxel autoencoder with a flow-matching generator that caches historical context to reduce iterative sampling latency. Extensive experiments demonstrate that LO-World outperforms prior state-of-the-art baselines, improving occupancy forecasting IoU by 10.9% and reducing LiDAR Chamfer Distance by 30.1%. Meanwhile, it sustains a real-time throughput exceeding $25$~FPS on a single RTX 4090. Our code is available at https://github.com/lzzzzzm/LO-World.
Related papers
- Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and Sensors
- Let LiDAR Decide Where to Look: Geometry-Measured Token Selection for Efficient Sparse Latent World Models
- Robust LIDAR Sensor-to-Sensor Domain Generalization via Accumulated-Point Scene Completion
- InfraWorld: Motion-Aware World Modeling for Roadside 4D Occupancy Forecasting
- UFMF: Radar-Guided Kinematic Priors for Real-Time 4D Occupancy Forecasting
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-09 · Indexed by Pub-AI: 2026-11-10 UTC
LO-World: Unified BEV Latent Modeling for Joint 4D Occupancy and LiDAR Forecasting
Preprint posted: 2026-11-09 · Original record ↗
- arxiv
- 2611.11912
Explicit source versions (1)
- v1 · 2026-11-09
- Preprint available
Source date: 2026-11-09 · First detected: 2026-11-10 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.11912v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
LiDAR point clouds and 3D semantic occupancy represent two distinct modalities capturing fine-grained metric geometry and dense volumetric semantics, respectively, holding natural potential for mutual complementarity. However, existing world models typically forecast these modalities independently or cascade them heuristically, missing the opportunity for reciprocal cross-modal regularization. To address this, we propose \textbf{LO-World}, a unified 4D world model that forecasts both modalities within a shared BEV latent space. This single-stream formulation enables dense semantics and metric geometry to mutually reinforce each other, while eliminating multi-pipeline redundancy. Specifically, LO-World couples an efficient hybrid range-voxel autoencoder with a flow-matching generator that caches historical context to reduce iterative sampling latency. Extensive experiments demonstrate that LO-World outperforms prior state-of-the-art baselines, improving occupancy forecasting IoU by 10.9% and reducing LiDAR Chamfer Distance by 30.1%. Meanwhile, it sustains a real-time throughput exceeding $25$~FPS on a single RTX 4090. Our code is available at https://github.com/lzzzzzm/LO-World.
