Driving · 2026-11-10

LO-World: Unified BEV Latent Modeling for Joint 4D Occupancy and LiDAR Forecasting

Zhimin Liao, Ping Wei, Kai Mao, Shuaijia Chen

arXiv:2611.11912PDFCode

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

LO-World is a unified 4D world model that forecasts 3D semantic occupancy and raw LiDAR point clouds in a shared BEV latent space. It combines LO-VAE (hybrid range-voxel autoencoder) with a flow-matching generator (LO-Former) that caches historical context to reduce iterative sampling latency. The authors report occupancy IoU improvements and LiDAR Chamfer Distance reductions over prior baselines, plus real-time throughput on an RTX 4090, and release code.

Why it matters

The authors target joint 4D forecasting of two complementary modalities (occupancy for dense semantics; LiDAR for metric geometry) and argue that existing approaches either forecast them independently or cascade them heuristically. LO-World replaces multi-pipeline inference with a single generative stream in a shared BEV latent, aiming to enable cross-modal regularization. The paper reports improvements on nuScenes occupancy and LiDAR forecasting metrics and includes efficiency claims and publicly released code.

Method

  • Unified tokenization: LO-VAE projects semantic occupancy and a hybrid range-voxel LiDAR representation into a single BEV latent space using a shared encoder and variational bottleneck.
  • Single-stream decoding: a shared trunk decodes forecasted latents into both occupancy (with binary and semantic heads) and LiDAR (ray-space lifting with stop-gradient occupancy conditioning).
  • Efficient temporal generation: LO-Former uses conditional flow matching with cached historical context (context encoder computed once per rollout) and a lightweight iterative flow network (L_flow=4 vs L_ctx=8).

Limitation

The authors state failure cases under reverse ego-motion trajectories and note their training set contains no reverse-motion trajectories, indicating limited generalization to motion patterns outside the training distribution.

Abstract (from arXiv)

LiDAR point clouds and 3D semantic occupancy represent two distinct modalities capturing fine-grained metric geometry and dense volumetric semantics, respectively, holding natural potential for mutual complementarity. However, existing world models typically forecast these modalities independently or cascade them heuristically, missing the opportunity for reciprocal cross-modal regularization. To address this, we propose \textbf{LO-World}, a unified 4D world model that forecasts both modalities within a shared BEV latent space. This single-stream formulation enables dense semantics and metric geometry to mutually reinforce each other, while eliminating multi-pipeline redundancy. Specifically, LO-World couples an efficient hybrid range-voxel autoencoder with a flow-matching generator that caches historical context to reduce iterative sampling latency. Extensive experiments demonstrate that LO-World outperforms prior state-of-the-art baselines, improving occupancy forecasting IoU by 10.9% and reducing LiDAR Chamfer Distance by 30.1%. Meanwhile, it sustains a real-time throughput exceeding $25$~FPS on a single RTX 4090. Our code is available at https://github.com/lzzzzzm/LO-World.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-09 · Indexed by Pub-AI: 2026-11-10 UTC

preprint

LO-World: Unified BEV Latent Modeling for Joint 4D Occupancy and LiDAR Forecasting

Zhimin Liao, Ping Wei, Kai Mao, Shuaijia Chen

Preprint posted: 2026-11-09 · Original record ↗

arxiv
2611.11912
Explicit source versions (1)
  • v1 · 2026-11-09
  1. Preprint available

    Source date: 2026-11-09 · First detected: 2026-11-10 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

LiDAR point clouds and 3D semantic occupancy represent two distinct modalities capturing fine-grained metric geometry and dense volumetric semantics, respectively, holding natural potential for mutual complementarity. However, existing world models typically forecast these modalities independently or cascade them heuristically, missing the opportunity for reciprocal cross-modal regularization. To address this, we propose \textbf{LO-World}, a unified 4D world model that forecasts both modalities within a shared BEV latent space. This single-stream formulation enables dense semantics and metric geometry to mutually reinforce each other, while eliminating multi-pipeline redundancy. Specifically, LO-World couples an efficient hybrid range-voxel autoencoder with a flow-matching generator that caches historical context to reduce iterative sampling latency. Extensive experiments demonstrate that LO-World outperforms prior state-of-the-art baselines, improving occupancy forecasting IoU by 10.9% and reducing LiDAR Chamfer Distance by 30.1%. Meanwhile, it sustains a real-time throughput exceeding $25$~FPS on a single RTX 4090. Our code is available at https://github.com/lzzzzzm/LO-World.