Latent Dynamics · 2026-10-09
CRISP: Fixing Flying Pixels in Latent LiDAR Generation via Diffusion Decoding
Andrea Ceron, Michael Schmidt, Alvaro Marcos-Ramiro, Sebastian Schmidt, Benjamin Busam
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
CRISP replaces the decoder in latent LiDAR pipelines with a pixel-space diffusion decoder to fix “flying pixels” from convolutional VAE decoders blurring depth discontinuities. It uses a backbone-agnostic latent adapter, a DiT denoiser, and a support mask predictor, while keeping the encoder/latent generator fixed. The authors report large reductions in FSVD/FPVD/FRID and improvements in edge geometry, including zero-shot decoder swaps inside a pretrained LiDM world model.
Why it matters
Decoder oversmoothing is identified as a directly correctable bottleneck in latent LiDAR generation: blurred depth boundaries back-project to floating 3D points. CRISP targets this stage specifically, enabling decoder-only replacements across frozen latent backbones and improving paired 3D metrics and distributional metrics, including in a pretrained LiDM world model.
Method
- Identifies convolutional VAE/LiDAR decoders as a geometry bottleneck causing flying pixels at depth discontinuities, proposing a decoder-only fix.
- Introduces CRISP: a pixel-space diffusion decoder with a backbone-agnostic latent adapter (latent tokens via channel projection + token-length reduction), DiT denoiser, and support mask predictor for sparse valid returns.
- Uses a geometry-aware diffusion objective (valid-only boundary-aware velocity regression plus spatial regularizers) and composes dense depth with predicted validity masks.
Limitation
The decoder swap yields the only perceptual exception noted as a small FPVD regression on the LiDM backbone.
Abstract (from arXiv)
Latent LiDAR pipelines suffer from flying pixels: convolutional VAEs blur sharp radial depth discontinuities, yielding edge depths that back-project to points floating between surfaces. We identify this as a major, directly correctable decoder bottleneck and introduce CRISP: a pixel-space diffusion decoder with a backbone-agnostic latent adapter, DiT-based denoiser, and support mask predictor. CRISP replaces video-VAE and LiDAR-native decoders alike while keeping the encoder and latent generator fixed. Across KITTI-360, SemanticKITTI, and nuScenes, replacing only the decoder reduces FSVD/FPVD by 50.5% on average across frozen backbones; for generic video VAEs, the reductions reach 71%/74%. On the LiDAR-native LiDM backbone, FRID drops by 71%, with the largest gains at depth discontinuities. In a pretrained LiDM world model, the same zero-shot replacement improves FSVD by 15.5%, narrowing the sim-to-real gap. Code and checkpoints will be released at https://andrea25512.github.io/CRISP/.
Related papers
- Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and Sensors
- Let LiDAR Decide Where to Look: Geometry-Measured Token Selection for Efficient Sparse Latent World Models
- LO-World: Unified BEV Latent Modeling for Joint 4D Occupancy and LiDAR Forecasting
- Robust LIDAR Sensor-to-Sensor Domain Generalization via Accumulated-Point Scene Completion
- Cache-Aware Conv3D Lowering Across Embedded World-Model Decoders
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC
CRISP: Fixing Flying Pixels in Latent LiDAR Generation via Diffusion Decoding
Preprint posted: 2026-10-08 · Original record ↗
- arxiv
- 2610.11376
Explicit source versions (1)
- v2 · 2026-10-20
- Preprint available
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (3)
- Official evidence ↗ · OAI arXiv created; not header datestamp (official metadata)
- Official evidence ↗ · atom:published (official metadata)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · atom:id explicit version (official metadata)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Latent LiDAR pipelines suffer from flying pixels: convolutional VAEs blur sharp radial depth discontinuities, yielding edge depths that back-project to points floating between surfaces. We identify this as a major, directly correctable decoder bottleneck and introduce CRISP: a pixel-space diffusion decoder with a backbone-agnostic latent adapter, DiT-based denoiser, and support mask predictor. CRISP replaces video-VAE and LiDAR-native decoders alike while keeping the encoder and latent generator fixed. Across KITTI-360, SemanticKITTI, and nuScenes, replacing only the decoder reduces FSVD/FPVD by 50.5% on average across frozen backbones; for generic video VAEs, the reductions reach 71%/74%. On the LiDAR-native LiDM backbone, FRID drops by 71%, with the largest gains at depth discontinuities. In a pretrained LiDM world model, the same zero-shot replacement improves FSVD by 15.5%, narrowing the sim-to-real gap. Code and checkpoints will be released at https://andrea25512.github.io/CRISP/.
