Latent Dynamics · 2026-10-09

CRISP: Fixing Flying Pixels in Latent LiDAR Generation via Diffusion Decoding

Andrea Ceron, Michael Schmidt, Alvaro Marcos-Ramiro, Sebastian Schmidt, Benjamin Busam

arXiv:2610.11376PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

CRISP replaces the decoder in latent LiDAR pipelines with a pixel-space diffusion decoder to fix “flying pixels” from convolutional VAE decoders blurring depth discontinuities. It uses a backbone-agnostic latent adapter, a DiT denoiser, and a support mask predictor, while keeping the encoder/latent generator fixed. The authors report large reductions in FSVD/FPVD/FRID and improvements in edge geometry, including zero-shot decoder swaps inside a pretrained LiDM world model.

Why it matters

Decoder oversmoothing is identified as a directly correctable bottleneck in latent LiDAR generation: blurred depth boundaries back-project to floating 3D points. CRISP targets this stage specifically, enabling decoder-only replacements across frozen latent backbones and improving paired 3D metrics and distributional metrics, including in a pretrained LiDM world model.

Method

  • Identifies convolutional VAE/LiDAR decoders as a geometry bottleneck causing flying pixels at depth discontinuities, proposing a decoder-only fix.
  • Introduces CRISP: a pixel-space diffusion decoder with a backbone-agnostic latent adapter (latent tokens via channel projection + token-length reduction), DiT denoiser, and support mask predictor for sparse valid returns.
  • Uses a geometry-aware diffusion objective (valid-only boundary-aware velocity regression plus spatial regularizers) and composes dense depth with predicted validity masks.

Limitation

The decoder swap yields the only perceptual exception noted as a small FPVD regression on the LiDM backbone.

Abstract (from arXiv)

Latent LiDAR pipelines suffer from flying pixels: convolutional VAEs blur sharp radial depth discontinuities, yielding edge depths that back-project to points floating between surfaces. We identify this as a major, directly correctable decoder bottleneck and introduce CRISP: a pixel-space diffusion decoder with a backbone-agnostic latent adapter, DiT-based denoiser, and support mask predictor. CRISP replaces video-VAE and LiDAR-native decoders alike while keeping the encoder and latent generator fixed. Across KITTI-360, SemanticKITTI, and nuScenes, replacing only the decoder reduces FSVD/FPVD by 50.5% on average across frozen backbones; for generic video VAEs, the reductions reach 71%/74%. On the LiDAR-native LiDM backbone, FRID drops by 71%, with the largest gains at depth discontinuities. In a pretrained LiDM world model, the same zero-shot replacement improves FSVD by 15.5%, narrowing the sim-to-real gap. Code and checkpoints will be released at https://andrea25512.github.io/CRISP/.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC

preprint

CRISP: Fixing Flying Pixels in Latent LiDAR Generation via Diffusion Decoding

Andrea Ceron, Michael Schmidt, Alvaro Marcos-Ramiro, Sebastian Schmidt, Benjamin Busam

Preprint posted: 2026-10-08 · Original record ↗

arxiv
2610.11376
Explicit source versions (1)
  • v2 · 2026-10-20
  1. Preprint available

    Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (3)
  2. Explicit source version detected · v2

    Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Latent LiDAR pipelines suffer from flying pixels: convolutional VAEs blur sharp radial depth discontinuities, yielding edge depths that back-project to points floating between surfaces. We identify this as a major, directly correctable decoder bottleneck and introduce CRISP: a pixel-space diffusion decoder with a backbone-agnostic latent adapter, DiT-based denoiser, and support mask predictor. CRISP replaces video-VAE and LiDAR-native decoders alike while keeping the encoder and latent generator fixed. Across KITTI-360, SemanticKITTI, and nuScenes, replacing only the decoder reduces FSVD/FPVD by 50.5% on average across frozen backbones; for generic video VAEs, the reductions reach 71%/74%. On the LiDAR-native LiDM backbone, FRID drops by 71%, with the largest gains at depth discontinuities. In a pretrained LiDM world model, the same zero-shot replacement improves FSVD by 15.5%, narrowing the sim-to-real gap. Code and checkpoints will be released at https://andrea25512.github.io/CRISP/.