Driving · 2026-10-20

GeoV2V: Geometry-Grounded Video Diffusion Model for Driving Scene Generation

Yuchen Xi, Yuhan Guo, Chenwei Hou, Zixu Liu, Jin Fang, Jason Liu, Ruigang Yang

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

GeoV2V is a geometry-grounded video-to-video diffusion model for driving scene generation under novel trajectories (e.g., lane shifts). It integrates LiDAR- and depth-derived geometric priors with full-source video conditioning in the Wan 2.1 backbone. The authors introduce Para4D, a synthetic multi-lane synchronized dataset, to explicitly supervise lane-shift generation and improve cross-lane geometric consistency.

Why it matters

The paper targets lane-shift novel view synthesis where extrapolation to unseen viewpoints must preserve spatial layout and temporal dynamics. The authors argue that real-world datasets lack synchronized multi-trajectory supervision, and propose Para4D plus GeoV2V to jointly improve geometric consistency, temporal coherence, and visual quality under large lateral trajectory shifts on Waymo, nuScenes, and Para4D.

Method

Introduce Para4D: a synthetic CARLA dataset with synchronized multi-lane driving videos from parallel trajectories in the same dynamic scenes for explicit lane-shift supervision.
Build GeoV2V: a geometry-grounded video-to-video diffusion model based on Wan-2.1 that conditions on the full source video plus geometry priors from LiDAR and depth-derived reprojected points.
Inject two geometric priors hierarchically into the denoising network (dense depth-completed first 15 layers; sparse LiDAR-informed last 15 layers) using lightweight ControlNets; train with a two-stage strategy for LiDAR branch transferability.

Abstract (from arXiv)

Novel view synthesis under trajectory changes is essential for autonomous driving simulation, yet existing methods struggle to generate consistent videos when extrapolating to unseen viewpoints such as lane shifts. Reconstruction-based approaches maintain geometric consistency but degrade in visual quality under large viewpoint changes, while reconstruct-then-restore methods often produce visually plausible frames yet suffer from geometric and temporal inconsistencies. We present GeoV2V, a geometry-grounded video-to-video diffusion framework that synthesizes temporally coherent driving videos along novel trajectories by integrating LiDAR and depth-derived geometric priors with full-video conditioning in the Wan 2.1 backbone. A key challenge in learning cross-lane transformations is the lack of synchronized multi-trajectory supervision in real-world datasets. To address this, we also introduce Para4D, a synthetic dataset capturing synchronized multi-lane driving videos within dynamic scenes, which provides explicit supervision for lane-shift generation and serves as a strong training prior for geometry-consistent view synthesis. Experiments on Waymo, nuScenes, and Para4D demonstrate state-of-the-art performance in visual quality, geometric consistency, and robustness to complex lighting dynamics under large trajectory shifts. Project page: https://xiyuche.github.io/GeoV2V/

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-18 · Indexed by Pub-AI: 2026-10-20 UTC

preprint