Video World Models · 2026-10-16

Orbia: Do Generated Videos Form a Consistent 3D World?

Dongyue Lu, Guian Fang, Hur Lim, Yonggang Wu, Yanrong Wang, Hongshan Chen, Shengju Qian, Yue Liao, Wei Chow, Lingdong Kong, Tianxin Huang, Yu Yang, Hongji Yang, Junchao Huang, Ao Liang, Yihao Wang, Zihan Wang, Rong Li, Hanlin Chen, Xin Wang, Wei Tsang Ooi, Mike Zheng Shou, Shuicheng Yan

arXiv:2610.20693 PDF Code

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Orbia is presented as the first benchmark dedicated to systematic, quantitative evaluation of 3D consistency in interactive video world models. It evaluates three dimensions—input preservation, generated-content persistence, and 3D self-consistency—using controlled exploration-and-revisit protocols with registered RGB-D references and annotated anchors, plus minute-long extensions. The authors build 720 evaluation instances and evaluate 27 models across text-, action-, and camera-conditioned interfaces.

Why it matters

The authors argue that visually convincing frames do not ensure a consistent 3D world across viewpoints and repeated visits, and that existing benchmarks do not provide systematic 3D evaluation. Orbia aims to jointly diagnose (1) preservation of the input scene, (2) persistence of newly generated regions after leaving and returning, and (3) whether generated views describe the same 3D scene, enabling more persistent 3D world models.

Method

Decompose 3D consistency into input preservation, generated-content persistence, and 3D self-consistency, evaluated within exploration-and-revisit trajectories.
Use registered RGB-D references and annotated anchors to measure appearance, geometry, and spatial stability, and compare revisits to regions absent from the input.
Test cross-view agreement and scene-level coherence using cross-view depth/feature correspondence, regional scale agreement, and held-out Gaussian rendering, with short and minute-long revisits.

Abstract (from arXiv)

Video generators serve as interactive worlds for exploration and simulation, yet realistic and visually appealing frames do not necessarily depict a consistent 3D world. Existing benchmarks assess selected geometric properties or use vision-language models to judge memory and consistency, leaving systematic 3D evaluation underexplored. We introduce Orbia, the first benchmark dedicated to a systematic and comprehensive evaluation of 3D consistency in interactive world models. Through controlled exploration and revisit, we organize the evaluation into three dimensions: input preservation, generated-content persistence, and 3D self-consistency. For input preservation, registered RGB-D references and annotated anchors support appearance, geometry, and spatial stability measurements. For generated-content persistence, we compare revisits to regions absent from the input image with their first generated observations to test whether models preserve what they create. For 3D self-consistency, cross-view depth and feature correspondence, regional scale agreement, and held-out Gaussian rendering test whether generated views describe the same 3D scene. Minute-long exploration and revisits further test long-term consistency. Our construction and annotation pipeline brings together public datasets, self-captured scenes, and simulation, yielding 720 instances across indoor and outdoor environments and six exploration-and-revisit protocols. Evaluation spans 27 text-, action-, and camera-conditioned models through native-interface adapters. Our Orbia benchmark provides a systematic basis for diagnosing 3D inconsistency and developing more persistent video world models.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-16 UTC

preprint