Video World Models · 2026-11-02

Dreams Are Cheaper Than Flights: Curated World-Model Data Improves Aerial Detection

Rugved Katole, Vedant Patil, Jenna Kline, Christopher Stewart

arXiv:2610.41030PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors seed a 7B video world model (Cosmos) with drone wildlife frames, then use an expert to reject deformed generations and annotate accepted frames. On held-out field videos from the same drones, mAP@50–95 rises 4.3% when curated synthetic frames are added to real data and 10.3% when they replace it; on an urban benchmark it reaches +33.5%. They also estimate synthetic generation is cheaper than field labor.

Why it matters

Field data collection is slow, costly, and disruptive for aerial wildlife/urban detection. The paper tests whether seeded, curated synthetic data can match real field data for training detectors evaluated on held-out field videos, and reports both detection gains and a compute-vs-field-labor cost comparison. It highlights that even with high rejection rates, compute-based synthetic data can be a practical substitute when transfer is verified and the output is curated.

Method

  • Field-to-simulation-to-field protocol: seed a world model only with training footage from the same drones, then have an expert curate generated clips and train detectors using real vs curated synthetic as the only variable.
  • Generate with Cosmos-Predict1-7B (Video2World): each clip is produced from nine consecutive training frames plus a short text prompt; accept clips only if a domain expert finds no deformities.
  • Evaluation: fine-tune YOLO detectors (YOLO11m on MMLA, YOLOv8m on VisDrone) and report mAP@50–95 on held-out field test/validation splits from the same drones.

Limitation

Synthetic clips must be curated and transfer verified per class; the authors note world models can produce deformities and poorly rendered classes can vanish in aggregate accuracy (e.g., bus on VisDrone).

Abstract (from arXiv)

Robot perception is increasingly data driven, yet field data is slow, costly, and sometimes disruptive to collect. Video world models offer a cheaper source: seeded with real frames, they extend a recorded scene into new footage with the deployment's sensor, altitude, and environment. We ask whether such synthetic data matches the utility of real field data and whether detectors trained with it perform in the field. Seeding a 7B-parameter world model with wildlife drone video, we more than double the training set with curated frames. On held-out videos from the same drones, mAP@50--95 rises 4.3\% when these frames are added to real data and 10.3\% when they replace it, and up to 33.5\% on an urban benchmark. Although an expert rejects an estimated 87.5\% of generated clips, the data costs about \$1,100 of GPU time, against more than \$3,200 of field labor alone for a less useful real training set. Synthetic data thus shifts data costs from fieldwork to compute.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-02 · Indexed by Pub-AI: 2026-11-02 UTC

preprint

Dreams Are Cheaper Than Flights: Curated World-Model Data Improves Aerial Detection

Rugved Katole, Vedant Patil, Jenna Kline, Christopher Stewart

First announced by arXiv: 2026-11-02 · Original record ↗

arxiv
2610.41030
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-02 · First detected: 2026-11-02 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2610.41030v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Robot perception is increasingly data driven, yet field data is slow, costly, and sometimes disruptive to collect. Video world models offer a cheaper source: seeded with real frames, they extend a recorded scene into new footage with the deployment's sensor, altitude, and environment. We ask whether such synthetic data matches the utility of real field data and whether detectors trained with it perform in the field. Seeding a 7B-parameter world model with wildlife drone video, we more than double the training set with curated frames. On held-out videos from the same drones, mAP@50--95 rises 4.3\% when these frames are added to real data and 10.3\% when they replace it, and up to 33.5\% on an urban benchmark. Although an expert rejects an estimated 87.5\% of generated clips, the data costs about \$1,100 of GPU time, against more than \$3,200 of field labor alone for a less useful real training set. Synthetic data thus shifts data costs from fieldwork to compute.