Driving · 2026-10-07

WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses

Thinh D. Le, Son T. Nguyen, Duong Q. Nguyen, Dung D. Le, Ngo Anh Vien, H. Nguyen-Xuan

arXiv:2610.08526PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

WareFly-VLA is a photorealistic UAV Vision–Language–Action (VLA) framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It includes 507 human-teleoperated flight episodes and 8,504 non-terminal RGB transitions with appearance-based language and synchronized 4-DoF body-frame control commands, plus a leakage-free benchmark over four open-source VLA architectures.

Why it matters

The authors report that language-conditioned UAV control in warehouses is “far from solved” and highlight gaps in existing benchmarks: they lack continuous low-level flight actions, fine-grained language target descriptions, and realistic industrial settings. WareFly-VLA provides synchronized video-language-action data, task coverage (approach-and-locate and person-following), challenge annotations, and an evaluation protocol intended to support language-grounded aerial autonomy research.

Method

  • Create a UAV VLA dataset from human-teleoperated quadrotor episodes in a photorealistic warehouse simulator (Pegasus Simulator / NVIDIA Isaac Sim), with synchronized high-resolution RGB, appearance-grounded language, and continuous 4-DoF body-frame actions.
  • Train/evaluate four open-source VLA architectures (SmolVLA, GR00T N1.7, π0/OpenPI, OpenVLA) under a leakage-free episode-level protocol at two control rates.
  • Use difficulty-tagged episodes and per-step prediction metrics (MAE and Pearson correlation) plus open-loop rollout error (ADE/FDE).

Limitation

The authors report that “performance drops substantially under strict generalization settings,” indicating limited robustness beyond the benchmark’s evaluated splits.

Abstract (from arXiv)

Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flight actions, fine-grained natural-language target descriptions, and realistic industrial environments. This paper introduces WareFly-VLA, a photorealistic UAV VLA framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It contains 507 human-teleoperated flight episodes and 8,504 high-resolution RGB transitions collected in NVIDIA Isaac Sim, each paired with a human-written appearance description of the target worker and a synchronized four-degree-of-freedom control command. Two aerial tasks are covered: target approach and person following, under occlusion, long-range search, altitude variation, and clutter. A unified benchmark of four open-source VLA architectures (SmolVLA, GR00T N1.7, pi_0 and OpenVLA) is established under a leakage-free episode-level protocol at two control rates. The results show that language-conditioned aerial control in warehouses is far from solved: performance drops substantially under strict generalization settings, continuous action modeling consistently outperforms discrete action tokenization, only the forward channel is reliably learnable from a single frame, and current foundation-model interfaces transfer poorly from ground and humanoid embodiments to aerial platforms. The synchronized video, language, action, pose, and difficulty annotations further support world-model research. The dataset, baselines, and evaluation protocol are released to support language-grounded aerial autonomy in smart warehouses.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC

preprint

WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses

Thinh D. Le, Son T. Nguyen, Duong Q. Nguyen, Dung D. Le, Ngo Anh Vien, H. Nguyen-Xuan

Preprint posted: 2026-10-06 · Original record ↗

arxiv
2610.08526
Explicit source versions (1)
  • v1 · 2026-10-06
  1. Preprint available

    Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flight actions, fine-grained natural-language target descriptions, and realistic industrial environments. This paper introduces WareFly-VLA, a photorealistic UAV VLA framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It contains 507 human-teleoperated flight episodes and 8,504 high-resolution RGB transitions collected in NVIDIA Isaac Sim, each paired with a human-written appearance description of the target worker and a synchronized four-degree-of-freedom control command. Two aerial tasks are covered: target approach and person following, under occlusion, long-range search, altitude variation, and clutter. A unified benchmark of four open-source VLA architectures (SmolVLA, GR00T N1.7, pi_0 and OpenVLA) is established under a leakage-free episode-level protocol at two control rates. The results show that language-conditioned aerial control in warehouses is far from solved: performance drops substantially under strict generalization settings, continuous action modeling consistently outperforms discrete action tokenization, only the forward channel is reliably learnable from a single frame, and current foundation-model interfaces transfer poorly from ground and humanoid embodiments to aerial platforms. The synchronized video, language, action, pose, and difficulty annotations further support world-model research. The dataset, baselines, and evaluation protocol are released to support language-grounded aerial autonomy in smart warehouses.