Driving · 2026-10-07
WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses
Thinh D. Le, Son T. Nguyen, Duong Q. Nguyen, Dung D. Le, Ngo Anh Vien, H. Nguyen-Xuan
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
WareFly-VLA is a photorealistic UAV Vision–Language–Action (VLA) framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It includes 507 human-teleoperated flight episodes and 8,504 non-terminal RGB transitions with appearance-based language and synchronized 4-DoF body-frame control commands, plus a leakage-free benchmark over four open-source VLA architectures.
Why it matters
The authors report that language-conditioned UAV control in warehouses is “far from solved” and highlight gaps in existing benchmarks: they lack continuous low-level flight actions, fine-grained language target descriptions, and realistic industrial settings. WareFly-VLA provides synchronized video-language-action data, task coverage (approach-and-locate and person-following), challenge annotations, and an evaluation protocol intended to support language-grounded aerial autonomy research.
Method
- Create a UAV VLA dataset from human-teleoperated quadrotor episodes in a photorealistic warehouse simulator (Pegasus Simulator / NVIDIA Isaac Sim), with synchronized high-resolution RGB, appearance-grounded language, and continuous 4-DoF body-frame actions.
- Train/evaluate four open-source VLA architectures (SmolVLA, GR00T N1.7, π0/OpenPI, OpenVLA) under a leakage-free episode-level protocol at two control rates.
- Use difficulty-tagged episodes and per-step prediction metrics (MAE and Pearson correlation) plus open-loop rollout error (ADE/FDE).
Limitation
The authors report that “performance drops substantially under strict generalization settings,” indicating limited robustness beyond the benchmark’s evaluated splits.
Abstract (from arXiv)
Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flight actions, fine-grained natural-language target descriptions, and realistic industrial environments. This paper introduces WareFly-VLA, a photorealistic UAV VLA framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It contains 507 human-teleoperated flight episodes and 8,504 high-resolution RGB transitions collected in NVIDIA Isaac Sim, each paired with a human-written appearance description of the target worker and a synchronized four-degree-of-freedom control command. Two aerial tasks are covered: target approach and person following, under occlusion, long-range search, altitude variation, and clutter. A unified benchmark of four open-source VLA architectures (SmolVLA, GR00T N1.7, pi_0 and OpenVLA) is established under a leakage-free episode-level protocol at two control rates. The results show that language-conditioned aerial control in warehouses is far from solved: performance drops substantially under strict generalization settings, continuous action modeling consistently outperforms discrete action tokenization, only the forward channel is reliably learnable from a single frame, and current foundation-model interfaces transfer poorly from ground and humanoid embodiments to aerial platforms. The synchronized video, language, action, pose, and difficulty annotations further support world-model research. The dataset, baselines, and evaluation protocol are released to support language-grounded aerial autonomy in smart warehouses.
Related papers
- Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models
- Aero-VPA: Few-Step Flow-Matching Control over Non-Reconstructive JEPA Latents for Agile Flight on Edge Hardware
- PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
- RoboSeed: Multidimensional Evaluation of Vision-Language-Action and World-Action Models: Capability, Resource Affordability, and Research Evidence
- SearchWorld: Spatial Value-Grounded Imagination for UAV Object Search via World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC
WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses
Preprint posted: 2026-10-06 · Original record ↗
- arxiv
- 2610.08526
Explicit source versions (1)
- v1 · 2026-10-06
- Preprint available
Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flight actions, fine-grained natural-language target descriptions, and realistic industrial environments. This paper introduces WareFly-VLA, a photorealistic UAV VLA framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It contains 507 human-teleoperated flight episodes and 8,504 high-resolution RGB transitions collected in NVIDIA Isaac Sim, each paired with a human-written appearance description of the target worker and a synchronized four-degree-of-freedom control command. Two aerial tasks are covered: target approach and person following, under occlusion, long-range search, altitude variation, and clutter. A unified benchmark of four open-source VLA architectures (SmolVLA, GR00T N1.7, pi_0 and OpenVLA) is established under a leakage-free episode-level protocol at two control rates. The results show that language-conditioned aerial control in warehouses is far from solved: performance drops substantially under strict generalization settings, continuous action modeling consistently outperforms discrete action tokenization, only the forward channel is reliably learnable from a single frame, and current foundation-model interfaces transfer poorly from ground and humanoid embodiments to aerial platforms. The synchronized video, language, action, pose, and difficulty annotations further support world-model research. The dataset, baselines, and evaluation protocol are released to support language-grounded aerial autonomy in smart warehouses.
