Driving · 2026-10-14
DR-VLA: Learning Human-Like Quadruped Navigation from Egocentric Web Videos via Domain-Invariant Scene Representations
Jun Tian, Yingjie Niu, Ruo Chen, Hongyi Lin, Chao Li
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
DR-VLA is a Mixture-of-Transformers framework for quadruped instruction-conditioned navigation learned from egocentric web videos via a domain-invariant scene representation (DISR). The authors pretrain understanding and world experts on 10,000h EgoLynx-Web without action labels, adapt to robot observations, and train a flow-matching action expert on 2,000 teleoperated clips (EgoLynx-Robot). On held-out EgoLynx-Robot, it improves QNS and reduces FHE vs competing methods, and achieves 92.1 PDMS on NAVSIM v1.
Why it matters
The authors aim to transfer human navigation priors from large-scale egocentric web video to legged robots, addressing viewpoint/motion and missing action-label gaps. Their DISR removes appearance factors by construction (semantic segmentation + relative depth, excluding RGB) and supports latent future prediction of scene structure, which they use to train a robot policy with only 2,000 action-labeled robot clips.
Method
- Introduce domain-invariant scene representation (DISR) as a compact spatio-temporal latent z capturing semantic, spatial (relative depth), and relational structure; DISR is trained as an autoencoder using segmentation (SAM) and relative depth (Depth Anything) as inputs (no RGB).
- Build DR-VLA as Mixture-of-Transformers with three experts: a vision-language understanding expert (Qwen3-VL-2B) generates captions from image views + instruction; a world expert predicts future DISR latents for K future frames; an action expert generates robot trajectories via conditional flow matching conditioned on predicted scene evolution.
- Train in two stages using EgoLynx: pretrain understanding/world on EgoLynx-Web (10,000h) without action labels, then adapt and train the action expert on EgoLynx-Robot (2,000 teleoperated clips, ~17 hours); evaluate with QNS and open-loop ADE/FDE and heading errors.
Abstract (from arXiv)
Quadruped robots performing outdoor deliveries must interpret complex scenes, anticipate changes, and navigate in socially compliant ways. Robot demonstrations are costly to collect at the scale needed for generalization. Egocentric web videos offer abundant human navigation experience, but differences in viewpoint and motion make this experience difficult to transfer to robots. We introduce DR-VLA, a Mixture-of-Transformers framework that transfers human navigation experience through a domain-invariant scene representation (DISR). The framework combines vision-language understanding and latent world modeling with a flow-matching action expert. We introduce EgoLynx, comprising 10,000 hours of curated egocentric web video (EgoLynx-Web) and 2,000 clips of teleoperated robot navigation collected during real-world deliveries (EgoLynx-Robot). We pretrain the understanding and world experts on EgoLynx-Web without action labels, then adapt the model to robot observations and train the action expert on EgoLynx-Robot to predict executable trajectories. We also propose the Quadruped Navigation Score (QNS), a pseudo-closed-loop metric combining safety, instruction compliance, comfort, and progress. On the held-out EgoLynx-Robot test split, DR-VLA improves QNS by approximately 5% and reduces open-loop final heading error (FHE) by 14.6%, each relative to the best competing result for that metric. Ablations show that web-video pretraining improves generalization and that the model's key components contribute to performance. As a future-prediction target, DISR outperforms five generic visual representations and yields performance close to task-specific 3D perception without dedicated geometric labels or point clouds for target construction. Trained separately on NAVSIM v1 without web pretraining, DR-VLA achieves 92.1 PDMS, suggesting that the framework also applies to autonomous driving.
Related papers
- ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
- UniFlowWAM: A Unified World-Action Model with Visual Semantics and 4D Geometry
- Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?
- EWAM: Emergent Depth-Wise Specialization in a Unified Embodied Model -- From Semantic Understanding through Visual Foresight to Action
- World Action Learning via Interaction-Centric Spectral Latent Guidance
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC
DR-VLA: Learning Human-Like Quadruped Navigation from Egocentric Web Videos via Domain-Invariant Scene Representations
Preprint posted: 2026-10-13 · Original record ↗
- arxiv
- 2610.18373
Explicit source versions (1)
- v1 · 2026-10-13
- Preprint available
Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Quadruped robots performing outdoor deliveries must interpret complex scenes, anticipate changes, and navigate in socially compliant ways. Robot demonstrations are costly to collect at the scale needed for generalization. Egocentric web videos offer abundant human navigation experience, but differences in viewpoint and motion make this experience difficult to transfer to robots. We introduce DR-VLA, a Mixture-of-Transformers framework that transfers human navigation experience through a domain-invariant scene representation (DISR). The framework combines vision-language understanding and latent world modeling with a flow-matching action expert. We introduce EgoLynx, comprising 10,000 hours of curated egocentric web video (EgoLynx-Web) and 2,000 clips of teleoperated robot navigation collected during real-world deliveries (EgoLynx-Robot). We pretrain the understanding and world experts on EgoLynx-Web without action labels, then adapt the model to robot observations and train the action expert on EgoLynx-Robot to predict executable trajectories. We also propose the Quadruped Navigation Score (QNS), a pseudo-closed-loop metric combining safety, instruction compliance, comfort, and progress. On the held-out EgoLynx-Robot test split, DR-VLA improves QNS by approximately 5% and reduces open-loop final heading error (FHE) by 14.6%, each relative to the best competing result for that metric. Ablations show that web-video pretraining improves generalization and that the model's key components contribute to performance. As a future-prediction target, DISR outperforms five generic visual representations and yields performance close to task-specific 3D perception without dedicated geometric labels or point clouds for target construction. Trained separately on NAVSIM v1 without web pretraining, DR-VLA achieves 92.1 PDMS, suggesting that the framework also applies to autonomous driving.