Driving · 2026-10-14
FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning
Junhyeong Go, Mingyu Jung, Woobin Im, Jongbin Ryu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
FlowVLA is a training-only framework that injects action-relevant 3D dynamics into 2D Vision-Language-Action (VLA) models using reconstruction-derived, temporally aligned 3D flow from RGB. It uses dual-level supervision: point-level flow predicts future motion of selected 3D points, and TCP-level flow aggregates predicted point motions to align with robot Tool-Center-Point trajectories. Auxiliary heads are discarded at inference, adding zero inference-time overhead.
Why it matters
The authors argue existing VLA models rely on 2D image-text priors and can overfit to static visual cues, missing action-relevant 3D dynamics. FlowVLA aims to bridge scene-level 3D motion and control-relevant end-effector displacement while avoiding extra 3D sensor inputs or inference-time computation, making 3D-aware dynamics training usable in practice.
Method
- Use VDPM reconstruction to derive temporally consistent, reference-anchored 3D point trajectories from RGB; compute 3D point flow as positional differences and align points to the robot base coordinate.
- Construct training targets by magnitude-based selection of top-K motion-salient 3D points, then supervise point-level future displacements for selected points.
- Use dual-level flow supervision: point-level flow learns dense scene motion in action latent space, and TCP-level flow aggregates point motions into TCP-aligned flow aligned with robot end-effector displacement; auxiliary heads are discarded at inference.
Limitation
FlowVLA requires an offline reconstruction step to generate 3D flow supervision.
Abstract (from arXiv)
Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. This allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework with dual-level flow supervision. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot's TCP trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.
Related papers
- UniFlowWAM: A Unified World-Action Model with Visual Semantics and 4D Geometry
- Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer
- Evo-World: A Lightweight Vision-Language-Action Model with Future Motion Prediction via Optical-Flow Supervision
- UNITAS: A 3D-Native World Action Model for Embodied Manipulation
- ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-14 UTC
FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning
Preprint posted: 2026-10-12 · Original record ↗
- arxiv
- 2610.16160
Explicit source versions (1)
- v1 · 2026-10-12
- Preprint available
Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. This allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework with dual-level flow supervision. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot's TCP trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.