Driving · 2026-10-30
HiFOR: Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model
Rongxing Ding, Hongyu Qu, Kehong Wu, Xudong Wang, Fan Zhang, Xinguang Xiang, Xiangbo Shu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
HiFOR is a hierarchical foresight VLA framework for robotic manipulation. It jointly predicts short-term state evolution, intermediate semantic milestones, and final goal states using a Hierarchical Dependency Mask (HDM). Predicted futures are organized into a structured latent world (horizon/view/spatial) and retrieved by Action-Guided Factorized World Retrieval that uses denoising-step action representations to select relevant future information for action generation.
Why it matters
The authors aim to improve how VLA policies translate multi-horizon future predictions into actionable guidance by explicitly modeling cross-horizon dependencies and selectively retrieving futures relevant to the current action requirements during denoising.
Method
- Hierarchical Foresight Prediction: jointly predicts short-term state evolution, semantic milestones, and final goal states using a Hierarchical Dependency Mask (HDM) to connect dependencies across temporal horizons.
- Structure-Aware Latent World Modeling: organizes predicted future latent representations along horizon, view, and spatial dimensions, and derives horizon-level and view-level summaries for retrieval.
- Action-Guided Factorized World Retrieval: adaptively retrieves action-relevant future information via factorized attention over horizon/view/spatial, using action-expert hidden representations at each denoising step.
Abstract (from arXiv)
Vision-Language-Action (VLA) models have demonstrated promising capabilities in robotic manipulation by integrating visual perception, language understanding, and action generation. Recent advances have further incorporated future prediction to enhance the understanding of environmental dynamics and task progression. However, existing methods often model short-term dynamics and long-term task objectives separately, while directly incorporating predicted futures into action policies without sufficiently considering their relevance to current action requirements. To address these limitations, we propose HiFOR, a hierarchical foresight framework with action-guided world retrieval. Specifically, Hierarchical Foresight Prediction jointly predicts short-term state evolution, semantic milestones, and final goal states through a Hierarchical Dependency Mask (HDM), establishing dependencies across temporal horizons. Building upon these predictions, Structure-Aware Latent World Modeling organizes future latent representations along the horizon, view, and spatial dimensions to facilitate fine-grained retrieval. Furthermore, Action-Guided Factorized World Retrieval adaptively retrieves relevant future information from the structured latent world based on evolving action representations during denoising, providing targeted guidance for action generation. Extensive experiments on two simulation benchmarks and six real-world robotic manipulation tasks demonstrate the effectiveness of our approach.
Related papers
- Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
- Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation
- HiLo-WAM: Decoupling Long-Horizon Task Evolution and Local Dynamics in World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-29 · Indexed by Pub-AI: 2026-10-30 UTC
HiFOR: Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model
Preprint posted: 2026-10-29 · Original record ↗
- arxiv
- 2610.40373
Explicit source versions (1)
- v1 · 2026-10-29
- Preprint available
Source date: 2026-10-29 · First detected: 2026-10-30 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.40373v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision-Language-Action (VLA) models have demonstrated promising capabilities in robotic manipulation by integrating visual perception, language understanding, and action generation. Recent advances have further incorporated future prediction to enhance the understanding of environmental dynamics and task progression. However, existing methods often model short-term dynamics and long-term task objectives separately, while directly incorporating predicted futures into action policies without sufficiently considering their relevance to current action requirements. To address these limitations, we propose HiFOR, a hierarchical foresight framework with action-guided world retrieval. Specifically, Hierarchical Foresight Prediction jointly predicts short-term state evolution, semantic milestones, and final goal states through a Hierarchical Dependency Mask (HDM), establishing dependencies across temporal horizons. Building upon these predictions, Structure-Aware Latent World Modeling organizes future latent representations along the horizon, view, and spatial dimensions to facilitate fine-grained retrieval. Furthermore, Action-Guided Factorized World Retrieval adaptively retrieves relevant future information from the structured latent world based on evolving action representations during denoising, providing targeted guidance for action generation. Extensive experiments on two simulation benchmarks and six real-world robotic manipulation tasks demonstrate the effectiveness of our approach.
