Latent Dynamics · 2026-10-20
Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation
Yutang Lin, Guanya Shi, C. Karen Liu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
FIBER uses a frozen behavioral foundation model (BFM) latent space as a shared executable representation for diverse humanoid loco-manipulation skills. It projects each independently acquired interaction policy into BFM latents via action-aligned latent policy projection (LPP), then trains a single perceptive flow planner offline from aggregated closed-loop latent rollouts. An adaptive-horizon BFM (AH-BFM) supports both precise control and longer-horizon behavior.
Why it matters
The authors target the representation gap between high-level planning and whole-body control in contact-rich humanoid interaction. By projecting experts into an executable latent space of a frozen BFM, the planner can operate at lower rate while preserving the BFM’s high-frequency state-dependent physical feedback. The unified offline planner can chain skills and steer execution using the same latent interface across simulation and a real humanoid.
Method
- Use a frozen BFM as an executable interface: given robot state and a latent command, the BFM outputs high-frequency low-level actions.
- Action-aligned latent policy projection (LPP): independently acquired interaction experts are mapped into BFM latents by aligning induced BFM actions to each expert’s local state-dependent control responses, with semantic anchoring in the BFM’s native representation.
- Train a single perceptive latent flow planner entirely offline from aggregated closed-loop latent rollouts; deploy with receding-horizon replanning while the frozen BFM provides feedback. AH-BFM supports adaptive horizons via multi-γ FB learning.
Abstract (from arXiv)
Learning a single perceptive policy for diverse humanoid loco-manipulation requires unifying high-level task decisions without losing the state-dependent physical responses that make each interaction succeed. This makes the representation between high-level planning and whole-body control critical: kinematic or contact targets provide compact abstractions but do not directly encode interaction-specific control responses, while directly predicting low-level actions requires the planner to model high-frequency closed-loop feedback. We propose FIBER, a framework that uses the executable latent space of a frozen behavioral foundation model (BFM) as a shared planning representation for heterogeneous interaction skills. Our latent policy projection (LPP) maps independently acquired task-specific policies into this representation by identifying latent commands whose induced actions match their local control responses, enabling lower-rate planning while retaining the BFM's high-frequency state feedback. After projection, we aggregate closed-loop latent rollouts from all skills and train a single perceptive flow planner entirely offline. We further introduce AH-BFM, an adaptive-horizon BFM trained with multi-gamma forward-backward (FB) learning to support both precise control and temporally extended whole-body behavior. We evaluate FIBER on box pickup, door opening, and object grasping in simulation and on a real Unitree G1 humanoid. Across 128 diverse variations per task in simulation, FIBER achieves 42.0% mean success, more than tripling projected latent replay (11.6%) and substantially outperforming kinematic planning (3.5%). Real-world experiments further demonstrate sim-to-real transfer, skill chaining, and conditional steering through the same unified planner.
Related papers
- Being-M0.7: A Latent World-Action Model for Humanoid Robots
- Interaction Latents as Actions for Contact-Rich Manipulation
- Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
- VioLA: Learning Generalist Humanoid Control Policies from Human Data
- Language-Forcing: Advancing Humanoid Loco-Manipulation via Semantic Alignment of Whole-Body World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-17 · Indexed by Pub-AI: 2026-10-20 UTC
Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation
Preprint posted: 2026-10-17 · Original record ↗
- arxiv
- 2610.24675
Explicit source versions (1)
- v2 · 2026-10-20
- Preprint available
Source date: 2026-10-17 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · OAI arXiv created; not header datestamp (official metadata)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Learning a single perceptive policy for diverse humanoid loco-manipulation requires unifying high-level task decisions without losing the state-dependent physical responses that make each interaction succeed. This makes the representation between high-level planning and whole-body control critical: kinematic or contact targets provide compact abstractions but do not directly encode interaction-specific control responses, while directly predicting low-level actions requires the planner to model high-frequency closed-loop feedback. We propose FIBER, a framework that uses the executable latent space of a frozen behavioral foundation model (BFM) as a shared planning representation for heterogeneous interaction skills. Our latent policy projection (LPP) maps independently acquired task-specific policies into this representation by identifying latent commands whose induced actions match their local control responses, enabling lower-rate planning while retaining the BFM's high-frequency state feedback. After projection, we aggregate closed-loop latent rollouts from all skills and train a single perceptive flow planner entirely offline. We further introduce AH-BFM, an adaptive-horizon BFM trained with multi-gamma forward-backward (FB) learning to support both precise control and temporally extended whole-body behavior. We evaluate FIBER on box pickup, door opening, and object grasping in simulation and on a real Unitree G1 humanoid. Across 128 diverse variations per task in simulation, FIBER achieves 42.0% mean success, more than tripling projected latent replay (11.6%) and substantially outperforming kinematic planning (3.5%). Real-world experiments further demonstrate sim-to-real transfer, skill chaining, and conditional steering through the same unified planner.