Driving · 2026-10-23
Adaptive Aerial Loco-Manipulation Policy Learning via LLM-Assisted Proactive Task Generation
Jingyu Ru, Xiangyuan Xie, Lixuan Jin, Jian An, Xingming Zhang, Binghao Li, Siyu Wang, Xiang Zhou, Lei Yuan, Hongli Xu, Yang Yu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ALMA learns an adaptive aerial loco-manipulation flight policy under external arm commands. It uses a contextual (GRU) policy trained with predictive context learning (action-conditioned prediction of future measurement representations in a JEPA-like setup). It also uses an LLM-assisted evolutionary process to generate proactive arm-motion tasks targeting observed tracking difficulties. In simulation it reports up to 24.7% lower position RMSE vs the strongest baseline and real-world zero-shot with 79.6% lower hover position RMSE vs PID.
Why it matters
The authors address aerial loco-manipulation where arm motion changes mass distribution and induces reaction forces/moments, making control history-relevant but not directly supervised by rewards. ALMA uses predictive supervision to shape what the policy context retains for control, and an LLM-assisted proactive task generation process to broaden training experience toward persistent tracking problems, aiming for simulation performance and zero-shot transfer to real hardware.
Method
- Contextual flight policy: recurrent (GRU) policy uses previewed flight reference, current measurements, and previous action; context is updated from history to adapt rotor commands under external arm commands.
- Predictive context learning: JEPA-like action-conditioned prediction of learned future measurement representations from context; optimized with cosine prediction loss and EMA target encoder; deployment uses policy alone.
- LLM-assisted proactive task generation: LLM analyzes execution records and proposes new arm motion structures; selected tasks are prioritized by episode tracking fraction (with tie-breaking via position/yaw RMSE).
Abstract (from arXiv)
Loco-manipulation extends the workspace of conventional manipulation by combining mobility with physical interaction, while aerial loco-manipulation further expands this capability into three-dimensional space, enabling robots to reach environments inaccessible to ground-based systems. Moving to an aerial platform, however, gives rise to a demanding control problem, since the vehicle must track trajectories accurately in three dimensions while the motion of the arm and its interaction with the environment continuously shift the center of mass, alter the inertia, and impose force and moment disturbances on the body. Conventional controllers address these effects through explicit models and estimation, whereas learning-based controllers infer dynamics from experience, yet control rewards provide no direct target for the response information retained from history, and fixed task sampling can leave persistent tracking difficulties insufficiently explored. We present ALMA, an aerial loco-manipulation framework for adaptive flight control under external arm commands. ALMA employs a contextual flight policy that adapts from recent measurements and actions, using action-conditioned prediction of future measurement representations to train its context to retain response information relevant to flight control. To expand the experience available for this adaptation, an evolutionary process uses a large language model (LLM) to analyze observed control difficulties and propose new structures of arm motion for subsequent training. In simulation, ALMA reduces position root-mean-square error (RMSE) by up to 24.7% relative to the strongest baseline, while the same policy transfers zero-shot to the aerial manipulator and reduces hover position RMSE by 79.6% during periodic arm motion relative to the real-world baseline.
Related papers
- LocoWM: High-Precision Locomotion through World-Model-Guided Residual Adaptation
- Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
- PILOT: Guiding Long-Horizon World-Model Planning for Robot Manipulation with Visual Subgoals and Policy Priors
- Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation
- World Action Learning via Interaction-Centric Spectral Latent Guidance
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-22 · Indexed by Pub-AI: 2026-10-23 UTC
Adaptive Aerial Loco-Manipulation Policy Learning via LLM-Assisted Proactive Task Generation
Preprint posted: 2026-10-22 · Original record ↗
- arxiv
- 2610.30810
Explicit source versions (1)
- v1 · 2026-10-22
- Preprint available
Source date: 2026-10-22 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Loco-manipulation extends the workspace of conventional manipulation by combining mobility with physical interaction, while aerial loco-manipulation further expands this capability into three-dimensional space, enabling robots to reach environments inaccessible to ground-based systems. Moving to an aerial platform, however, gives rise to a demanding control problem, since the vehicle must track trajectories accurately in three dimensions while the motion of the arm and its interaction with the environment continuously shift the center of mass, alter the inertia, and impose force and moment disturbances on the body. Conventional controllers address these effects through explicit models and estimation, whereas learning-based controllers infer dynamics from experience, yet control rewards provide no direct target for the response information retained from history, and fixed task sampling can leave persistent tracking difficulties insufficiently explored. We present ALMA, an aerial loco-manipulation framework for adaptive flight control under external arm commands. ALMA employs a contextual flight policy that adapts from recent measurements and actions, using action-conditioned prediction of future measurement representations to train its context to retain response information relevant to flight control. To expand the experience available for this adaptation, an evolutionary process uses a large language model (LLM) to analyze observed control difficulties and propose new structures of arm motion for subsequent training. In simulation, ALMA reduces position root-mean-square error (RMSE) by up to 24.7% relative to the strongest baseline, while the same policy transfers zero-shot to the aerial manipulator and reduces hover position RMSE by 79.6% during periodic arm motion relative to the real-world baseline.