Latent Dynamics · 2026-10-09

Being-M0.7: A Latent World-Action Model for Humanoid Robots

Junpeng Yue, Boyuan Li, Yuxuan Wang, Zepeng Wang, Yuhui Fu, Feiyang Xie, Yu Zhang, Jing Zhang, Xianqi Zhang, Weibo Li, Xiaofei Zheng, Yuming Fang, Jiangxing Wang, Zongqing Lu

arXiv:2610.11283 PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Being-M0.7 is a latent world-action model for humanoid loco-manipulation that transfers visual–motion priors from mixed-modality human video and motion data to robot control. It uses a Mixture of Transformers prior pretrained on latent visual-state and motion prediction, adapted by robot mid-training, then grounded into executable actions via a gated action expert post-training. The authors report the highest aggregate simulation success on SIMPLE and matching the strongest baseline on real-world Unitree G1 tasks.

Why it matters

Humanoid loco-manipulation is temporal and whole-body, but robot demonstrations are scarce and human motion does not directly provide executable robot actions. The authors present a training framework that (1) learns future kinematics from mixed-modality human video/motion streams without pixel reconstruction, (2) adapts the learned prior to robot viewpoints and dynamics, and (3) grounds predictive latent context into actionable commands using gated cross-attention during post-training.

Method

Pretrain a latent video–motion prior (MoT) on egocentric video, motion-only, and paired video-motion streams by jointly predicting future latent visual states and motion via flow matching (DINO latents; avoid pixel-level reconstruction).
Robot mid-training adapts the visual–motion prior to robot trajectories and dynamics using the same robot dataset (loss applied where motion is available).
Action post-training freezes the adapted prior and trains a flow-based action expert that injects prior-generated world-context via gated cross-attention using current images and proprioception; explicit motion prediction is disabled in the prior during action learning/deployment.

Limitation

The paper mentions possible task interference under joint multi-task training in the ablation/discussion: compared with language-only context, the full model improves substantially on XMovePick and BendPick but underperforms on XMoveBendPick, where the authors state that these tasks share similar visual observations and may introduce interference.

Abstract (from arXiv)

Humanoid loco-manipulation requires coordinated locomotion and manipulation informed by future scene evolution and whole-body motion, yet learning these capabilities is constrained by scarce robot demonstrations. Human video and motion datasets offer scalable supervision, but many contain only video or motion rather than paired video-motion data. Moreover, human motion does not directly specify executable robot actions. We present Being-M0.7, a latent world-action model that transfers visual-motion priors learned from mixed-modality human data to humanoid control through pre-training, robot mid-training, and action post-training. We curate a corpus from more than 10,000 hours of raw human-centric data, integrating video-only, motion-only, and paired video-motion streams to learn complementary visual dynamics and whole-body kinematic structure. Joint prediction of future latent visual states and motion encourages visual representations to encode future kinematics. Robot mid-training adapts this coarse-grained prior to robot viewpoints and body dynamics. During action post-training, an action expert combines visual predictive representations from the frozen, adapted prior with current images and proprioception through gated cross-attention, grounding predictive context in executable whole-body commands. Being-M0.7 achieves the highest aggregate success rate among the compared baselines on SIMPLE and matches the strongest baseline on real-world Unitree G1 loco-manipulation tasks.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC

preprint