Robotics · 2026-10-09
VioLA: Learning Generalist Humanoid Control Policies from Human Data
Mert Albaba, Jens Bei{\ss}wenger, Anna Manasyan, Daniel Marta, Michael J. Black, Wieland Brendel, Andreas Krause, Georg Martius, Martin Riedmiller
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
VioLA learns a generalist humanoid policy by predicting body and hand motion latents (instead of joint commands) in a shared latent space used by frozen human/robot motion encoders and decoders. On a Unitree G1, the authors report zero-shot locomotion success (100%) and zero-shot manipulation success (88.6%) without task-specific fine-tuning, using human demonstrations alone; code and checkpoints are released.
Why it matters
The paper addresses two issues in humanoid instruction following: large, coupled action spaces and scarce humanoid demonstrations. VioLA changes the policy’s action representation to motion latents that are directly labelable from human recordings, enabling learning from human data without requiring physical robot demonstrations for each human motion. The authors’ results show zero-shot task execution on real hardware and a recipe that transfers across multiple VLA/WAM backbones.
Method
- Predict body and hand motion latents (128-D action = concatenated 64-D body + 64-D hand) from egocentric image, instruction, and proprioception; frozen controllers execute the predicted latents as coordinated joint commands.
- Use motion encoders that map both human and robot demonstrations into the same latent spaces, so human recordings supervise the policy in the deployed action space.
- Train only the generalist policy; encoders/decoders remain frozen. Use real-time chunking (RTC) in motion-latent space to reduce discontinuities at chunk switches.
Limitation
Including a separate bottle-carrying task lowers VioLA’s manipulation success from 31/35 (88.6%) to 31/40 (77.5%).
Abstract (from arXiv)
Teaching a humanoid to follow instructions with its whole body runs into two obstacles. Its action space is large and tightly coupled: legs, arms, and fingers must move together while the robot keeps its balance, which makes joint-level actions hard to learn. And humanoid demonstrations are scarce, so current humanoid generalist policies do not follow new instructions out of the box and are fine-tuned on teleoperated demonstrations of each task before deployment. Human demonstrations exist in far larger numbers, but a person's motion is not a robot command. We remove both obstacles by changing what the generalist policy predicts. We introduce VioLA, a generalist humanoid policy that predicts body and hand motion latents instead of joint commands. A pretrained body- and hand-controller execute these latents on the robot. Their corresponding motion encoders map human and robot motion into the same latent spaces. A human recording is therefore labeled in the policy's action space, and the training demonstration pool contains 140.6 million frames, 93.2% of them human. As a result, VioLA follows locomotion instructions on the real robot zero-shot, without task-specific fine-tuning, reaching 100% success where GR00T N1.7 and $\Psi_0$ reach 16.7% and 0%, respectively. It also reaches 88.6% manipulation success without task-specific fine-tuning. The same approach works across two VLA and one world-action model backbones. A generalist policy trained on human demonstrations alone performs locomotion tasks on the real robot zero-shot. Code and checkpoints will be released.
Related papers
- Being-M0.7: A Latent World-Action Model for Humanoid Robots
- BeJEPA: A Goal-Conditioned Humanoid Behavior World Model via Unsupervised RL
- Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation
- Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulation
- SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC
VioLA: Learning Generalist Humanoid Control Policies from Human Data
Preprint posted: 2026-10-08 · Original record ↗
- arxiv
- 2610.12435
Explicit source versions (1)
- v1 · 2026-10-08
- Preprint available
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Teaching a humanoid to follow instructions with its whole body runs into two obstacles. Its action space is large and tightly coupled: legs, arms, and fingers must move together while the robot keeps its balance, which makes joint-level actions hard to learn. And humanoid demonstrations are scarce, so current humanoid generalist policies do not follow new instructions out of the box and are fine-tuned on teleoperated demonstrations of each task before deployment. Human demonstrations exist in far larger numbers, but a person's motion is not a robot command. We remove both obstacles by changing what the generalist policy predicts. We introduce VioLA, a generalist humanoid policy that predicts body and hand motion latents instead of joint commands. A pretrained body- and hand-controller execute these latents on the robot. Their corresponding motion encoders map human and robot motion into the same latent spaces. A human recording is therefore labeled in the policy's action space, and the training demonstration pool contains 140.6 million frames, 93.2% of them human. As a result, VioLA follows locomotion instructions on the real robot zero-shot, without task-specific fine-tuning, reaching 100% success where GR00T N1.7 and $Ψ_0$ reach 16.7% and 0%, respectively. It also reaches 88.6% manipulation success without task-specific fine-tuning. The same approach works across two VLA and one world-action model backbones. A generalist policy trained on human demonstrations alone performs locomotion tasks on the real robot zero-shot. Code and checkpoints will be released.