Latent Dynamics · 2026-10-14
UniLAP: A Unified Latent Action Policy for Cross-Embodiment Manipulation
Lei Yao, Qiao Feng, Mutian Tong, Qihao Yun, Lap-Pui Chau, Lingjie Liu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
UniLAP introduces a unified continuous latent-action policy for cross-embodiment manipulation. It models latent actions with flow matching distilled from a frozen LAM teacher, “bypassing reliance on future frames” during inference. It uses label-free motion supervision from cross-frame correspondences in frozen DINOv2 to isolate kinematic cues. With an embodiment-agnostic latent manifold and lightweight adaptation, the authors report improvements on hand pose forecasting, single-arm manipulation, humanoid manipulation, and real-world deployment.
Why it matters
The authors target two issues in latent-action pipelines: discrete latents that bottleneck action granularity, and continuous alternatives that often require future-frame access and planning. UniLAP aims to produce a causal latent-action policy via flow-matching distillation and correspondence-based motion supervision, then transfer it across embodiments with a shared latent space and per-embodiment decoding/adaptation.
Method
- Model latent actions as a continuous flow-matching process conditioned on current observation and instruction, distilled from a frozen LAM teacher using a flow-adaptive objective.
- Use label-free motion supervision by deriving a dense correspondence residual from cross-patch correspondences of a frozen DINOv2 encoder (no explicit factorization or external optical flow estimator).
- Adapt across embodiments using an embodiment-specific action decoder trained from scratch and lightweight policy fine-tuning on target labeled actions (Stage II).
Limitation
The paper text does not state a specific limitation.
Abstract (from arXiv)
Latent Action Models (LAMs) can capture manipulation priors from action-free videos, but discrete latents usually impose a granularity bottleneck, limiting their ability to represent fine-grained action. Recent continuous alternatives mitigate this, but they are typically used as proxies for world modeling and require a further planning step to produce executable actions. We thus introduce UniLAP, a unified continuous latent-action policy that models the latent space via flow matching and bypasses reliance on future frames during inference. By distilling a LAM teacher with a flow-adaptive objective, UniLAP inherits the teacher's action representation while retaining the multimodal distribution of feasible behaviors. To decouple latent representations from task-irrelevant appearance, we extract a label-free motion target to isolate relevant kinematic cues from spurious visual dynamics. With the established embodiment-agnostic latent manifold, UniLAP can be easily adapted to novel platforms. Pretrained on multi-source videos at a compact 1.1B parameter scale, UniLAP outperforms state-of-the-art latent-action and Vision-Language-Action (VLA) baselines across hand pose forecasting, single-arm and humanoid manipulation, and real-world deployment.
Related papers
- Being-M0.7: A Latent World-Action Model for Humanoid Robots
- OneLA: Learning Many Worlds from Latent Actions
- Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling
- Cross-Embodiment Robot Foundation World Models with Latent Actions
- UNITAS: A 3D-Native World Action Model for Embodied Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-14 UTC
UniLAP: A Unified Latent Action Policy for Cross-Embodiment Manipulation
Preprint posted: 2026-10-12 · Original record ↗
- arxiv
- 2610.16969
Explicit source versions (1)
- v1 · 2026-10-12
- Preprint available
Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Latent Action Models (LAMs) can capture manipulation priors from action-free videos, but discrete latents usually impose a granularity bottleneck, limiting their ability to represent fine-grained action. Recent continuous alternatives mitigate this, but they are typically used as proxies for world modeling and require a further planning step to produce executable actions. We thus introduce UniLAP, a unified continuous latent-action policy that models the latent space via flow matching and bypasses reliance on future frames during inference. By distilling a LAM teacher with a flow-adaptive objective, UniLAP inherits the teacher's action representation while retaining the multimodal distribution of feasible behaviors. To decouple latent representations from task-irrelevant appearance, we extract a label-free motion target to isolate relevant kinematic cues from spurious visual dynamics. With the established embodiment-agnostic latent manifold, UniLAP can be easily adapted to novel platforms. Pretrained on multi-source videos at a compact 1.1B parameter scale, UniLAP outperforms state-of-the-art latent-action and Vision-Language-Action (VLA) baselines across hand pose forecasting, single-arm and humanoid manipulation, and real-world deployment.