Robotics · 2026-10-14

PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation

Xinyu Guo, Chao Jing, Li Yi, Xingyu Chen, Zhengxing Wu

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

PINT is a unified robot manipulation model that shares a learned chunk-level “Predictive Intent” across a vision-language model, a video world model, and an action expert. It inserts learnable Intent Tokens into the VLM, uses them to condition only the VGM (VGM remains active at inference), and trains with a two-stage schedule. The authors report strong ID and improved OOD generalization without embodied pretraining.

Why it matters

Unified models that fuse VLM/VGM/action components through shared attention can make it harder to use each expert’s specialized capabilities. PINT’s Predictive Intent provides a compact shared interface while preserving expert-specific pathways, aiming to coordinate semantic reasoning, future video dynamics, and action generation—especially for OOD generalization in robot manipulation.

Method

Insert learnable Intent Tokens at the end of the VLM multimodal prefix; the final hidden states of these tokens form Predictive Intent.
Condition the pretrained VGM using Predictive Intent by replacing only the VGM’s prompt conditioning; keep the VGM active at inference and provide one-way VGM representations to the action expert via one-way attention.
Train in two stages: Stage 1 freezes the VGM so future video prediction shapes Predictive Intent; Stage 2 fixes Intent and adapts the VGM and action expert.

Limitation

The authors state their evaluation focuses on robot manipulation and a single pairing of VLM and VGM backbones; combining PINT with embodied pretraining and extending it to broader embodied settings and larger-scale training are future work.

Abstract (from arXiv)

Generalist robot manipulation can benefit from pretrained vision-language models (VLMs) for semantic reasoning and video generation models (VGMs) for world modeling. However, existing unified models that directly fuse them through shared attention at multiple layers can make it harder to draw on their specialized capabilities. We introduce PINT, a unified model that coordinates semantic reasoning, world modeling, and action generation through a shared chunk-level Predictive Intent while preserving expert-specific pathways. PINT inserts learnable Intent Tokens into the VLM sequence to form this representation, conditioning both the VGM and an Action Expert (AE). It replaces only the VGM's prompt conditioning and retains the VGM at inference, where the AE reads its representations through one-way attention. Rather than prescribing what Predictive Intent should encode, PINT learns it jointly from the native prediction objectives of the VLM, VGM, and AE, which provide complementary views of the same upcoming task execution. Training uses two stages: Stage 1 freezes the VGM so that future video prediction shapes Predictive Intent with task-relevant future dynamics, while Stage 2 fixes the learned Intent and adapts the VGM and AE. Without embodied pretraining, PINT achieves strong in-distribution (ID) performance and substantially improves out-of-distribution (OOD) generalization, reaching 15.5% on RoboTwin 2.0-Hard compared with 1.8% for Fast-WAM and 3.2% for StarVLA, and 79.5% on LIBERO-Plus compared with 51.5% and 75.0%, respectively. Across five real-world bimanual tasks, PINT reaches 68% ID and 24% OOD success, compared with 35%/9% for pretrained Motus and 67%/35% for pretrained $\pi_{0.5}$.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-14 UTC

preprint