Latent Dynamics · 2026-10-01
GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
Qize Yu, Lianrui Fan, Boyu Chen, Jiaqi Liang, Xini Ding, Yue Chen, Zetian Song, Yuran Wang, Yi Zou, Kaixuan Wang, Tianxing Chen, Wenxuan Song, Bohan Zhou, Mingleyang Li, Siqiao Huang, Yuqi Ye, Caigao Jiang, Wei Wei, Ruihai Wu, Hang Zhang, Yixiao Ge, Shuchang Zhou, Shilong Liu, Xianming Liu, Ping Luo, Shiyu Huang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
GroundingPI is a 4B grounding foundation model that outputs points and boxes as quantized coordinate tokens in a shared vocabulary. Trained with multimodal/spatial pretraining, supervised fine-tuning, and GRPO reinforcement learning, it improves grounding accuracy across 34 benchmarks and transfers to physical-intelligence tasks (autonomous driving and robot manipulation) as a visual backbone.
Why it matters
The authors argue that general-purpose VLA/WAM backbones are not optimized for precise language-conditioned grounding needed for physical interaction, especially in dense scenes and with tiny objects. GroundingPI targets grounding natively via visual primitives (points/boxes), and evaluates both broad grounding performance and downstream transfer to driving/manipulation, including OOD robustness and action-data efficiency improvements.
Method
- Autoregressive grounding model that generates variable-length responses using shared vocabulary tokens for labels and 1,000 quantized coordinate tokens (<0>–<999>), with absent targets output as None.
- Staged training: projector alignment with image-caption supervision (freezing backbones), then joint multimodal pretraining with causal next-token prediction, followed by supervised fine-tuning for structured coordinate outputs.
- Reinforcement post-training with GRPO (G=8 responses/image-query), optimizing task-specific rewards (coverage/localization for grounding; text–geometry consistency for OCR) while updating only language parameters.
Limitation
The authors note a frontier-model gap on professional GUI interfaces: for ScreenSpot-Pro, Astra’s overall accuracy (93.17) remains substantially higher than GroundingPI’s (65.78).
Abstract (from arXiv)
Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.
Related papers
- Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models
- Grounded-Dreamer: Robot Learning from World Model Synthetic Data with Physical Grounding
- A Persian Vision-Language-Action Dataset and Benchmark for Ambiguity- and Infeasibility-Aware Instruction-to-Plan Grounding
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- Encoded but Not in Control: Revealing the Grounding Gap in Vision-Language Robot Policies
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.39601
Explicit source versions (1)
- v2 · 2026-10-16
- Explicit source version detected · v2
Source date: 2026-10-16 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.