Driving · 2026-11-06

SLA: Scaling Spatial-Language-Action Model via Continual Pre-training

Fuhao Li, Wenxuan Song, Qizhou Huang, Ziyang Zhou, Jingbo Wang, Junwu E, Pengwei Wang, Yan Wang, Shunbo Zhou, Haoang Li

arXiv:2611.08373PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

SLA is a Spatial-Language-Action model that learns transferable 3D spatial understanding by continual pre-training: it jointly optimizes action prediction and implicit alignment between intermediate VLA visual embeddings and spatial representations from a frozen 3D perception model. It uses feature-dependent gates to aggregate multiple VLA layers and probabilistic state dropout for generalization, while keeping the original inference architecture (auxiliary components used only in training).

Why it matters

The authors target reliable robotic manipulation in 3D by addressing the mismatch where VLA foundations inherit mostly 2D visual representations and receive spatial supervision mainly during task-specific adaptation. SLA aims to turn spatial understanding into a transferable foundation capability via continual pre-training, then retain it during downstream post-training without changing the inference architecture.

Method

  • Initialize from a pretrained π0.5 policy and continue training with an added 3D spatial alignment loss: align intermediate VLA visual tokens to spatial representations from a frozen 3D perception foundation model using cosine-similarity-based loss.
  • Use shared layer-norm + 2-layer MLP to project VLA visual tokens from each attention layer; apply feature-dependent token-and-channel-wise gates to adaptively aggregate embeddings from multiple VLA layers for spatial supervision.
  • Use partially available geometric annotations (e.g., depth and camera calibration with RGB) through the external 3D perception model during training; use probabilistic state dropout to mitigate overfitting to demonstrations; discard auxiliary components at inference.
Abstract (from arXiv)

Reliable robotic manipulation requires both semantic understanding and precise control in three-dimensional space. However, vision-language-action (VLA) foundations largely inherit visual representations from 2D vision-language pre-training, while spatial supervision is often introduced only during task-specific adaptation. Developing spatial understanding as a transferable foundation-model capability therefore calls for learning from broader robot experience. We present SLA, a Spatial-Language-Action Model that scales implicit spatial alignment to continual pre-training and maintains it during downstream post-training. Initialized from a pretrained pi05 policy, SLA jointly optimizes action prediction and alignment between intermediate visual embeddings and spatial representations from a frozen 3D perception model. We curate and unify heterogeneous robot datasets to support spatial learning across diverse embodiments, tasks, and scenes. To accommodate this diversity, we introduce feature-dependent gates that adaptively aggregate visual embeddings from multiple VLA layers for spatial supervision. The teacher further exploits partially available depth and camera calibration alongside RGB observations, enabling effective use of heterogeneous geometric annotations. Probabilistic state dropout encourages reliance on visual cues and mitigates overfitting to demonstration trajectories. All auxiliary components are used only during training, allowing direct reuse of pretrained VLA weights and preserving the original inference architecture.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-05 · Indexed by Pub-AI: 2026-11-06 UTC

preprint

SLA: Scaling Spatial-Language-Action Model via Continual Pre-training

Fuhao Li, Wenxuan Song, Qizhou Huang, Ziyang Zhou, Jingbo Wang, Junwu E, Pengwei Wang, Yan Wang, Shunbo Zhou, Haoang Li

Preprint posted: 2026-11-05 · Original record ↗

arxiv
2611.08373
Explicit source versions (1)
  • v1 · 2026-11-05
  1. Preprint available

    Source date: 2026-11-05 · First detected: 2026-11-06 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Reliable robotic manipulation requires both semantic understanding and precise control in three-dimensional space. However, vision-language-action (VLA) foundations largely inherit visual representations from 2D vision-language pre-training, while spatial supervision is often introduced only during task-specific adaptation. Developing spatial understanding as a transferable foundation-model capability therefore calls for learning from broader robot experience. We present SLA, a Spatial-Language-Action Model that scales implicit spatial alignment to continual pre-training and maintains it during downstream post-training. Initialized from a pretrained pi05 policy, SLA jointly optimizes action prediction and alignment between intermediate visual embeddings and spatial representations from a frozen 3D perception model. We curate and unify heterogeneous robot datasets to support spatial learning across diverse embodiments, tasks, and scenes. To accommodate this diversity, we introduce feature-dependent gates that adaptively aggregate visual embeddings from multiple VLA layers for spatial supervision. The teacher further exploits partially available depth and camera calibration alongside RGB observations, enabling effective use of heterogeneous geometric annotations. Probabilistic state dropout encourages reliance on visual cues and mitigates overfitting to demonstration trajectories. All auxiliary components are used only during training, allowing direct reuse of pretrained VLA weights and preserving the original inference architecture.