Driving · 2026-11-09

SURE:Spatiotemporal Uncertainty Representation for Efficient Vision-Language-Action Learning

Jingye Zhang, Choo Sin Wai, Dake Zhong, Zhifang Liu, Yuhong Zhang, Siyao Xiao, Guorui Song, Guocun Wang, Dongbin Zhang, Peng Jiao, Haoqian Wang

arXiv:2611.09684PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

SURE is a vision-language-action training framework for robots that models structured spatial and temporal uncertainty in demonstrations. It adds a lightweight transposed-convolutional probabilistic action decoder to capture spatial uncertainty with locally correlated action bins, and uses Soft-DTW alignment supervision to tolerate local timing variations between predicted and demonstrated action chunks. The authors report higher manipulation success with reduced data/step requirements on LIBERO, RoboTwin 2.0, and real-world tasks.

Why it matters

The authors target a key limitation of standard VLA imitation losses under spatiotemporal uncertainty: pointwise supervision penalizes correct actions that differ in pose or execution timing. SURE explicitly represents spatial uncertainty in the action decoder and aligns temporally misordered action chunks via Soft-DTW, aiming to improve data and optimization efficiency while maintaining or improving success rates across simulation and real-world manipulation benchmarks.

Method

  • Use an OpenVLA-7B backbone with a probabilistic action decoder: discretize translation/rotation into R=256 bins, predict per-bin logits with a transposed-convolutional head to encourage local correlations, then decode continuous actions via expectation.
  • Add Soft-DTW alignment supervision between predicted and demonstrated action chunks to handle semantically consistent trajectories executed at different local speeds.
  • Train primarily with differentiable L1 action supervision (via expectation decoding) plus lightly weighted auxiliary KL regularization for spatial distributions and a Soft-DTW divergence plus endpoint term for temporal alignment.
Abstract (from arXiv)

Vision-language-action (VLA) policies have become a promising paradigm for end-to-end robot control, but their effectiveness still depends heavily on large volumes of demonstration data. In manipulation tasks, this dependence is compounded by structured spatiotemporal uncertainty in the action space: multiple nearby actions or trajectories may successfully solve the same task, while semantically equivalent demonstrations often unfold at different speeds. Standard pointwise supervision treats each demonstrated action as an exact target, imposing an unnecessary learning burden under such uncertainty. We present SURE, a data-efficient VLA training framework that explicitly models both spatial and temporal uncertainty. SURE augments a VLA backbone with a lightweight transposed-convolutional probabilistic action decoder that captures spatial uncertainty through local correlations among neighboring action bins. It further introduces Soft-DTW alignment supervision to accommodate local timing variations between predicted and demonstrated action chunks. Together, these two components reduce sensitivity to demonstrator-specific poses and execution timing, enabling more data-efficient action learning. We evaluate SURE on LIBERO, RoboTwin 2.0, and real-world manipulation tasks. Even with only 10% of the demonstration data or 20% of the training steps, SURE achieves manipulation success rates that match or surpass baseline VLAs.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-06 · Indexed by Pub-AI: 2026-11-09 UTC

preprint

SURE:Spatiotemporal Uncertainty Representation for Efficient Vision-Language-Action Learning

Jingye Zhang, Choo Sin Wai, Dake Zhong, Zhifang Liu, Yuhong Zhang, Siyao Xiao, Guorui Song, Guocun Wang, Dongbin Zhang, Peng Jiao, Haoqian Wang

Preprint posted: 2026-11-06 · Original record ↗

arxiv
2611.09684
Explicit source versions (1)
  • v1 · 2026-11-06
  1. Preprint available

    Source date: 2026-11-06 · First detected: 2026-11-09 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Vision-language-action (VLA) policies have become a promising paradigm for end-to-end robot control, but their effectiveness still depends heavily on large volumes of demonstration data. In manipulation tasks, this dependence is compounded by structured spatiotemporal uncertainty in the action space: multiple nearby actions or trajectories may successfully solve the same task, while semantically equivalent demonstrations often unfold at different speeds. Standard pointwise supervision treats each demonstrated action as an exact target, imposing an unnecessary learning burden under such uncertainty. We present SURE, a data-efficient VLA training framework that explicitly models both spatial and temporal uncertainty. SURE augments a VLA backbone with a lightweight transposed-convolutional probabilistic action decoder that captures spatial uncertainty through local correlations among neighboring action bins. It further introduces Soft-DTW alignment supervision to accommodate local timing variations between predicted and demonstrated action chunks. Together, these two components reduce sensitivity to demonstrator-specific poses and execution timing, enabling more data-efficient action learning. We evaluate SURE on LIBERO, RoboTwin 2.0, and real-world manipulation tasks. Even with only 10% of the demonstration data or 20% of the training steps, SURE achieves manipulation success rates that match or surpass baseline VLAs.