Robotics · 2026-10-30

BURGER: Benchmarking and Understanding Robotic Generalization via Embodied Drawing

Xiangyu Xu, Bangshuai Peng, Ziyi Liu, Mengjie Zhao, Chenhao Lu, Zhengyang Fan, Huazhe Xu, Kaizhe Hu

arXiv:2610.40532PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

BURGER introduces a language-conditioned robotic drawing benchmark and prompt-to-trajectory pipeline that generates continuous end-effector drawing trajectories in simulation and the real world. It uses 28,200 simulated demonstrations across 282 drawing tasks to evaluate five VLA models on data scaling, language grounding, spatial relations, and multimodal behavior. The authors also report real-robot deployment on a Franka Panda arm with reactive human intervention.

Why it matters

The paper argues that robotic drawing provides a controlled, inspectable setting for studying VLA generalization under scalable language-conditioned multimodal actions, avoiding some visual shortcuts and enabling fine-grained diagnosis of semantic, spatial, and action-generation failures. BURGER includes an evaluation metric aimed to better reflect sketch quality than image-text alignment metrics, and supports sim-to-real and closed-loop human modifications during drawing.

Method

  • Introduces BURGER: a language-conditioned robotic drawing benchmark where a policy draws a prompt-specified object or spatial composition from a blank canvas using sequential drawing actions.
  • Builds a prompt-to-trajectory pipeline by generating sketch candidates from GPT-Image 2, converting to ordered SVG strokes, replaying SVGs in physics simulators (e.g., ManiSkill with SAPIEN backend), recording observation-action pairs, and producing robot-executable trajectories.
  • Evaluates drawing quality with a sketch-specific contrastive evaluator built on a frozen DINOv3 backbone and a user study to compare correlation with human judgments vs CLIPScore.
Abstract (from arXiv)

How effectively can vision-language-action (VLA) models learn diverse language-conditioned behaviors as training data scales? Investigating this question requires large and varied robot datasets, whose collection on physical robots remains costly. We introduce BURGER, a language-conditioned robotic drawing benchmark with a scalable, prompt-driven pipeline for generating continuous drawing trajectories, both in simulation and the real world. In BURGER, an agent starts from the same blank canvas and controls a robot arm to draw the object or spatial composition specified by language, while each instruction allows an open-ended variety of valid drawings and stroke orders. Using 28,200 simulated demonstrations spanning 282 drawing tasks, we evaluate five representative VLA models across data scales, language grounding, and multimodal behavior. BURGER supports fine-grained analysis of language grounding, spatial composition, and action generation, revealing distinct failure patterns across VLA models. Our experiments suggest that some VLA models can benefit from greater task diversity and demonstration scale, capture multimodal drawing behaviors, and generalize to unseen compositions of familiar objects. Finally, we deploy a trained policy on a Franka Panda arm, demonstrating the ability to draw reactively under human intervention. Project homepage and code can be found at https://siyingzhao.github.io/BURGER/.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-29 · Indexed by Pub-AI: 2026-10-30 UTC

preprint

BURGER: Benchmarking and Understanding Robotic Generalization via Embodied Drawing

Xiangyu Xu, Bangshuai Peng, Ziyi Liu, Mengjie Zhao, Chenhao Lu, Zhengyang Fan, Huazhe Xu, Kaizhe Hu

Preprint posted: 2026-10-29 · Original record ↗

arxiv
2610.40532
Explicit source versions (1)
  • v1 · 2026-10-29
  1. Preprint available

    Source date: 2026-10-29 · First detected: 2026-10-30 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

How effectively can vision-language-action (VLA) models learn diverse language-conditioned behaviors as training data scales? Investigating this question requires large and varied robot datasets, whose collection on physical robots remains costly. We introduce BURGER, a language-conditioned robotic drawing benchmark with a scalable, prompt-driven pipeline for generating continuous drawing trajectories, both in simulation and the real world. In BURGER, an agent starts from the same blank canvas and controls a robot arm to draw the object or spatial composition specified by language, while each instruction allows an open-ended variety of valid drawings and stroke orders. Using 28,200 simulated demonstrations spanning 282 drawing tasks, we evaluate five representative VLA models across data scales, language grounding, and multimodal behavior. BURGER supports fine-grained analysis of language grounding, spatial composition, and action generation, revealing distinct failure patterns across VLA models. Our experiments suggest that some VLA models can benefit from greater task diversity and demonstration scale, capture multimodal drawing behaviors, and generalize to unseen compositions of familiar objects. Finally, we deploy a trained policy on a Franka Panda arm, demonstrating the ability to draw reactively under human intervention. Project homepage and code can be found at https://siyingzhao.github.io/BURGER/.