Robotics · 2026-11-10
VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess
Mat\v{e}j Kripner, Milan Straka
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors fine-tune a vision-language-action policy (π0.5) for robotic chess king moves using teleoperated demonstrations and simulated demonstrations from a randomized scripted expert. In simulation, success scales with both data and updates; checkpoint comparisons require a learning-rate cool-down, while validation loss and action error are unreliable for model selection. On the real arm, partial results suggest simulation pretraining helps most when real data is scarce. The authors release the simulator, the expert, and real demonstrations.
Why it matters
The paper addresses practical VLA adaptation questions: how much data and update budget are needed, how to select checkpoints when closed-loop success is expensive, and whether simulation can reduce real-robot data collection. It also reports negative results for several learning-from-rollouts variants. The released simulator and datasets enable comparable evaluation on a graded protocol.
Method
- Fine-tune π0.5 for chess king moves between named squares using teleoperated real demonstrations and simulated demonstrations from a randomized scripted expert.
- In simulation, sweep dataset sizes and update budgets; evaluate with 1,000 closed-loop trials per model and test learning-rate cool-down vs constant-rate checkpoints and SWA for checkpoint selection.
- On the real arm, fine-tune on subsets of 32/128/256 teleoperated demonstrations (up to 25 trials per model for evaluation) and compare base-initialized vs sim-initialized policies.
Limitation
The authors report results from one task and with a single training run per setting; scripted-expert similarity to human demonstrations is not rigorously measured; real-arm statistical evidence is limited with 25 trials per model.
Abstract (from arXiv)
Vision-language-action (VLA) models are trained as general-purpose foundation models for robot learning, and require fine-tuning for each downstream task and robot. This raises practical questions such as how much data and training updates are needed, how checkpoints should be selected, and whether simulation can be leveraged to alleviate the cost of real data collection. We study these questions on a task that requires both precise manipulation and spatial understanding of the scene. In this task, an SO-101 arm moves a chess king between two squares that are given only by their names, such as e4 and a5. We fine-tune $\pi_{0.5}$ with various dataset sizes and update budgets, using teleoperated demonstrations as well as simulated demonstrations from a randomized scripted expert that imitates the variability of human demonstrations. In simulation, the success rate scales with both data and updates. Importantly, we find that a short learning-rate cool-down adds up to 31 points and is necessary for comparing checkpoints, and that validation loss and action error are unreliable for model selection. On the real arm, we fine-tune on up to 256 teleoperated demonstrations and evaluate ten policies in 25 trials each. Partial results indicate that pretraining on the simulated demonstrations helps substantially, most clearly when real data is scarce. We also present a negative result, in which training on the policy's own rollouts with filtered behavior cloning, RECAP-style advantage conditioning, corrections from an automatic coach, or residual reinforcement learning did not yield a measurable improvement over plain behavior cloning in our setting. We release the simulator, the expert and the real demonstrations.
Related papers
- DreamVLA: From Generated Human Motions to Humanoid Vision-Language-Action Policies
- DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
- Causal Analysis of Robot Foundation Models Enables Efficiency Gains
- APS-VLA: Action-Guided and Future-Aware Token Pruning for Efficient Vision-Language-Action Models
- Fast-VLA: You Only Think Once for Robot Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-09 · Indexed by Pub-AI: 2026-11-10 UTC
VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess
Preprint posted: 2026-11-09 · Original record ↗
- arxiv
- 2611.12676
Explicit source versions (1)
- v1 · 2026-11-09
- Preprint available
Source date: 2026-11-09 · First detected: 2026-11-10 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.12676v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision-language-action (VLA) models are trained as general-purpose foundation models for robot learning, and require fine-tuning for each downstream task and robot. This raises practical questions such as how much data and training updates are needed, how checkpoints should be selected, and whether simulation can be leveraged to alleviate the cost of real data collection. We study these questions on a task that requires both precise manipulation and spatial understanding of the scene. In this task, an SO-101 arm moves a chess king between two squares that are given only by their names, such as e4 and a5. We fine-tune $π_{0.5}$ with various dataset sizes and update budgets, using teleoperated demonstrations as well as simulated demonstrations from a randomized scripted expert that imitates the variability of human demonstrations. In simulation, the success rate scales with both data and updates. Importantly, we find that a short learning-rate cool-down adds up to 31 points and is necessary for comparing checkpoints, and that validation loss and action error are unreliable for model selection. On the real arm, we fine-tune on up to 256 teleoperated demonstrations and evaluate ten policies in 25 trials each. Partial results indicate that pretraining on the simulated demonstrations helps substantially, most clearly when real data is scarce. We also present a negative result, in which training on the policy's own rollouts with filtered behavior cloning, RECAP-style advantage conditioning, corrections from an automatic coach, or residual reinforcement learning did not yield a measurable improvement over plain behavior cloning in our setting. We release the simulator, the expert and the real demonstrations.
