Robotics · 2026-11-02
CaPtain: A Pipeline for Post-Training Code-as-Policy Robot Agents
Yijia Fan, Xilin He, Yilei Jiang, Dongming Wu, Jiaxu Wang, Xiangyu Yue
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
CaPtain is an open post-training pipeline that turns a small open VLM into a code-as-policy (CaP) robot agent without using a VLA, frontier model, or privileged state at test time. It trains inside CaPtain-Harness via SFT on GPT-6 Astra synthetic trajectories and multi-turn RL using a simulator physical critic for turn-level credit. CaPtain-4B improves LIBERO success substantially and transfers after simulation-only adaptation.
Why it matters
The authors target a missing SFT-and-RL recipe for training small open VLMs to manipulate robots end-to-end with low-level primitives. Their approach removes test-time reliance on VLAs/frontier models and uses a simulator-only physical critic to address multi-turn delayed physical effects, enabling RL gains to persist under perturbations and transfer after simulation-only adaptation.
Method
- Use CaPtain-Harness, a portable execution environment exposing low-level perception, measurement, and guarded-motion primitives with grounded text receipts; no grasp/place skills and no VLA.
- Stage 1 SFT: align the base model (Qwen3.5-4B) on in-harness trajectories in the exact multi-turn format used at deployment.
- Stage 2 RL: apply multi-turn RL with a simulator physical critic that reads privileged physical state (hidden from the policy) and credits each request by how much it changes predicted chance of success; critic is removed at test time.
Limitation
Transfer remains limited on RoboTwin 2.0 and RoboDojo (22.6% and 0.8% success), which require dual-arm coordination and unfamiliar arm kinematics.
Abstract (from arXiv)
Across coding, web, and tool-use tasks, language-model agents are now routinely trained with the same recipe: supervised fine-tuning (SFT) to learn the interface, then multi-turn reinforcement learning (RL) to master it. Code-as-policy (CaP) casts robot manipulation in the same form, as the policy writes code against robot APIs and reads back the results, so the same recipe should apply. Yet existing CaP agents sidestep training: they either prompt closed frontier models or delegate manipulation to pre-trained vision-language-action models (VLAs) and high-level skills, while small open vision-language models (VLMs) fail outright. We present CaPtain, an open post-training pipeline that turns a small open VLM into a capable CaP agent, with no VLA, frontier model, or privileged state at test time. CaPtain trains and deploys the policy inside CaPtain-Harness, which exposes low-level perception, geometry, and guarded-motion primitives across simulation and real hardware. We first align the base model via SFT on synthetic trajectories generated with GPT-6 Astra, then apply multi-turn RL. Multi-turn RL must credit tens of agent requests, whose physical effects often surface turns later, from a single end-of-episode outcome. We therefore introduce a physical critic that reads privileged simulator state, never seen by the policy, and credits each request by how much it changes the predicted chance of success. Starting from Qwen3.5-4B (0% zero-shot), CaPtain-4B reaches 55.4% held-out LIBERO success after SFT and 73.8% after RL. On LIBERO-PRO, it reaches 78.5%, surpassing several frontier-model agents, and its RL gains persist across three perturbation suites (+17.6 to +21.2 points). Without target-domain training, it attains 83.3% on SimplerEnv WidowX, exceeds frontier models on CaP-Bench, and runs on a real robot after simulation-only adaptation. We release all code, data, and model weights.
Related papers
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-02 · Indexed by Pub-AI: 2026-11-02 UTC
CaPtain: A Pipeline for Post-Training Code-as-Policy Robot Agents
First announced by arXiv: 2026-11-02 · Original record ↗
- arxiv
- 2610.40799
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-02 · First detected: 2026-11-02 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.40799v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Across coding, web, and tool-use tasks, language-model agents are now routinely trained with the same recipe: supervised fine-tuning (SFT) to learn the interface, then multi-turn reinforcement learning (RL) to master it. Code-as-policy (CaP) casts robot manipulation in the same form, as the policy writes code against robot APIs and reads back the results, so the same recipe should apply. Yet existing CaP agents sidestep training: they either prompt closed frontier models or delegate manipulation to pre-trained vision-language-action models (VLAs) and high-level skills, while small open vision-language models (VLMs) fail outright. We present CaPtain, an open post-training pipeline that turns a small open VLM into a capable CaP agent, with no VLA, frontier model, or privileged state at test time. CaPtain trains and deploys the policy inside CaPtain-Harness, which exposes low-level perception, geometry, and guarded-motion primitives across simulation and real hardware. We first align the base model via SFT on synthetic trajectories generated with GPT-6 Astra, then apply multi-turn RL. Multi-turn RL must credit tens of agent requests, whose physical effects often surface turns later, from a single end-of-episode outcome. We therefore introduce a physical critic that reads privileged simulator state, never seen by the policy, and credits each request by how much it changes the predicted chance of success. Starting from Qwen3.5-4B (0% zero-shot), CaPtain-4B reaches 55.4% held-out LIBERO success after SFT and 73.8% after RL. On LIBERO-PRO, it reaches 78.5%, surpassing several frontier-model agents, and its RL gains persist across three perturbation suites (+17.6 to +21.2 points). Without target-domain training, it attains 83.3% on SimplerEnv WidowX, exceeds frontier models on CaP-Bench, and runs on a real robot after simulation-only adaptation. We release all code, data, and model weights.
