Robotics · 2026-11-04
CRONOS: Benchmarking Non-Episodic Reinforcement Learning for Robotic Manipulation
Po-Yi Wu, Djengo Cyun-Jyun Fang, Dian Cheng, Tsung-Wei Ke
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
CRONOS is a simulation benchmark for fine-tuning vision-language-action (VLA) policies with non-episodic scene resets. The authors report that naive non-episodic fine-tuning is ineffective due to drift in object configurations and end-effector poses, but targeted reset mechanisms (end-effector resets, curriculum reset schedules, and heuristic resets of unrecoverable scenes) improve training and generalization.
Why it matters
The paper addresses a practical mismatch between standard RL training (frequent episodic resets) and real-world deployment (resets require human intervention). By focusing on non-episodic scene reset RL for pre-trained VLAs and using a shared-scene, high-fidelity simulator setting, CRONOS makes it possible to study how reset strategies affect learning efficiency and generalization under continual, long-horizon task execution.
Method
- Introduce CRONOS: a shared-scene multi-task simulation benchmark with infrequent/non-episodic scene resets for RL fine-tuning of pre-trained VLAs (OpenVLA, SpatialVLA).
- Evaluate success-rate under independent-task and sequential-task protocols for in-domain vs out-of-domain spatial configurations.
- Study reset mechanisms: end-effector resets (EER), curriculum learning that allocates more frequent resets early (CL), and heuristic scene reset (HSR) that resets when scenes are unrecoverable (e.g., objects fall off table or leave reachable workspace).
Limitation
Experiments are primarily conducted in simulation benchmarks, as they are less time-consuming and offer better reproducibility than real-world benchmarks.
Abstract (from arXiv)
Reinforcement learning (RL) is promising for adapting robot policies to unstructured real-world environments. However, standard RL pipelines rely on episodic training with frequent scene resets, which is impractical for real-world deployment due to the need for substantial human intervention. While prior work has studied non-episodic RL in relatively small policies trained from scratch, its behavior when fine-tuning modern large-scale pre-trained vision-language-action (VLA) models remains underexplored. In this work, we introduce CRONOS, a simulation benchmark for RL fine-tuning of VLAs under non-episodic scene resets. CRONOS leverages high-fidelity physics simulators and adopts shared-scene multi-task settings. Using CRONOS, we show that naively fine-tuning pre-trained policies fails in non-episodic settings due to shifts in object configurations and end-effector poses; however, these challenges can be mitigated through targeted reset allocation and by addressing biases in pre-trained models. Moreover, we find that non-episodic training enhances long-horizon manipulation and improves generalization to previously unseen object configurations and task sequences. Project page: https://embodiedai-ntu.github.io/cronos/
Related papers
- MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation
- AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation
- VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess
- DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
- World In-Context Learning for Agentic Robot Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-03 · Indexed by Pub-AI: 2026-11-04 UTC
CRONOS: Benchmarking Non-Episodic Reinforcement Learning for Robotic Manipulation
Preprint posted: 2026-11-03 · Original record ↗
- arxiv
- 2611.05080
Explicit source versions (1)
- v1 · 2026-11-03
- Preprint available
Source date: 2026-11-03 · First detected: 2026-11-04 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.05080v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Reinforcement learning (RL) is promising for adapting robot policies to unstructured real-world environments. However, standard RL pipelines rely on episodic training with frequent scene resets, which is impractical for real-world deployment due to the need for substantial human intervention. While prior work has studied non-episodic RL in relatively small policies trained from scratch, its behavior when fine-tuning modern large-scale pre-trained vision-language-action (VLA) models remains underexplored. In this work, we introduce CRONOS, a simulation benchmark for RL fine-tuning of VLAs under non-episodic scene resets. CRONOS leverages high-fidelity physics simulators and adopts shared-scene multi-task settings. Using CRONOS, we show that naively fine-tuning pre-trained policies fails in non-episodic settings due to shifts in object configurations and end-effector poses; however, these challenges can be mitigated through targeted reset allocation and by addressing biases in pre-trained models. Moreover, we find that non-episodic training enhances long-horizon manipulation and improves generalization to previously unseen object configurations and task sequences. Project page: https://embodiedai-ntu.github.io/cronos/
