Robotics · 2026-11-04

CRONOS: Benchmarking Non-Episodic Reinforcement Learning for Robotic Manipulation

Po-Yi Wu, Djengo Cyun-Jyun Fang, Dian Cheng, Tsung-Wei Ke

arXiv:2611.05080PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

CRONOS is a simulation benchmark for fine-tuning vision-language-action (VLA) policies with non-episodic scene resets. The authors report that naive non-episodic fine-tuning is ineffective due to drift in object configurations and end-effector poses, but targeted reset mechanisms (end-effector resets, curriculum reset schedules, and heuristic resets of unrecoverable scenes) improve training and generalization.

Why it matters

The paper addresses a practical mismatch between standard RL training (frequent episodic resets) and real-world deployment (resets require human intervention). By focusing on non-episodic scene reset RL for pre-trained VLAs and using a shared-scene, high-fidelity simulator setting, CRONOS makes it possible to study how reset strategies affect learning efficiency and generalization under continual, long-horizon task execution.

Method

  • Introduce CRONOS: a shared-scene multi-task simulation benchmark with infrequent/non-episodic scene resets for RL fine-tuning of pre-trained VLAs (OpenVLA, SpatialVLA).
  • Evaluate success-rate under independent-task and sequential-task protocols for in-domain vs out-of-domain spatial configurations.
  • Study reset mechanisms: end-effector resets (EER), curriculum learning that allocates more frequent resets early (CL), and heuristic scene reset (HSR) that resets when scenes are unrecoverable (e.g., objects fall off table or leave reachable workspace).

Limitation

Experiments are primarily conducted in simulation benchmarks, as they are less time-consuming and offer better reproducibility than real-world benchmarks.

Abstract (from arXiv)

Reinforcement learning (RL) is promising for adapting robot policies to unstructured real-world environments. However, standard RL pipelines rely on episodic training with frequent scene resets, which is impractical for real-world deployment due to the need for substantial human intervention. While prior work has studied non-episodic RL in relatively small policies trained from scratch, its behavior when fine-tuning modern large-scale pre-trained vision-language-action (VLA) models remains underexplored. In this work, we introduce CRONOS, a simulation benchmark for RL fine-tuning of VLAs under non-episodic scene resets. CRONOS leverages high-fidelity physics simulators and adopts shared-scene multi-task settings. Using CRONOS, we show that naively fine-tuning pre-trained policies fails in non-episodic settings due to shifts in object configurations and end-effector poses; however, these challenges can be mitigated through targeted reset allocation and by addressing biases in pre-trained models. Moreover, we find that non-episodic training enhances long-horizon manipulation and improves generalization to previously unseen object configurations and task sequences. Project page: https://embodiedai-ntu.github.io/cronos/

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-03 · Indexed by Pub-AI: 2026-11-04 UTC

preprint

CRONOS: Benchmarking Non-Episodic Reinforcement Learning for Robotic Manipulation

Po-Yi Wu, Djengo Cyun-Jyun Fang, Dian Cheng, Tsung-Wei Ke

Preprint posted: 2026-11-03 · Original record ↗

arxiv
2611.05080
Explicit source versions (1)
  • v1 · 2026-11-03
  1. Preprint available

    Source date: 2026-11-03 · First detected: 2026-11-04 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Reinforcement learning (RL) is promising for adapting robot policies to unstructured real-world environments. However, standard RL pipelines rely on episodic training with frequent scene resets, which is impractical for real-world deployment due to the need for substantial human intervention. While prior work has studied non-episodic RL in relatively small policies trained from scratch, its behavior when fine-tuning modern large-scale pre-trained vision-language-action (VLA) models remains underexplored. In this work, we introduce CRONOS, a simulation benchmark for RL fine-tuning of VLAs under non-episodic scene resets. CRONOS leverages high-fidelity physics simulators and adopts shared-scene multi-task settings. Using CRONOS, we show that naively fine-tuning pre-trained policies fails in non-episodic settings due to shifts in object configurations and end-effector poses; however, these challenges can be mitigated through targeted reset allocation and by addressing biases in pre-trained models. Moreover, we find that non-episodic training enhances long-horizon manipulation and improves generalization to previously unseen object configurations and task sequences. Project page: https://embodiedai-ntu.github.io/cronos/