Driving · 2026-10-20

Arctic-SWE: Revisiting the Long Horizon Agent Training by Modeling-System Co-Design

Boyi Liu, Yite Wang, Xinyu Lian, Fan Shu, Sida Li, Feng Yan, Mingjia Zhang, Yuxiong He, Zhewei Yao

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Arctic-SWE is a fully asynchronous RL framework built via model–system co-design, using long training on a small overfit task pool to expose instabilities and guide fixes. No single component is new, but together they form a reproducible baseline. On unchanged R2E-Gym tasks, Arctic-SWE trains Qwen3.5-4B to 62.0% pass@1 on SWE-bench Verified, without new data algorithms.

Why it matters

The authors report that early reward gains can conceal recipe failures in long-horizon, asynchronous agent RL. Arctic-SWE aims to establish a reproducible baseline by addressing both algorithmic feedback (termination/reward gating) and system-level issues (numerical alignment, cache invalidation on refresh, and scheduling). This enables clearer attribution when evaluating new algorithms, data, or environments for asynchronous agent RL.

Method

Fully asynchronous RL framework (built on Prime-RL) with an overfitting diagnostic: filter 160 R2E-Gym tasks where Qwen3.5-4B passes unit tests at least once out of four, and train on them to expose instabilities during prolonged training.
Reward and execution gating: require explicit submission; terminate duplicate tool calls/invalid formats/exhausted budgets immediately and assign zero reward before group-relative advantages; add raw token format checks.
System-layer co-design for async training: match training and inference numerics (reduce rollout–trainer mismatch KL), invalidate caches and re-prefill after weight refresh, and use dispatcher load balancing to reduce stale drops and policy lag.

Abstract (from arXiv)

Long-horizon agent training, including software engineering, is advancing rapidly, but the recipe and infrastructure needed for a strong reinforcement-learning baseline remain difficult to reproduce. This makes gains from new algorithms or data hard to distinguish from improvements to an under-tuned baseline. We present Arctic-SWE, a fully asynchronous RL framework that establishes such a baseline through model-system co-design, using overfitting on a small task pool as the testbed. This testbed exposes failures that early reward gains conceal and guides the integration of established components: gated reward, importance-weight control, aligned training and inference engines, and consistent state across policy updates. No single component is new, yet together they form the missing foundation. On raw R2E-Gym data, without new algorithms or data work, Arctic-SWE trains Qwen3.5-4B to 62.0% pass@1 on SWE-bench Verified. This is comparable to concurrent FrogNano (61.5%), which relies on online task synthesis, after about one-sixth as many training rollouts. Building on this foundation, we further filter and refine R2E-Gym, SWE-rebench V2, and Scale-SWE. With this curated subset, the 4B model improves from 47.4%/25.0%/37.9% to 65.0%/38.3%/59.7% on SWE-bench Verified/Pro-v1/Pro-v2, and the recipe also improves Qwen3.5-9B from 56.2%/31.9%/48.8% to 65.7%/39.6%/70.2% on Verified/Pro-v1/Pro-v2. Controlled comparisons show that both curation and test strengthening improve held-out performance, although stronger tests reduce apparent training-task mastery. Arctic-SWE provides a documented starting point for studying better algorithms, data and environments for asynchronous agent RL.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-17 · Indexed by Pub-AI: 2026-10-20 UTC

preprint