Driving · 2026-10-09

Can Jev be Your Q or Policy in Reinforcement Learning?

Yi Ma, Tianpei Yang, Yaodong Yang, Weixun Wang, Hongyao Tang

arXiv:2610.11692PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors study whether Jev, a frozen decision model that returns calibrated typed answers in a single forward pass, can serve as Q-like or policy-like components inside RL training. They formalize Jev’s usable roles (policy, score rater, meta regulator, transition; excluding cardinal value arithmetic) and propose algorithms using Jev as reference policy, exploration judge, and replay rater. Across nine MiniGrid tasks and three Atari games, Jev training outperforms a standard RL learner.

Why it matters

Token-by-token LLM querying is costly in RL. Jev instead generates no text and answers typed questions in one forward pass. The paper formalizes which RL objects Jev can represent and builds RL training algorithms that use a frozen decision model to guide learning, exploration, and replay prioritization—without training Jev. This positions a decision-model primitive as an RL component rather than a generator to be prompted or a model to be trained.

Method

  • Formal framework mapping Jev’s three answer forms (choice/score/noul) to RL objects (policy, value, reward, transition), and identifies positions usable in RL: Jev as Policy, Score Rater, Meta Regulator, and Transition; excludes cardinal value arithmetic.
  • Algorithms that freeze Jev and use it in three roles: Jev-BC/Jev-PEX/Jev-Mixed Sampling as policy reference; Jev-Gate and Jev-Bonus for exploration; Jev-Reweighted Replay (intuitive/success importance) for replay sampling distribution.
  • Experiments on nine MiniGrid tasks and three Atari games, evaluating learned policies without Jev or auxiliary rewards and measuring success at 10k MiniGrid steps and native return at 50k Atari decisions.

Limitation

The authors state that among the four RL objects, only the cardinal use of a value function is excluded; a score answer supports ranking but not adding/multiplying/averaging for value-function updates.

Abstract (from arXiv)

Foundation models supply reinforcement learning (RL) with priors that mitigate its longstanding weaknesses in sample efficiency and transfer, but their token-by-token generation makes queries sequential and costly. Jev, a recently released decision model, generates nothing and returns calibrated, typed answers in a single forward pass. Existing work studies foundation models in RL either as models to be trained or as generators to be prompted, and Jev belongs to neither category, having so far served only as a black box in single domains. How well such a model decides on its own in RL environments, and how it can improve RL as a component of training, therefore remain unaddressed. To this end, in this paper we first examine the requirements that the objects of an RL system place on the answers they consume, and establish that Jev can fulfill all of them except the cardinal use of a value function. The remaining objects form positions that admit several roles each. We then construct algorithms that employ Jev at three of these positions, as a reference policy, an exploration judge, and a replay rater, to improve sample efficiency, exploration, and learning performance. Across nine MiniGrid tasks and three Atari games, training with Jev outperforms a standard RL learner, including where the learner makes no progress alone, while the model itself remains untrained. To our knowledge, we present the first use of Jev within the RL learning process and establish a frozen decision model as a usable component of RL training, inviting further exploration of how Jev and other advanced decision models can improve RL.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC

preprint

Can Jev be Your Q or Policy in Reinforcement Learning?

Yi Ma, Tianpei Yang, Yaodong Yang, Weixun Wang, Hongyao Tang

Preprint posted: 2026-10-08 · Original record ↗

arxiv
2610.11692
Explicit source versions (1)
  • v1 · 2026-10-08
  1. Preprint available

    Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Foundation models supply reinforcement learning (RL) with priors that mitigate its longstanding weaknesses in sample efficiency and transfer, but their token-by-token generation makes queries sequential and costly. Jev, a recently released decision model, generates nothing and returns calibrated, typed answers in a single forward pass. Existing work studies foundation models in RL either as models to be trained or as generators to be prompted, and Jev belongs to neither category, having so far served only as a black box in single domains. How well such a model decides on its own in RL environments, and how it can improve RL as a component of training, therefore remain unaddressed. To this end, in this paper we first examine the requirements that the objects of an RL system place on the answers they consume, and establish that Jev can fulfill all of them except the cardinal use of a value function. The remaining objects form positions that admit several roles each. We then construct algorithms that employ Jev at three of these positions, as a reference policy, an exploration judge, and a replay rater, to improve sample efficiency, exploration, and learning performance. Across nine MiniGrid tasks and three Atari games, training with Jev outperforms a standard RL learner, including where the learner makes no progress alone, while the model itself remains untrained. To our knowledge, we present the first use of Jev within the RL learning process and establish a frozen decision model as a usable component of RL training, inviting further exploration of how Jev and other advanced decision models can improve RL.