Driving · 2026-10-14
RoboFQE: Rollout-Free Evaluation of Visual Robot Policies from Offline Trajectories
Yunfu Deng, Sangwoo Shin, Brahma S. Pavse, Josiah P. Hanna
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
RoboFQE is a rollout-free off-policy evaluator for visual robot policies from offline trajectories. It fits fitted Q-evaluation (FQE) without rolling out the target policy, using a pretrained DINOv3 vision backbone for high-dimensional observations and a learned terminal success detector (LRM) to provide reward signals when ground-truth labels are absent. The authors report accurate discounted return estimates and preserved policy rankings on LIBERO and CALVIN and on real robot block-stacking tasks.
Why it matters
Evaluating robot policies often requires costly rollouts in real worlds, simulators, or learned world models, where prediction errors can compound. The authors propose an alternative that estimates discounted expected return directly from offline trajectories using value-based OPE, aiming to avoid rollout infrastructure and reduce error accumulation over long horizons for visual manipulation.
Method
- Introduces RoboFQE: fitted Q-evaluation that estimates discounted return from offline visual robot trajectories, without learning a dynamics model or estimating behavior-policy density.
- Uses a pretrained vision foundation model (DINOv3) as the action-value representation backbone, and fuses proprioception/actions (via learned embeddings) with language for Qθ(x,a,l).
- When rewards are missing, uses a learned terminal success detector (LRM) to infer terminal success labels and constructs reward-augmented transitions for FQE training.
Limitation
The authors report that RoboFQE can fail when the evaluated policy selects out-of-support actions not covered by the offline dataset: the Q-function extrapolates and can assign positive value to actions with no basis in observed outcomes, inverting pairwise orderings (e.g., adding a random zero-success policy).
Abstract (from arXiv)
Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon. In this paper, we study off-policy evaluation as an alternative to rollout-based robot-policy evaluation methods. Specifically, we introduce RoboFQE, an evaluator that estimates the discounted expected return of a target policy from offline robot trajectories without estimating a behavior-policy density, combining fitted Q-evaluation with a pretrained vision foundation model and learned terminal success annotation to bridge the gap between standard OPE benchmarks and continuous-control visual manipulation. Controlled experiments across simulated and real-world manipulation tasks show that RoboFQE produces accurate discounted-return estimates and preserves policy rankings even on long-horizon manipulation tasks where video world-model evaluators are degraded by autoregressive prediction errors. Ablation experiments show that pretrained vision encoders substantially outperform from-scratch alternatives, particularly on long-horizon tasks, and that learned terminal success annotation enables evaluation without ground-truth reward labels with only slight reduction of accuracy.
Related papers
- WorldLine: Action-Driven Visual Simulation for Robotic Manipulation
- Learning Robotic World Models with Semantic and Visual Feedback
- RoboSeed: Multidimensional Evaluation of Vision-Language-Action and World-Action Models: Capability, Resource Affordability, and Research Evidence
- ProWorld: Multi-View World Model with Proprioceptive Feedback for Closed-Loop Policy Evaluation
- RoboRender: Robot-Oriented Video Generation for Visual Sim-to-Real Transfer
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC
RoboFQE: Rollout-Free Evaluation of Visual Robot Policies from Offline Trajectories
Preprint posted: 2026-10-13 · Original record ↗
- arxiv
- 2610.18699
Explicit source versions (1)
- v1 · 2026-10-13
- Preprint available
Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon. In this paper, we study off-policy evaluation as an alternative to rollout-based robot-policy evaluation methods. Specifically, we introduce RoboFQE, an evaluator that estimates the discounted expected return of a target policy from offline robot trajectories without estimating a behavior-policy density, combining fitted Q-evaluation with a pretrained vision foundation model and learned terminal success annotation to bridge the gap between standard OPE benchmarks and continuous-control visual manipulation. Controlled experiments across simulated and real-world manipulation tasks show that RoboFQE produces accurate discounted-return estimates and preserves policy rankings even on long-horizon manipulation tasks where video world-model evaluators are degraded by autoregressive prediction errors. Ablation experiments show that pretrained vision encoders substantially outperform from-scratch alternatives, particularly on long-horizon tasks, and that learned terminal success annotation enables evaluation without ground-truth reward labels with only slight reduction of accuracy.
