JEPA · 2026-10-14

Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?

Qingan Liu, Juntong Feng, Yue Zhu, Yujie Cheng, Xinzhe Han, Shuai Hao, Yunzhi Zhuge, Haiwen Diao, Huchuan Lu

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Game2Real-Bench studies whether toy-game visual reasoning reflects and transfers to real-world visual reasoning in VLMs. The authors build a 1,000-sample procedurally generated, programmatically verified toy-game set and align 400 real-world samples by shared reasoning requirements. Evaluating 25 VLMs, they report strong game–real model-level correlations and selective one-shot transfer effects, plus a nearly 24K-sample Game2Real-Dataset.

Why it matters

The authors aim to connect evaluation and training signals between simplified procedural games and real-world visual reasoning. By aligning tasks via reasoning requirements (not visual similarity) and providing executable verification, Game2Real-Bench tests whether performance correspondence and in-context transfer hold, and how effects differ by capability (fine-grained perception, spatial, constraint, sequential planning).

Method

Construct Game2Real-Bench by aligning toy-game and real-world samples through shared reasoning requirements across four capability groups, not visual similarity.
Generate toy-game samples from executable task specifications and verify answers programmatically; curate real-world samples from images/videos/multi-view observations with automated checks and manual review.
Evaluate 25 VLMs on both settings zero-shot, then test one-shot game-to-real transfer via in-context learning on four models and analyze capability-wise changes.

Limitation

The one-shot demonstration results are based on randomly assigned game demonstrations from a fixed pool with the same demonstration-target pairs used for all models, and intermediate solution traces are not included in the demonstration presented to the model.

Abstract (from arXiv)

Visual reasoning is an important capability for vision-language models (VLMs) to understand visual evidence, spatial structure, constraints, and state changes. Existing evaluations mainly rely on real-world data or procedurally generated tasks. Real-world data preserve natural complexity, while procedural games provide controllable generation and verifiable solutions. However, it remains unclear whether abilities measured in toy games correspond to real-world visual reasoning and whether game examples can support transfer. We introduce Game2Real-Bench, a benchmark that studies these connections across four capability groups, including fine-grained perception, spatial reasoning, constraint reasoning, and sequential planning. We construct 1,000 procedurally generated game samples with executable verification and curate 400 real-world samples aligned by shared reasoning requirements rather than visual similarity. We evaluate 25 representative VLMs and observe strong game-to-real consistency, with Pearson and Spearman correlations of 0.935. We further study one-shot game-to-real transfer on four models. Constraint reasoning improves for several models, while fine-grained perception declines with game demonstrations. We also construct Game2Real-Dataset with nearly 24K verified game samples across 18 tasks and 65 QA families. Our benchmark, dataset, and code are available at https://github.com/QingAnLiu/Game2Real-Bench.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC

preprint