Driving · 2026-10-15
Player-1: Learning to Play from Synthetic Experience
Mingyu Ouyang, Yiran Wang, Siyuan Hu, Xingyilang Yin, Huiqiang Sun, Guangzhi Wang, Junfu Pu, Cheng Cheng, Honghao Chen, Ying Shan, Mike Zheng Shou, Hwee Tou Ng
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Player-1 is a post-training framework for game control beyond demonstrations using synthetic experience. It extends a computer-use action space with relative pointer motion, sustained inputs, and composite actions, then fine-tunes on 6,044 curated gameplay trajectories. It constructs Player-1-Gym with 16 synthetic 2D/3D environments and programmatically verifiable rewards for multi-environment PPO (200 RL steps), improving MineDojo-300 success and transferring beyond the RL suite.
Why it matters
The authors target gaps in game-agent training: demonstrations cover limited dynamic states and scalable interactive environments are difficult to obtain. Player-1 aims to learn transferable keyboard/mouse control from synthetic environments with programmatically verifiable rewards, improving performance beyond supervised fine-tuning and generalizing to external benchmarks.
Method
- Extend computer-use actions with relative pointer motion, sustained inputs (holds), and composite actions executed concurrently in one control step.
- Initialize via SFT on 6,044 curated gameplay trajectories annotated with task instructions and per-turn visual reasoning; action-balanced sampling.
- Train with PPO on Player-1-Gym: 16 synthetic 2D/3D environments with randomized observations/control dynamics and programmatically verifiable, privileged-state-free rewards; use per-environment advantage normalization.
Abstract (from arXiv)
Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain. We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience. We extend the computer-use action space with relative pointer motion, sustained inputs, and composite actions. For supervised initialization, we fine-tune computer-use models on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning. We construct Player-1-Gym, sixteen synthetic 2D and 3D environments designed around control demands shared across games. Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment reinforcement learning with PPO. The resulting Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points, respectively. Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld. Further analyses show that broader synthetic environment coverage improves transfer, while the use of composite actions that combine multiple keyboard and mouse inputs increases during RL. We demonstrate that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite.
Related papers
- AgentGarten: Code Worlds for Evolving Agents
- LiveAct: Towards Real-Time Multi-Agent World Models with Heterogeneous Action Interfaces
- Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition
- DashVMC: Real-Time Discrete World Model Control in Geometry Dash
- Code to Control: Synthesizing Parameterized Reactive Controllers
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-15 UTC
Player-1: Learning to Play from Synthetic Experience
Preprint posted: 2026-10-14 · Original record ↗
- arxiv
- 2610.20584
Explicit source versions (1)
- v1 · 2026-10-14
- Preprint available
Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain. We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience. We extend the computer-use action space with relative pointer motion, sustained inputs, and composite actions. For supervised initialization, we fine-tune computer-use models on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning. We construct Player-1-Gym, sixteen synthetic 2D and 3D environments designed around control demands shared across games. Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment reinforcement learning with PPO. The resulting Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points, respectively. Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld. Further analyses show that broader synthetic environment coverage improves transfer, while the use of composite actions that combine multiple keyboard and mouse inputs increases during RL. We demonstrate that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite.