Video World Models · 2026-10-16

RIGen: Beyond Action Following with Reciprocal Interaction in Generative Gameplay

Mengfei Li, Yi Zhang, Xingtong Ge, Lunjie Zhu, Fangyu Lin, Peng Li, Guanglu Song, Yu Liu, Wei Xue, Wenhan Luo, Yike Guo

arXiv:2610.20811 PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

RIGen (Reciprocally Interactive Generation) is a framework for reciprocal interaction in generative gameplay. It couples a state/rule engine with an interaction reasoning agent and an action-conditioned audiovisual generator. In an FPS setup, the stateful rules and agent planning determine interaction consequences before generation, then are reconciled using generated visual outcomes. RIGen is trained on a Xonotic gameplay dataset and post-trained for an interactive prototype, RIGen-Arena.

Why it matters

The authors target sustained reciprocal interaction in generative gameplay, where player actions have persistent consequences and the generated world acts according to its own rules and state. RIGen’s closed-loop design—predict interaction events, update an executable state machine, generate audiovisual chunks with action-conditioned guidance, and reconcile with committed visuals—aims to produce more reliable state-dependent combat outcomes than approaches based on audiovisual prediction alone.

Method

State and rule engine maintains persistent numerical variables (health, ammo, strategy modes) and updates them via rules mapping state+events to successor state.
Interaction agent proposes events and actor behavior using recent video, player controls, and persistent state; rule engine computes next planned state and local guidance.
Action-conditioned audiovisual generation produces chunk-wise video/audio under global prompt and local guidance; after generation, verification and state correction reconcile planned state with committed visual outcomes.

Limitation

RIGen is not yet real-time, supports only single-opponent FPS gameplay, and relies on VLM perception and reasoning for interaction decisions and state verification.

Abstract (from arXiv)

Generative gameplay requires more than immediate responses to player actions.Actions should have persistent consequences, while the generated world should act according to its own state and rules, shaping the player's situation and subsequent choices. Video generation captures local dynamics, but limited history, partially observed game state, and the need for contextual behavior decisions make such interactions difficult to sustain through audiovisual prediction alone. We introduce \rig (Reciprocally Interactive Generation), a framework for reciprocal interaction in generative gameplay. \rig combines persistent state and configurable rules, contextual interaction reasoning, and action-conditioned audiovisual generation. Together, these components make interaction consequences and actor behavior jointly determine generated content while maintaining consistency between game state and visible outcomes. We construct a Xonotic gameplay dataset to train an audiovisual generator for learning dynamics and interactions. Further post-training enables \rigarena, an interactive generative first-person shooter (FPS) prototype. Experiments demonstrate more reliable combat outcomes and reciprocal interaction between the player and the generated world. Project page: https://murphylmf.github.io/RIGen/.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-16 UTC

preprint