- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper argues that generative gameplay requires reciprocity: player actions should have persistent consequences, and the generated world should act according to its own rules and state, in turn shaping the player's situation. RIGen couples three layers: (i) a state-and-rule engine that maintains persistent gameplay variables (player/enemy health, ammunition, strategy modes) and applies configurable transition rules; (ii) an interaction agent (InternVL3-8B) that infers action consequences, plans actor behavior, and reconciles predicted state with the generated video; and (iii) an audiovisual generator (LTX-2.3 22B, with FiLM-based movement/camera conditioning, button cross-attention, chunk-wise prompt routing, and causal sliding-window generation). The authors build a synchronized Xonotic dataset (95,731 recordings, 176.8 h, with audio, controls, engine state, and interaction annotations), post-train a few-step distilled generator, and demonstrate RIGen-Arena, an interactive one-on-one FPS prototype. On 100 controlled combat scenarios (50 survival / 50 death targets), RIGen reaches 81% Final-state Agreement (FSA) vs. 29% for LingBot-World 2.0 and 4% for SCOPE; fixed-generator ablations reduce FSA to 39% (motion guidance only), 35% (semantic prompting), 64% (post-hoc updates), and 76% (without reconciliation). Visual-quality metrics are largely unchanged across ablations.
Critical strengths
-
The system decomposition and the "predict-then-reconcile" loop are a concrete answer to persistent consequences. Separating executable rules from learned generation, predicting interaction outcomes before the chunk is generated, and reconciling predicted state with committed visual evidence is a sensible architecture, and the fixed-generator ablations substantiate that each part matters: pre-generation prediction + rules + reconciliation reaches 81% FSA, while post-hoc updates reach 64% and removing reconciliation reaches 76%. This is consequential because it identifies which mechanism enables state-dependent outcomes, rather than attributing gains to a stronger generator.
-
The outcome-control margin and the dataset are substantial contributions. 81% vs. 29%/4% FSA on prescribed combat outcomes, combined with a 176.8-hour synchronized dataset spanning local dynamics, controlled interactions, and sustained arena combat (with audio, controls, poses, and state labels), gives the community both a demonstrated capability and the data to study it. The qualitative state-dependent branches (ammo gating, low-health retreat) are convincing illustrations of gameplay that is genuinely state-driven rather than text-steered.
Critical weaknesses
-
The "reciprocal" claim is only partially evaluated. In the quantitative benchmark, enemy motion is fixed to source annotations and all methods receive prescribed outcomes as text; this measures realization of specified outcomes, not the world acting on the player through autonomous behavior. The genuinely reciprocal part - RIGen-Arena with autonomous opponent decisions, respawns, and player defeat - is shown only qualitatively (Fig. 5). Since "reciprocal interaction" is the paper's central framing and title, the absence of any quantitative evaluation of Arena (e.g., state-transition accuracy, consistency between HUD/state and visible events, survival/death statistics, or human play ratings over multiple sessions) is a material gap.
-
The primary metrics are LLM-judged with no human validation. FSA and Enemy Behavior are produced by Qwen3.8-27B on generated video, with unjudgeable outcomes counted as failures. This conflates generation failure with judging failure, and a 5-point difference such as 76% vs. 81% may be within judge noise. Because FSA is the main evidence for the paper's core claim, a human agreement audit on a subset (even 100 clips, reporting agreement with the judge and the frequency of unjudgeable cases) is needed. The privileged reference-guidance variant reaching 89/90/88 also suggests considerable headroom that is not discussed.
Questions for authors
-
Can you provide any quantitative evaluation of RIGen-Arena with autonomous opponent behavior - e.g., over N sessions, the rate at which rule-engine state (health, ammo, respawns) matches visible outcomes, or human ratings of state-dependent behavior? Even a small study would directly support the reciprocity claim.
-
What is the human-judge agreement for FSA/EB on a held-out subset, and what fraction of FSA failures are "unjudgeable" rather than visibly incorrect outcomes?
-
Since the benchmark fixes enemy motion to source annotations, how much does the interaction agent's behavior planning contribute to FSA? Can you run the same 100 scenarios with autonomous behavior and report outcome realization?
-
FSA-D (72%) is much lower than FSA-S (90%). What are the dominant failure modes for death targets - e.g., delayed collapse, occlusion treated as death, or respawn artifacts?
-
The generator receives local guidance that already describes outcomes (e.g., "enemy retreats", "player is hit"). How do you ensure the generator is not merely following text, and how much of the state consistency comes from the prompt compiler rather than learned dynamics?
2 comments
Please let us know if you have any remaining questions or concerns. We would be happy to provide further clarification.
We thank the reviewer for the detailed feedback on reciprocal interaction and evaluation. We respond below with additional Arena results, human evaluations, and clarifications of the benchmark setting and failure modes.
Q1. Can you quantitatively evaluate reciprocal interaction in RIGen-Arena with autonomous opponents?
We evaluate 20 continuous RIGen-Arena rollouts of 60s each with autonomous opponents and repeated encounters after respawn. We extract chunks corresponding to the events and behaviors below and use Qwen3.8-27B to check whether the generated visuals match the engine state after reconciliation or the active enemy strategy.
Player HP is updated by the rule engine. The hit/injury metric checks only whether injury feedback appears in the video.
The rule engine manages ammunition and deducts one round per permitted shot, regardless of whether it hits. When ammo is depleted, firing inputs are blocked before reaching the generator.
Q2. How well do FSA/EB judgments agree with human labels, and what fraction of FSA failures are unjudgeable?
Human and VLM evaluations yield similar overall FSA/EB scores and the same method ranking, as shown in the table below. SCOPE has difficulty preserving enemy appearance after attacks, often producing blurred results that the VLM marks as unjudgeable. Human review also cannot reliably classify these cases as alive or dead, so they receive no FSA credit.
Q3. How does autonomous planning affect outcome realization on the same 100 scenarios?
The 100 scenarios use fixed player controls and prescribed enemy motion. Once autonomous planning changes the enemy trajectory, the resulting hits and damage can change, making the original engine labels inapplicable. FSA therefore cannot be reliably evaluated against those labels.
Q4. Why is FSA-D lower than FSA-S, and what are the dominant failure modes?
Q5. Is the generator merely following outcome guidance, and how much of the state consistency comes from the prompt compiler versus learned dynamics?
This decoupling of the prompt compiler and video dynamics is intentional. The engine computes state transitions, and the prompt compiler provides outcome guidance that mainly contributes to state consistency. The generator learns to realize these outcomes under player controls. Learned dynamics also constrain the visual outcome. For example, a visibly missed shot may leave the enemy alive despite incorrect death guidance, triggering reconciliation of the engine state. Table 3 reports 89% FSA with reference guidance. With the same generator, full RIGen achieves 81% FSA versus 39% with motion guidance alone.
Q6. What explains the FSA gap between RIGen and the reference-guidance variant?
Reference guidance uses source hit annotations and rules to determine outcomes in advance, bypassing online hit assessment. In full RIGen, missed hit detections can undercount damage and prevent a kill within the limited firing budget in the original action trajectories.