acceptodds
Under review as a conference paper at ICLR 2027

RSIGame: Controllable Agentic Game Development with Recursive Self-improvement

12 views

Abstract

Recent advances in large language models have made automatic game generation increasingly feasible, yet reliably improving generated games beyond a playable version remains challenging. Naive iterative refinement can easily overfit a small set of test cases, producing fragile games with unresolved bugs, missing behaviors, and poor generalization to broader player interactions. We introduce RSIGame, an autonomous agentic game development framework with recursive self-improvement. RSIGame organizes development into complementary local and global loops. Concretely, a local explore-diagnose-improve loop broadly explores the executable game, diagnoses and prioritizes discovered issues, and performs evidence-grounded revision, where an evolving checklist continually accumulates new testing and improvement guidance. A global loop tracks overall quality, preserves the best checkpoint, and detects saturation or regression over long-horizon development. Beyond test-time improvement, RSIGame further internalizes successful development experience into the generator through training. Across 140 GameCraft-Bench tasks, two game engines, and five generators, consistently improves game quality under matched development budgets. Notably, experience internalization enables Qwen3.8-27B to reach 61.38 on Godot and 58.53 on Phaser, exceeding GPT-5.5 one-shot scores while reducing Qwen's generation tokens by 11 times.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Multimodal Large Language Models, Multi-Agent Systems, Game Generation Agent, Coding Agent, Recursive Self-Improvement

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NFnXOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a structured workflow for agents implementing games from provided text prompts: first identify and diagnose problems, then propose and verify fixes. The workflow couples this with a persistent memory - a checklist - and uses a backup of best performing games when iterating. Validation uses GameCraft-Bench and compares to baseline models with a fixed harness (Play2Code) or no harness. This shows the proposed harness impproves in terms of the benchmark metrics, while reducing cost compared to the fixed harness. Finetuning on traces from the iteration process improves over tuning on good initialization traces alone.

Critical strengths

Shows this harness can improve over baseline harnesses for generating games. Comparisons used existing models in a relatively fair benchmark, offering hope these results hold.

Critical weaknesses
  • New models and harnesses - GPT 6.0 astra, GPT 6.1 sol, Claude Fable 5.1, Claude Opus 5.5 - have improved substantially on this task. It is not clear whether the proposed method still improves over them. This matters as many previously helpful workflows have shown diminishing returns or to be outright harmful to the stronger models, so it is no longer obvious this additional workflow structure would help.
  • The GameCraft-Bench website reports performance from several models (not as new as above) that are competitive with or surpass RSIGame: https://tongxuluo.github.io/gamecraft-bench-website/
    • Opus 5 xhigh (68.44), Fable 5 high (65.72)
  • Evaluations only use a single run for generation. We do not know how much variability there is in outputs across generation runs, only evaluations of the same artifacts. This makes it hard to have high certainty in the results.
Questions for authors

The GameCraft-Bench public leaderboard results conflict those in the paper for GPT 5.5 on high using codex. Why is there this discrepancy?

Does this mean we should expect variance in performance across runs of similar magnitude? The paper reports only a single run of each method (though variability in evaluations is measured), hence my inquiry to understand how much variation in generation to expect and whether differences observed are a strong signal of underlying performance differences.

5 comments
AuthorsOpenReview ↗

Response to Reviewer NFnX (1/3)

We sincerely thank the reviewer for the constructive feedback! We address all concerns raised in the weaknesses and questions below.

Weakness1&2 RSIGame Improvement over New Models

New models and harnesses - GPT 6.0 astra, GPT 6.1 sol, Claude Fable 5.1, Claude Opus 5.5 - have improved substantially on this task. It is not clear whether the proposed method still improves over them. This matters as many previously helpful workflows have shown diminishing returns or to be outright harmful to the stronger models, so it is no longer obvious this additional workflow structure would help. The GameCraft-Bench website reports performance from several models (not as new as above) that are competitive with or surpass RSIGame: https://tongxuluo.github.io/gamecraft-bench-website/ Opus 5 xhigh (68.44), Fable 5 high (65.72)

Following the reviewer's suggestion, we conduct additional experiments to evaluate whether RSIGame remains effective on games generated by stronger new models. Table R1 shows that the improvement of RSIGame consistently persists. On GPT-6.1 Sol, our method RSIGame improves game quality by +15.40 points over the initial generation. Notably, on the stronger Claude Opus 5.5 initialization, the baseline Play2Code reduces game quality by 1.70 points, while our method RSIGame still achieves a +6.85 points improvement. These results demonstrate that RSIGame still remains effective on games generated by stronger new models.

Table-R1. RSIGame Improvement over new models over 3 runs.

GeneratorBase+ Play2Code+ RSIGameΔ vs. Base
Qwen3.8-27B39.26 ± 1.7544.23 ± 1.7852.82 ± 2.87+13.56
GLM-5.3-Flash33.44 ± 2.5438.94 ± 1.6850.38 ± 1.83+16.94
Kimi-K2.629.43 ± 2.0437.61 ± 0.4447.79 ± 3.37+18.36
GPT-5.552.25 ± 0.7748.73 ± 2.8766.75 ± 2.02+14.50
GPT-6.1 sol62.10 ± 1.4761.62 ± 3.0277.50 ± 1.90+15.40
Claude Opus 5.577.28 ± 0.4975.58 ± 2.1984.13 ± 1.54+6.85
AuthorsOpenReview ↗

Response to Reviewer NFnX (2/3)

Weakness3 & Question2: Variability Across Independent Generation Runs

Evaluations only use a single run for generation. We do not know how much variability there is in outputs across generation runs, only evaluations of the same artifacts. This makes it hard to have high certainty in the results. Does this mean we should expect variance in performance across runs of similar magnitude? The paper reports only a single run of each method (though variability in evaluations is measured), hence my inquiry to understand how much variation in generation to expect and whether differences observed are a strong signal of underlying performance differences.

Following the reviewer's suggestion, we conduct additional experiments to evaluate variability across game generation runs. Table-R2 reports the performance across repeated runs. We find that game generation shows small variability across independent runs: across six generators with three repeated-generation runs, the standard deviations of Overall scores range from 0.44 to 3.37 points. In particular, RSIGame's improvement on GPT-5.5 achieves 14.50, the improvement on GPT-6.1 Sol achieves 15.40, and the improvement on Claude Opus 5.5 achieves 6.85.

Conclusion. Generation quality across repeated runs are stable. We have added results and analysis of generation variability in paper's Appendix D5 Line 4607 Page 86 and Table15 in Page 87. We open-source all game generation and scores results across different models over 3 runs in this anonymous website for a more detailed understanding.

Table-R2. RSIGame Stability across different models over 3 runs.

MethodMechanicsDepthVisualsArtOverall ↑
Generator: Codex+Qwen3.8-27B
Base46.0 ± 2.637.5 ± 2.939.1 ± 2.938.1 ± 2.639.26 ± 1.75
+ Play2Code53.8 ± 2.339.6 ± 2.244.4 ± 1.844.5 ± 1.444.23 ± 1.78
+ RSIGame60.9 ± 3.544.1 ± 1.352.0 ± 1.658.3 ± 5.252.82 ± 2.87
Δ+14.9+6.6+12.9+20.2+13.56
Generator: Codex+GLM-5.3-Flash
Base38.2 ± 1.532.5 ± 1.833.1 ± 2.832.4 ± 4.433.44 ± 2.54
+ Play2Code49.4 ± 1.437.7 ± 4.739.1 ± 1.335.6 ± 0.938.94 ± 1.68
+ RSIGame56.9 ± 2.642.6 ± 3.549.8 ± 0.855.5 ± 2.350.38 ± 1.83
Δ+18.7+10.1+16.7+23.1+16.94
Generator: Codex+Kimi-K2.6
Base39.5 ± 3.930.4 ± 3.132.2 ± 1.922.9 ± 0.529.43 ± 2.04
+ Play2Code50.3 ± 1.238.3 ± 1.240.7 ± 0.830.1 ± 0.237.61 ± 0.44
+ RSIGame56.4 ± 4.341.7 ± 1.349.4 ± 3.149.5 ± 5.247.79 ± 3.37
Δ+16.9+11.3+17.2+26.6+18.36
Generator: Codex+GPT-5.5 (high)
Base58.9 ± 1.552.2 ± 2.352.3 ± 0.449.2 ± 1.552.25 ± 0.77
+ Play2Code56.8 ± 4.248.0 ± 2.850.2 ± 2.645.3 ± 2.548.73 ± 2.87
+ RSIGame76.6 ± 2.761.2 ± 3.367.5 ± 0.467.7 ± 2.466.75 ± 2.02
Δ+17.7+9.0+15.2+18.5+14.50
Generator: Codex+GPT-6.1 sol
Base67.7 ± 2.161.0 ± 2.762.3 ± 1.060.6 ± 0.962.10 ± 1.47
+ Play2Code68.7 ± 3.160.6 ± 3.161.7 ± 2.859.5 ± 3.461.62 ± 3.02
+ RSIGame87.1 ± 2.073.6 ± 1.377.5 ± 3.177.2 ± 2.277.50 ± 1.90
Δ+19.4+12.6+15.2+16.6+15.40
Generator: Claude Code+Claude Opus 5.5 (high)
Base80.8 ± 1.875.4 ± 1.076.2 ± 0.878.1 ± 0.977.28 ± 0.49
+ Play2Code82.3 ± 2.073.5 ± 2.773.3 ± 2.275.5 ± 1.875.58 ± 2.19
+ RSIGame91.9 ± 1.583.0 ± 0.481.5 ± 1.582.9 ± 3.184.13 ± 1.54
Δ+11.1+7.6+5.3+4.8+6.85
AuthorsOpenReview ↗

Response to Reviewer NFnX (3/3)

Question1: The discrepancy of GPT5.5 with GameCraft-Bench public leaderboard

The GameCraft-Bench public leaderboard results conflict those in the paper for GPT 5.5 on high using codex. Why is there this discrepancy?

Thanks for the reviewer's careful comparison! We respectfully specify that our evaluation builds upon the official GameCraft-Bench code and method described in paper Section3.2, and further improve gameplay replay reliability and address issues we identified during evaluation. Our paper reports 50.26 for Codex + GPT-5.5 (high), compared with 39.49 on the official GameCraft-Bench leaderboard. However, we also have concerns in the GPT-5.5 performance provided by the public leaderboard, because Codex GPT-5.5 high (39.49) is lower than Deepseek-V4-Flash (40.61) and Kimi-K3 (56.96). We carefully analyze possible factors in the following:

1. Differences in harness: Our experiments use Codex CLI 0.150.1, whereas the official implementation does not specify its Codex version. Its Codex + GPT-5.5 result is dated 2026-04-23, and the newest Codex release on that date was 0.124.0.

2. Effective development budgets: GameCraft-Bench imposes a two-hour wall-clock limit without a fixed tool-call limit. Under this constraint, execution speed, model latency, and system load can affect how much development an agent completes. As shown in Table-R3, our runs consume 3.83M total tokens per task, approximately 3.5× the official 1.08M. The token usage difference indicates substantially different execution characteristics.

Table-R3. Codex + GPT-5.5 (high) on the GameCraft-Bench: public leaderboard vs. our paper.

SourceTasksOverall ScoreTokens / task
Public leaderboard14039.49 [1]1.08M [2]
Ours (paper)14050.26 (Table 1)3.83M (Table 9)
Ratio (ours / leaderboard)3.5×

3. System Load and Gameplay Replay Reliability: We observed that high CPU utilization can slow down game execution and cause stalls during gameplay replay, potentially affecting evaluation scores. To ensure stable gameplay recordings, we carefully controlled execution concurrency and machine load. We also made several adjustments to improve replay reliability, including waiting for the first frame to render before starting replay, extending short demonstrations, and re-recording failed demos caused by environment-related issues. We have explicitly document these modifications in the revised manuscript at Line 4454 Page83.

In conclusion, the discrepancy between the two GPT-5.5 scores may stem from differences in harness versions and execution environments, as discussed above. Importantly, we maintained consistent harness configurations and carefully controlled execution conditions across our experiments, supporting the reliability of our experimental comparisons and conclusions.

[1] GameCraft-Bench Leaderboard, Codex + GPT-5.5 high. https://tongxuluo.github.io/gamecraft-bench-website/

[2] GameCraft-Bench official repository, README, "Token usage" table, GPT-5.5 (Codex): 148.5M input and 2.8M output tokens over 140 tasks. Commit a433475, 2026-08-13. https://github.com/FreedomIntelligence/gamecraft-bench/blob/a43347534374df9a0c1a6c001aa9380862783f6d/README.md#L68-L77

Reviewer_YH2VOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RSIGame combines a local controller-explorer-editor-verifier loop with a global monitor that compares checkpoints, retains improvements, and stops when progress plateaus. It also fine-tunes a Qwen3.8-27B initial generator on generation, planning, and verified improvement trajectories; the development agents remain fixed across generator variants. Experiments use 140 GameCraft-Bench tasks with Godot and Phaser implementations, comparing continued development from identical initial projects against Play2Code. The paper reports improved game quality, stronger initial generation after fine-tuning, and reduced token use under its accounting convention, which excludes cache reads. Policy and training-data ablations, replay variability measurements, an alternative judge, free-play evaluation, and paired statistical analyses further examine the results.

Critical strengths
  1. The local/global division addresses two practical difficulties of iterative development: edits can degrade a functional project, and further editing can cease to improve quality (Section 3; Figures 3 and 5). Checkpoint retention, adaptive focus, and saturation control form a coherent method. The policy and monitor ablations help distinguish these mechanisms from the benefit of simply allowing more editing rounds.
  2. The empirical safeguards are stronger than a single automatic-score comparison. Appendix D includes independently replayed evaluations, alternative-judge comparisons, free-play checks, and paired task-level confidence intervals. Table 1 shows substantial gains in several settings, such as 64.53 versus 50.74 for GPT-5.5-initialized Godot projects against Play2Code. The paper also acknowledges that the Qwen Phaser improvement over Play2Code is not statistically conclusive, appropriately limiting that claim.
Critical weaknesses
  1. The training-data provenance and attribution of the fine-tuning gains need clarification. Table 15 lists 284 released Phaser generation trajectories, whereas Table 16 lists 418 Phaser agent trajectories used for generation training; the mapping between these corpora is not explained. The nested data ablation also trains each progressively larger corpus for one epoch. Adding verified improvement data raises the score from 41.77 to 48.22, but changes both supervision type and training exposure. This supports the combined recipe without isolating the benefit of improvement supervision from additional data or optimization steps.
  2. The headline token-efficiency claim needs qualification. The roughly 11-fold reduction, from 6.41M to 0.57M, concerns the paper's "billable tokens" metric: uncached input plus output. Total tokens including cache reads decline from 11.62M to 8.21M, approximately 1.42-fold, while the cache fraction changes from 46% to 95%. Appendix D.2 transparently defines this convention and notes that cache reads still incur costs. The headline should preserve that distinction: this metric does not establish an 11-fold reduction in total tokens, monetary cost, or execution time.
Questions for authors
  1. How do the released corpus in Table 15 and the actual training mixture in Table 16 map to each other, especially the Phaser counts? Please specify the filtering, versioning, and any additional source data.
  2. Does verified improvement supervision still outperform generation/planning-only training under matched training-token or optimizer-step budgets?
5 comments
AuthorsOpenReview ↗

Response to Reviewer YH2V (1/2)

We sincerely thank the reviewer for the insightful comments and positive feedback! We address all concerns raised in the weaknesses and questions below.

Weakness1.1 & Question1: Training Data Provenance Clarification

The training-data provenance and attribution of the fine-tuning gains need clarification. Table 15 lists 284 released Phaser generation trajectories, whereas Table 16 lists 418 Phaser agent trajectories used for generation training; the mapping between these corpora is not explained. How do the released corpus in Table 15 and the actual training mixture in Table 16 map to each other, especially the Phaser counts? Please specify the filtering, versioning, and any additional source data.

We clarify that the discrepancy between Tables 15 and 16 comes from our data balancing strategy, instead of additional training data sources. As shown in Table-R1, to balance the training data across the two game engines, we downsample Godot trajectories from 1,834 to 876 and upsample Phaser trajectories from 226 to 418. This reduces the Godot-to-Phaser ratio from 8.1:1 to 2.1:1. Thus, Table 15 reports the original released trajectories, while Table 16 reports the training instances after sampling. We have clarified this data balancing procedure in Appendix E1 Line 4802 Page 89 and Table 17 in Page 90, and release the training data IDs at our anonymous repository for reproducibility.

Table-R1. Generation trajectories released in Table 15 vs. used for training in Table 16.

EngineReleased in Table 15Held-out tasks removedAvailable for trainingUsed for training in Table 16Sampling
Godot1,929951,834876Down-sampled 0.48×
Phaser28458226418Up-sampled 1.85×
Godot : Phaser6.8 : 18.1 : 12.1 : 1

Weakness1.2 & Question2: Benefits of Improvement Supervision under Matched Training-token or Optimizer-step Budgets

The nested data ablation also trains each progressively larger corpus for one epoch. Adding verified improvement data raises the score from 41.77 to 48.22, but changes both supervision type and training exposure. This supports the combined recipe without isolating the benefit of improvement supervision from additional data or optimization steps. Does verified improvement supervision still outperform generation/planning-only training under matched training-token or optimizer-step budgets?

Following the reviewer's suggestion, we conduct additional experiments to evaluate the benefits of verified improvement supervision under matched training-token and optimizer-step budgets. Table-R2 shows that the gains persist under both controlled settings: (a) Under matched training token budgets, the outperformance of verified improvement supervision still persists. When increasing generation trajectories to match the same 63.0M training tokens, our full training setting still outperforms Gen by +5.14 points and Gen + Plan by +4.07 points. (b) Under matched optimizer-step budgets, the outperformance of verified improvement supervision still persists. When increasing optimization steps to match the same 2111 training steps, our full training setting still outperforms Gen by +6.06 points and Gen + Plan by +5.02 points. Together, these results show the benefits of improvement supervision collected through our RSI loop. We have added this experiment results in paper's Appendix E.3 Line 4952 Page 92 and Table 21 in Page 93.

Table-R2. Game generation quality under Token-matched and Optimizer-step matched training budgets.

VariantTraining dataOptimizer stepsTraining Data tokensOverall
Qwen3.8-27B (no fine-tuning)–––37.07
Token-matched to 63.0M
+ Gen1,294 + 77 extra generation trajectories1,37163.0M43.08
+ Gen + Plan1,602 + 60 extra generation trajectories1,66263.0M44.15
+ Gen + Plan + Improveunchanged2,11163.0M48.22
Step-matched to 2,111 optimizer steps
+ Gen1,294 generation trajectories, 1.63 epochs2,11159.9M42.16
+ Gen + Plan1,602 rows, 1.32 epochs2,11160.5M43.20
+ Gen + Plan + Improveunchanged2,11163.0M48.22
Original budgets (Table 18)
+ Gen1,294 generation trajectories1,29459.9M41.45
+ Gen + Plan1,602 rows1,60260.5M41.77
+ Gen + Plan + Improve2,111 rows2,11163.0M48.22
AuthorsOpenReview ↗

Response to Reviewer YH2V (2/2)

Weakness 2: Token Reduction and Cache Usage Qualification

The headline token-efficiency claim needs qualification. The roughly 11-fold reduction, from 6.41M to 0.57M, concerns the paper's "billable tokens" metric: uncached input plus output. Total tokens including cache reads decline from 11.62M to 8.21M, approximately 1.42-fold, while the cache fraction changes from 46% to 95%. Appendix D.2 transparently defines this convention and notes that cache reads still incur costs. The headline should preserve that distinction: this metric does not establish an 11-fold reduction in total tokens, monetary cost, or execution time.

We thank the reviewer for highlighting the need to clarify our token-efficiency claim. In Table-R3 we provide a detailed breakdown of token usage and generation costs for the base and fine-tuned Qwen models, inclding: Billable tokens (uncached input tokens + output tokens), Total tokens (billable tokens + cached input tokens), and monetary costs. We report Billable tokens because prefix caching is widely used in real-world agent deployment to reduce inference costs, making this metric relevant to practical usage. Following the reviewer's suggestion, we have explicitly clarified this metric as 11.2× fewer billable tokens, alongside 1.42× fewer total tokens and 2.4× lower estimated API-equivalent cost, in paper's Abstract Lines 3053 Page 57, Introduction Line 3122 Page 58, and Section 4.2 Line 3375 Page 63.

Table-R3. Token usage, Generation cost, and Model Configuration.

GeneratorBillable tokensTotal tokensCostCache hit rateHarnessPrompt cachingContextServing
Qwen3.8-27B6.41M11.62M$1.4746%Codex CLI 0.150.1Yes262kOpenRouter, pinned to DeepInfra
Qwen3.8-27B (SFT)0.57M8.21M$0.6195%Codex CLI 0.150.1Yes262kSelf-hosted vLLM
Reduction11.2×1.42×2.4×

Thank you for the responses and clarifications. The rebuttal has largely addressed my concerns, although I remain cautious about the interpretation of the reported efficiency gains. I also consider the concerns raised by the other reviewers important, so I will maintain my current score. I would also understand if the final decision were not to accept the paper.

Reviewer_R2iFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RSIGAME improves generated games through a local exploration–editing–verification loop and a global monitor for checkpoint selection and stopping. It also fine-tunes a generator on development trajectories. Experiments on 140 tasks across Godot and Phaser show improvements over initial games and Play2Code, supported by ablations, cross-judge evaluation, independent free play, and statistical analyses.

Critical strengths
  1. The improvements are substantial, particularly on Godot, and hold across several initial generators. Starting each method from the same projects makes the comparisons useful, and the development trajectories show why preserving earlier checkpoints matters.
  2. The additional evaluation is convincing. Results remain favorable with a different judge and independently chosen gameplay, which reduces concerns about dependence on a particular evaluator or fixed demonstration scripts.
Critical weaknesses
  1. Play2Code is evaluated on its final checkpoint, while RSIGAME returns a checkpoint selected by its monitor. Giving Play2Code the same monitor and stopping rule would help establish how much of the gain comes from the proposed local development process.
  2. The 11× token reduction needs qualification: it excludes cache reads, while total tokens decrease from 11.62M to 8.21M (Table 9). Also, the fine-tuned model generates the initial game, but GLM remains the editor during development (Table 6). The results therefore demonstrate better initialization more directly than improved refinement ability.
Questions for authors
  1. Does RSIGAME retain its advantage when Play2Code receives the same monitor and stopping policy under comparable total expenditure?
  2. Were caching policies identical for base and fine-tuned Qwen, and what explains the cache-rate change?
  3. Does the training advantage persist when supervision types are compared under matched training-token budgets?
5 comments
AuthorsOpenReview ↗

Response to Reviewer R2iF (1/3)

We sincerely thank the reviewer for the insightful comments and constructive feedback! We address all concerns raised in the weaknesses and questions below.

Weakness 1 & Question1: Performance of Play2Code with the same Monitor and Stopping Rule

Play2Code is evaluated on its final checkpoint, while RSIGAME returns a checkpoint selected by its monitor. Giving Play2Code the same monitor and stopping rule would help establish how much of the gain comes from the proposed local development process.

Following your advices, we conduct the Play2Code with Monitor experiment: we provide Play2Code with the same Global Quality Monitor and saturation-aware Stopping Rule used by RSIGame. We compare both methods with and without the monitor to select checkpoints, starting from identical frozen initial games. Table R1 below reports the results.

We highlight two key findings: (a) RSIGame consistently outperforms Play2Code under identical monitoring policies. With the same Monitor and stopping rule, RSIGame achieves advantage over Play2Code of +19.05 points on GPT-5.5 and +10.08 points on Qwen3.8-27B. The advantage remains positive consistently. (b) Regardless of monitoring, RSIGame's development loop is substantially more effective than Play2Code. RSIGame without the Monitor still outperforms Play2Code with the Monitor by +16.91 points on GPT-5.5 and +8.46 points on Qwen3.8-27B. This demonstrates that RSIGame's gains are not merely due to better checkpoint selection, but stem substantially from its more effective local development process.

Conclusion. These results demonstrate that RSIGame's advantage persists even when both methods share the same monitoring and stopping policy.

Table R1. Comparison of RSIGame and Play2Code under w. and w/o. Monitor Settings

ModelMonitorMethodP0612182430
GPT-5.5w/oPlay2Code51.9651.9948.2049.3846.2751.57
RSIGame51.9660.8762.5661.8861.2168.90
Δ+0.00+8.88+14.36+12.50+14.94+17.33
w/Play2Code51.9651.9951.9951.9951.9951.99
RSIGame51.9665.0467.8570.9570.7071.04
Δ+0.00+13.05+15.86+18.96+18.71+19.05
Qwen3.8-27Bw/oPlay2Code41.7445.5844.0043.4846.8347.00
RSIGame41.7451.7253.2156.1050.1555.46
Δ+0.00+6.14+9.21+12.62+3.33+8.46
w/Play2Code41.7445.5845.5845.5846.8347.00
RSIGame41.7448.5153.9555.6656.3357.07
Δ+0.00+2.92+8.37+10.08+9.50+10.08
AuthorsOpenReview ↗

Response to Reviewer R2iF (2/3)

Weakness 2.1 & Question2: Token Reduction and Cache Usage Clarification

The 11× token reduction needs qualification: it excludes cache reads, while total tokens decrease from 11.62M to 8.21M (Table 9). Were caching policies identical for base and fine-tuned Qwen, and what explains the cache-rate change?

We thank the reviewer for highlighting the need to clarify our token-efficiency claim. In Table-R2 we provide a detailed breakdown of token usage and generation costs for the base and fine-tuned Qwen models, including: Billable tokens (uncached input tokens + output tokens), Total tokens (billable tokens + cached input tokens), and monetary costs. We report Billable tokens because prefix caching is widely used in real-world agent deployment to reduce inference costs, making this metric relevant to practical usage. Following the reviewer's suggestion, we have explicitly clarified this metric as 11.2× fewer billable tokens, alongside 1.42× fewer total tokens and 2.4× lower estimated API-equivalent cost, in paper's Abstract Line 3053 Page 57, Introduction Line3122 Page 58, and Section 4.2 Line 3375 Page 63.

For reviewer's question 2, We ensure consistent agent configurations: both models use the same Codex CLI, context length, and enabled prefix caching. The difference in cache hit rates only arises from how caching operates in different model-serving systems. Specifically, the base model runs on DeepInfra, while the fine-tuned model is hosted on our own GPUs using vLLM. Both systems automatically reuse cached input when possible, but their cache hit rates depend on the serving system and the input patterns during execution. We keep all agent-side configurations consistent and transparently report the resulting cache rates and serving setups in Table R2. Specifically, the cache hit rate for the base Qwen is 46%, and that for fine-tuned Qwen is 95%.

Table-R2. Token usage, Generation cost, and Model Configuration.

GeneratorBillable tokensTotal tokensCostCache hit rateHarnessPrompt cachingContextServing
Qwen3.8-27B6.41M11.62M$1.4746%Codex CLI 0.150.1Yes262kOpenRouter, pinned to DeepInfra
Qwen3.8-27B (SFT)0.57M8.21M$0.6195%Codex CLI 0.150.1Yes262kSelf-hosted vLLM
Reduction11.2×1.42×2.4×

Weakness 2.2 Refinement Ability of Fine-tuned Model

Also, the fine-tuned model generates the initial game, but GLM remains the editor during development (Table 6). The results therefore demonstrate better initialization more directly than improved refinement ability.

We clarify that “the fine-tuned model demonstrates better initialization ability” is precisely the goal of our experience internalization: to enable the model to learn from the iterative game improvement process, internalize knowledge about game development, and ultimately enhance its ability to generate high-quality games directly. As shown in Table-R3, fine-tuning improves Qwen3.8-27B's game initialization quality by 11.15 points, approaching Codex + GPT-5.5 at 50.26. We have clarified this training objective in paper's Abstract Lines 3050 Page 57, and Introduction Line 3116 Page 58, and revise the description in Section 3.2 Line 3277 Page 61 to avoid misunderstanding.

Table-R3. Game Initialization Quality of Qwen3.8-27B before and after Fine-Tuning.

SettingMechanicsDepthVisualsArtOverall
Qwen3.8-27B41.233.438.638.337.07
Qwen3.8-27B (fine-tuned)56.147.549.844.848.22
Δ+14.9+14.1+11.2+6.5+11.15
Codex + GPT-5.558.951.751.444.650.26
AuthorsOpenReview ↗

Response to Reviewer R2iF (3/3)

Question3 Training Advantage under Matched Training-token Budgets

Does the training advantage persist when supervision types are compared under matched training-token budgets?

Following the reviewer's suggestion, we conduct additional experiments to compare the effectiveness of different supervision types under matched training token budgets. Table-R4 shows that (a) Under matched training token budgets, the training advantage of improvement data still persists. When increasing generation trajectories to match the same 63.0M training tokens, our full training setting still outperforms Gen by +5.14 points and Gen + Plan by +4.07 points. (b) Improvement experience provides substantial gains with only a small increase in training tokens. Under the original setting, improvement supervision raises the overall score from 41.77 to 48.22, with only 2.5M additional training tokens. We have added this experiment results in paper's Appendix E.3 Line 4952 Page 92 and Table 21 in Page 93.

Table-R4. Game generation quality under the original and token-matched training budgets.

VariantTraining dataRowsTraining Data tokensOverall
Qwen3.8-27B (no fine-tuning)–––37.07
Token-matched to 63.0M
+ Gen1,294 + 77 extra generation trajectories1,37163.0M43.08
+ Gen + Plan1,602 + 60 extra generation trajectories1,66263.0M44.15
+ Gen + Plan + Improveunchanged2,11163.0M48.22
Original budgets (Table 18)
+ Gen1,294 generation trajectories1,29459.9M41.45
+ Gen + Plan+ 308 plans1,60260.5M41.77
+ Gen + Plan + Improve+ 509 verified improvement rounds2,11163.0M48.22

Comments on the paper

8 comments
AuthorsOpenReview ↗

Response to Reviewer NFnX: Further Discussions (1/2)

Dear reviewer NFnX,

Thank you so much for your reply and follow up questions! We address all questions below.

Results for the strongest models GPT6-Astra and Fable-5.2

We initially prioritized GPT-6.1 Sol and Claude Opus 5.5 due to the high evaluation costs of GPT6-Astra and Fable-5.2. According to the official OpenAI pricing and Anthropic pricing, GPT-6 Astra costs USD 10 per million input tokens and USD 50 per million output tokens, while Claude Fable 5.2 costs USD 8 and USD 40, respectively. Their average game generation costs in our experiments are USD 4.58 and USD 18.60 per task.

Following your suggestions, we extend our experiments to both models, comparing the initial games with Play2Code and RSIGame under the same experimental protocol. Table R7 highlights two key findings.

  • Firstly, RSIGame remains effective on the strongest frontier models. On GPT-6 Astra, RSIGame improves the Overall score from 68.51 to 87.41, achieving a gain of 18.90 points and recovering 60% of the remaining improvement headroom. On Claude Fable 5.2, RSIGame also improves the Overall score from 80.24 to 89.96, achieving a gain of 9.72 points and recovering 49% of the remaining improvement headroom.
  • Secondly, RSIGame substantially improves game quality with little additional cost. For GPT-6 Astra, RSIGame requires only USD 1.24 of additional development cost to achieve an 18.90-point improvement. In comparison, Play2Code requires an additional USD 1.60 per task while reducing game quality. We also observe that GPT-6.1 Sol with RSIGame reaches 77.50 at only USD 1.86 per task, comparable to the 77.28 achieved by Claude Opus 5.5 initial generation at USD 10.96 per task. We add all results for these two models to our anonymous page for a clear understanding.

Conclusion. RSIGame continues to provide substantial improvements on strong frontier models, while requiring relatively little additional development cost compared with expensive initial generation.

Table-R7 Performance and cost of RSIGame on frontier models.

GeneratorSettingOverallΔ vs. BaseHeadroom Improvement RatioCost per task
GPT-6.1 Sol (from Table-R1)Base62.10 ± 1.47––$0.76
+ Play2Code61.62 ± 3.02-0.48-1%$2.43
+ RSIGame77.50 ± 1.90+15.4041%$1.86
GPT-6 AstraBase68.51––$4.58
+ Play2Code67.92-0.59-2%$6.18
+ RSIGame87.41+18.9060%$5.82
Claude Opus 5.5 (from Table-R1)Base77.28 ± 0.49––$10.96
+ Play2Code75.58 ± 2.19-1.70-7%$12.56
+ RSIGame84.13 ± 1.54+6.8530%$12.04
Claude Fable 5.2Base80.24––$18.60
+ Play2Code77.25-2.99-15%$20.20
+ RSIGame89.96+9.7249%$20.81
AuthorsOpenReview ↗

Response to Reviewer NFnX: Further Discussions (2/2)

How much is differences in tokens used by the different models. In Table-R8 below, we compare the token usage, cost, and early stopping rate across different models. Here, ‘early stopping’ refers to the generator terminating a task before producing a runnable game. We highlight two key findings:

  • First, token usage varies substantially across models and does not directly predict game quality. Different models exhibit distinct generation behaviors. GPT models achieve high scores with relatively few tokens, suggesting more efficient task execution. Claude Opus 5.5 takes a more extensive approach, consuming substantially more tokens while producing high-quality games. In contrast, weaker models such as Kimi, GLM, and Qwen consume considerable tokens but achieve lower scores, with 5% to 12% of runs stopping prematurely. This suggests that much of their token usage does not translate into effective task progress.
  • Second, RSIGame improves game quality and reduces premature termination at low additional cost. For example, on Kimi-K2.6, RSIGame reduces the rate of terminating a task before producing a runnable game from 12% to 3% and improves the Overall score from 29.43 to 47.79, with only USD 1.18 in additional development cost. Moreover, even when stronger models already achieve a zero early stopping rate, RSIGame still substantially improves their game quality.

Table-R8 Token Usage, Cost, and Early Stopping Rate across different models.

GeneratorBase total tokensBase costAdditional RSIGame costBase OverallRSIGame OverallEarly stopping rate
Kimi-K2.68.98M$1.87$1.1829.4347.7912% → 3%
GLM-5.3-Flash11.43M$0.20$1.2533.4450.385% → 1%
Qwen3.8-27B11.62M$1.47$1.0939.2652.828% → 0%
GPT-5.53.83M$3.77$0.8852.2566.750% → 0%
GPT-6.1 Sol1.87M$0.76$1.1062.1077.500% → 0%
GPT-6 Astra2.17M$4.58$1.2468.5187.410% → 0%
Claude Opus 5.529.46M$10.96$1.0877.2884.130% → 0%
Claude Fable 5.214.16M$18.60$2.2180.2489.960% → 0%

Diversity Across Independent Generation Runs. We respectfully clarify that similar scores do not necessarily mean that independent runs produce the same game. Each GameCraft-Bench task provides a detailed game design document in the task instruction, specifying the required mechanics, content, controls, and win and loss conditions. The scoring rubrics evaluate these requirements, so independently generated games can receive similar scores even when they differ in other aspects.

To check this, we compare the three independently generated games for each task across six aspects. We assess whether each aspect is the same, partly similar, or different across runs, and whether the shared elements are explicitly required by the task. Table-R9 reveals two key findings.

  • Firstly, similarities mainly come from explicit task requirements. Core mechanics are identical across all three runs for every task, and all shared mechanics are specified in the design document. Since the scoring rubrics focus on these requirements, the games can achieve similar scores despite differences in their implementations.
  • Secondly, differences mainly appear in aspects that the task leaves open. Visual style and art differ across runs in 55% of tasks, theme and names are 35% and UI and screen layout are 25%. These aspects are less constrained by the design document, and their differences are not always reflected in the scoring rubrics.

For a closer look, our anonymous results page provides ten detailed case studies. Each case includes the task design document with explicit requirements highlighted, an analysis of which aspects are similar or different and how they relate to the requirements, and gameplay replays from all three Base and RSIGame runs. We have also added this analysis and annotated screenshots to Appendix F.4 at Line 5778 Page 108 of the revised paper.

Conclusion. Similar scores reflect consistent fulfillment of task requirements, not identical game generation. The games share the required mechanics but differ in aspects where the design document allows more flexibility.

Table-R9. Game Diversity Analysis of games in three runs per task.

AspectSame in all three runsPartly sameDifferentShared part required by the design document
Core mechanics100%0%0%100%
Win/Lose Conditions65%35%0%95%
Content (levels, enemies, items)15%75%10%94%
UI Design10%65%25%87%
Theme and names5%60%35%54%
Visual style10%35%55%67%
All aspects34%45%21%86%
AuthorsOpenReview ↗

Response to Reviewer NFnX: Game Similarity Evaluation Method (1/2)

Dear Reviewer NFnX,

Thank you for your reply and clarification about your question! In this response we focus on this question you mentioned:

Is there any easy way to tell if the games are very different from one another or if the solution is converging to the same game across runs? The paper did not make claims about diversity - I'm just curious to know.

We agree that quantitatively evaluating the similarity between independently generated games is an interesting and meaningful problem. Motivated by your question, we surveyed several related studies to investigate how game similarity can be measured. We summarize three key insights from prior work.

First, game similarity should be evaluated from multiple perspectives. Li et al. [1] survey game diversity evaluation and distinguish between content-centered representations, which capture game elements and structures, and player-centered representations, which capture gameplay behaviors and interaction trajectories. Berns et al. [2] further investigate visual similarity in tile-based games using level screenshots and abstract color patterns, comparing computational metrics against human similarity judgments. These studies suggest that game similarity involves multiple aspects, including content, gameplay, and visual presentation.

Second, game similarity can be quantified through pairwise comparisons using appropriate metrics. Khalifa et al. [3] introduce a pairwise diversity function that measures differences between generated game content on a scale from 0 to 1, with evaluation criteria tailored to different game problems. Beukman et al. [4] propose a gameplay-based diversity metric that compares agent action trajectories using edit distance. Li et al. [1] also summarize metrics such as Jaccard distance for feature sets and edit distance for gameplay sequences. These works highlight the importance of selecting appropriate representations and metrics for different aspects of a game.

Third, diversity across multiple generations can be measured by aggregating pairwise distances. Li et al. [1] summarize Average Pairwise Distance as a widely used approach for quantifying diversity across generated content. For three independent runs, this can be expressed as $ D_{\mathrm{task}} = \frac{d(G_1,G_2)+d(G_1,G_3)+d(G_2,G_3)}{3}, $ where d(Gi,Gj)d(G_i,G_j) measures the difference between two generated games. A higher value indicates greater diversity across runs.

References

[1] Y. Li, Z. Wang, Q. Zhang, B. Yuan, and J. Liu, “Measuring Diversity of Game Scenarios,” IEEE Transactions on Games, vol. 17, no. 3, pp. 558–581, 2025. https://doi.org/10.1109/TG.2025.3543135

[2] S. Berns, V. Volz, L. Tokarchuk, S. Snodgrass, and C. Guckelsberger, “Not All the Same: Understanding and Informing Similarity Estimation in Tile-Based Video Games,” in Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 2024), Article 366, pp. 1–23, 2024. https://doi.org/10.1145/3613904.3642077

[3] Khalifa, A., Gallotta, R., Barthet, M., Liapis, A., Togelius, J., & Yannakakis, G.N. (2025). The Procedural Content Generation Benchmark: An Open-source Testbed for Generative Challenges in Games. Proceedings of the 20th International Conference on the Foundations of Digital Games. https://arxiv.org/abs/2503.21474

[4] M. Beukman, S. James, and C. W. Cleghorn, “Towards Objective Metrics for Procedurally Generated Video Game Levels,” arXiv preprint arXiv:2201.10334, 2022. https://arxiv.org/abs/2201.10334

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.