acceptodds
Withdrawn from ICLR 2027

RSI-GameCreator: Product-Level AI-Native Game Creation with Self-Evolution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Coding agents significantly accelerate game creation. Coding agents can now build basic draft games quickly. However, there is still a huge gap between a quick AI-generated draft and enjoyable, product-level game. In this work, we introduce RSI-GameCreator, a self-evolving agent team with workflow that mirrors a real game studio. Our system uses four agent roles: Designer, Builder, Player, and Reviewer. The Designer and Builder create the game content and code. The Player and Reviewer test the game and provide feedback. This creates a closed loop that improves the gaming experience round by round. Our results show that this self-evolving mechanism successfully builds mature games that human players actually enjoy.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM agents, multi-agent systems, self-evolving agents, agentic game generation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_qK8uOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Recursive Game Creator, a workflow that refines games using LLM agents. The workflow employs a "player" model that writes executable proxy policies to play the game, and a "reviewer" model that guides the loop toward revising the game to make it more enjoyable.

Critical strengths

It automates the entire game-refinement process without human involvement.

Critical weaknesses

It appears to be an engineering tool for game development that directly replaces human players and developers with LLM agents in the standard game-development workflow; the research question and novelty are unclear.

The empirical evidence is currently unconvincing. The evaluated benchmarks, such as GAMECRAFT-BENCH and GameASG-Bench, and the Harness-of-Harness measurement used in Table 1, have not been peer-reviewed or widely used in peer-reviewed academic papers. It is unclear whether these benchmarks can support a scientific contribution. In Table 1, the proposed Recursive Game Creator is compared with other harness softwares using different foundation models, which makes it difficult to draw reliable conclusions from this comparison.

Since the goal is to "produce complete games that people enjoy and want to keep playing", which is subjective, the most reliable way I am aware of to verify this is a user study. However, this paper does not include such a study

Questions for authors
  • Could authors explain how "This provides a concrete step toward producing complete games that people enjoy and want to keep playing." (in abstract)? Especially, there are several statement claimed that the "reviewer" does not evaluate human enjoyment (e.g., line 157, line 293)
Reviewer_YAVGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper covers a new agentic harness using programatic policies. The authors briefly describe the harness and present some outputs. They also present the results of running the harness with GPT-6 on two existing benchmarks.

Critical strengths

It is a benefit of the paper that they make use of two existing benchmarks rather than inventing their own for the paper.

Critical weaknesses

This current paper draft has a number of major weaknesses holding it back.

The first major weakness is a lack of engagement with prior work. The authors essentially re-invent programmatic policies [1] as their major technical contribution. However, no part of the paper acknowledges this or engages with this area. Further, the paper mischaracterizes prior work. The authors cover Yannakakis & Togelius, 2011 and Barthet et al., 2024 as if these are comparable works. However Yannakakis & Togelius 2011 define the area of experience-driven PCG, which this paper is an example of though they make use of simulated users.

A second major weakness is the lack of any reproducibility details. There is no appendix with prompts, no code presented, and no in-depth description of said code. It is impossible to reproduce this work from the paper description, which is a pillar of good science.

A third major weakness is the presentation. The authors appear to have just copy and pasted output from an LLM in multiple places, with section 4.5 being the most egregious example. Even when output does not appear to be directly copy and pasted it is very unclear. For example, section 4.4 does not describe what model is the "expert" or where these scores arose from. If this was a human subject study that is especially unclear and it's unclear what the methodology was, if so.

A fourth major weakness is the results, which are mixed at best compared to prior work. The clarity issue arises again, with no explanation for things why RGC was evaluated on fewer tasks than the baselines on GameASG-Bench or why GPT-6 Astra (high) was not run for Simulation or Adventure in GameCraft-Bench.

  1. Carvalho, Tales, Kenneth Tjhia, and Levi Lelis. "Reclaiming the source of programmatic policies: Programmatic versus latent spaces." International Conference on Learning Representations. Vol. 2024. 2024.
Questions for authors
  1. Can the authors clarify what their evaluation methodology was, and how this impacted how they presented their results?
Reviewer_cqJuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Recursive Game Creator, a game development harness designed to address the question of how to evaluate player experience. They build it with four roles: Designer, Builder, Player, and Reviewer, mirroring a real game studio. They use two levels of evaluation: a meta-evaluation protocol uses success rates and play time as shared signals, and the Reviewer interprets them with a game-specific rubric, a set of checks suited to each game. To support these checks, the Player writes proxy policies, or programs that play the game, to collect play records and screenshots without a language-model call at each step. The Reviewer compares the candidate and retained game under anonymous labels using the same game-specific experience criteria, then returns a preference judgment and concrete revision objectives. The Designer and Builder turn the feedback into changes, and the next round checks the result, which evolves the game artifact while keeping the underlying language-model parameters fixed. They evaluate Recursive Game Creator on GameCraft-Bench and GameASG-Bench, and then examine Player exploration, game examples, and expert scores across refinement rounds.

Critical strengths
  • The coding-native Player is a good fit for the problem. Writing policies once and running them many times at game speed, in parallel, removes the per-step model call that makes screenshot agents slow.

  • The full harness reaches strong scores on GameCraft-Bench, improves steadily across refinement rounds in every category, and is competitive on GameASG-Bench runtime checks while using a lower reasoning setting. Even with the caveats below, this shows the loop produces working games that score well under external rubrics.

Critical weaknesses
  • The paper does not show that its key parts work. All reported gains come from the full workflow, so we cannot tell whether the coding-native Player, the game-specific rubric, or simply more rounds of agent editing drive the improvement. A same-model, same-budget run without the Player, and one without the Reviewer, are the minimum needed to support the stated contributions. Without them, the paper is closer to a system report than a validated method.

  • The paper is framed around player experience and human enjoyment, but provides almost no human evidence. Proxy play time is admitted not to show human interest, the expert scoring is unspecified and very small, and the comparisons to prior systems use different task sets and settings. Combined with the unfinished sections and boilerplate statements, the current version does not yet back its title claim of product-level, experience-oriented games.

Questions for authors
  • How many experts rated the games in Table 3, how many games were rated, were raters blind to the round order, and how was average playtime measured?

  • How were the 21 GameCraft-Bench tasks and the 46 GameASG-Bench tasks chosen out of 45 and 47, and what happened to the missing GameASG task?

  • Do the proxy policies read privileged game state that a human player would not see, and how is the command-line interface added to each generated game?

  • What is the total cost per game in tokens, wall-clock time, and rounds, including policy writing and repair?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.