- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Recursive Game Creator, a workflow that refines games using LLM agents. The workflow employs a "player" model that writes executable proxy policies to play the game, and a "reviewer" model that guides the loop toward revising the game to make it more enjoyable.
Critical strengths
It automates the entire game-refinement process without human involvement.
Critical weaknesses
It appears to be an engineering tool for game development that directly replaces human players and developers with LLM agents in the standard game-development workflow; the research question and novelty are unclear.
The empirical evidence is currently unconvincing. The evaluated benchmarks, such as GAMECRAFT-BENCH and GameASG-Bench, and the Harness-of-Harness measurement used in Table 1, have not been peer-reviewed or widely used in peer-reviewed academic papers. It is unclear whether these benchmarks can support a scientific contribution. In Table 1, the proposed Recursive Game Creator is compared with other harness softwares using different foundation models, which makes it difficult to draw reliable conclusions from this comparison.
Since the goal is to "produce complete games that people enjoy and want to keep playing", which is subjective, the most reliable way I am aware of to verify this is a user study. However, this paper does not include such a study
Questions for authors
- Could authors explain how "This provides a concrete step toward producing complete games that people enjoy and want to keep playing." (in abstract)? Especially, there are several statement claimed that the "reviewer" does not evaluate human enjoyment (e.g., line 157, line 293)