acceptodds
Withdrawn from ICLR 2027

GameHatch3D: An End-to-End Agentic Framework for 3D Game Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Traditional 3D game development requires substantial human effort in asset creation, programming, and playtesting. Recent agentic frameworks have shown promise in automating the planning, execution, and revision of complex systems. However, end-to-end automated 3D game generation remains at an early stage and presents additional challenges. It requires agents to jointly construct complex scenes and characters, model physical behavior, and implement gameplay logic. The resulting combinatorial complexity increases reasoning overhead and reduces generation reliability. To address these challenges, we propose GameHatch3D, an end-to-end agentic framework for generating 3D games from a text prompt. GameHatch3D first extracts game-design cues from the input prompt and consolidates them into a structured game specification. Guided by this specification, HatchCraft, our game-construction harness, coordinates reusable Environment, Character, Physics, and Rule Skills to construct the core components of a functional game. These skills encode domain-specific construction knowledge, enabling more reliable and efficient game construction. To support automated refinement, we further equip the agent with an engine-guided playtesting mechanism that enables dual-path self-correction. Engine-provided state information guides navigation planning and action generation, while playtesting observations and failures are used to diagnose visual and gameplay defects. For evaluation, we establish a benchmark comprising diverse gameplay demonstrations and curated high-level game descriptions, together with a comprehensive evaluation protocol. Experiments show that GameHatch3D generates higher-quality and more playable games than general-purpose agentic baseline while consuming fewer tokens.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Game Agent, Game Generation, End-to-End Agentic Framework

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_MfvPOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper starts by mining traces from agents tasked with creating games from prompts to produce 4 skills for game authoring. These are incorporated into a harness for game creation that uses playtesting of the games for feedback to improve the games.

Validation compares the harness to calling models via API and using an existing harness (Cursor). On the custom set of tasks the proposed harness improves scores according to tests on types of interactions, playtest with automated controller, and visual assessments. A small user study indicates preference for games from the harness compared to Cursor.

Critical strengths

None.

Critical weaknesses

The paper does not compare to other models using their native harnesses. New models and harnesses - GPT 6.0 astra, GPT 6.1 sol, Claude Fable 5.1, Claude Opus 5.5 - have improved substantially on game generation. It is not clear whether the proposed method still improves over them. This matters as many previously helpful workflows have shown diminishing returns or to be outright harmful to the stronger models, so it is no longer obvious this additional workflow structure would help.

Evaluation does not assess if users consider the games to be complete / "done". Even if they prefer a result it does not mean it is considered a complete game. Ideally the model would have been evaluated on a game generation benchmark like GameCraft-Bench or GameDevBench.

Questions for authors

What effort level was used for models? This makes a substantial difference to performance.

What variability was there over the repeated runs? Results only report means.

Reviewer_eRMFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents GameHatch3D, a framework for generating playable Three.js games from text specifications. Four reusable skills cover environments, characters, physics, and rules. HatchCraft coordinates their sequential execution, permits cross-component updates, and assembles structured outputs into executable games. Engine-guided playtesting supplies observations for model-generated gameplay and aesthetic feedback, followed by iterative repair. Evaluation uses 210 video-derived specifications across 21 genres, two model backends, comparisons with direct generation and general-purpose coding agents, component ablations, and a 20-participant preference study. The reported results show higher physical correctness, objective completion, and visual quality with lower token use than the agentic baselines.

Critical strengths
  1. The framework addresses coupled 3D implementation failures through an explicit shared representation and cross-component correction, rather than treating scene appearance, physics, and rules independently (Sections 3.2-3.3). Table 4 is particularly useful: with identical skills and assembler and without refinement, removing cross-component updates reduces physical correctness from 68.4 to 52.1. This directly supports the value of coordination within the proposed system.
  2. The experimental coverage is reasonably broad for an end-to-end system. Table 1 reports consistent improvements across two backends, including total scores of 73.1 versus 54.6 for GPT-5.5 and 74.5 versus 55.7 for Claude against Cursor. Appendix A.2 adds Codex, and Tables 2-3 examine skills, feedback, and refinement rounds. Appendix A.1 explicitly describes development/evaluation separation and library freezing.
Critical weaknesses
  1. Key details needed to reproduce the comparison remain underspecified (Sections 3.2-3.3, 4.1-4.2; Appendix A.1). The appendix describes skills distilled from 100 manually refined games, but does not provide their full contents or a complete runtime/schema specification. Although three refinement rounds and a shared evaluation budget are reported, the per-run generation limits and playtesting horizon are not specified. It is also unclear how baseline projects are instrumented to expose equivalent state and objective information to the shared controller. These omissions affect the reproducibility and interpretation of the quality-efficiency comparison.
  2. The accuracy of the automated gameplay measurements needs independent validation (Sections 3.3 and 4.1; Table 5). Objective completion is checked using runtime state, with objectives registered by generated rule code; physical correctness combines runtime observations with model-assisted assessment. An audit against independently established specification-level outcomes is not reported. The preference study offers complementary evidence, but game selection, participant assignment, blinding, and uncertainty are unspecified. Moreover, three playthroughs per artifact assess replay variability, not variation across independent generation runs. The aggregate means therefore leave uncertainty about the reliability and generality of the reported advantages.
Questions for authors
  1. Can you provide the skill files, runtime interface and component schema, controller configuration, and baseline instrumentation procedure? What checks ensure that reported objective completion agrees with the requested rules for every method?
  2. How were games and participants assigned in the preference study? An independent audit of physical and gameplay outcomes, paired task-level confidence intervals, and repeated generation on a representative subset would strengthen confidence in the results.
Reviewer_DmunOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduced a new Harness for generating 3D games in threes and compared it to direct agentic agents and Cursor harness with the same models. It shows that games created by GameHatch3D are overall visually better and more playable and use fewer tokens overall than directly using Claude or GPT with/without cursor. Also, it ran a small user study that shows that people prefered the generated games from the other system.

Critical strengths

The paper tackles a new area of using LLMs to generate fully playable games in 3D in the Three.js engine. Having a study that compares different models and different harness skills is important to understand how to move that forward. They also generated tons of games in different genres to show the coverage.

Critical weaknesses

The background ignores a huge amount of previous work in multiple facets, such as rule generation from the PCG community; it also ignores orchestration work by Antonios and others, like generating GVGAI games, puzzle script games, etc., especially ones with domain-specific language.

Since the main contribution is the harness and the skills, I would focus more on making part of the paper focused on what the most important points in these skills are and why are they since it said that these points came from analyzing and generating tons of games with agentic systems.

Also, I would go a bit deeper in analyzing the generated games with the system through qualitative analysis, as it is not clear if any of the generated games are covered by something in the skills. What happens if you generate a game that is not similar to any other known game or niche game?

Another issue is that using LLMs and big models as the judges might reinforce games in a specific direction. Not having a lot of qualitative analysis for the results make me wonder if the LLMs are generating anything new or just rehashing old games.

Questions for authors

I would ask to try to include stuff from the appendix that makes the paper clearer and fix the weaknesses.

Reviewer_TwUgOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents GameHatch3D, a system that uses large language models to build playable 3D games from a text description. Instead of asking the model to write everything from scratch, it gives the model skills for the environment, characters, physics and game rules. After building a game, the system automatically plays it, looks for visual and gameplay problems, and fixes them. On a new benchmark of 210 game descriptions, the authors report clearly better games than the coding assistants Cursor and Codex, while using about three times fewer tokens.

Critical strengths
  • The motivation is clear and convincing.
  • Having the system play its own games to find bugs is an intuitive and practical idea.
  • The ablation studies are easy to follow and show what each part contributes.
  • Results hold across two different language models.
Critical weaknesses
  • The skills were built from 100 games the authors manually debugged, but Cursor and Codex received only the text prompt. When the skills are removed, the system performs about the same as Cursor. So I could not tell whether the gains come from the framework itself or from the expert knowledge written into the skills. Giving Cursor the same skill would make the comparison more convincing.
  • The individual techniques (task-specific skills, self-testing and repair) are known to help LLM agents, and the ablations suggest most of the gain comes from the hand-written skill content. The paper would be stronger if it clarified what is new beyond applying these ideas to 3D games.
  • The aesthetic metric (from VBench) was designed for natural videos. I am not sure it is meaningful for simple, stylized game graphics.
  • The paper reports no variation across repeated runs and no results per game genre.
  • The user study was too shallow (20 users) and it lacks basic details, such as how many games were shown.
Questions for authors
  • How do the baselines perform when given the same skills?
  • Do results hold for game genres that were not used to build the skills?
  • How much human effort went into building the four skills (e.g., hours spent manually debugging the 100 development games), and would the skills need to be rebuilt for new game types or a different engine such as Unity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.