acceptodds
Withdrawn from ICLR 2027

GameCrafter: An Agentic Framework for Multi-Gameplay Game World Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent advances in generative models and agentic workflows are making it increasingly feasible to automate game creation. These workflows offer a path from high-level user intent to playable worlds. However, existing systems largely focus on isolated assets, scene construction, or games centered on a single gameplay pattern, leaving coherent multi-gameplay 3D world generation underexplored. Generating such worlds requires coordinating scene construction, character creation, gameplay implementation and different softwares. To this end, we present GameCrafter, an agentic framework for scalable multi-gameplay game world generation. The long and interdependent production process makes consistent coordination across specialized agents and tools particularly challenging; GameCrafter addresses this with Specification-Guided Orchestration under shared production requirements. Moreover, visual plausibility alone does not guarantee gameplay readiness, motivating Structure-Aware Asset Synthesis that jointly models appearance and executable structure for environments and characters. These assets are then grounded in a shared executable runtime through Multi-Gameplay World Realization, allowing them to coexist consistently within the same world. Since errors can accumulate across stages and only become apparent after runtime integration, GameCrafter further employs Closed-Loop Quality Control, combining visual inspection, playtesting, and iterative repair. Beyond playable worlds, GameCrafter also produces synchronized multi-view, multimodal data aligned within the same simulation. Experiments demonstrate strong performance in visual quality, gameplay fidelity, and overall playability compared with existing game-generation approaches. Project Page: https://anonymous.4open.science/w/GameCrafter_27805

Primary area:
generative models
Keywords:
Coding Agent, Game World Generation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_HS5iOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Authors introduce GameCrafter, a hierarchical approach to generating playable 3D game worlds through a prompt via a VLM. The first stage is Specification-guided orchestration where the user inputs a prompt, optionally some reference images and assets, and the system extracts game specifications, artifact graphs and designs a workflow orchestrator. The second stage, named structure-aware asset synthesis, synthesises the environment by generating art plans, scene layouts etc.. It also creates or imports character meshes, cleans it up and performs important operations on them to check motion etc. Multi-gameplay world realization is the third step where GameCrafter sets up an UnrealEngine project and intergrates all necessary assets and checks gameplay and physics routines. Lastly, * Closed-loop quality control* reviews visual aspects and playtests the game. These steps results in playable and editable game world, 2 gameplay modes, 3 camera views and 5 modalities, such as RGB, depth etc.

The authors create a benchmark of 10 tasks, and compare there methods with three baselines. The evaluation and playtesting is also done by VLM. The final metrics contain Core Mechanics (M), Content Depth (D), Functional Visuals (V), and Art and Presentation (A). M has 6 checks, D, V and A has 4 checks each, with A having more atomic checks as well.

Critical strengths

The strengths are:

  • Very easy to follow. No unnecessary mathematical jargon. Clearly states what the pipeline is and what it is supposed to do.
  • Very good introduction and related works section.
  • Results are qualitatively great. Editable small worlds are much better than Genie-type uneditable larger worlds as game designers can use them for quick prototyping.
  • Multi-gameplay modes is the strongest contribution as this can be used to create a full game with exploration and boss fights etc.
Critical weaknesses

The weaknesses are:

  • It is not clear if ALL of the metrics are judged via VLM
  • No evaluations of playtesting in the main text.
  • Needs ablations.
Questions for authors
  • Playtesting via VLM is very risky, how do you evaluate the playtester itself?
  • Are all evaluations done by a VLM?
  • Can you measure the diversity of generated worlds?
  • Can you ablate each stage, where necessary?
Reviewer_SATcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GameCrafter, an agentic framework for scalable multi-gameplay game world generation. The framework contains four components: specification-guided orchestration, structure-aware asset synthesis, multi-gameplay world realization, and closed-loop quality control. Specialized agents coordinate through shared specifications and an artifact dependency graph, while external generation tools, Blender, and Unreal Engine provide asset creation, scene construction, animation, gameplay execution, and rendering. The generated worlds also support aligned RGB, depth, semantic, skeleton, and structural-whitebox observations. Empirically, GameCrafter achieves better performance compared to Codex, AutoUE, 3AGameFactory. The paper demonstrates substantial engineering effort and addresses practical challenges in producing usable interactive worlds.

Critical strengths
  1. This paper provides an effective engineering pipeline to produce Multi-Gameplay Game World Generation. GameCrafter coordinates specialized agents and heterogeneous software across asset creation, scene construction, character rigging and animation, game-engine integration, and quality control.
  2. The design of shared specifications provides a sensible mechanism for maintaining consistency across stages.
  3. Empirically, the visual-quality improvement is substantial compared to strong baselines (e.g., 3AGameFactory). The quality improvement is measured both quantitatively and qualitatively.
  4. This paper is well written and easy to follow.
Critical weaknesses
  1. The game setup evaluated seems to be pretty similar: a controllable humanoid player, a boss, and a scene that can be explored and fought. Consequently, the experiments demonstrate diverse worlds built around a particular gameplay configuration, while leaving generalization to different mechanics and their combinations unresolved. Furthermore, the evaluation primarily assesses functional correctness, content coverage, readability, and visual quality. Whether the generated exploration and combat provide meaningful choices, appropriate challenge, or sustained player interest remains unevaluated.
  2. The evaluation benchmark is also relatively small (i.e., 10 tasks), so the variance might be large.
  3. There are no controlled ablations of the central components.
  4. While claiming that GameCrafter produces synchronized multimodal game-world data, the usefulness of the generated data remains unvalidated. The experiments didn't measure downstream benefits for world-model training or other learning tasks.
Questions for authors

N/A

Reviewer_tVSZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes GameCrafter, an agentic pipeline that turns a prompt into an editable UE project. In the generated world, a humanoid player explores the environment and fights one humanoid melee boss, viewed from first-person, third-person, or 2.5D cameras. GameCrafter follows a four-stage pipeline. A design agent writes a structured game specification, and an orchestrator schedules specialist agents over an artifact dependency graph. Environments and characters are then built “structure-aware”: Tripo-generated or Blender-authored geometry is paired with metric layouts, route plans, whitebox proxies, rigs, skin weights, and retargeted animations. These assets are integrated into one UE runtime shared by both gameplay modes and all cameras. Finally, VLM visual review and automated playtests localize failures to the responsible artifact and recompute only its dependents. The same runtime captures aligned RGB, whitebox, depth, semantic, and skeleton channels. On a self-built benchmark scored by a VLM judge with an 18-item rubric adapted from GameCraft-Bench, Codex + GameCrafter scores 87.77 overall, vs. 80.14 (Codex + 3AGameFactory), 76.75 (Codex), and 48.65 (AutoUE).

Critical strengths
  1. I like how clearly the paper explains why “looks right” isn’t the same as “plays right.” A character can look fine in a neutral pose and still deform badly once combat starts, and a scene can look finished while the path to the boss is blocked. Treating game creation as a structured, executable, and verifiable process instead of one-shot generation feels like the right framing to me, and I think it’s a useful perspective for the community.

  2. The pipeline covers a lot of ground: concept art, 3D asset generation, Blender authoring, rigging and skinning, motion retargeting, UE integration, playtesting, and data capture. I think the output is valuable. It’s an editable UE project, not a video or an isolated scene, so people can actually open it, play it, and keep building on it.

  3. Each item is broken down into atomic checks, and unobserved checks count as zero instead of being dropped. Flying a free camera or teleporting doesn’t count as proof that a route is traversable, and build failures are kept separate from missing evidence. I also appreciate that the authors are upfront about the limits of their own evidence.

  4. Potential as a data engine. Because the worlds are executable, the same simulation can output RGB, depth, semantics, skeletons, and whitebox renders that are aligned frame by frame within each camera view. The storage formats are also clearly documented. With more work on the data side, I can see this being useful for research on world models and inverse rendering.

Critical weaknesses
  1. The paper claims that this is the first agentic framework for generating coherent 3D game worlds that support multiple gameplay modes within a shared executable environment. But every benchmark task already asks for exploration plus a boss fight, and the baselines handle it fine: Codex alone gets 98.33 on M and 92.03 on D. Against Codex + 3AGameFactory, GameCrafter only gains 0 / +0.86 / +1.65 on M/D/V, so the abstract’s claim of strong gameplay fidelity feels like an overstatement.

  2. I’m not sure the main gain can be trusted yet. About 7 of the 7.63 points over Codex + 3AGameFactory come from the A score. That score is given by the same model that generates the games, and it hasn’t been calibrated against human ratings. The authors’ own protocol says this should happen before a formal comparison. GameCrafter’s built-in visual reviewer also checks much the same things as the art rubric. On top of that, GameCrafter gets Tripo assets, gpt-image-2 concept art, PBR materials, and ready-made UE animations, and it’s unclear whether the baselines had any of these. Before I fully buy the claim, I’d want to see a human study and an equal-resource baseline.

  3. The paper has four components but never tests any of them on its own, and the authors admit their diagnostic examples aren’t controlled ablations. We also don’t know how often QC catches failures, how well it localizes them, or how many repair rounds it usually takes. A few simple variants would help: no QC, visual review only, playtesting only, no spec/artifact graph, and no structural checks. Each should be reported with M/D/V/A, failure rate, and cost. If these show that each part matters, I’d consider raising my score.

  4. Some numbers are missing. There’s also no variance, no per-difficulty breakdown, no build-failure count, and no evidence coverage, even though the authors’ own protocol asks for some of these. With ten tasks and gaps under 2 points on D and V, it’s hard to tell what’s actually significant. Cost (e.g., time, tokens, API spend) is missing too, so we can’t rule out that the gains just come from more compute.

  5. Some core parts of the method are vague. Who does the orchestration: plain code, a separate LLM, or Codex itself? Is the artifact graph fixed, or built per task? Is failure localization a lookup table like Table 5, or the agent’s own judgment? A full example spec would help a lot.

Questions for authors
  1. Beyond exploration plus a single melee boss, what kinds of gameplay can GameCrafter support that the baselines can’t? If there are such cases, could you report results on them?

  2. Does GameCrafter’s internal visual reviewer use the same model and criteria as the judge? Could you add a blinded human study?

  3. Which generators, asset libraries, PBR materials, and motion packs could each baseline use? Could you add an equal-resource baseline, e.g., Codex with the same Tripo, gpt-image-2, PBR, and motion access but without the GameCrafter workflow?

  4. Could you add the ablations?

  5. How many generation runs and evaluation sessions were used per task? Please report variance, per-task and per-difficulty scores, build failures, and evidence coverage.

  6. Is the orchestrator deterministic code, a separate LLM, or Codex itself?

Reviewer_ZnRGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a system that mimics a game production studio to generate an actually playable game from a prompt. The process proceeds in four main stages. The first is creating a design specification; the second is producing gameplay-ready assets; the third is realizing multiple gameplay modes within a single shared world; and the fourth is closed-loop quality control, in which the system plays the game itself and fixes what it finds. This closely resembles a conventional game development process.

Critical strengths

The paper tackles a problem that naturally arises when using AI to develop games: visually plausible assets alone do not yield a playable product. GameCrafter is, to my knowledge, the first agentic framework that generates a coherent 3D world in which multiple gameplay modes (exploration and boss combat) and multiple camera views coexist within a single shared runtime, rather than a single-pattern or 2D game. The framework deliberately mirrors a real game-production pipeline: a shared specification and artifact dependency graph coordinate heterogeneous tools (Tripo, Blender, Unreal Engine), assets are synthesized with both appearance and executable structure (whitebox proxies, rigs, animations), and closed-loop quality control combines visual review with actual playtesting, repairing only the responsible stage and re-running its dependents. This design is what makes the “playable” claim credible. As a by-product, the verified worlds yield synchronized multimodal data (RGB, whitebox, depth, semantics, skeleton), which is a useful contribution for world-model research.

Critical weaknesses

(1) The evaluation covers only a single gameplay pattern — exploration plus one humanoid melee boss in a single level — with a six-minute playtest budget per game. Nothing is shown about longer, multi-stage games with narrative or progression, so the claim of emulating a full production studio remains unverified beyond short, self-contained levels. (2) There is no human evaluation. All scores come from an LLM judge (Codex on gpt-6-astra), the same model family used for generation, which raises concerns about judge bias; the appendix itself notes that the art-rating anchors were not calibrated against human ratings. With only ten tasks and no reported variance or significance tests, the statistical validity of the reported margins is unclear, and a user study with a reasonable number of participants would be needed. (3) The formalization in Section 3 (Eqs. 1–6) is largely notational: the symbols are introduced but never used in any derivation, objective, or algorithm. Only the evaluation weighting (Eq. 7) is operational. The equations therefore add little beyond the prose and arguably make the method harder, not easier, to follow.

Questions for authors

Have you tried generating longer games — for example, worlds with multiple levels, narrative progression, or longer play sessions beyond the single exploration-plus-boss-fight setting evaluated here? If so, how did the system behave as the scenario length and complexity increased? Have you conducted any user testing with human players? How did users respond to the generated games in terms of playability and visual quality, and how do their judgments compare with the LLM-based evaluator’s scores?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.