- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Summary: This paper introduces GameBoy Worlds, the first benchmark platform designed to evaluate and foster autonomous self-improvement in embodied agents within complex video games. Traditional embodied AI heavily relies on human supervision, expert demonstrations, or hand-crafted reward signals. To bypass this bottleneck, GameBoy Worlds tests whether an agent can start with zero game-specific knowledge, demonstrations, or external rewards, and progressively acquire actionable skills and long-horizon capabilities purely through self-directed interaction and experience.
Contributions:
- GameBoy Worlds-Execution (Short-Horizon Task Execution): A benchmark comprising 10 games across 5 distinct series and 500 hand-crafted tasks. Agents are given access to a training game per series without any explicit task examples or guidance, and must subsequently execute short-horizon tasks (navigation, combat, interaction) in test environments.
- GameBoy Worlds-Playthrough (End-to-End Gameplay): A long-horizon challenge evaluating start-to-finish game completion across two fan-made Pokémon ROM hacks (Pokémon Brown and Pokémon Prism) designed to minimize pretraining data contamination and test true deployment-time learning.
- Evaluation of Existing Paradigms: Systematic implementation and benchmarking of current self-improvement methods—including world modeling, autonomous skill discovery (PAE), and a novel curiosity-driven exploration pipeline that distills game insights into guidance documents.
Validation:
- Execution Results: Out-of-the-box frontier models (such as Gemini-3.6-Flash, GPT-5-Mini, and Claude-4.5-Haiku) struggle significantly, achieving less than 50% average task completion due to multimodal grounding and perception failures. Standard self-improvement methods either fail or cause catastrophic performance degradation (e.g., skill discovery dropping performance by over 40%), while the proposed curiosity-to-documentation pipeline yields only modest and inconsistent gains (+2.0% average), proving the execution leg remains largely unsolved.
- Playthrough Results: Through rigorous contamination checks using 400 questions, the authors confirm that while models possess high parametric knowledge of official games like Pokémon Red, they lack procedural knowledge for the fan-made hacks (<30% accuracy). When deployed on the playthrough challenge, an advanced agentic pipeline with hierarchical subgoals and memory successfully navigated early milestones in contaminated official games but completely failed to clear even the first major milestone (obtaining the starter Pokémon) in the unfamiliar fan-made games.
Critical strengths
-
Novel Benchmark Design for Self-Improvement: The introduction of GameBoy Worlds provides a rigorous and much-needed testbed to evaluate autonomous, zero-shot self-improvement in embodied agents without relying on human demonstrations, hand-crafted external rewards, or expert priors. This addresses a critical bottleneck in scaling embodied AI.
-
Rigorous Handling of Data Contamination: The inclusion of the Playthrough leg utilizing fan-made ROM hacks (Pokémon Brown and Pokémon Prism) alongside systematic contamination checks effectively isolates true deployment-time learning and active knowledge acquisition from mere memorization of pretraining corpora.
Critical weaknesses
-
Limited Novelty in Methodological Outcomes: Despite introducing a comprehensive benchmark, the evaluated self-improvement methods (world modeling, skill discovery, and curiosity-guided documentation) either fail catastrophically or yield only marginal, inconsistent gains, leaving the benchmark largely unsolved without providing a robust, generalizable algorithmic solution to overcome these failure modes.
-
Evaluation Constraints Driven by Emulator Parsing: Relying on screen-region captures and visual state comparisons restricts task verification to events with immediate visual feedback, potentially filtering out complex, long-horizon internal state transitions that lack distinct graphical representations.
Questions for authors
-
Impact of Verification Bottlenecks on Task Distribution: In Appendix A.5, the authors note that task verification relies on immediate visual feedback (e.g., distinct screen changes or dialogue boxes), which inherently excludes long-horizon internal state transitions. Could the authors clarify how severely this constraint skews the execution benchmark away from complex, non-visual strategic progress (such as inventory management or hidden status modifications)? Furthermore, what fraction of potential tasks had to be discarded due to this limitation?
-
Generalization vs. Overfitting in Curiosity-Based Exploration: While the curiosity-to-documentation pipeline achieved modest gains on certain games, it resulted in severe performance degradation on others (e.g., -29% on Deja Vu 1). Can the authors clarify whether the negative transfer is primarily caused by noisy insight extraction from the VLM or by the inherent brittleness of PPO-discovered trajectories when applied across different task distributions?
Summary of revisions, and request for engagement
We thank all the reviewers for their comments. As highlighted in the individual responses below, here are the full summaries of how we have revised the paper and added concrete experiments to address the reviewers' concerns:
Reviewer CLYD
Weakness 1: The experiments show that the tested methods struggle, but do not clearly explain why.
Response: We respond with concrete quantitative comparisons for each method.
We show:
World Models:
Autonomous Skill Acquisition:
Curiosity-based guidance documents:
This also answers (Question 1 and 2)
Weakness 2: The playthrough results depend on one model and one agent design, with run counts, budgets, and stopping criteria insufficiently reported.
Response:
Reviewer us1E
Weakness 1: Models are inconsistent across experiments
Response:
Weakness 2: The Playthrough benchmark evaluation is limited
Response:
Weakness 3: No discussion on contamination in execution benchmark
Response: Showed that contamination is not the primary determinant of performance in the execution benchmark (unlike with the Playthrough benchmark) with multiple experiments.
Weakness 4: Related work lacks discussion
Response: Discussed related work in more detail and added citations
Question 1: Can Self-Improvement be run on other models
Response: Done, results are consistent across 3 models.
Question 2: How does contamination affect the Execution benchmark?
Response: Answered with Weakness 3. Contamination is not the primary determinant of performance in GameBoyWorlds-Execution
Question 3: Could the authors define an evaluation protocol for the Playthrough benchmark
Response: Yes. Done above in response to weakness 2.
Reviewer Wm3c:
Limited Informativeness:
Response: We have provided a list of insights we think one may draw from our work
Weakness 1: All self-improvement methods fail. Limited methodological novelty.
Response: Stressed that the failure of standard approaches is a meaningful finding, and we shed light on which parts of each pipeline goes wrong.
Weakness 2: Limited task distribution
Response: Clarified the limitation regarding task distribution. We can and do support long-horizon tasks.
Ethics Review:
Response: Ethics Statement added to the paper. The statement explicitly discusses the risks of copyright and provides information to help researchers engage in more responsible research practices.
We have made an effort to respond to all of the reviewers comprehensively, with additional experiments when possible. Seeing how the discussion period is nearly over, we ask the reviewers to let us know whether we have answered their questions and concerned, and if not, give us a chance to follow up.