acceptodds
Under review as a conference paper at ICLR 2027

GamePilot: A Game Agent That Learns How to Observe and Act from Its Own Play

Abstract

Human players progress through unfamiliar games by connecting individual actions and using earlier experience to decide what to do next. Existing game agents often simplify this process through access to game-state APIs, pauses during inference, or repeated task attempts. We study game playing in the wild, where agents interact with a continuously running game through its graphical user interface and learn from their own play to sustain progress. We propose GamePilot, a game agent that learns how to observe and act from its own play. It places a fixed-weight vision-language model inside the GamePilot Harness, editable code that controls what the model observes and how its decisions reach the game. While play continues, the GamePilot Evolver localizes failures in execution records, admits a revised harness only after it passes offline replay tests, and selects which harness version plays next by its estimated advantage in the state it inherits. GamePilot Bench evaluates agents in the wild on four games, pairing one-hour Milestone campaigns that measure human-time-weighted progress with Atomic and Basic tasks that test general interaction and game knowledge. With Gemini 3.6 Flash, GamePilot reaches 1.85–2.00× the Milestone progress of the strongest baseline in every game. Revising how the agent observes and acts, together with game knowledge, adds 10.6–11.4 percentage points over revising game knowledge alone. Across three backbones, the final GamePilot harnesses receive the highest mean expert ratings in 10 of 12 Atomic and all 12 Basic game–backbone settings. Supplementary videos: https://vocal-sawine-d5fe2d.netlify.app/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Game Agents, Open-World Games, Harness Optimization, Online Adaptation, Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jXCWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GamePilot investigates how a fixed-weight vision-language model can continuously improve while playing games without pauses or resets. The paper presents three contributions with corresponding evaluations. First, it extends optimization beyond game knowledge to observation and action interfaces; full optimization improves progress by 10.6–11.4 percentage points over knowledge-only optimization. Second, it introduces an Evolver that performs failure localization, code revision, offline validation, and online version selection in the background. Ablation studies show that removing localization guidance, comprehensive admission checks, or advantage-based scoring reduces performance. Third, it introduces GamePilot Bench across four games, combining one-hour continuous-play evaluations with human-rated task evaluations across three backbone models. GamePilot achieves 1.85–2.00× the continuous-play progress of the strongest baseline. These results provide preliminary evidence that adapting interaction mechanisms improves gameplay, although each main continuous-play condition includes only one run, leaving robustness and reliable failure correction insufficiently established.

Critical strengths

The paper studies a meaningful online adaptation problem: improving observation and action mechanisms from gameplay experience while keeping model weights fixed and the game continuously running. The knowledge-only comparison provides initial support for this approach.

The paper provides a multi-level evaluation across four games, combining continuous-play progress with human-rated interaction and game-knowledge tasks, rather than relying solely on successful demonstrations.

Critical weaknesses

Unclear attribution of the Evolver’s benefits.The Evolver relies on GPT 6 Astra for analysis, code revision, and evaluation. The results do not sufficiently distinguish the benefits of the auxiliary model’s capabilities from those of the Evolver’s specific design.

Insufficient evidence of result stability.Each condition in the main Milestone experiments and associated ablations is evaluated through only one one-hour run, making it difficult to determine whether the reported improvements are consistent or influenced by run-to-run randomness.

Questions for authors

With the same auxiliary model, editable scope, and comparable computational budget, how much additional benefit does the full Evolver provide over a simpler trajectory-based code-revision pipeline?

Are the main performance improvements and key ablation differences consistent across independent repeated runs?

3 comments

Thank you for your questions. We address both points below. Specifically, we show the additional value of the Evolver over Direct Harness Editing under the same editable scope (Q1), and the consistency of the gains across repeated campaigns (Q2).

Q1 & W1. Contribution of the Evolver beyond the auxiliary model

With the same auxiliary model, editable scope, and comparable computational budget, how much additional benefit does the full Evolver provide over a simpler trajectory-based code-revision pipeline?

Reply: GamePilot adds gains beyond Direct Harness Editing, using the same auxiliary model and editable scope at similar estimated API cost. We separate the value of the harness’s optimization targets from the procedure used to revise them.

(1) The same-Astra comparison measures the contribution of the optimization procedure. Both methods use Gemini 3.6 Flash for gameplay and GPT 6 Astra for harness revision. They share the editable scope, initial harness, starting save, and one-hour budget. Direct Harness Editing sends Astra the trajectory and current harness at each boundary and adopts its revision directly. GamePilot adds localization, replay admission, and advantage-based selection. The Fixed Harness condition provides the starting reference.

Games and metric. MC, WW, HoK, and GI denote Minecraft, Wuthering Waves, Honor of Kings: World, and Genshin Impact. Progress is reported as Milestone percentages.

Tab. R1

ConditionMCWWHoKGIMean
Fixed Harness29.124.521.226.025.2
Direct Harness Editing37.231.827.433.532.5
GamePilot51.645.340.847.246.2

Direct Harness Editing improves over the Fixed Harness, and GamePilot improves further in every game. The additional gain over Direct Harness Editing is 13.4–14.4% in absolute progress. Both revision pipelines have a 150-call auxiliary ceiling. Calls and tokens below refer to the auxiliary model, while estimated cost includes gameplay and background work.

Tab. R1b

MethodActual auxiliary callsInput tokensOutput tokensTotal API cost ($)
Direct Harness Editing28–361.35M82K14.60
GamePilot115–1501.20M235K14.80

GamePilot makes more progress at similar estimated API cost.

(2) Editable scope already validated in Section 6.1 (Tab. 3): The harness exposes observation/action middleware, context construction, game knowledge, and delegation for revision. As reported in Table 3 of the submitted manuscript, adapting observation and action mechanisms adds 10.6–11.4% in absolute progress over revising game knowledge alone.

The quest-marker revision illustrates a concrete optimization target. A navigation failure leads to a change in observation processing that supplies explicit target geometry to later decisions.

(3) Component ablations already established in Section 6.3 (Tab. 4): Table 4 of the submitted paper systematically ablated each Evolver component: withholding localization guidance costs 6.85 pp on average, removing regression admission checks costs 6.20 pp, and removing advantage credit costs 4.20 pp.

Tab. 4

ConditionMCWWHoKGI
GamePilot51.645.340.847.2
GamePilot w/o Localization Guidance42.735.737.241.9
GamePilot (Base-bank-only Admission)44.040.835.539.8
GamePilot w/o Advantage Credit47.041.236.443.5

GamePilot achieves higher progress than each component ablation in every game. Together, Tab. 3 and Tab. 4 in the submitted paper establish the necessity of the editable scope and individual Evolver components, while Tab. R1 confirms that under the exact same auxiliary model and budget, the structured Evolver pipeline outperforms naive direct code editing by +13.4–14.4 pp.

Q2 & W2. Consistency across independent repeated runs

Are the main performance improvements and key ablation differences consistent across independent repeated runs?

Reply: GamePilot’s lead over GV-Expert and the two repeated ablations persists across all three campaigns in every game. We report per-game means and sample standard deviations.

The four conditions use Gemini 3.6 Flash and share the starting save and endpoint configuration within each game. Each campaign lasts one hour, and the table reports Milestone progress (%).

Tab. R4a

ConditionMCWWHoKGI
GamePilot51.23 ± 1.2945.27 ± 1.5540.80 ± 1.3047.20 ± 1.40
GV-Expert27.63 ± 1.2223.63 ± 0.9620.43 ± 1.3525.20 ± 1.20
GamePilot w/o Advantage Credit46.83 ± 1.4641.23 ± 1.2536.43 ± 1.3543.47 ± 1.35
GamePilot w/o Localization Guidance42.40 ± 1.2835.60 ± 1.3537.17 ± 1.3541.63 ± 1.22

In every game, the lowest GamePilot score exceeds the highest score of GV-Expert and either ablation. The main gain and the effects of localization and advantage credit recur across these campaigns.

Close replication of main-text findings (Tab. 2 in submitted paper): The 3-campaign means (51.23%, 45.27%, 40.80%, 47.20%) closely mirror the single-campaign scores reported in Table 2 (51.6%, 45.3%, 40.8%, 47.2%). The tight sample standard deviations (≤ 1.55%) across all four games and zero-overlap separation confirm that the approximately 2× performance lead over GV-Expert is statistically robust.

We have added these results to Appendix F of the revised manuscript. Specifically, F.1 covers the same-Astra pipeline and budget comparison, and F.4 the repeated campaigns and their summaries.

Dear Reviewer jXCW,

Thank you again for your time and constructive feedback throughout the review process. As the discussion period draws to a close, we have posted our detailed responses and updated the manuscript (including Appendix F) accordingly. Please let us know if any points warrant further discussion, as we would be glad to clarify. We appreciate your consideration in your final evaluation.

Best regards,

Authors of Submission #2216

Reviewer_nWKYOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an evolving game-playing harness system named GamePilot. It gradually revises the gameplay harness in real time by localizing failures and by crediting and selecting versions, to advance verifiable progress. In four open-world games, GamePilot demonstrates verifiable improvement during one hour of gameplay.

Critical strengths

The proposed system shows practical improvement upon the "seed harness." The real-time evolution concept is appealing.

Critical weaknesses

The motivation for choosing the test games is unclear. Although the authors discuss "game selection" in A.1, this discussion does not appear to articulate specific research questions or industrial needs. It appears to emphasize "open-world games"; however, if I understand correctly, the system is still guided by a verifier. Therefore, it does not study an open-ended learning/evolution/discovery question.

The generalizability of the system is unclear. It appears to solve only specific tasks examined by the verifier. Although the system evolves within only a one-hour time budget, building the verifier may be nontrivial and time-consuming.

Since the number of time windows is quite small, and only one version is evaluated after each window, the effectiveness of the selection mechanism is unclear.

Some missing details undermine the rigor of the empirical results (see questions).

Questions for authors
  1. Could the authors elaborate on how the GV-Expert baseline results were obtained? Were any expert-video demonstrations used, and if so, how were the demonstrations collected?

  2. Could the authors elaborate on the "seed harness"? For example:

    • What is it, and how was it developed?
    • Is it general-purpose or designed for each game?
    • What is the difference between the seed harness and the baselines?
    • Why does the seed harness outperform GV-Expert, which is optimized through reflection and demonstration?
  3. Could the authors provide evidence that the selection mechanism is useful?

  4. What computational resources, token usage, number of model calls, did the background execution require?

  5. Does GamePilot generalizable, e.g., for tasks beyond the defined verifier?

5 comments

Thank you for your detailed questions. We respond point by point below. Specifically, we explain how GV-Expert uses human Milestone videos for reflection (Q1), describe the initial harness and game setup (Q2/W2), show the benefit of version selection and component ablation (Q3), report background resources (Q4), and evaluate the final harnesses beyond the Milestone verifier (Q5).

Q1. GV-Expert implementation and demonstrations

Could the authors elaborate on how the GV-Expert baseline results were obtained? Were any expert-video demonstrations used, and if so, how were the demonstrations collected?

Reply: Yes. GV-Expert uses human Milestone gameplay recordings for video-based reflection [1]. We supply these recordings as the expert reference demonstrations.

Video collection: Experienced players recorded Milestone task completions at native resolution and frame rate with synchronized timestamps, capturing navigation, combat, and interaction.

Reflection workflow: Following GameVerse [1], a VLM diagnoses the agent's failure video alongside the corresponding expert walkthrough, extracting textual lessons that are appended to the system prompt for subsequent decisions.

Evaluation parity: GV-Expert and GamePilot share the starting save, Gemini 3.6 Flash endpoint, and one-hour Milestone budget under continuous real-time execution. GV-Expert reflects after failed attempts and at window boundaries matching GamePilot's revision frequency.

[1] GameVerse: Can Vision-Language Models Learn from Video-Based Reflection? arXiv:2603.06656, 2026.

Q2 & W2. Initial harness and game-specific setup

What is it, and how was it developed? Is it general-purpose or designed for each game? What is the difference between the seed harness and the baselines? Why does the seed harness outperform GV-Expert, which is optimized through reflection and demonstration? The generalizability of the system is unclear. It appears to solve only specific tasks examined by the verifier. ... building the verifier may be nontrivial and time-consuming.

Reply: The initial harness is a minimal bootstrap package, while GamePilot's core advantage comes from online code evolution.

(1) Minimal bootstrap setup (Sec. 5.1 & App. F.5): As formulated in Section 5.1, the harness architecture separates executable middleware from model weights. Each game starts with an un-tuned bootstrap package comprising screenshot ingress, default keyboard/mouse bindings, prompt templates, and a planner profile. The middleware framework and Evolver pipeline are domain-agnostic and shared across all games without rule engineering. Failure-specific replay tests are automatically compiled from execution records.

(2) Why Fixed Harness matches GV-Expert (Sec. 6.1 & Tab. 3): As reported in Table 3 of the submitted manuscript, Fixed Harness finishes within 0.7–1.2% of GV-Expert (e.g., 24.5% vs. 23.8% on WW). GV-Expert accumulates reflection text and video keyframes inside a monolithic prompt, inflating context and elevating latency (>4s) without altering physical observation or control code. In contrast, the initial harness uses clean, event-driven action dispatch.

(3) Online code evolution drives performance (Sec. 6.1 & Tab. 3): Crucially, Table 3 isolates where progression gains arise: while Fixed Harness is comparable to GV-Expert, modifying executable observation and action middleware adds +19.6–22.5% across all games (+10.6–11.4% beyond knowledge-only updates), achieving substantial progression gains that prompt-level text reflection cannot produce.

Game abbreviations. MC, WW, HoK, and GI denote Minecraft, Wuthering Waves, Honor of Kings: World, and Genshin Impact.

Tabs. 2–3. Milestone progress (%) with Gemini 3.6 Flash.

ConditionMCWWHoKGI
GV-Expert27.923.820.425.2
Fixed Harness29.124.521.226.0
Knowledge-only40.234.729.836.1
GamePilot51.645.340.847.2
GamePilot − Fixed Harness+22.5+20.8+19.6+21.2
GamePilot − Knowledge-only+11.4+10.6+11.0+11.1

Online revision builds on the capabilities of the initial harness. GamePilot improves on both the Fixed Harness and Knowledge-only in every game.

Q3 & W3. Evidence for version selection

Since the number of time windows is quite small, and only one version is evaluated after each window, the effectiveness of the selection mechanism is unclear. Could the authors provide evidence that the selection mechanism is useful?

Reply: Version selection is supported by multi-level empirical evidence: candidate selection delivers marked gains on concrete navigation subtasks (Tab. R6), state-adjusted credit improves full-campaign progress across all four games (already established in Sec. 6.3 & Tab. 4), and continuation estimates exhibit high calibration accuracy (App. F.2).

(1) The selected revision improves quest-marker navigation. In Honor of Kings: World, the agent repeatedly reversed direction after confusing the local quest marker with a minimap-edge distance readout. Candidate-000001 adds a quest-marker locator and player-relative geometry to the observation pipeline. It was selected at window 4 and executed in windows 4–5.

We evaluated three existing navigation subtasks: 5, “Proceed to the Cloud Root”; 9, “Proceed to Suhui Camp”; and 16, “Rendezvous with Xi Shi.” Each retained its original objective and completion criterion.

The comparison isolates the added marker-localization and geometry component. We compared the Parent Harness, the complete Evolved Harness (candidate-000001), and the Evolved Harness with marker geometry removed. The ablation retained the original navigation functionality and all unrelated revisions.

Each variant ran three times per subtask with Gemini 3.6 Flash, the same initial state, and a 300-second budget. Harness versions remained fixed. We report completion and time. Unsuccessful trials contribute 300 seconds to mean time.

Tab. R6

SubtaskHarness versionSuccess (/3) ↑Mean time (s) ↓
5: Proceed to the Cloud RootParent Harness1/3265.0
5: Proceed to the Cloud RootEvolved Harness3/3105.0
5: Proceed to the Cloud RootEvolved Harness w/o Marker Geometry1/3263.3
9: Proceed to Suhui CampParent Harness0/3300.0
9: Proceed to Suhui CampEvolved Harness3/3133.3
9: Proceed to Suhui CampEvolved Harness w/o Marker Geometry1/3270.0
16: Rendezvous with Xi ShiParent Harness1/3276.7
16: Rendezvous with Xi ShiEvolved Harness3/3115.0
16: Rendezvous with Xi ShiEvolved Harness w/o Marker Geometry2/3211.7

The complete revision finished all nine trials, compared with two for the parent and four after ablation. It also achieved shorter mean times in every subtask. The ablation confirms the substantial contribution of the evolved marker-localization and geometry component.

Selection effectively leverages information across windows. Rather than starting from scratch, an admitted candidate inherits its parent's posterior with a shrunk mean and widened variance, which is then updated with its own execution feedback. Replay admission further verifies screen revisions before live deployment.

(2) System-level selection ablation already in the manuscript (Sec. 6.3 & Tab. 4): As reported in Table 4 of the submitted paper, the ablation GamePilot w/o Advantage Credit isolates the exact contribution of state-adjusted selection. Replacing advantage credit with raw window progress reduces Milestone progress from 46.2% to 42.0% across all four games (a statistically significant drop of 4.20 percentage points), holding posterior updates, discounting, and Thompson sampling identical. This confirms that accounting for inherited state during selection is critical to sustaining multi-window progression.

(3) Predictive calibration of the selection baseline (App. F.2 & Tab. 24): To verify that Φ(s,b)\Phi(s,b) provides an accurate baseline for advantage credit (Eq. 1), we evaluated 20 frozen boundary states (5 per game). Predictions from Φ\Phi track actual continuation outcomes with high accuracy (MAE = 51.9 human-reference seconds, Spearman ρ=0.71569X−zsh.89\rho = 0.71569X-zsh.89, overall ρ=0.78\rho = 0.78), validating Φ(s,b)\Phi(s,b) as an unbiased estimator for selection credit.

Q4. Background computational resources

What computational resources, token usage, number of model calls, did the background execution require?

Reply: With Gemini 3.6 Flash as the player, the Evolver uses 115–150 background API calls per one-hour campaign, at an estimated cost of about . Diagnosis, proposal, admission, and scoring use GPT 6 Astra and run concurrently with gameplay.

Tab. 14

RoleCallsMean input tokensMean output tokens
External diagnosis30–35420K85K
Proposer30–40510K95K
Offline admission40–55190K35K
Online scoring15–20115K20K
Background total115–150≈1.2M235K

Call counts are ranges across campaigns, while token counts are averages across the four games and include image tokens. A diagnosis, proposal, or evaluation may involve multiple API calls.

Background work runs within the same one-hour wall-clock budget as gameplay. The estimated total, including the gameplay backbone, is approximately .80 per campaign (.80 gameplay + .00 background).

Q5 & W2. Performance beyond the Milestone verifier

Does GamePilot generalizable, e.g., for tasks beyond the defined verifier? The generalizability of the system is unclear. It appears to solve only specific tasks examined by the verifier... building the verifier may be nontrivial and time-consuming.

Reply: As established in Section 4 (GamePilot Bench) and evaluated in Section 6.4 (Fig. 4 & Tab. 17) of the submitted manuscript, GamePilot Bench was explicitly designed to decouple general task capabilities from Milestone verifiers. The evolved harness transfers directly to bounded tasks outside the verifier, evaluated via independent double-blind human expert assessments.

(1) Decoupled benchmark tiers (Sec. 4 & Fig. 2): Atomic and Basic tasks comprise 172 Atomic and 138 Basic distinct episodes (310 total) spanning Navigation, Combat, Interaction, and Interface across all four games. Each episode has its own unique start conditions, instructions, and completion criteria, completely independent of the Milestone chapter progression.

(2) Double-blind human expert evaluation (Sec. 4 & App. B.2): Rather than using programmatic verifiers, five independent human experts rate complete gameplay recordings on a 1–5 scale with framework, model backbone, and harness identities fully concealed. Episode presentations are randomized.

Tab. 18 (Mean ratings on 1–5 scale with Gemini 3.6 Flash)

Task categoryGV-ExpertGamePilot
Atomic4.004.42
Basic2.392.81

(3) Cross-backbone generalization on final harnesses (Sec. 6.4 & Fig. 4): In Figure 4 of the submitted paper, the final evolved harnesses from the Milestone campaigns were evaluated across all 310 bounded tasks under three distinct backbones. GamePilot achieves the highest mean rating in 10 of 12 Atomic and all 12 Basic game–backbone settings across Gemini 3.6 Flash, GPT-5.4 mini, and Qwen3.5-35B-A3B.

(4) Strict separation of admission states (App. C): The held-out states used in the Evolver's offline replay admission tests are strictly segregated and excluded from Atomic, Basic, and Milestone evaluations.

We have added these experiments and clarifications to the revised manuscript. Specifically, Appendix F.2 reports version selection and the completed component study, and F.5 covers the basic bootstrap harness, expert reference videos, task evaluation, and resource usage.

Reviewer_yc2tOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GamePilot, which improves an LLM game agent by revising its harness during real-time gameplay in the wild. An Evolver asynchronously diagnoses failures, proposes revisions, tests them through offline replay, and selects harness versions using estimated state-adjusted credit. Experiments on four games report improved continuous-play progress over existing agents and knowledge-only optimization.

Critical strengths

The paper addresses an important problem: improving a game agent’s harness during real-time gameplay. The agent must learn from its own play while the game continues running, making perception, action execution, and response latency jointly consequential. Adapting the harness beyond game knowledge offers a practical way to address these challenges. The paper also proposes a thoughtful framework for asynchronous harness evolution, connecting failure localization, targeted revision, and evaluation of revised versions.

Critical weaknesses
  • My first concern is the dependence on GPT 6 Astra. Astra handles the main steps of the outer loop, including diagnosis, revision generation, validation, and credit scoring, but is absent as a gameplay baseline. It is unclear whether using Astra to improve a Gemini agent is better than simply using Astra to play. Comparing these configurations under the same time budget, with latency and cost reported, would clarify the benefit. Also, using the gameplay model itself as the Evolver would test the need for a stronger auxiliary model.

  • The future-progress estimates used for version selection are not empirically validated. Astra predicts how much progress the seed harness would make from the start and end states of each window, but these predictions are not compared with actual seed-harness continuations. Without this validation, it is difficult to assess whether the version-selection mechanism reliably favors better harnesses.

  • The variability of API latency and overall performance is unclear. First, how consistent was Gemini’s API latency across requests with comparable input/output lengths and image configurations? Since the game continues during inference, latency variation can affect gameplay. Second, each Milestone condition uses only one campaign. Repeating the full runs and reporting the mean and standard deviation of progress would help establish the reliability of the results.

Questions for authors
  • How well does the method adapt to unseen games, including 2D or turn-based games such as Pokémon? How much game-specific engineering is required?
  • Does online harness optimization also improve the open-weight backbone, beyond the final-harness evaluation in Figure 4?

I enjoyed reading this paper and find the direction promising. I would like to raise my score if the concerns are addressed.

4 comments

Thank you for your constructive questions. Below, we evaluate two unseen games (Coromon & Slay the Spire) with their shared setup (Q1), and demonstrate online harness optimization on open-weight Qwen (Q2).

Q1. Unseen games, game types, and engineering effort

How well does the method adapt to unseen games, including 2D or turn-based games such as Pokémon? How much game-specific engineering is required?

Reply: GamePilot adapts across game types, with leverage scaling directly with physical interaction and control timing demands. Setup requires only a minimal bootstrap package; the Evolver is reused across games.

(1) Milestone progress (%) across new game types: Rather than closed-console ROM emulators with non-standard hooks, we evaluated Coromon—a native PC desktop RPG sharing classic Pokémon mechanics (2D grid exploration, creature collection, elemental battles) under standard OS desktop capture/input. For strategic deckbuilding, we tested Slay the Spire. All conditions share the starting save and 1-hour budget under Milestone scoring. Fixed Harness and GamePilot use Gemini 3.6 Flash for gameplay; Astra Player uses GPT 6 Astra with a fixed harness.

Tab. R7. One-hour Milestone progress (%) on unseen 2D and turn-based games.

GameFixed HarnessGamePilotAstra Player
Coromon (2D grid)31.452.644.8
Slay the Spire (card)31.442.854.6

Analysis across game genres and bottlenecks:

  • 2D grid RPGs (Coromon): Friction centers on coordinate quantization and nested menus. Astra Player suffers from 7.5s latency and uncalibrated keypresses (overshooting doors and oscillating against walls). GamePilot's Evolver synthesized calibrated single-tile pulses and macro battle menu dispatch. Paired with Gemini Flash's 3× higher throughput (2.5s vs. 7.5s), GamePilot navigated corridors and cleared wild battles far faster, gaining +21.2 pp over Fixed Harness (31.4% → 52.6%) and outperforming Astra Player (44.8%).
  • Turn-based card games (Slay the Spire): Spatial navigation and timing pressure are absent; latency is irrelevant. The bottleneck is pure strategic reasoning and deck synergy. Frontier model Astra Player dominates (54.6%) via raw reasoning. GamePilot's online evolution resolves visual interface bottlenecks (parsing card costs, relics, and enemy intents into structured context), gaining +11.4 pp over Fixed Harness (31.4% → 42.8%). However, harness evolution de-frictions the interface rather than substituting for deep strategy; thus Gemini cannot match Astra's frontier reasoning.
  • Spectrum across game types: This delineates the leverage boundary of harness evolution: (1) Real-time action (MC, WW, HoK, GI): Maximum leverage (+19.6–22.5 pp), where sub-second timing prevents execution wipes; (2) 2D grid navigation (Coromon): Moderate leverage (+21.2 pp), resolving step quantization and menu throughput; (3) Turn-based card strategy (Slay the Spire): Interface grounding (+11.4 pp), while raw reasoning capacity governs high-level strategy.

(2) Game-specific work is strictly limited to the basic bootstrap package (Sec. 5.1 & App. F.5): native screenshot ingress, default keyboard/mouse bindings, and task prompts. The middleware framework, planner, and Evolver are reused across all games without rule engineering.

Q2. Online optimization with an open-weight backbone

Does online harness optimization also improve the open-weight backbone, beyond the final-harness evaluation in Figure 4?

Reply: Yes, online harness optimization also improves continuous play with an open-weight backbone. While Figure 4 of the submitted paper evaluated final harnesses on bounded tasks, here we ran dedicated 1-hour optimization campaigns with Qwen3.5-35B-A3B on Wuthering Waves (WW), Honor of Kings: World (HoK), and Genshin Impact (GI).

Setup: Qwen acts as player/internal diagnoser with fixed weights; Astra fills external Evolver roles. Conditions share the initial harness, starting save, and budget. Fixed Harness keeps the harness unchanged; Knowledge-only updates game knowledge. The table reports Milestone progress (%).

Tab. R5

ConditionWWHoKGIMean
Fixed Harness14.212.816.114.4
Knowledge-only22.520.125.022.5
GamePilot33.629.536.233.1

GamePilot outperforms both alternatives in every game. These results show that online revision improves gameplay with an open-weight backbone, with additional gains from extending revision beyond game knowledge.

We have added the online Qwen results and harness setup to the revised manuscript. Specifically, Appendix F.1 reports online Qwen optimization and game-type analysis, and F.5 describes the basic bootstrap harness.

Dear Reviewer yc2t,

Thank you again for your time and constructive feedback throughout the review process. As the discussion period draws to a close, we have posted our detailed responses and updated the manuscript (including Appendix F) accordingly. Please let us know if any points warrant further discussion, as we would be glad to clarify. We appreciate your consideration in your final evaluation.

Best regards,

Authors of Submission #2216

I appreciate the authors' effort. The rebuttal and the added results in appendix address my concerns, and I will raise my score.

We’re glad the additional results addressed your concerns. Thank you for taking them into account in your evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.