acceptodds
Withdrawn from ICLR 2027

Evolving Gaming Harnesses via Knowledge–Probing Co-Evolution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-horizon gaming requires agents to diagnose stalled execution, recover from failures, and reuse experience across tasks. Existing execution harnesses typically rely on fixed intervention rules, limiting their ability to adapt to visually ambiguous failures. We introduce Knowledge–Probing Co-Evolution Harness (), a framework that improves the execution harness surrounding a frozen vision-language-action model through interaction. separates fast action execution from a slow research loop that diagnoses failures, proposes harness modifications, and verifies them through isolated paired sandbox evaluations. Its central mechanism is knowledge–probing co-evolution, which turns outcome-driven harness search into evidence-guided evolution: transferable recovery knowledge provides intervention hypotheses, active probes test when they apply, and validated outcomes improve future knowledge and probe selection. We instantiate with a gaming policy trained through supervised fine-tuning and reinforcement learning, and keep its parameters frozen throughout harness adaptation. Experiments on 149 Minecraft tasks spanning mining, combat, and crafting show that knowledge-guided harness evolution substantially improves task coverage over both the bare actor and a manually specified fixed harness, while the complete framework further improves over knowledge-only evolution. Frozen evaluation on unseen tasks further shows that the evolved harness transfers beyond tasks encountered during adaptation. These results support external harness evolution as a means of converting grounded interaction feedback into transferable execution capability without further updating the underlying actor. The project page is available at https://gaming-harness.github.io/adaptive-gaming-harnesses/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Gaming Agent, Harness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2TBMOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes an evolving gaming harness framework based on knowledge-probing co-evolution. When an agent encounters execution failures, an action probe executes exploratory actions in the environment to collect failure evidence. A knowledge probe then converts this evidence into knowledge. Subsequently, a Vision-Language Researcher leverages the informations to propose update strategies for the harness. Finally, a Paired Validation mechanism evaluates whether these proposed updates should be updated to the harness.

While the high-level intuition is digestible, the paper suffers from pervasive presentation deficiencies and technical ambiguities:

  1. The paper states that "Auto-K searches for harness modifications from accumulated knowledge but does not improve its probing policy from probe outcomes." However, what constitutes this "accumulated knowledge"? How is it represented and updated? How does it concretely parameterize modifications to the harness? None of these critical details are explained.

  2. My understanding is that Section 4.2 intends to deploy an action policy and a knowledge policy to execute actions and converts into knowledge. However, lines 273–310 are exceedingly convoluted and poorly explained, making it nearly impossible to grasp the underlying technical formulation.

  3. In Section 4.3, what exactly is the "proposal" generated by the Vision-Language Researcher? The manuscript fails to define the configuration space of the harness.

  4. The paper is replete with incomprehensible statements, such as "legacy protocol" and "mtv1" (Lines 377–379), "fixed" and "Auto-K" baselines (Lines 419–421).

Due to severe presentation and clarity issues, I cannot review this work. Moreover, the anonymous code repository contains files like auto_research_claw.py, indicating that this may be an auto-generated research project. However, the authors have failed to include this in AI Use Statement. Therefore, I believe this manuscript should be desk-rejected, and the PC should strictly penalize this irresponsible behavior.

Critical strengths

None

Critical weaknesses

See summary.

Questions for authors

None

8 comments

We thank the reviewer for raising concerns about clarity and transparency. The relevant definitions were distributed across the main text and appendix; we consolidate their operational meaning below and in the revised manuscript.

For Comment 1: Knowledge representation and use

  • Recovery knowledge has an explicit representation and promotion rule. Section 4.4 defines records through triggering context, recovery strategy, and execution evidence. After task-specific details are removed, a recovery is promoted when it improves performance on a validation seed or succeeds on another task.

    During execution, the knowledge policy selects and instantiates these strategies. During search, accumulated knowledge joins failure trajectories and probe evidence in guiding configuration edits. Figure 5 provides examples; Appendix F and Table 14 trace backup_and_turn and reacquire_and_center through sand-mining execution and feedback updates. Table 12 distinguishes selecting existing knowledge from creating or revising records.

For Comment 2: Action policy, knowledge policy, and equations

  • The two policies select different interventions and maintain separate learning statistics. The action policy selects diagnostic actions. The knowledge policy selects recovery strategies conditioned on the failure context; generating or revising knowledge is handled through proposals, validation, and promotion.
EquationOperational role
Eq. (7)Contextual UCB selection using local utility, a global prior, and an exploration bonus
Eq. (8)Immediate utility from progress, reward, novelty, cost, and failure
Eq. (9)Terminal-success credit discounted by subsequent selection events within the same module

Knowledge probing uses no novelty term in our configuration. Each selection increments its count once; immediate and delayed feedback accumulate in the local and global value statistics used by later UCB decisions. These updates change the external harness state while VLA parameters remain frozen.

The credit clock is now explicit. Action and knowledge probing use separate event sequences, with discount distance Mm−jM_m-j, rather than elapsed VLA decision steps. Consequently, the two knowledge selections at turns 23 and 24 receive 0.80.8 and 1.01.0 when success occurs at turn 29. The eight earlier action probes receive total delayed credit 4.1611392. Section 4.2, Appendix G.3, and Table 14 consistently describe this implemented rule.

For Comment 3: Proposals and the configuration space

  • A proposal is a bounded, executable configuration edit. Section 4.3 and Appendix D specify the diagnosis, evidence-linked rationale, and editable fields: task/decomposition/recovery prompts, runtime instructions, and designated probing, memory, verification, recovery, and runtime settings.

Every candidate must include a prompt or instruction edit. VLA parameters, the action space, task definitions, environment, and evaluation protocol are excluded. Appendix D provides a concrete sand-mining JSON patch:

Edited fieldSand-mining example
runtime.instructionLocate and center the sand block before breaking it
recovery_promptReorient and retry when the target is not centered

The same frozen VLA executes the candidate, and paired comparison with the incumbent determines acceptance. This specifies both the editable scope and how a proposed modification is tested.

For Comment 4: Protocol and baseline terminology

  • Input protocol and harness method are separate choices.
TermDefinitionLocation
legacyCurrent observation and task instructionSection 5.1, Appendix A.1
mtv1Adds up to ten recent observation–action turnsSection 5.1, Appendix A.1, Figure 7
BareDirect execution of the frozen VLASection 5.2
FixedManually specified retry and recovery rulesSection 5.2
Auto-KSearches and validates recovery knowledge with predefined/coverage-based probingSection 5.2
KPHarnessAdditionally learns action- and knowledge-selection policies through execution feedbackSections 4.2 and 5.2

Both protocols output four 50-ms actions. Auto-K also performs probing; the distinction concerns feedback-driven selection updates. Figure 7 illustrates the input formats, while Figure 6 and Appendix F provide execution examples.

For the concern about auto_research_claw.py and AI disclosure

  • We clarify that auto_research_claw.py is a separate utility for automated experimentation with gaming world models. It was not used in any experiment reported in this manuscript and is not part of the evaluated KPHarness pipeline. Its presence in the repository does not establish that the scientific study or manuscript was autonomously generated.

  • The work reported here uses Gemini-2.5-Flash as the vision-language researcher and harness proposer, as disclosed in Section 5 and Appendix I. We have clarified the repository structure, the AI-use statement, and the novelty claims to distinguish the reported experimental use from other support utilities.

The authors remain responsible for the methodology, code, results, novelty claims, and manuscript. We respectfully ask the reviewer and AC to reconsider the desk-rejection recommendation in light of these concrete implementation details and disclosures.

Dear Reviewer 2TBM,

Thank you again for your comments on clarity and transparency. Our response provides concrete definitions and examples of recovery knowledge, the two probing policies, executable proposals, and the input protocols. It also clarifies the role of the experimental LLM, the authors’ contributions to the implementation, and the scope of AI assistance.

We would be grateful if you could review these explanations and share any remaining concerns, particularly regarding the implementation and AI-use disclosure. Your follow-up would help us identify and resolve any remaining misunderstandings during the discussion.

Best regards, The Authors

Reviewer_HKwbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces KPHarness, a framework that improves game-task execution through knowledge–probing co-evolution without updating the underlying VLA parameters. When execution fails, the system performs targeted probes, uses interaction feedback to select recovery knowledge, and proposes harness modifications that are validated through paired sandbox evaluations. Validated experience and updated probe and knowledge selection policies are reused across tasks. In post-adaptation evaluation on 149 Minecraft tasks, KPHarness improves task coverage from 63 tasks with Auto-K to 70. Evaluation on unseen tasks with all components frozen also demonstrates transfer benefits.

Critical strengths

The method directly addresses the identified problem. For visually ambiguous failures, it actively gathers evidence before selecting recovery strategies, offering a targeted alternative to proposing modifications based solely on failure outcomes.

The evaluation includes actual interaction and transfer tests. Paired sandbox evaluations assess candidate modifications, while unseen-task evaluations with all components frozen provide initial support for overall effectiveness and experience reuse.

Critical weaknesses

The specific contribution of bidirectional co-evolution is not sufficiently isolated. The results support the full system’s advantage over Auto-K, but do not sufficiently distinguish the individual contributions and synergy of knowledge improving probing and probing improving knowledge.

Adaptation conditions differ in the cross-seed evaluation. Auto-K remains frozen while KPHarness retains feedback updates. Consequently, the observed advantage cannot be directly interpreted as greater cross-seed robustness of the learned harness or knowledge itself.

Questions for authors

Which experimental results separately support knowledge improving probe selection and probe feedback improving recovery knowledge, rather than only demonstrating better overall system performance?

Why does the cross-seed evaluation freeze Auto-K while retaining feedback updates for KPHarness? Does the advantage persist when both systems are frozen or evaluated with comparable adaptation budgets?

3 comments

We thank the reviewer for recognizing the motivation for active evidence collection, environment-grounded validation, and frozen transfer.

For W1&Q1: Knowledge–probing feedback and co-evolution

  • Co-evolution links recovery knowledge and both probing policies through shared execution feedback. Action probing collects diagnostic evidence; knowledge probing selects and instantiates recovery strategies. Execution updates action- and knowledge-selection statistics, while validated proposals and promotion add or revise recovery records. Appendix D, Table 12, distinguishes these interfaces and update objects.

    Appendix A.5, Table 7, documents cross-task knowledge use:

Knowledge-reuse statisticResult
Prior-knowledge selections840
Cross-family reuse relations101
Initial-rollout successes before local search12
Successful trajectories explicitly using prior knowledge9
  • The sand trajectory directly records learning-state updates in both modules. Action probes occur at decision turns 7, 8, 9, 10, 11, 12, 19, and 20. The system selects backup_and_turn at turn 23 and reacquire_and_center at turn 24, instantiates their recovery instructions, and succeeds at turn 29. Comparing the input snapshot with episode-end statistics gives:
Module or recordSelection-count changeDelayed-credit change
Eight action probes, aggregate+8+4.1611392
backup_and_turn+1+0.8
reacquire_and_center+1+1.0
Unselected approach_and_act00

Table 14 reports these changes. Implementation inspection confirms that immediate and delayed feedback enter local and global value statistics used in subsequent contextual UCB scores. The trace therefore documents actual learning-state changes. Recovery knowledge participates in execution, and the terminal outcome updates both earlier action probes and knowledge selections; another action probe after knowledge selection in that episode is not necessary.

  • Delayed credit uses a separate selection-event clock for each module. For module m∈{a,z}m\in\{a,z\} with MmM_m selections, event jj receives
=R_{\mathrm{succ}}\gamma^{M_m-j} \mathbf{1}[\mathrm{success}], \qquad j=1,\ldots,M_m.$$ The last event has discount distance zero. With $R_{\mathrm{succ}}=1$ and $\gamma=0.8$, eight action events receive $\sum_{k=0}^{7}0.8^k=4.1611392$ in total, while the two knowledge events receive $0.8$ and $1.0$. Turn 29 triggers retrospective credit assignment; elapsed VLA decision counts are not the discount distances. Section 4.2, Eq. (9), Appendix G.3, and the case description now use this same definition. This clarifies the existing implementation without changing the algorithm or rerunning experiments. The complete-system results, fixed-bank pairing, and historical deployment diagnostic provide complementary outcome evidence: **70/149 versus 63/149**, **15/60 versus 7/60**, a local knowledge-injection gain of **13/110 versus 9/110**, and historical selection at **9/10 versus 7/10** on its selected task subset. The records directly support the operational co-evolution mechanism. **For W2&Q2: Frozen evaluation and seed provenance** - **Both methods are frozen during the reported final tests.** Section 5.1, Table 5, and Appendix E disable further search and persistent writeback. Each method uses one final configuration across all 149 tasks; the 60-task transfer evaluation also freezes all evolved components. |Evaluation|Auto-K|KPHarness| |-|-|-| |Final coverage, Table 1|63/149|**70/149**| |Frozen transfer, Table 3|7/60|**15/60**| |110-task evaluation, seed 101|24/110|**31/110**| |110-task evaluation, seed 202|8/110|**30/110**| |110-task evaluation, seed 303 | 15/110 | **25/110** | - **Final cross-seed evaluation loads retained skills that have already undergone multi-seed validation.** During adaptation, recovery skills are validated across multiple seeds before being retained for subsequent use. After adaptation, we freeze each method's final harness configuration, knowledge bank containing these retained skills, and probing statistics. Auto-K and KPHarness each reuse their own fixed set of artifacts under test seeds 101, 202, and 303. During final evaluation, the system can retrieve the retained skills and execute probes, but performs no further harness search, knowledge-bank modification, or persistent update of selection statistics. Multi-seed skill validation belongs to adaptation and skill retention; the reported final cross-seed results measure execution of the resulting frozen systems. ACCEPT and HO-win counts describe adaptation and promotion-validation, not learning during final testing. Task-level pairing gives **24 versus 2 exclusive successes** on seed 202 and **12 versus 2** on seed 303, favoring KPHarness. These results support the observed cross-seed deployment gains with each method's evolved artifacts held fixed. They do not imply equal preceding adaptation costs.

Dear Reviewer HKwb,

Thank you again for your thoughtful feedback. We would appreciate the opportunity to continue the discussion on knowledge–probing co-evolution and frozen evaluation. Our response includes an execution trace documenting feedback updates to both probing modules and clarifies how each method’s fixed artifacts are reused across evaluation seeds.

Could you please share your thoughts on these clarifications and indicate whether any aspects of the mechanism or evaluation protocol remain unclear? We would be happy to provide further details and discuss any outstanding concerns.

Best regards, The Authors

AuthorsOpenReview ↗

Response to Reviewer HKwb's Follow-up

We thank the reviewer for acknowledging that our rebuttal has clarified the main concerns to some extent:

Thank you to the authors for their efforts in addressing my comments. My main concern has been clarified to some extent. However, I still have some reservations about the overall novelty and contribution of the work, so I have decided to maintain my original score.

We respect the reviewer's remaining reservations regarding novelty and contribution. However, we would appreciate further clarification of which specific aspects of our proposed method are considered insufficiently novel, particularly as the latest comment does not identify overlapping prior work or explain which technical contributions remain unconvincing.

Our central contribution is not the individual use of probing or recovery knowledge, but an execution-feedback-driven framework that adapts both contextual action-probing and knowledge-selection policies, integrating these mechanisms with evidence-guided harness modification and paired sandbox validation around a frozen VLA.

Unlike Auto-K, which relies on predefined or coverage-based probing, KPHarness learns from interaction outcomes to guide subsequent diagnostic and recovery decisions. The resulting system improves task coverage from 63/149 to 70/149 and frozen-transfer performance from 7/60 to 15/60. We recognise that the component-level diagnostics do not independently establish the causal contribution of every feedback mechanism, and we have clarified the scope of our claims accordingly.

We acknowledge that the reviewer may still consider the contribution insufficient for acceptance. Nevertheless, we respectfully ask whether the remaining concern relates primarily to methodological novelty relative to existing work or the experimental evidence supporting the proposed co-evolution mechanism, as these are distinct issues.

We appreciate the reviewer's consideration and hope this clarification assists the reviewers and the Area Chair in assessing the technical contribution.

Reviewer_6JCNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces KPHarness, a framework that improves the execution harness surrounding a frozen gaming VLA through adaptive probing, recovery knowledge accumulation, and sandbox validation. It separates fast action execution from a slower loop for failure diagnosis and harness modification. Immediate interaction feedback and delayed success credit update the selection of action probes and recovery knowledge. On 149 Minecraft tasks, KPHarness achieves 47.0% task coverage, compared with 26.2% for Bare, 33.6% for Fixed, and 42.3% for Auto-K. In a frozen evaluation on 60 unseen tasks, it completes 15 tasks, compared with eight for Bare and seven for Auto-K. The problem is practically relevant, although aspects of the evaluation and ablation design limit the conclusions that can be drawn about the proposed co-evolution mechanism.

Critical strengths
  1. Clear motivation and practical relevance. Gaming agents frequently fail because of target loss, insufficient interaction range, or ineffective recovery behavior. Improving the external execution harness offers a useful way to address these failures without further updating the underlying VLA.

  2. Environment-grounded validation of proposed modifications. The framework connects failure diagnosis, active probing, recovery knowledge, and harness revision, and evaluates candidate modifications through actual sandbox execution. Comparing candidates and incumbents from identical external-state snapshots, with persistent updates disabled, is a sensible design that reduces interference between evaluations.

  3. Useful supplementary analyses and transparency. Beyond task coverage, the paper reports frozen transfer to unseen tasks, sequential knowledge reuse, static-knowledge diagnostics, and computational costs. It also distinguishes search-time success aggregates from final frozen-harness performance and acknowledges the limitations of heuristic credit assignment and the assumptions underlying its theoretical analysis.

Critical weaknesses
  1. The multi-seed comparison uses asymmetric adaptation settings. Appendix B and Table 6 indicate that Auto-K is evaluated with a frozen harness and Knowledge Search disabled on seeds 202 and 303, whereas KPHarness continues knowledge search and adaptation. Consequently, the difference in Table 2 reflects both the algorithmic design and whether further adaptation is permitted. This comparison does not isolate the benefit of knowledge–probing co-evolution or establish its advantage under equally frozen evaluation. Comparisons with both methods frozen, and with both allowed matched adaptation budgets, are needed.

  2. The ablations do not sufficiently isolate the contribution of co-evolution. Moving from Auto-K to KPHarness introduces adaptive action probing, adaptive knowledge selection, and delayed credit updates together. The results therefore do not establish which components drive the improvement or whether their interaction provides additional benefits. Appendix C offers a useful static-knowledge diagnostic, but its execution protocol differs from the complete search pipeline. As the authors acknowledge, comparisons across these protocols are not controlled effect estimates.

  3. The unseen-task transfer results are promising but insufficiently characterized. Table 3 is valuable because it evaluates the evolved components without further adaptation. However, the manuscript does not sufficiently explain the relationship between these 60 tasks, the 149 harness-evaluation tasks, and the 32 GRPO training tasks. It is unclear whether “unseen” refers to new objectives, task compositions, or environment instances. Repeated evaluations and uncertainty estimates are also missing. Auto-K performing slightly below Bare on this split further motivates an analysis of when transferred recovery knowledge helps or hurts.

  4. The baseline and efficiency comparisons remain incomplete. The complete method is primarily evaluated under the legacy protocol, which conditions on the current observation without interaction history. The history-aware mtv1 protocol includes only partial comparisons for Bare and Fixed, leaving the effectiveness of the proposed method under that protocol unresolved. Although Appendix H reports computational costs, it does not provide a sufficiently detailed per-method breakdown of adaptation costs and deployment overhead, or performance comparisons under matched total budgets.

  5. The acceptance criterion and robustness of accepted modifications require clarification. Appendix D uses a single search seed for paired acceptance. In the primary setting, only five of the 14 tasks with accepted patches show a holdout improvement, indicating that acceptance on one seed does not necessarily imply a robust gain. Furthermore, the main text describes success as the primary criterion, while the implemented score combines success with unnormalized environment reward and explicitly does not guarantee a strictly success-first ordering. These descriptions should be reconciled, and the practical effect of reward scaling should be examined.

Questions for authors
  1. Could the authors report multi-seed comparisons with Auto-K and KPHarness both fully frozen, and separately with both allowed to adapt under identical budgets? How much of the advantage in Table 2 remains in each setting?

  2. With initial knowledge, proposal counts, and interaction budgets controlled, what happens when action-probe learning, knowledge-selection learning, delayed success credit, or knowledge-content updates are individually disabled? How do random or fixed probing policies perform under the same budgets?

  3. How were the 60 unseen tasks constructed and separated from actor-training and harness-adaptation tasks? Could the authors provide task identities, split criteria, and repeated evaluation results to distinguish transfer to new tasks from generalization to similar task instances?

  4. Could all methods be evaluated under mtv1 at the same decoding temperature? Please also report per-method environment interactions, researcher-model calls, adaptation time, and frozen execution overhead.

  5. What is the observed range of raw environment rewards? Were any unsuccessful candidates accepted because they obtained a higher weighted score? Would the conclusions hold with multiple search seeds, a success-first acceptance rule, and final test seeds that never participate in knowledge promotion?

7 comments

We thank the reviewer for recognizing the practical motivation, environment-grounded validation, and transfer analyses. We address the corresponding weaknesses and questions together below.

For W1&Q1

  • The reported final evaluations use frozen learned systems. Section 5.1, Table 5, and Appendix E specify that each method uses one final harness across all 149 tasks, including a fresh execution of the 39 tasks originally solved by Bare. The harness configuration, knowledge bank, and probing statistics remain frozen, with no further search or persistent writeback. Frozen policies can still execute probes and select accumulated skills.
EvaluationAuto-KKPHarnessSetting
Final coverage, Table 163/14970/149One final configuration per method; includes adaptation tasks
Transfer, Table 37/6015/60All evolved components frozen
  • Final cross-seed evaluation loads retained skills that have already undergone multi-seed validation. During adaptation, recovery skills are validated across multiple seeds before being retained for subsequent use. After adaptation, we freeze each method's final harness configuration, knowledge bank containing these retained skills, and probing statistics. Auto-K and KPHarness each reuse their own fixed set of artifacts under test seeds 101, 202, and 303. During final evaluation, the system can retrieve the retained skills and execute probes, but performs no further harness search, knowledge-bank modification, or persistent update of selection statistics. Multi-seed skill validation belongs to adaptation and skill retention; the reported final cross-seed results measure execution of the resulting frozen systems.
SeedAuto-KKPHarnessNet gain
10124/11031/110+7
2028/11030/110+22
30315/11025/110+10

Tables 2 and 8 report these results. Task-level pairing further shows 24 KPHarness-only versus 2 Auto-K-only successes on seed 202, and 12 versus 2 on seed 303. Thus, exclusive gains substantially exceed regressions on both additional seeds. With one fixed set of artifacts per method, these results describe cross-seed execution performance on the evaluated task set. Repeated task–seed observations are not pooled as independent samples for significance claims.

For W2&Q2

  • Co-evolution is implemented through coupled feedback updates. Action probes collect evidence, knowledge probing selects and tests recovery strategies, and execution outcomes update both selection policies. Validated proposals and knowledge promotion additionally change the harness and the recovery records available to later tasks. Table 12 separates these operations, including selection of existing knowledge and creation or revision of knowledge.

    The audited sand trajectory provides direct implementation evidence: eight action probes precede two knowledge selections, and terminal success updates both modules' local and global value statistics used by subsequent contextual UCB decisions. Table 14 reports eight additional action selections with total delayed credit 4.1611392, and one additional selection for each chosen knowledge record with credits 0.8 and 1.0. This establishes updates to the learning state of both modules. Another action probe after knowledge selection within the same episode is not required for terminal feedback to update earlier probes.

  • The fixed-bank diagnostic distinguishes knowledge injection from online selection updates. Appendix C, Table 9(b), holds the task set, seed, base harness, and eight-probe budget fixed while disabling knowledge-content revision and harness search.

Fixed-bank diagnosticSuccess
Probe-only, no knowledge bank15/110
Static K, no knowledge injection9/110
Static K, injection without knowledge-selection updates13/110
Static K, injection with knowledge-selection updates13/110

Table 11 expands the existing results through task-level pairing:

Controlled comparisonSuccess before / afterNewly successfulRegressionsNet gain
Enable knowledge injection, retain selection updates9/110 / 13/11084+4
Enable selection updates, retain knowledge injection13/110 / 13/110550

Knowledge injection therefore provides an observed net benefit in its corresponding comparison; online selection updates show no aggregate coverage gain in this restricted setting. Probe-only reaches 15/110, so these results do not establish that static knowledge universally improves probing.

  • Learned-policy deployment is a separate diagnostic. Appendix F.2, Table 16, compares historical-statistics selection with random selection using the same frozen actor, harness, knowledge bank, and candidates. On ten tasks previously solved by that historical configuration, success is 9/10 versus 7/10, with three gains and one regression. This supports a local deployment benefit from learned selection, while its selected task subset limits generalization. Both conditions retain the previously evolved skills, so it is not a full comparison with and without co-evolution.

Together with 70/149 versus 63/149 coverage and 15/60 versus 7/60 frozen transfer, these results support the complete system and specific roles of recovery knowledge and learned selection. The trace establishes the coupled update mechanism.

For W3&Q3

  • Frozen transfer improves outcomes across all three task families. Table 3 evaluates 20 mining, 20 combat, and 20 crafting tasks with all evolved components fixed.
MethodMineKillCraftOverall
Bare7/201/200/208/60
Auto-K6/201/200/207/60
KPHarness9/205/201/2015/60

The eight-task gain over Auto-K consists of three mining, four combat, and one crafting success. Most improvement comes from mining and combat; crafting provides limited positive evidence. Auto-K's one-task deficit relative to Bare does not distinguish knowledge mismatch from execution stochasticity.

  • The revised split description includes a task-by-task configuration audit. The recorded target/task names of all 20 mining, 20 crafting, and 20 combat transfer tasks are absent from the corresponding families in the original 149-task evaluation set. This supports transfer beyond the harness-adaptation tasks and explicitly establishes name disjointness.

    We perform cold-start supervised fine-tuning on approximately 765K multimodal Minecraft training samples. These include multi-turn demonstrations of crafting, smelting, and inventory operations, together with single-turn auxiliary samples for GUI mouse positioning and camera movement. Each sample contains a task instruction and one or more observation–action turns. At each turn, the model predicts four consecutive low-level control steps, each lasting 50 ms, covering a 200-ms execution horizon. A control step may combine mouse movement, mouse clicks, and key presses. Multi-turn samples retain preceding observations and demonstrated actions as context. Thus, “four steps” specifies the output format of each prediction; the number of turns in a training sample can vary. Complete long-horizon task execution is evaluated through sustained closed-loop environment interaction.

    GRPO uses 32 interactive tasks. These training–evaluation distinctions do not establish semantic disjointness from all actor-training data. Appendix A.2 now states the relationships explicitly. Table 3 remains one frozen transfer evaluation, and evaluation seeds should not be interpreted as defining new Minecraft worlds.

For W4&Q4

  • All four methods were evaluated under both input protocols at the same decoding temperature. Table 1 reports the following results at T=0T=0:
MethodLegacy, T=0mtv1, T=0
Bare23/14926/149
Fixed31/14934/149
Auto-K41/14942/149
KPHarness44/14946/149

For the mtv1 results, Auto-K and KPHarness first complete harness adaptation under mtv1, then freeze the resulting harness configurations, knowledge banks, and probing states for evaluation. Under mtv1, which includes interaction history, KPHarness solved 46 of 149 tasks, compared with 42 for Auto-K. The observed advantage was four tasks under mtv1 and three under legacy at the same decoding temperature.

  • Method-specific interaction counts quantify the additional adaptation cost. For the two reported legacy settings, both methods have the same number of proposal opportunities:
SettingMethodEpisodesEnvironment stepsProposal opportunities
Legacy, T=0Auto-K50947,900309
Legacy, T=0KPHarness57155,252309
Legacy, T=0.7Auto-K36733,307222
Legacy, T=0.7KPHarness41438,554222

The KPHarness totals include 103 learning episodes and 10,266 steps at T=0, and 74 learning episodes and 6,748 steps at T=0.7. These learning interactions are already included in the totals above. With equal proposal opportunities, KPHarness uses 15.3% more environment steps at T=0 and 15.8% more at T=0.7. These counts explicitly quantify the additional interaction cost of the complete adaptation system.

  • Appendix H reports resource use by experimental stage. Table 17 includes:
StageEpisodesGPU-hours
Knowledge search and co-evolution1,861≥547.5
Fixed-knowledge diagnostic440≥124.6
Frozen transfer, all three methods18055.5

The first two estimates sum single-GPU worker time. Frozen transfer takes 6.94 wall-clock hours on eight GPUs. Across these stages, retained records contain 233,499 environment steps, 19,310 probes, 831 proposer responses, at least 1,654 external request attempts, and approximately 5.76 million reported tokens. Actor training is shared across all harness variants. These stage totals characterize the computation used; they do not establish an equal-budget adaptation-efficiency advantage.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.