acceptodds
Under review as a conference paper at ICLR 2027

First Things First: Priority Scheduling with On-Demand Skill Evolution in Minecraft

Abstract

As autonomous agents move toward practical use, reliably handling long-horizon, complex tasks remains a key challenge. Minecraft provides a controllable open-world testbed, yet existing benchmarks largely focus on isolated goals such as finding, collecting, or crafting, leaving the changing priorities and capability gaps of complex execution underexplored. Open-world agents may fail because they execute the wrong available behavior at the wrong time, or because they lack a reliable capability for a necessary subgoal. To probe this gap, we introduce seven long-horizon tasks, such as preparing from scratch and defeating a zombie, or repeatedly chopping trees until 100 logs are collected, and evaluate agents by both task success and completion efficiency. We introduce PRIME (Priority-guided Runtime Scheduling with Importance-aware Missing-skill Evolution), which couples runtime priority scheduling with selective skill evolution. PRIME maintains a priority-based schedule that adapts to task state and progress, while task-critical missing skills trigger validated evolution for future reuse. Across seven Minecraft tasks and three high-level model families, PRIME consistently improves task success and completion efficiency, with the largest gains on long-horizon and dynamically changing tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Minecraft Agents, LLM Agents, Open-World Planning, Priority Scheduling, Skill Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_svpkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

PRIME is a Minecraft agent that separates scheduling failures from capability-grounding failures. It combines a priority agenda that is rescored after every bounded skill call with a gated skill-evolution path, where generated Mineflayer programs are admitted only after passing at least 7 of 10 independent validation trials.

Critical strengths
  1. The distinction between scheduling failures and capability-grounding failures is a clear and reasonable way to frame open-world execution. It gives a concrete criterion for when an agent should reschedule and when it should extend its skill library.

  2. The evaluation uses external hard checkers shared across methods, and the paper reports its limitations openly. The skill admission protocol with independent validation trials is a careful design choice.

Critical weaknesses
  1. The work is incomplete and the amount of experimental work is insufficient. The paper states that the matched no-seed ablation and component ablations are not measured, and the execution example in Figure 3 is a schematic, so no experiment tests the central claim about separating the two failure modes.

  2. The difference from prior Minecraft agents is unclear. Voyager and Odyssey already provide skill libraries with verification and DEPS provides feedback-driven replanning, and the paper does not report whether online skill evolution was triggered in the formal episodes.

  3. The evaluation only contains seven tasks, one world seed, and five trials per cell, reported only as thresholded pass/fail. With Gemini, PRIME_skill ties three baselines at 3/7.

Questions for authors
  1. In the 735 formal episodes, how often was online skill evolution triggered, and how many new skills were admitted?

  2. Please report the raw five-trial success rate for every cell. The majority threshold hides differences between 3/5 and 5/5.

  3. How do the baselines perform when given the same 20-skill seed?

4 comments
AuthorsOpenReview ↗

Response 1 to Reviewer svpk

Thank you for your constructive feedback. We have expanded the evaluation across seven tasks, six baselines, and three backbones with additional world seeds and repeated trials over two rounds. GPT uses five seeds and ten trials per seed per round; Gemini and Grok each use three seeds and five trials per seed per round. We also added component ablations and analyses of skill evolution, reuse, and execution success. These additions are incorporated into the revised paper, with changes highlighted in blue.

W1: Component evidence and recorded examples of the two recovery paths

1) Scheduling, online acquisition, and persistence comparisons

Section 3.4 and Table 3 compare the strongest baseline with scheduling-only PRIMEsch\mathrm{PRIME}_{\mathrm{sch}}, first-round PRIME1\mathrm{PRIME}_{1}, and inherited-library PRIME2\mathrm{PRIME}_{2}. Values are task success rates (%).

ConditionGPTGeminiGrok
Highest-SR baseline64.448.656.2
PRIMEsch\mathrm{PRIME}_{\mathrm{sch}}68.053.362.9
PRIME1\mathrm{PRIME}_{1}71.155.265.7
PRIME2\mathrm{PRIME}_{2}87.778.191.4

Scheduling alone improves success by 3.6–6.7 percentage points over the strongest baseline. Adding the skill module yields another 1.9–3.1 points in the first round. With the controller and evaluation setting unchanged, retaining validated skills improves second-round SR by 16.6–25.7 points. The scheduling-only condition uses five seeds for GPT and three for Gemini/Grok, with five trials per seed; the full conditions follow the main protocol.

2) Recorded executions beyond the schematic

Figure 3 remains explicitly labeled as a schematic. Section 3.5 adds two recorded GPT executions, with step-level evidence in Appendix E:

CaseObserved obstacleRecoveryOutcome
Task 1: collect 100 logsWater interrupts collection with 22 logs retainedPrioritize an existing water-exit skill, then resume collection100 logs collected; no skill evolution
Task 3: craft an iron pickaxeExisting coal-mining behavior obtains no coal despite available toolsGenerate and validate alternative-target mining behaviorCoal obtained; iron smelted; pickaxe completed

The first case changes execution priority; the second introduces a new execution behavior. In Task 1, the recovery capability already exists, and the scheduler preserves inventory progress while bringing it forward. In Task 3, PRIME generates a candidate that excludes the failed mining target and searches other deposits. It passes 10/10 independent validation trials and then obtains coal in the live task. These traces illustrate the two recovery paths; the component comparisons evaluate their contribution to task performance.

W2 & Q1: Online evolution and the distinction from existing agents

1) Evolution is measured in the revised evaluation

Table 2 reports episode-level triggers and subsequent reuse/call statistics. All values are percentages.

MetricGPTGeminiGrok
Trigger (Round 1)78.979.068.6
Trigger (Round 2)30.961.048.6
Later-episode reuse69.868.372.4
Round-2 reuse78.583.276.8
Skill-call success87.177.683.8

Evolution becomes less frequent as the validated library accumulates, while task success increases. Later-episode reuse and live call success show that acquired capabilities remain useful beyond their creation episodes. These results concern the expanded evaluation, not the original 735-episode cohort. Trigger rates measure episodes invoking evolution; they are not counts of distinct admitted skills.

2) Priority-conditioned acquisition and independent admission

DEPS combines feedback-driven planning with subgoal ranking by estimated completion steps [1]. Voyager develops reusable programs through iterative prompting and self-verification [2], while Odyssey provides primitive and compositional skills [3]. PRIME connects these concerns through an explicit runtime decision: after each bounded call, it reprioritizes feasible subgoals; synthesis becomes eligible only when an evidence-revised agenda retains a necessary subgoal whose prerequisites are met but which lacks suitable executable grounding. Persistent admission requires at least seven successes in ten independent, environment-checked validation trials. The contribution is the connection between runtime priority selection, capability acquisition, and independently validated reuse, clarified in Section 5.

References

[1] Wang, Z., et al. Describe, Explain, Plan and Select: Interactive Planning with LLMs Enables Open-World Multi-Task Agents. NeurIPS, 2023.

[2] Wang, G., et al. Voyager: An Open-Ended Embodied Agent with Large Language Models. TMLR, 2024.

[3] Liu, S., et al. Odyssey: Empowering Minecraft Agents with Open-World Skills. IJCAI, 2025.

AuthorsOpenReview ↗

Response 2 to Reviewer svpk

W3 & Q2: Multiple seeds and success rates without majority thresholding

1) Expanded replication

GPT uses five world-generation seeds and ten trials per seed per round. Gemini and Grok use three world seeds and five trials per seed per round. Each independent experiment runs two seven-task curricula. Task worlds reset, while validated skills accumulate only within the corresponding experiment and carry into its second round. Separate libraries prevent cross-experiment injection.

2) Raw success counts underlying the revised comparison

Table 1 reports trial-level success percentages, and Appendix Table 4 gives aggregate success counts. For the strongest baseline and PRIME conditions:

ConditionGPTGeminiGrok
Highest-SR baseline451/700102/210118/210
PRIME1\mathrm{PRIME}_{1}249/35058/10569/105
PRIME2\mathrm{PRIME}_{2}307/35082/10596/105

Baseline rows aggregate both rounds; each PRIME row reports one round. The counts above aggregate across tasks. The majority threshold no longer obscures differences in the main comparison. For Gemini, the strongest baseline achieves 48.6%, compared with 55.2% for PRIME₁ and 78.1% for PRIME₂.

Across all three backbones, PRIME2\mathrm{PRIME}_{2} exceeds the strongest baseline by 23.3–35.2 percentage points and achieves the lowest penalized active and wall times. The abstract and main-result analysis now reflect this expanded evaluation.

Q3: Matched initial skill availability

The revised evaluation addresses this concern by removing PRIME's additional pre-populated skills. All baselines, PRIME_sch, and PRIME₁ now start from the same frozen base skill library, with PRIME's learned-skill store initially empty. The comparison therefore aligns initial skill availability rather than retaining the previous 20-skill advantage.

Table 3 shows a consistent ordering in task success across all three backbones: best baseline < PRIME_sch < PRIME₁ < PRIME₂. Scheduling alone improves success over the strongest baseline by 3.6–6.7 percentage points; enabling online skill acquisition adds 1.9–3.1 points in Round 1; inheriting the skills validated during that experiment's Round 1 adds a further 16.6–25.7 points in Round 2. Overall, PRIME₂ exceeds the strongest baseline by 23.3–35.2 points. These results demonstrate gains from scheduling under matched initial skills, followed by additional benefits from online acquisition and persistent reuse.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer svpk,

Thank you again for your time and effort in reviewing our submission.

As the discussion period will end in 24 hours, we hope you can review our rebuttal and share your feedback on whether our responses have addressed your concerns. We would be happy to clarify any remaining questions or issues.

Thank you for your time and consideration.

Dear Reviewer svpk,

Thank you again for your time and effort in reviewing our submission.

As the discussion period will end in 2 hours, we hope you can review our rebuttal and share your feedback on whether our responses have addressed your concerns. We would be happy to clarify any remaining questions or issues.

Thank you for your time and consideration.

Reviewer_KvGLOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PRIME, which combines priority-based scheduling, evidence-gated skill generation, and independent skill validation for Minecraft agents. The distinction between scheduling failures and missing capabilities is useful. However, the evaluation does not adequately establish the contributions of the proposed mechanisms.

Critical strengths

The architecture is clearly described, and environment-based success checks improve evaluation credibility. PRIME achieves encouraging results on repeated collection across three backbones. The authors also transparently acknowledge important experimental limitations.

Critical weaknesses
  1. The comparison does not isolate the effects of the scheduling mechanism from those of the available skills. Specifically, the proposed method is equipped with a library of 20 previously developed skills, whereas the baseline methods do not have access to these skills. The authors should compare different controllers under the same skill library and conduct targeted component ablation studies.

  2. The current evaluation primarily demonstrates the reuse of previously accumulated skills, without adequately validating the selective online skill evolution mechanism. Beyond the existing discussion of Voyager, further comparisons with feedback-driven planning in AdaPlanner (Sun et al.) and task-activated knowledge extension in TALKER (Lou et al.) would help clarify the unique contributions of PRIME's capability gap detection and validation mechanisms.

  3. The current experimental results do not sufficiently support the claims regarding success rates and efficiency in the abstract. With only five trials per setting and a single fixed environment seed, the results remain subject to considerable uncertainty. For example, on the Gemini model, PRIME achieves the same number of successfully completed tasks as several baseline methods.

Questions for authors

1、How much of the advantage remains when competing controllers receive the same frozen skill library?

2、In the reported evaluation, how frequently does online evolution produce a new skill that is subsequently used to complete the task?

5 comments
AuthorsOpenReview ↗

Response 1 to Reviewer KvGL

Thank you for your feedback. We now evaluate seven tasks, six baselines, and three backbones over two rounds: GPT uses 5 world seeds and 10 trials per seed per round; Gemini/Grok each use 3 seeds and 5 trials per seed per round. We added scheduling and skill ablations, and analyzed skill evolution and reuse. The revised paper is uploaded, with changes in blue.

W1 & Q1: Separating priority scheduling from learned skills

1) Initial libraries and controlled PRIME conditions

The revised evaluation removes the previous pre-populated learned-skill initialization: PRIME1\mathrm{PRIME}_{1} starts with the shared frozen base skill library and an empty learned-skill store. PRIMEsch\mathrm{PRIME}_{\mathrm{sch}} retains priority scheduling over the base repertoire while disabling online evolution, persistence, and learned-skill reuse. PRIME2\mathrm{PRIME}_{2} inherits skills admitted during PRIME1\mathrm{PRIME}_{1}, with the controller, tasks, model, and budgets unchanged.

Each independent (world seed, trial) experiment has its own library. Skills can be reused by later tasks within Round 1 and are carried into Round 2 of that same experiment, with no cross-experiment injection (Appendix B).

2) Contributions of scheduling, online acquisition, and persistence

Table 3 reports task success rates (%):

ConditionGPTGeminiGrok
Highest-SR baseline64.448.656.2
PRIMEsch\mathrm{PRIME}_{\mathrm{sch}}68.053.362.9
PRIME1\mathrm{PRIME}_{1}71.155.265.7
PRIME2\mathrm{PRIME}_{2}87.778.191.4

Priority scheduling improves success before adding learned skills. PRIMEsch\mathrm{PRIME}_{\mathrm{sch}} exceeds the strongest baseline by 3.6, 4.8, and 6.7 percentage points for GPT, Gemini, and Grok. Enabling the skill module adds another 3.1, 1.9, and 2.9 points in PRIME1\mathrm{PRIME}_{1}. Retaining validated skills then improves PRIME2\mathrm{PRIME}_{2} over PRIME1\mathrm{PRIME}_{1} by 16.6, 22.9, and 25.7 points.

The scheduling-only condition uses five seeds for GPT and three for Gemini/Grok, with five trials per seed; the two full PRIME conditions follow the main protocol. All baselines, PRIME_sch, and PRIME₁ start from the same frozen base skill library. The scheduling-only condition therefore evaluates the scheduling framework without an initial learned-skill advantage.

W2 & Q2: Online evolution produces useful and reusable capabilities

1) Evolution triggers and subsequent use

We analyze this question in detail in Section 3.3 (Online Skill Evolution and Reuse) of the revised paper. Table 2 reports the following percentages:

MetricGPTGeminiGrok
Trigger (Round 1)78.979.068.6
Trigger (Round 2)30.961.048.6
Later-episode reuse69.868.372.4
Round-2 reuse78.583.276.8
Skill-call success87.177.683.8

Trigger measures the percentage of episodes invoking evolution. Skill-call success measures successful executions among logged evolved-skill calls. Evolved skills achieve 77.6-87.1% success in live execution. Together with the 16.6-25.7-point improvement from Round 1 to Round 2, frequent reuse and fewer evolution triggers demonstrate the practical value of retaining validated capabilities.

2) A concrete acquisition that enables task completion

We discuss two recorded recovery cases in Section 3.5 (Execution Analysis) of the revised paper. In the iron-pickaxe case, the agent already has a stone pickaxe, a furnace, and three raw iron, but its existing coal-mining behavior obtains no coal. The tool prerequisite is satisfied; the unresolved obstacle is obtaining coal from a workable target. PRIME therefore generates an alternative-target mining skill that excludes the failed target, tries up to three other deposits, and checks coal inventory to verify success. After passing 10/10 independent skill-validation trials, the skill obtains coal in the live task, enabling smelting and completion of the iron pickaxe. The new skill addresses a specific execution gap revealed by feedback, rather than simply repeating the unsuccessful target choice. Appendix E provides the step-level trace; Appendix F gives further context-driven examples, including mixed-fuel smelting, direct crafting from available intermediates, and recovery from repeated submersion.

3) Relationship to AdaPlanner and TALKER

We have added these comparisons in Section 5. AdaPlanner uses feedback-driven plan refinement and reuses successful plans as exemplars. TALKER extends and reuses its skill library through feedback-based authorization. PRIME instead ties capability acquisition to a necessary subgoal that remains ungrounded after agenda revision, and uses independent environment-checked validation to decide persistent admission. The distinction is priority-conditioned capability acquisition coupled with independently validated reuse.

AuthorsOpenReview ↗

Response 2 to Reviewer KvGL

W3: Expanded evaluation, higher success, and improved efficiency

1) Multiple world seeds and repeated trials

GPT uses five world-generation seeds and ten trials per seed per round; Gemini and Grok use three seeds and five trials per seed per round. Each experiment contains two rounds of seven tasks. Worlds and episode state reset between tasks, while PRIME's validated skills persist only within the corresponding independent curriculum. Baseline rows aggregate both rounds; the two PRIME rows report their respective rounds separately.

2) Higher success and improved efficiency through skill reuse

Table 3 summarizes success and efficiency against the highest-SR baseline for each backbone (DEPS for GPT/Grok; ReAct for Gemini). We reproduce its full metrics below, omitting the scheduling-only condition already discussed above. Higher SR and lower AtryA_{\mathrm{try}}/WtryW_{\mathrm{try}} are better; times are in seconds.

BackboneMethodSR (%)AtryA_{\mathrm{try}}WtryW_{\mathrm{try}}
GPTBest baseline64.44622.411045.8
GPTPRIME1\mathrm{PRIME}_{1}71.14175.09599.9
GPTPRIME2\mathrm{PRIME}_{2}87.71870.33797.3
GeminiBest baseline48.65215.512516.3
GeminiPRIME1\mathrm{PRIME}_{1}55.25780.513718.6
GeminiPRIME2\mathrm{PRIME}_{2}78.14548.68212.1
GrokBest baseline56.25263.012786.4
GrokPRIME1\mathrm{PRIME}_{1}65.74736.810527.3
GrokPRIME2\mathrm{PRIME}_{2}91.43167.46270.2

Successful trials retain their measured times; failures receive uniform penalties of 9000 active seconds and 21600 wall seconds. PRIME2\mathrm{PRIME}_{2} achieves both the highest success rate and the lowest penalized times across all three backbones. Relative to each backbone's highest-SR baseline, SR increases by 23.3-35.2 percentage points, penalized active time decreases by 12.8-59.5%, and penalized wall time by 34.4-65.6%. Gemini's earlier majority-count tie is resolved: its SR rises from 48.6% for the strongest baseline to 55.2% for PRIME1\mathrm{PRIME}_{1} and 78.1% for PRIME2\mathrm{PRIME}_{2}.

3) Actual execution-time savings through skill reuse

The initial cost of skill acquisition yields benefits in subsequent execution: skill reuse enables faster completion of the same task in Round 2. For example, the following Grok trajectories under the same experimental setting show reductions in actual episode wall time:

TaskRound 1Round 2Time reduction
Collect 100 logs1778.4 s1066.7 s40.0%
Craft a wooden axe615.1 s190.0 s69.1%
Collect 20 logs at midnight669.6 s407.8 s39.1%

Round 2 reduces completion time by 39.1-69.1% in these examples, with both rounds successfully completing each task. Taking Collect 100 logs as an example, the original logging behavior fails in Round 1. PRIME generates collectOneObservedLog, uses it to resume collection, and subsequently admits it to the persistent library. In Round 2, PRIME reuses this skill to complete the same objective, with no additional in-episode skill generation or validation. Episode wall time falls from 1778.4 to 1066.7 seconds. This case illustrates how validated skill persistence avoids repeated acquisition and supports faster subsequent completion, complementing the aggregate success-efficiency results above.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer KvGL,

Thank you again for your time and effort in reviewing our submission.

As the discussion period will end in 24 hours, we hope you can review our rebuttal and share your feedback on whether our responses have addressed your concerns. We would be happy to clarify any remaining questions or issues.

Thank you for your time and consideration.

Reviewer_2YDwOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents a long horizen scheduling and skill evolution method for language-based agents playing Minecraft. The paper formulates a method for Open-world agent scheduling and capability grounding , combining stateful priority scheduling with independently validated mechanisms and evaluate on mask model pairs with five trials. The paper connects scheduling, skill development and grounded feedback to allow language models to play Minecraft.

Critical strengths
  1. Technical contribution:
    • Presents a method for scheduling and skill development for open-world agents in the Minecraft domain, showing that the model is able to outperform existing baselines for 3 different modern language models.
  2. Empirical strength:
    • Research Questions are clearly presented and answered in experimental results.
    • 3 different models are used for evaluation.
    • Compared against 6 different baselines
    • Dedicated limitations section
  3. Relevance and impact:
    • Open-world agent planning and skill development is an open-problem with on-going research. Allowing System 2 thinking for Language Models and learning of skills is both areas that have broad applications beyond the presetned domain of Minecraft.
Critical weaknesses
  • The novelty of the method beyond a agent harness for skill learning and task prioritization is unclear. What makes this work unique compared to pre-existing work in Lifelong Learning?
  • It is not clear if 5 trials per configuration is sufficient analysis to provide a significant conclusion of the provided model. Beyond the time it takes to run more experiments, what is the justification of limiting to 5 trials?
  • The metrics in Table 1 should be clearly labeled for which one is the best for each metric.
  • The title on the paper and the OpenReview do not match.
  • Figure 2 has edit artifacts.
Questions for authors
  • Does the improvement over the performance justify the higher Wall-clock time justify?
  • Could the authors clarify what are some broader applications of this work beyond the tested Minecraft domain?
4 comments
AuthorsOpenReview ↗

Response 1 to Reviewer 2YDw

Thank you for your constructive feedback. We have expanded the evaluation across seven tasks, six baselines, and three model backbones by adding more world seeds and repeated trials over two rounds. GPT uses five world seeds and ten trials per seed per round; Gemini and Grok each use three seeds and five trials per seed per round. We also added ablations of priority scheduling and the skill module, together with analyses of skill evolution, reuse, and execution success. These additions are incorporated into the revised paper, which has been uploaded with changes highlighted in blue.

W1: Contributions beyond an agent harness or a reusable skill library

1) A decision mechanism connecting scheduling and capability acquisition

PRIME distinguishes two interventions during execution: reallocating control among available behaviors and acquiring a missing capability. After each bounded call, its stateful agenda updates priorities using prerequisites, verified progress, and environmental risk. If feedback-driven revision leaves an important subgoal without suitable executable grounding, PRIME generates a candidate skill. Persistent admission then requires at least seven successes in ten independent validation trials.

DEPS ranks subgoals by estimated completion steps [1], while AdaPlanner refines plans through environmental feedback [2]; PRIME connects runtime priority rescheduling to an explicit gate for acquiring a capability still missing from the revised agenda. Unlike Voyager’s LLM-based self-verification [3], PRIME bases persistent admission on repeated independent tests against environment-state predicates.

The contribution is the connection between when to execute, when to acquire a capability, and when that capability is sufficiently validated for reuse. Completed progress and unfinished alternatives remain available throughout this process. Section 3.5 illustrates the distinction with recorded recovery through an existing water-exit skill and recovery through a newly generated alternative-target mining skill.

2) Component evidence

Table 3 compares priority scheduling alone, online skill acquisition, and accumulated skills. Values are task success rates (%).

ConditionGPTGeminiGrok
Highest-SR baseline64.448.656.2
PRIMEsch\mathrm{PRIME}_{\mathrm{sch}}68.053.362.9
PRIME1\mathrm{PRIME}_{1}71.155.265.7
PRIME2\mathrm{PRIME}_{2}87.778.191.4

Scheduling alone improves success over the strongest baseline for every backbone. Enabling the skill module adds a further 1.9–3.1 percentage points in the first round, while retaining validated skills improves second-round success by 16.6–25.7 points. These comparisons support the complementary roles of priority scheduling and skill acquisition, together with the cumulative value of persistence (Section 3.4).

W2: Evaluation beyond five trials

The revised evaluation uses multiple Minecraft world-generation seeds and repeated trials:

BackboneWorld seedsTrials per seed per roundRounds
GPT5102
Gemini352
Grok352

Each independent experiment runs Task 1 through Task 7 in each round. PRIME starts with an empty learned-skill store, accumulates validated skills within that curriculum, and carries them into its second round. Independent experiments use isolated libraries. Table 1 reports trial-level success percentages rather than majority-solved task counts; Appendix Table 4 provides the underlying aggregate success counts. The revised conclusions therefore draw on repeated trials across different generated worlds, rather than five trials under one world seed.

As Table 3 shows, both PRIME rounds outperform the strongest baseline for every backbone under the expanded multi-seed evaluation, with accumulated skills yielding gains of 23.3–35.2 percentage points. These results provide consistent empirical support for PRIME’s effectiveness across the three evaluated backbones.

W3–W5: Presentation corrections

These issues have been corrected in the revised manuscript.

References

[1] Wang, Z., et al. Describe, Explain, Plan and Select: Interactive Planning with LLMs Enables Open-World Multi-Task Agents. NeurIPS, 2023.

[2] Sun, H., et al. AdaPlanner: Adaptive Planning from Feedback with Language Models. NeurIPS, 2023.

[3] Wang, G., et al. Voyager: An Open-Ended Embodied Agent with Large Language Models. Transactions on Machine Learning Research, 2024.

AuthorsOpenReview ↗

Response 2 to Reviewer 2YDw

Q1: Skill efficiency and the cost of acquisition

1) Faster successful execution across rounds

The initial cost of skill acquisition yields benefits in subsequent execution: skill reuse enables faster completion of the same task in Round 2. For example, the following Grok trajectories under the same experimental setting show substantial reductions in actual episode wall time:

TaskRound 1Round 2Time reduction
Collect 100 logs1778.4 s1066.7 s40.0%
Craft a wooden axe615.1 s190.0 s69.1%
Collect 20 logs at midnight669.6 s407.8 s39.1%

Round 2 reduces completion time by 39.1–69.1% in these examples, with both rounds successfully completing each task.

Taking Collect 100 logs as an example, the original logging behavior fails in Round 1. PRIME generates collectOneObservedLog, uses it to resume collection, and subsequently admits it to the persistent library. In Round 2, PRIME reuses this admitted skill to complete the same 100-log objective, with no additional in-episode skill generation or validation. Episode wall time falls from 1778.4 to 1066.7 seconds. This case illustrates how validated skill persistence avoids repeated acquisition and supports faster subsequent completion.

2) Aggregate success-efficiency comparison

Table 1 reports penalized active and wall times. Successful trials retain their measured times; failed trials receive the same 9000-second active and 21600-second wall penalties across methods. This prevents early termination from appearing efficient merely because a task failed.

PRIME2\mathrm{PRIME}_{2} achieves the lowest penalized active and wall times for all three backbones. Relative to each backbone’s highest-SR baseline, penalized active time decreases by 12.8–59.5% and wall time by 34.4–65.6%. These aggregate results and the successful-execution examples address complementary aspects of the performance-cost tradeoff.

Q2: Broader applications beyond Minecraft

The same decision problem arises in software automation and embodied control: an agent must determine whether a blocked task requires a different arrangement of existing actions or a genuinely new executable capability.

For example, a software agent may need to reorder tool calls to satisfy a dependency, or construct a missing API wrapper and validate it with tests before reuse. An embodied agent may similarly need to prioritize an existing recovery behavior or develop a new task-specific routine.

PRIME offers a framework for connecting these decisions while preserving completed progress and accumulating validated capabilities. Applying it beyond Minecraft would require domain-specific execution interfaces and independently checkable success criteria.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer 2YDw,

Thank you again for your time and effort in reviewing our submission.

As the discussion period will end in 24 hours, we hope you can review our rebuttal and share your feedback on whether our responses have addressed your concerns. We would be happy to clarify any remaining questions or issues.

Thank you for your time and consideration.

Dear Reviewer 2YDw,

Thank you again for your time and effort in reviewing our submission.

As the discussion period will end in 2 hours, we hope you can review our rebuttal and share your feedback on whether our responses have addressed your concerns. We would be happy to clarify any remaining questions or issues.

Thank you for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Additional Experiments

Dear Area Chair and Reviewers,

Thank you for your feedback and the discussion so far. We have uploaded a revised manuscript incorporating the additional experiments and clarifications presented in our responses. The main updates are summarized below, with substantive additions and changes marked in blue.

1) Expanded evaluation with multiple seeds and more trials. We expanded the evaluation across seven tasks, six baselines, and three backbones: GPT uses five world seeds and ten trials per seed per round; Gemini and Grok each use three seeds and five trials per seed per round, over two rounds. We also replaced majority-solved task counts with trial-level success rates. PRIME$_2$ achieves 87.7%, 78.1%, and 91.4% success, exceeding the strongest corresponding baseline by 23.3–35.2 percentage points. The expanded results show a consistent advantage across all three backbones, including Gemini, where the previous evaluation showed a tie (Table 1).

2) Component ablations without pre-populated learned skills. We removed the previous pre-populated learned-skill initialization and added a scheduling-only ablation alongside the two full PRIME conditions. Scheduling alone improves success over the strongest baseline by 3.6–6.7 percentage points; enabling online skill acquisition adds another 1.9–3.1 points in Round 1. With validated skills carried into Round 2, task success increases by 16.6–25.7 points over Round 1. These results support the contributions of both priority scheduling and the skill module (Section 3.4, Table 3).

3) Success–efficiency analysis and concrete reuse benefits. We added penalized active and wall times to account for both successful and failed trials. PRIME$_2$ achieves the lowest values across all three backbones, reducing penalized active time by 12.8–59.5% and penalized wall time by 34.4–65.6% relative to each backbone’s highest-SR baseline (Table 1). Our responses further analyze successful Grok trajectories with 39.1–69.1% lower actual wall time in Round 2 than in Round 1; the 100-log case directly illustrates reuse of a previously acquired skill without repeating in-episode generation and validation.

4) Online evolution statistics and recorded execution cases. We added analyses of evolution triggers, subsequent reuse, and live skill-call success. Round 2 combines fewer evolution triggers with higher task success, while evolved skills achieve 77.6–87.1% success in live execution (Section 3.3, Table 2). We also added two recorded cases: recovery through an existing water-exit skill without evolution, and recovery through a newly generated and validated mining skill. These cases show how PRIME selectively acquires behavior when an execution gap remains, rather than treating every interruption as a need for new code (Section 3.5; Appendices E–F).

We have incorporated these findings into the paper and expanded the related-work comparisons and reproducibility details. Our individual responses provide the supporting evidence for each reviewer’s concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.