- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a hierarchical skill self-improving method for LLM, where Meta-Skills guide the revision of lower-level Task-Skills. The paper further introduce Hindsight Meta-Experience Distillation, which re-evaluates Skills through state replay and stores the comparison results as historical memory. The method is evaluated on models including Qwen3-32B and DeepSeek-v4-Flash, achieving better performance than methods such as MetaSkill-Evolve.
Critical strengths
Major Strengths
The idea of the paper is interesting, and the proposed method brings performance improvements over existing methods.
Minor Strengths
The paper is clearly written.
Critical weaknesses
1.The proposed method relies on state replay to re-evaluate Skills. However, accurately reproducing historical states can be difficult in many real-world scenarios, such as Agent tasks involving external APIs or robotic environments. This may limit the applicability of the proposed method.
- The contribution is largely incremental compared with MetaSkill-Evolve. In particular, the performance improvements in Table 4 are very limited, while the proposed method incurs an additional 27% computational cost. Without sufficient statistical evidence and comparisons under matched computational budgets, the effectiveness of the proposed method remains unclear.
Questions for authors
-
How can the proposed method be applied to environments where historical states cannot be accurately reproduced?
-
Can the authors report the standard errors of the results in Table 4 to further demonstrate the performance improvements?
-
Can the authors increase the number of search iterations and candidate evaluations for MetaSkill-Evolve to match the computational cost of the proposed method?
4 comments
The authors have not addressed my question regarding standard errors. My concern is that the performance differences between the methods are very small, and the observed improvements may simply result from randomness across multiple runs. Reporting standard errors would help address this concern. Furthermore, the applicability of the proposed method remains limited in tasks where complete replay is not possible, such as robotics. In addition, the poor clarity of the main figure negatively affected my impression of the paper. Therefore, I have decided to maintain my rating.
Thank you again for the follow-up clarification. We sincerely apologize for overlooking the request for standard errors in our previous response. The reviewer’s concern is well taken, and we should have reported these statistics directly.
We retrieved and checked the original reuse experiment records and provide the run-level results and corresponding standard errors below:
| Model | Method | Successful tasks in each run | Mean ± SE (%) |
|---|---|---|---|
| Qwen3-32B | MetaSkill-Evolve | 64/134, 68/134 | 49.25 ± 1.49 |
| Qwen3-32B | HMED | 67/134, 69/134 | 50.75 ± 0.75 |
| DeepSeek-v4-Flash | MetaSkill-Evolve | 98/134, 106/134 | 76.12 ± 2.99 |
| DeepSeek-v4-Flash | HMED | 102/134, 104/134 | 76.87 ± 0.75 |
HMED achieves a higher mean in both reuse settings, and the reported standard errors quantify the observed variation across runs. These results support the reuse of the learned Meta-Skill after resetting the Task-Skill library, consistent with the broader improvements in the main experiments. Table 4 is intended to evaluate reuse across fresh discovery origins, while the main experiments provide the primary performance comparison.
Regarding robotics, we agree with the reviewer that irreversible physical interactions create a genuine applicability constraint. HMED requires a restorable discovery state and the ability to execute comparable continuations. Access to an external LLM API does not imply that a physical environment can be fully replayed. The evaluated protocol therefore applies to replayable settings; extending it to irreversible real-world interactions would require an appropriate replay mechanism or a validated approximation.
We also appreciate the reviewer’s observation about the main figure. This is a formatting and layout issue rather than a change to the method itself. We will enlarge the labels, reduce visual crowding, and distinguish the adoption and feedback paths more clearly so that the transition from the cited discovery event to matched continuation and subsequent update is easier to follow.
Thank you again for the careful reading and for identifying these specific points. We hope that the corrected statistics, the additional clarification of the replay scope, and the planned figure improvements address the remaining concerns and provide a stronger basis for reconsidering the rating.
Thank you to the authors for providing the standard errors. These results further reinforce my original concerns. The extremely small differences in mean performance, together with the large standard errors, make the experimental results highly unconvincing. Therefore, I have decided to maintain my rating.
We thank the reviewer for the careful and constructive review. The questions about replayability, framework scope, and additional computation helped us clarify the interface required by HMED and the role of its matched continuations.
1. Replayability and environment assumptions
HMED does not require bitwise restoration of an external model provider or of every hidden state of a physical environment. It requires a serializable discovery state sufficient to execute a continuation, including the current Task-Skill, origin tasks, learning frontier, directive archive, visible search history, task configuration, and a verifiable entry snapshot.
For DeepSeek-v4-Flash, the model is accessed through an external API. The replay interface restores the discovery state relevant to continuation; it does not restore the provider’s hidden sampler state or guarantee bitwise-identical responses.
HMED therefore applies directly when a benchmark or system provides a replay or memoization interface for the relevant discovery state. Environments without such an interface require an adapter or an approximate replay protocol. The requirement is an interface, not a fully reversible environment.
2. Framework scope
HMED is implemented within a two-timescale Skill-evolution framework, but its mechanism requires an editable improvement policy, a provenance-linked discovery event with a restorable entry state, and an executable continuation for comparing incumbent and candidate policies.
The method is defined at this interface level rather than by MetaSkill-Evolve’s particular prompt format. GEPA and PromptBreeder do not directly expose all of these replay interfaces, so applying HMED to them would require an adapter. We have not evaluated HMED on other self-improvement frameworks.
3. Additional evolution cost
HMED incurs approximately 1.27× the evolution cost of the original MetaSkill-Evolve protocol because revisions are evaluated through matched continuations.
We also tested a budget-augmented MetaSkill-Evolve baseline in the Qwen3-32B/ALFWorld setting. The augmented baseline used 1.21× the original baseline cost and achieved 49.25% on the reuse protocol, whereas HMED achieved 50.75%. Relative to the same original reference, the residual evolution-cost difference is approximately 1.27/1.21−1≈5%.
In the measured ScienceWorld downstream workload, the resulting policy reduced model calls by 6.9% and environment steps by 6.5%. The corresponding break-even estimate is approximately 6.2 equivalent execution rounds under that workload.
4. Is the gain only due to extra search budget?
The budget-sensitivity check shows that increasing ordinary proposal and discovery budget alone did not reproduce the reuse result in the tested setting. Additional search can increase coverage, but it does not directly construct paired evidence about the consequence of a particular Meta-Skill revision.
The two methods allocate their additional budget differently, so this check does not establish exact equality for every call, token, and environment component. It tests the simpler explanation that ordinary additional search alone explains the observed reuse gain.
5. What is specific to HMED
HMED treats the consequence of a particular Meta-Skill revision as the unit returned to future Meta-Skill updates. Matched continuation constructs this evidence; the Meta-Experience record serializes and reuses it.
This differs from retaining only a successful branch outcome. Valid non-adopted revisions also leave records that remain available to later updates. The contribution is the revision-centered, provenance-linked comparison record and its reuse in subsequent Meta-Skill updates.
We hope the replay clarification, budget check, and reuse results help clarify HMED's practical scope and contribution.