- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies inference-time adaptation of frozen language-model agents through an evolving procedural skill library and plan-driven correction. The Skill Engine creates skills from successful trajectories, revises retrieved skills after failures, and periodically consolidates failure lessons across trials. The Critique Engine uses the same frozen backbone in a separate context to verify plan-step completion and directly execute corrective actions. On ALFWorld, SKT achieves 58.96±1.50% success with Qwen2.5-7B-Instruct under the authors’ matched protocol, compared with 55.72±1.14% without the Critique Engine. The paper also evaluates the framework on multi-hop QA, WebShop, and additional backbones, and includes operator ablations and an advice-versus-execution comparison.
Critical strengths
1.The main empirical comparison controls important sources of variation. The inference-time baselines share the backbone, maximum number of attempts, and action budget, with experience sources documented separately (§4.1; Appendix B.2). Both full SKT and the library-only configuration outperform these baselines across all three reported evaluation seeds (Tables 2–3). This makes the observed improvement more persuasive than a comparison assembled from different repositories or evaluation protocols. 2.The paper makes a meaningful effort to identify which mechanisms contribute. Rebuilding the library without each operator establishes their practical contribution (Table 7). More distinctively, Table 15 compares neutral text, corrective advice, and direct execution using the same verifier design; direct execution exceeds advice on all three seeds, by 2.24 percentage points on average. This is useful evidence for the proposed correction interface and is the paper’s clearest empirical contribution.
Critical weaknesses
1.The distinctive contribution of the Skill Engine is not sufficiently established against closely related procedural-memory methods. The framing in the abstract and §2 understates prior work on structured experience and failure-driven revision. AutoManual creates, updates, and consolidates success/error rules; ReasoningBank extracts structured strategies from successful and failed experiences; and Memᵖ revises retrieved procedural memories using failed executions. The more specific contribution appears to be support-filtered, cross-task failure consolidation combined with plan-triggered direct correction. However, the current comparisons do not establish which benefits arise from this particular consolidation mechanism, rather than more generally from learning and revising procedural knowledge. This matters because it determines the extent of the methodological contribution beyond the demonstrated system improvement. Relevant sources: AutoManual, §3.3–3.4, ReasoningBank, §3.2–3.3, and Memᵖ, §4.3. 2.The central correction controller cannot be unambiguously reconstructed from the supplied specification. Appendix E permits next_action only when fix_worked=false. Yet the worked example requires a successful prerequisite correction—opening the microwave—to be followed by another corrective action before the original heating step is complete (pp.20–21, 29–30). Algorithm 1 also exits the loop on replanning, whereas Table 14 describes generating a fresh plan within the current attempt. These are inconsistencies in the written specification; they do not establish that the executed code is incorrect. Nevertheless, the alternatives produce different action sequences and budget handling, so resolving them is necessary to assess and reproduce the principal correction mechanism.
Questions for authors
1.Contribution clarification: What precisely distinguishes the failure-driven library updates from AutoManual, ReasoningBank, and Memᵖ? For the Distiller, is there existing evidence or a targeted control using the same failure lessons through sequential refinement, separating cross-trial consolidation from additional calls or changes in library size? 2.Implementation clarification: How does the actual controller handle fix_worked=true with now_landed=false? Does replanning continue the current attempt, and how are environment state, history, and remaining action budget preserved? Please reconcile the implementation with Algorithm 1 and the printed prompts. 3.Channel-control clarification: In Table 15, does “same firing schedule/context” mean the same triggering rule or matched realized events? Since direct execution changes subsequent observations, please clarify tracker advancement and verify-after-fix handling across arms, preferably with existing call-count statistics. 4.Robustness clarification: Appendix B.1 states that Tables 2 and 15 reuse one frozen library per configuration, while Tables 6 and 7 rebuild libraries per seed. Which results share library artifacts or evaluation runs, and what variability is observed across independently constructed libraries? This would clarify whether the reported robustness extends to the skill-evolution process itself.
7 comments
Q1 / W1 (2/2) — Evidence for the Distiller: existing evidence and a targeted control
Existing evidence. In the leave-one-out ablation (Table 7) the library is rebuilt without the Distiller while the Refiner stays, so every failure is still processed sequentially, one trial at a time. Success drops from to ( pp, 3/3 seeds), and the best ablated seed is below the worst seed of the full system. The three Distiller-origin skills (incorrect-task-decomposition, early-termination, lost-track-of-progress) are the only cross-family entries of the 19-skill library. This ablation compares sequential refinement alone with sequential refinement plus consolidation; it does not separate consolidation from the additional Distiller calls or from the size of the library, which is what the question asks.
Targeted control (new). We ran the variant described in the question, in which failure lessons are processed one at a time: every failure lesson of attempts 2 and 3 triggers its own Distiller call (fire threshold 1, minimum cluster size 1) with the library snapshot in context, instead of being buffered until 20 lessons accumulate and then clustered by failure axis (minimum cluster size 3). We compare with SKT as reported in Table 2.
| Configuration | Distiller calls | skills | seed 41 | seed 42 | seed 43 | mean ± std |
|---|---|---|---|---|---|---|
| original batch Distiller | 9 | 19 | 57.46 | 58.96 | 60.45 | 58.96 ± 1.50 |
| per-lesson Distiller | 139 | 24 | 52.24 | 61.94 | 55.97 | 56.72 ± 4.89 |
- Success. The control reaches against for SKT; the difference is , and pp for seeds 41, 42 and 43, so per-lesson processing does not improve on consolidation.
- Calls. The control makes one Distiller call per lesson, where the batch Distiller makes one call per 20 lessons.
- Library size. The control ends with 24 skills, against 19 in SKT's library.
More calls and a larger library therefore do not raise success above SKT's. We present consolidation as the low-cost form of turning failure lessons into skills. The revision will report this control in Appendix D.3.
Q2 / W2 — The correction controller
We thank the reviewer for reading the controller specification this closely. We answer the points of W2 and Q2 one by one, in the order of the review, and note where the paper already gives the rule.
1. fix_worked = true with now_landed = false (Q2). This is the case in which the corrective action took effect but the plan step itself is not yet achieved, for example when the corrective only opens a receptacle that the step needs. In this case the pointer stays at the current step, no further corrective action is issued, and the policy takes the next action from the corrected state. The paper already states both parts: the fix-verify prompt (Appendix E) gives next_action "Only when fix_worked=false AND should_escalate_replan=false. Else null", and in Algorithm 1 only now_landed advances the pointer (line 10); otherwise line 6 takes the next action from the policy.
2. Worked example and the prompt (W2). The reviewer notes that the prompt permits next_action only when fix_worked is false, while the worked example reads as if a second corrective action followed a successful one. The prompt is what applies here: after open microwave 1 succeeds, the subsequent heat tomato with microwave 1 is emitted by the policy, not by the verifier. The worked example writes the two actions as one sequence, which is why it reads as a second corrective; the revision will replace it by the annotated trace below.
3. Does replanning continue the current attempt? (W2, Q2) The reviewer also notes that Algorithm 1 leaves the loop at "flag replan; break", while Table 14 describes a fresh plan within the current attempt. Replanning does continue the current attempt. Table 14 (the events of the Critique Engine) in Appendix D.2 already gives the escalation ("Escalate": re-issue the first-turn plan directive, ); Algorithm 1 does not show it, so we clarify it here.
4. Environment state, history and action budget on replanning (Q2). Because replanning stays within the attempt, nothing is restarted: the environment is not reset, the action–observation history is kept, the action counter continues, and every action, corrective or not, is charged to the budget .
Complete rule. So that the controller can be reconstructed from the text alone, the table below lists every outcome of the verify-after-fix call that follows an executed corrective; the controller applies the first matching row. The revision will add this table to Appendix D.2 and an "otherwise" case to Algorithm 1:
| Row | Verifier output | Controller action |
|---|---|---|
| 1 | now_landed true | ; control returns to the policy |
| 2 | now_landed false, next_action given | queue it; it is executed as the next environment action and followed by another verify-after-fix call |
| 3 | now_landed false, no next_action, should_escalate_replan true | raise the replan flag, ; the first-turn plan directive is re-issued at the next policy prompt |
| 4 | otherwise (e.g. fix_worked true, now_landed false) | unchanged; control returns to the policy |
Annotated example. The case of point 2, written out event by event (illustrative, written by hand to follow the prompts of Appendix E; plan step 3 is heat tomato in microwave 1):
| Event | Verifier output | Controller |
|---|---|---|
The policy emits heat tomato with microwave 1; observation "Nothing happens." | step check: landed=false, corrective_action="open microwave 1" | execute open microwave 1 as the next environment action |
| Observation: the microwave is now open | fix-verify: fix_worked=true, now_landed=false, next_action=null, should_escalate_replan=false | row 4: stays 3; control returns to the policy |
The policy emits heat tomato with microwave 1; observation: the tomato is heated | step check: landed=true |
We thank Reviewer FiHz for the careful and precise review. We answer Questions 1–4 in 5 comments; W1 is answered with Q1 and W2 with Q2.
Q1 / W1 (1/2) — What distinguishes the failure-driven library updates from AutoManual, ReasoningBank and Mem$^p$
The difference is in the path a failure takes into the library, at two points: the Refiner decides between patching the retrieved skill and skipping, and the Distiller consolidates failures from different trials under a support threshold. The table sets this against the sections the reviewer cites ("—": not described in that paper).
early-terminationandlost-track-of-progress, which no success-triggered extractor can write. In the three papers we find no step that aggregates failures across tasks under a support threshold.Revised framing. The abstract, Section 2 and Table 1 will describe prior work on structured experience and failure-driven revision as follows.
Revised caption: "SKT combines natural-language procedural skills evolved from both success and failure, consolidation of failures across trials, and verification by direct actuation, with a frozen open-weight backbone; to our knowledge, none of the listed methods combines all three."