- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes an agent framework that decides how to fix a failure based on what caused it. An Answer Agent runs the task. After a failure, an Improvement Agent reads the verifier feedback, the execution trace, and the existing skills, and chooses one of three fixes: guidance for the next run, a memory update, or a lasting change to a skill. A proposed change is adopted only after it passes replay and regression checks. The authors also propose using the results of past fixes to change the Improvement Agent itself. The experiments consist of results from archived runs, an eight-case check of the diagnostic rules, and one PDF-form case that tests update validation.
Critical strengths
Strength 1: The paper points out that different failure causes need different fixes
The same missing output can come from a missing skill, a skill that was not retrieved, or a skill that was not followed, and each cause needs a different fix. The paper maps these cases to execution guidance, memory updates, and skill changes, and keeps proposing a change separate from adopting it. This avoids changing stored skills after every failure.
Strength 2: The PDF case shows errors that a pass/fail reward hides
In Figure 4 and Appendix D, a faulty helper that skips checkbox and radio fields still gets reward 1 from the original verifier, although the underlying checks drop from 47 passes to 35 passes and 12 skips. Stricter checks tell the faulty and intact outputs apart. So a candidate update should be judged by its actual output and skipped checks, not only by the final reward.
Critical weaknesses
Weakness 1: The paper does not test its central claim
The abstract claims that an agent can use the results of past fixes to improve the way it chooses fixes. The paper never shows a model-generated change to the Improvement Agent that was accepted, put into use, and then improved later decisions. The algorithm on page 14 only records a meta_proposal. Table 6 lists the benefit of an accepted policy change as an evaluation target, not a reported result. The conclusion also leaves this test to future work. The authors are open about this, but it means the central claim in the abstract has no direct evidence. The paper should separate what is implemented, the diagnosis and validation steps, from what is not yet shown: outer-loop learning that improves the Improvement Agent itself.
Weakness 2: The comparisons cannot isolate the effect of outer-loop learning
For Qwen, the fixed-strategy and RSI-enabled runs cover 38 and 81 instances with different tasks, run histories, and implementations, so their success rates (65.8% and 71.6%) are not comparable. The GPT evolution result counts a task as solved if any of several archived attempts succeeded, so the gain may come from extra attempts. In the PDF case, a hand-written adapter and a fixed rule that always requires replay pick the same intact helper, so nothing shows that a learned policy beats the fixed rule. The authors acknowledge these limits, but they affect the main contribution. Fixing them needs a paired comparison, not more unpaired success rates.
Weakness 3: The "first to define" claim conflicts with prior work such as STOP
Page 2 claims to be the first to define the problem of improving both task skills and the improvement process. Yet page 3 cites STOP and related work for the idea that the improvement process itself can be changed. STOP uses a seed improver to improve itself and reports that the updated improver beats the original on a few downstream tasks. So improving how an agent improves is not new. The new part may be the combination of failure attribution, fix selection, and candidate validation. The paper should say what this adds over prior work and drop or narrow the "first" claim.
Questions for authors
-
(Related to Weakness 1) Is there any model-generated change to the Improvement Agent that was accepted and put into use? If so, please show the change, why it was accepted, and how it affected fix decisions on later, independent tasks.
-
(Related to Weakness 2) Is there a comparison on the same tasks, with the same implementation and total budget, that separates a learned Improvement Agent from fixed diagnosis, the replay rule, and extra retries? If not, which finished experiment supports outer-loop learning?
-
(Related to Weakness 3) STOP already shows an improver modifying itself. Compared with STOP, what exactly is new in your two-level formulation?
5 comments
Response to JhCW (2/4)
W2&Q2: (W2) The comparisons cannot isolate the effect of outer-loop learning. For Qwen, the fixed-strategy and RSI-enabled runs cover 38 and 81 instances with different tasks, run histories, and implementations, so their success rates (65.8% and 71.6%) are not comparable. The GPT evolution results count a task as solved if any of several archived attempts succeeded, so the gain may come from extra attempts. In the PDF case, a hand-written adapter and a fixed rule that always requires replay pick the same intact helper, so nothing shows that a learned policy beats the fixed rule. The authors acknowledge these limits, but they affect the main contribution. Fixing them needs a paired comparison, not more unpaired success rates. (Q2) Is there a comparison on the same tasks, with the same implementation and total budget, that separates a learned Improvement Agent from fixed diagnosis, the replay rule, and extra retries? If not, which finished experiment supports outer-loop learning?
The added experiments use DeepSeek and compare the Hyperagents adaptation, frozen-strategy routing, and a local RSI configuration on the same task identifiers. Each condition is evaluated over the original attempt followed by one execution after improvement. Appendix E.2 thus compares recovery within a common two-round window. Appendix E.1 provides the internal diagnosis-and-routing ablations, while Appendices E.3–E.4 document the strategy change discussed in our response to Q1.
Each comparison addresses a different mechanism. Fixed diagnosis versus dynamic routing examines intervention selection. Frozen-strategy routing versus the local RSI configuration compares recovery outcomes. The candidate audit examines functional completeness. Restricting the analysis to two rounds excludes gains from third-round retries. Full cost accounting must additionally include diagnosis, proposal generation, and replay. Policy-activation records establish which strategy changes were used during execution.
Round-by-round success under fixed diagnosis and dynamic routing (Appendix E.1, excerpt)
| Condition | First round | Second round | Final |
|---|---|---|---|
| D: Fixed diagnosis | 55.56% | 37.50% | 72.22% |
| F: Dynamic routing | 55.56% | 62.50% | 83.33% |
The internal ablations evaluate the same selected instances from each of 18 task families, using shared task identifiers and inputs. First-round and final success rates use the full cohort in each condition. Second-round success uses only tasks entering that round. Skills, Memory, and intervention mechanisms follow their respective ablation definitions.
D and F have identical first-round success rates of 55.56%, followed by final rates of 72.22% and 83.33%. The difference is 11.11 percentage points. Their second-round rates are 37.50% and 62.50%, respectively. The difference therefore emerges during recovery rather than in first-round performance. The independent contribution of outer-loop learning requires a separate comparison of activated policies under matched budgets.
Recovery across three baseline configurations
We select initially unsuccessful tasks from the original task set and compare the same task identifiers across methods. The tasks cover court-form filling, dependency auditing, earthquake calculations, and video object counting. The reporting window contains the original attempt and one execution after improvement. First-round task success is 0.00% in each condition, and checkpoint success is 63.64%. The following comparison reports outcomes after the second round.
Task recovery within two rounds (Appendix E.2)
| Condition | Task success | Checkpoint success |
|---|---|---|
| Hyperagents task-domain adaptation | 16.67% | 63.64% |
| F: Dynamic routing, frozen strategy | 30.00% | 78.79% |
| G: Dynamic routing, local RSI configuration | 50.00% | 90.91% |
F achieves 30.00% task success and 78.79% checkpoint success; G achieves 50.00% and 90.91%; the Hyperagents adaptation achieves 16.67% and 63.64%. The differences between G and F are 20.00 and 12.12 percentage points. Task success reflects complete recovery, while checkpoint success captures partial functional completion. The contribution of policy revision must be assessed together with its activation record.
Checkpoint success aggregates all named checkpoints. Appendix E.2 specifies each condition’s initial trajectories, Memory configuration, and comparison protocol.
The internal ablations show how recovery varies with the intervention mechanism. The external comparison reports task success and checkpoint success within two rounds. Isolating policy learning additionally requires matched total model-call budgets and evidence that the revised policy was activated.
Response to JhCW (3/4)
W2 (PDF case): In the PDF case, a hand-written adapter and a fixed rule that always requires replay pick the same intact helper, so nothing shows that a learned policy beats the fixed rule.
In the PDF case, the fixed replay rule and our mechanism select the same intact helper. They therefore agree on candidate acceptance. Policy learning concerns the proposal and revision of interventions, rather than changes to an already correct acceptance decision. We retain fixed functional acceptance criteria while optimizing the procedure used to meet them. The PDF case examines the basis for adoption. The policy-revision case examines how historical outcomes influence the direction of improvement.
-
The replay rule determines whether a candidate is acceptable; the improvement policy determines what to do next. Replaying the two supplied helpers identifies the intact version through functional checks. LRSI makes a separate decision about whether the current failure calls for guidance, Memory, or Skill revision. Previous intervention outcomes then inform the next action. Different policies can use the same acceptance rule. Selecting the same helper does not establish which policy is better at producing effective interventions.
-
The PDF case establishes the need for functional checks beyond the final reward. Both helpers receive a reward of 1. The intact helper passes 47 checks without skips. The defective helper passes 35 checks and skips 12. The final reward cannot distinguish these candidates; detailed replay outcomes can. The agreement between the hand-written adapter and the fixed rule shows that candidate selection can follow explicit acceptance criteria. We retain these criteria outside the adaptive policy so that strategy changes remain subject to the same quality checks.
-
We provide separate execution evidence for policy revision. In the dependency-auditing case discussed in Q1, the revised candidate Skill achieves 80.00% checkpoint success with the original feedback. Strategy-shift feedback raises the rate to 100.00%. The change redirects the intervention from continued Skill rewriting toward source-evidence checks. The acceptance criteria remain unchanged. This local strategy evaluation is reported at the shadow-validation stage. A comparison that isolates its contribution should share the acceptance rule and match the starting state and total model-call budget. Subsequent actions and task outcomes would then provide the evaluation targets.
Functional completeness of the PDF candidates (Appendix D.1)
| Candidate | Original reward | Passed checks | Skipped checks |
|---|---|---|---|
| Intact helper | 1 | 47 | 0 |
| Defective helper | 1 | 35 | 12 |
The PDF audit establishes a functional basis for accepting persistent Skill updates when final rewards are identical. Acceptance checks assess the update’s reliability; intervention strategies determine how to produce an effective correction. LRSI therefore separates candidate generation from candidate adoption. The PDF values are from the case reported in the submitted manuscript.
Response to JhCW (1/4)
Dear Reviewer JhCW,
Thank you for recognizing the value of cause-dependent interventions and the PDF audit’s finer-grained assessment of functionality. Below, we explain what each experiment establishes about routing, policy revision, and candidate validation.
Appendix E.3 documents a versioned policy proposal and its use in shadow evaluation. The proposal shifts improvement from task-Skill content to feedback strategy. The record distinguishes this execution from formal promotion and subsequent evaluation on independent tasks.
The revised rule governs the next intervention after Skill revision stalls. It retains the useful candidate knowledge and redirects feedback toward verification against source evidence. Previous outcomes motivated this shadow evaluation. In the dependency-auditing case, retrying with the original feedback achieves 60.00% checkpoint success. The original candidate Skill with that feedback reaches 80.00%, and a revised candidate with unchanged feedback also achieves 80.00%. Strategy-shift feedback with the revised candidate reaches 100.00%. The policy therefore redirects the intervention from further Skill rewriting toward source-evidence checks.
Local RSI policy revision and execution record (Appendix E.3, excerpt)
feedback_shift_after_skill_stall_v1candidate_shadow_test_onlyThe execution record identifies the policy change and its validation stage. Appendix E.4 then compares task outcomes using the revised candidate Skill under two feedback conditions. The reported configurations retain the revised candidate Skill while changing the feedback condition.
Checkpoint success under different candidate Skills and feedback conditions (Appendix E.4)
The original candidate raises checkpoint success from 60.00% to 80.00%. Further Skill revision with unchanged feedback retains 80.00%, whereas strategy-shift feedback with the revised candidate reaches 100.00%, an increase of 20.00 percentage points. The sequence illustrates the intended role of local RSI. Previous revision outcomes motivate a shift from editing Skill content to improving feedback verification. The corresponding policy status is documented in Appendix E.3.
The case links a versioned policy proposal to a change in the next intervention. Useful candidate knowledge is retained while the strategy shifts toward feedback verification. The reported follow-up execution completes the remaining checkpoints.