- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper asks a simple question: if an agent learns a fix from one failure, when should it reuse that fix later? The authors show that the same correction often helps on some tasks and hurts on others. Their method, HCAR, runs each correction with and without it on calibration tasks, learns a predictor of its effect, and only applies it where the predicted benefit is high. In a controlled GLM-5 setup, this clearly beats random selection: precision goes from 51.9% to 87.8%, with much lower harm. On the real benchmarks (WebShop and LifelongAgentBench), though, the gains are small and not significant, especially when evidence is limited or the correction is new.
Critical strengths
- Most work on self-improving agents focuses on how to generate and store corrections, and assumes that a stored fix can be reused. This paper points out that fixing one failure doesn’t mean the fix will work elsewhere, and treats “when to reuse” as its own problem. As agent memory and skill libraries grow, this question will only matter more, so this research question is deserve to investigate.
2.The evaluation is careful, honest, and valuable (Although it is obvious that AI conducted the experiments). Activation-matched random is a fair baseline. The authors split data before evaluation, fixed their prediction rules before collecting test outcomes, and use BC-level bootstrap CIs. They also report negative results openly: weak gains with limited evidence and no clear transfer to unseen corrections.
Critical weaknesses
I only have one main concern:
The research question is important, and the method is intuitive, simple, and works well in the controlled setting. But it’s hard to see it carrying over to realistic environments. The method relies on a lot of direct evidence per correction: in the GLM-5 study, each BC gets 48 calibration contexts with 3 paired repeats. When evidence is limited (K=10 in WebShop), it doesn’t beat simple baselines like SupportMean. So in practice, the approach seems to depend on running many paired experiments for every single correction. That cost grows with the size of the correction library, which makes it hard to scale. Given this, I think the current experimental scale is too small (30 BCs and 5 synthetic task types in the main study), and the method doesn’t go deep enough. Fitting a separate scope for each correction won’t scale as the number of corrections grows.
What’s really needed is something closer to meta-learning: learning transferable structure across corrections, so that the scope of a new correction can be predicted from only a few observations. The paper’s own cross-correction transfer results (not significant on either LAB or WebShop) suggest that this is exactly the open problem, and the current method doesn’t address it.
Questions for authors
Overall, I think this work is valuable, but it would be better positioned as an exploratory, findings-driven paper than as a methodology paper. The proposed method is quite simple, and the paper’s own comparison (Table 3) shows that plain Ridge and partial pooling perform about the same as HCAR-Risk. This suggests that the gains come from the direct effect evidence rather than from the method itself. On the other hand, the analysis is a useful exploration for the recursive self-improvement (RSI) area. The finding that source-admitted corrections have mixed-sign effects on future tasks is worth sharing. If the authors go in this direction, I’d encourage them to broaden the empirical study (more corrections, more realistic task distributions) to make the findings more convincing.