- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper is based on the observation that failure modes in self-improvement tasks can be used for future task selection and proposes a teacher-student distillation method
Critical strengths
The observation that the authors make is that current self-improvement methods in coding tasks do not use failure types to curate next tasks for self-improvement, and it is a potential research gap, especially considering that evaluations are the costly part of the coding agents.
Critical weaknesses
I think the paper misrepresents the contribution as a self-improvement paper, whereas the entire method requires a stronger teacher model to supervise and propose a concurrent trajectory. Hence, the claim of the paper about improving self-improvement cannot be justified in this way. For example, the performance is by definition, limited of the fixed teacher model.
The paper is really hard to follow in depth. For example, the main figure is not helpful in understanding the main loop of the paper and feels like a set of buzzwords. It would be helpful to describe the process under the figure as well and justify the words in the boxes of the plot.
The comparisons in the main table are unfair because this is the only method that has access to a bigger model, which it can distill. AbsoluteZero and other baseline methods do not have bigger teacher models.
Questions for authors
I think the method itself could be useful for some distillation cases. I'd like to suggest that the authors reframe the paper from a self-improvement paper (which it is not) to a distillation paper and propose a situation in which the authors' method could be used.
I would also be interested in the performance of the model if the teacher model uses the same foundational model as the student model.
What is the relation of the method the authors propose to the on-policy distillation methods[1]
[1] - On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. Agarwal et al.2024