- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ARC (Accountable Repair Control), a train-free technique to improve model's solve rate across CLI tasks. The extensive experimental results show that ARC lifts the Qwen3-32B solve rate from 21.4% to 34.1%.
Critical strengths
- The problem solved in this paper is very important
- This paper tries to solve the problem with train-free techniques
- The experimental results shows that ARC improves the model's solve rate from 21.4% to 34.1%
Critical weaknesses
- ARC improves the Qwen3-32B solve rate from 21.4% to 34.1%. However, ARC cannot lifts the GPT-5.4-nano solve rate as shown in Figure 5. Could you explain it?
- The models used in this paper are not up to date. It looks like this paper has been rejected for several times, maybe I am wrong
- I cannot open the code link: https://anonymous.4open.science/r/Tracebench-cli/ Could you check it?
Questions for authors
I do not have any questions for the authors, since I also have a similar idea and I think the solution proposed in this paper is workable. However, the limitation for this paper is that if the model is not powerful enough, the solution proposed in this paper does not have marginal improvement.
Dear Reviewers,
The authors have now submitted their responses. Please review them and indicate whether your concerns have been addressed. Please also update your scores if the responses have changed your assessment.
Thank you.
AC