- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LeanRepair, a lightweight external controller for token-efficient iterative code repair. It uses edit-trajectory statistics to guide revisions, detect repeated failures, and stop unproductive search without additional model calls. Across six benchmarks and four backbones, LeanRepair reduces token usage relative to GoT in 23/24 settings (23.9% median reduction) while achieving a comparable or slightly higher aggregate pass rate (75.4% vs. 72.9%).
Critical strengths
- A simple and intuitive mechanism for reducing test-time compute.
- Broad empirical evidence for the headline efficiency claim, though the benchmarks are out-of-date by 2026 standards.
Critical weaknesses
- The evaluation focuses on relatively outdated code-repair tasks and benchmarks. Most of the benchmarks are based on short, self-contained programming problems or synthetic mutations, which may not adequately reflect modern repository-level coding agents. This limits the practical relevance of the empirical conclusions.
- The evaluation is largely restricted to small language models. The main experiments use models in roughly the 4B–9B range. Although the appendix includes Qwen3-14B, it remains unclear whether the proposed controller provides similar benefits for substantially stronger modern coding models, whose repair dynamics may differ considerably. In fact, the 14B results already suggest that the advantage can diminish as model capacity increases.
- Minor: There is an unresolved citation placeholder (“??”) on Line 35.
Questions for authors
See weaknesses
Thank you for your review, and for recognizing LeanRepair's simple, intuitive mechanism for reducing test-time compute and the broad empirical evidence for its efficiency claim. Below we quote each of your comments and respond to it. Changes in the revised PDF are marked in blue.
Weakness 1.
Response. Thank you for your comment. LeanRepair reduces tokens on every benchmark we evaluate. This includes QuixBugs, which contains real, human-written defects. The revision now states the repository-level scope explicitly.
Weakness 2.
Response. Thank you for your comment. LeanRepair's advantage holds at larger scale. At Qwen3-14B it has the highest average pass rate of all five methods (84.5%) and uses 22% fewer tokens than GoT.
Evaluating even larger and code-specialized models is a natural next step, which we list as future work.
Note on one Table 1 cell. While answering other reviewers, we found that in the Qwen3-8B / QuixBugs cell, the baselines had been run with Qwen3's thinking mode while LeanRepair had not. We will correct that cell and withdraw the QuixBugs-specific 82.4% figure. The headline result is unchanged: the median reduction across 24 cells remains 23.9%. Details are in the general response.
Minor.
Response. Thank you for your comment. The "??" came from a missing BibTeX pass. The recompiled PDF has no unresolved references.
Thank you again for your review. We hope the clarifications and the added scope discussion address your concerns, and we would be glad to answer any further questions.