- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces PAUSE, a program repair framework that decides whether to change code, accept a result, or continue trying. It uses five specialized agents and a repository of lessons from previous repair attempts. The authors evaluate PAUSE on 2,580 faults across three benchmarks, along with control cases involving correct programs and unreliable feedback. PAUSE achieves higher confirmed repair rates than the controlled baselines and reports fewer unnecessary changes, fewer unsupported acceptances, and lower model usage in some comparisons.
Critical strengths
-
Broad evaluation. The experiments cover three benchmarks, two programming languages, and three model backbones. The controlled baselines use the same backbone, task evidence, attempt budget, and validation procedure.
-
Evaluation beyond passing tests. The paper checks whether patches pass additional validation. It also measures unnecessary changes, unsupported acceptance, and early stopping.
Critical weaknesses
-
The paper does not quantify how often unreliable feedback causes unnecessary changes or wasted attempts in realistic repair workflows. The practical need for the proposed mechanism therefore remains insufficiently supported.
-
Most instances are small algorithmic programs. Although Defects4J includes real project bugs, the results do not establish whether PAUSE handles longer repository-level tasks, where several attempts without immediate progress may still be necessary.
-
Table 6 does not report acceptance, rejection, and successful repair rates under unreliable feedback. These results are needed to determine whether low unsupported acceptance comes at the cost of rejecting useful repair opportunities.
-
PAUSE uses five agents with separate prompts, yet averages only 4.1 model calls per fault. The paper should explain which roles require separate calls and whether any are skipped, combined, or reused.
Questions for authors
- How common is unreliable execution feedback in realistic repair workflows?
- How accurately does the Reflector predict whether further attempts will produce a confirmed repair?
- How does PAUSE perform under each of the four unreliable-feedback conditions?
- Does each repair iteration make a separate model call for all five roles? If so, how is the average of 4.1 calls per fault obtained?
- Does PAUSE still achieve higher repair rates than the controlled baselines under the same token budget?
- Can PAUSE be applied directly to repository-level repair tasks? If so, how?