acceptodds
Under review as a conference paper at ICLR 2027

Not Every Failure Needs a Patch: PAUSE for Selective Program Repair under Uncertain Execution Feedback

Abstract

LLM-based automated program repair increasingly relies on iterative execution feedback to guide patch generation. However, systems often treat each failure as evidence that the program should be modified. Because test and runtime signals may be incomplete, noisy, or environmentally induced, this assumption can cause unnecessary patches, ineffective retries, and unreliable termination. We present PAUSE, a selective program repair framework that determines whether modification is warranted, whether the current candidate is acceptable, and whether continued repair remains worthwhile. PAUSE coordinates specialized agents to analyze repair context, reason over execution evidence, decide between Patch and NoOp, and learn from repair trajectories. Its dual-condition governor separately evaluates candidate acceptability and continuation value. Informative trajectories are distilled into reusable decision constraints rather than patch templates. Experiments on three benchmarks spanning Python and Java and dedicated control cases show that PAUSE improves confirmed repair rate by 2.5–3.0 percentage points over the strongest controlled baseline, limits over-repair and unsupported acceptance to 1.7% and 2.0%, respectively, and reduces model calls by 44.6% relative to fixed-budget execution. These results demonstrate the value of selective action and trajectory-aware governance for LLM-based program repair.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Automated Program Repair, Selective Repair, Execution Feedback, Repair Governance, LLM Agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ywKDOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PAUSE, a program repair framework that decides whether to change code, accept a result, or continue trying. It uses five specialized agents and a repository of lessons from previous repair attempts. The authors evaluate PAUSE on 2,580 faults across three benchmarks, along with control cases involving correct programs and unreliable feedback. PAUSE achieves higher confirmed repair rates than the controlled baselines and reports fewer unnecessary changes, fewer unsupported acceptances, and lower model usage in some comparisons.

Critical strengths
  • Broad evaluation. The experiments cover three benchmarks, two programming languages, and three model backbones. The controlled baselines use the same backbone, task evidence, attempt budget, and validation procedure.

  • Evaluation beyond passing tests. The paper checks whether patches pass additional validation. It also measures unnecessary changes, unsupported acceptance, and early stopping.

Critical weaknesses
  • The paper does not quantify how often unreliable feedback causes unnecessary changes or wasted attempts in realistic repair workflows. The practical need for the proposed mechanism therefore remains insufficiently supported.

  • Most instances are small algorithmic programs. Although Defects4J includes real project bugs, the results do not establish whether PAUSE handles longer repository-level tasks, where several attempts without immediate progress may still be necessary.

  • Table 6 does not report acceptance, rejection, and successful repair rates under unreliable feedback. These results are needed to determine whether low unsupported acceptance comes at the cost of rejecting useful repair opportunities.

  • PAUSE uses five agents with separate prompts, yet averages only 4.1 model calls per fault. The paper should explain which roles require separate calls and whether any are skipped, combined, or reused.

Questions for authors
  1. How common is unreliable execution feedback in realistic repair workflows?
  2. How accurately does the Reflector predict whether further attempts will produce a confirmed repair?
  3. How does PAUSE perform under each of the four unreliable-feedback conditions?
  4. Does each repair iteration make a separate model call for all five roles? If so, how is the average of 4.1 calls per fault obtained?
  5. Does PAUSE still achieve higher repair rates than the controlled baselines under the same token budget?
  6. Can PAUSE be applied directly to repository-level repair tasks? If so, how?
Reviewer_VhQyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses an important question in automated program repair (APR): whether an observed execution failure warrants a code change. The authors introduce PAUSE, a multi-agent framework that uses execution feedback and repair trajectory to determine whether to modify the program, whether the current candidate is acceptable, and whether further repair attempts are worthwhile. It combines 5 specialized agents to guide the repair process.

Critical strengths

The problem is relevant and important.

Critical weaknesses

A major weakness of this paper is that the mismatch between the problem it aims to address and its experimental evaluation. It is ok to say that many patches/fixes in APR domain may 'over fix' the bug, while the paper does not show how frequently existing APR methods exhibit this behavior or demonstrated that PAUSE reduces it relative to those methods. Furthermore, debugging encompasses bug detection, localization, and repair. If APR assumes that the input program contains a confirmed bug, determining whether an observed failure actually warrants repair extends the original task. The paper should clarify whether PAUSE improves repair itself or introduces an additional decision stage, and compare with baselines in a fair setup (e.g., also provide baselines with a bug detection result as input).

Another major issue of this paper is the inconsistency of the evaluation results provided in the main pages and appendices. For example, Table 2 reports a QuixBugs plausible repair rate of 95.0%, whereas Table 20 reports 87.5%. The confirmed repair rates for ConDefects are 41.2% and 41.8%, respectively, and those for Defects4J are 38.9% and 37.3%. The governance results also disagree: PAUSE’s NoOp accuracy is 91.3% in Table 6 but 96.3% in Table 21, while its premature-stop rate is 3.1% and 1.3%, respectively... These differences affect the main effectiveness and claims, there are several similar issues about other RQs, please check.

I think Section 4.5 is attractive and match the proposed 'over fix' problem, however, there are some important missing details. Where these 300 verified-correct programs and faulty programs come from? Are they in a single programming language? What is their complexity (loc or Cyclomatic Complexity)? Can existing bug detection/APR methods 'over fix' them? Any showcases demonstrating the effectiveness of PAUSE? I think addressing these gaps would allow a more rigorous assessment of the proposed method’s effectiveness. Also, the ablations provide some evidence for the complete workflow, but the attribution of each component's benefits to the 'over fix' problem remains unclear.

The experimental settings of compared baselines, like fault-localization information and source context, are unclear.

The observation taxonomy in Section 3.3 differs from that in Appendix A.3 and the Executor prompt.

Questions for authors

Please see the detailed comments in the Weakness section.

Reviewer_zvBBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PAUSE, a selective program repair framework in which five roles, i.e., Profiler, Planner, Executor, Reflector, and Governor, work with a non-generative experience-retrieval module. At each iteration, the framework chooses PATCH or NOOP, separately assesses the acceptability of the current candidate and the value of continued repair, and ultimately issues AcceptPatch, Preserve, Continue, or Reject.

Critical strengths

The problem decomposition has practical value.

Critical weaknesses
  1. The main evidence cannot be reconciled between the main text and appendices, making it unclear what the paper has actually demonstrated. Which set of values in the main text and appendices comes from the final run?

  2. The ground truth for safe rejection and selective risk is not sufficiently operationalized, and the safety gains may still largely reflect low acceptance coverage. How were the labels for the 247/260 “safe rejections” obtained? Of the remaining 13 cases, how many are identified through successful paired continuation, and how many through other rules? Please provide complete definitions of the control perturbations and continuation protocol, and report comparisons with simple conservative decisions at the same acceptance coverage.

Questions for authors

See above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.