acceptodds
Under review as a conference paper at ICLR 2027

REFIXAGENT: ITERATIVE REFLECTION AND LOCAL- IZATION FOR AUTOMATED PROGRAM REPAIR

Abstract

Automated program repair via Large Language Models (LLMs) has achieved re- markable advancements, yet prevailing agent-centric paradigms exhibit limited exploration of diverse remediation strategies. Although multi-sampling method- ologies have been introduced to alleviate this constraint, they frequently converge toward homogeneous solutions owing to the stochastic nature of autoregressive language models. To mitigate these deficiencies, we introduce ReFixAgent, a col- laborative multi-agent architecture that enables systematic exploration of diverse candidate localization sets and implements iterative introspection and refinement for patch synthesis, thereby augmenting the strategic search landscape of program repair. Our framework initiates with multi-faceted localization set sampling, op- tionally augmented with code dependency graph-enhanced fault localization for complex remediation tasks. This is succeeded by iterative introspective refinement to generate high-fidelity patches, culminating in a final-generation phase guided by distilled knowledge from all historical remediation attempts. Empirical evaluations on the SWE-bench Verified benchmark demonstrate that ReFixAgent attains excep- tional performance, achieving 74.4% pass@1 with DeepSeek-V3.2—surpassing state-of-the-art methodologies under equivalent model configurations by up to 5.0%. The framework also exhibits robust generalization across diverse backbone models including Claude-4, GPT-5, and Qwen-Coder, while delivering superior fault localization precision compared to existing approaches.

Primary area:
datasets and benchmarks
Keywords:
Large Language Models, Automated Program Repair, Fault Localization, Multi-Agent Systems, Iterative Refine- ment

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_KdsPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

ReFixAgent explores multiple candidate edit-location sets, generates patches constrained to each set, and progressively selects and refines candidates using trajectory analysis and regression testing. Tasks with multiple distinct localization sets additionally receive graph-based localization information. A final generation stage synthesizes guidance from earlier attempts. On the 500-instance SWE-bench Verified benchmark, the paper reports 74.4% resolution with DeepSeek-V3.2, compared with 69.4% for the strongest reported same-backbone baseline. It also reports improvements with other backbones and when incorporating the mechanism into Mini-SWE-agent and Live-SWE-agent.

Critical strengths
  1. Exploring alternative localization hypotheses is a sensible way to address repeated repair attempts that remain trapped at the same incorrect edit location. The separation between search over locations and patch refinement makes the proposal understandable and potentially reusable.
  2. The reported gains appear across several model configurations. The integrations with two existing agent frameworks are useful evidence that the idea is not tied to one implementation.
Critical weaknesses
  1. The main comparison does not isolate the proposed search strategy from additional inference and selection. ReFixAgent internally samples multiple localization sets and patches, invokes several analysis or selection agents, and performs a final synthesis. Reporting one returned patch per task is compatible with a system-level pass@1 measure, but does not make this computationally comparable to a single baseline rollout. Appendix A.1 states a 50-agent-step budget without making sufficiently clear whether every agent call, graph exploration, candidate generation, and selection operation is charged to one shared budget, or how equivalent limits are applied to the baselines. No token, cost, or wall-time comparison establishes efficiency. The central experiment should compare against budget-matched independent sampling plus selection, and repeated refinement at one location. Without this control, the gains do not establish that localization diversity is better than allocating similar computation to a simpler search procedure.

  2. The proposed mechanism is plausible but insufficiently diagnosed. The paper motivates a lack of search diversity, yet does not quantify how often additional localization sets recover the correct region, how often the selector retains a correct candidate, or whether constrained editing rejects valid repairs outside the initial ranges. This matters because the procedure repeatedly discards half the candidate sets and cannot necessarily recover a discarded strategy. Section 3.2 also filters out existing tests deemed unrelated to the issue, which can remove evidence of regressions elsewhere; the cost-benefit of that filter is not evaluated. A focused analysis of localization diversity, candidate survival, and final selection accuracy would support the mechanism more directly than resolution-only ablations. The graph component improves the score by only 0.2 points on each reported backbone, equivalent to one issue in a single 500-task run, so its claimed contribution needs uncertainty or targeted evidence.

Questions for authors
  1. What is the complete per-task computation accounting, and how does ReFixAgent compare with independent sampling plus the same selector at an equal budget?
  2. What exactly is averaged in Table 2, and are additional predicted locations penalized? Can the authors provide counts, seeds, and corrected gain calculations?
  3. Are the 50 pilot tasks included in the main evaluation, and what is the result after excluding them?
  4. How frequently does selection discard a viable patch or location, and what happens when a correct repair requires editing outside the allowed ranges?
Reviewer_KjiBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the problem that APR agents may repeatedly generate patches around similar, but incorrect, code locations. The authors introduce ReFixAgent, which samples different localization sets and generates patches within these locations, followed by iterative reflection and patch selection. The paper reports a repair rate of 74.4% on SWE-bench Verified with DeepSeek-V3.2.

Critical strengths

The problem is relevant, and exploring different repair locations is a reasonable idea.

Critical weaknesses

The paper contains some unrelated content. Please check the supplementary material. Appendix A.5, Figure 5 and Table 8 discuss residual selection annealing, CAMU and MindSpore learning-rate schedules. However, I could not find their connection to the proposed APR method. I do not think these are related to this work.

The paper’s main motivation is that existing agents repeatedly focus on similar and possibly incorrect locations. However, it does not show how frequently this happens in existing methods. Does ReFixAgent actually produce more diverse locations? How often does it find the correct location after the first attempt fails? Also, CodeMonkeys and SE-Agent already explore multiple repair trajectories, but the paper mainly discusses their limitations without directly comparing with them. I think these comparisons and analyses are necessary to support the proposed motivation.

The paper does not adequately show how often existing methods suffer from the proposed problem, or whether ReFixAgent addresses it better than simply running more repair attempts. The comparison and ablations do not clearly separate the benefits of the proposed method from additional computation. Inconsistencies and missing details in the evaluation make the reported improvements difficult to assess.

Another issue is the fairness of the comparison. ReFixAgent generates multiple localization sets and patches, and uses additional agents for analysis, testing and selection. Although Appendix A.1 mentions a budget of 50 agent steps, how are these steps counted across different agents? Do the baselines receive the same total budget? Using the same LLM does not necessarily make the comparison fair. The same issue applies to the ablations: removing sampling or reflection also reduces the number of attempts. It would be useful to compare with baselines that spend the same budget on ordinary retries.

There are some inconsistencies between the metric definition and the reported results. Section 4.1 defines Acc@1 as whether each issue’s prediction covers all required modification locations. For 500 issues, a single-run accuracy should change in increments of 0.2 percentage points. However, Table 2 reports values such as 85.64% and 76.82%. Are these averaged over multiple runs, or calculated over individual code entities? This needs clarification. Furthermore, lines 343–346 interchange the improvements for DeepSeek and Qwen. According to Table 2, the gains should be 2.60, 2.60 and 3.22 percentage points for DeepSeek, and 2.62, 2.24 and 3.18 for Qwen. These are also absolute differences, rather than relative gains.

I am not convinced by the rule for identifying simple and complex tasks. The paper treats an issue as simple when repeated localization produces only one set after deduplication. However, the introduction specifically argues that agents may repeatedly agree on the same incorrect location. Would these issues also be classified as simple? If so, the graph-based localization would not be activated, and the editing constraints could keep the agent at the wrong location. The paper should explain how it handles this case and provide results showing whether the proposed difficulty rule is reliable.

The patch selection procedure is important to the whole workflow, but several details are missing. How are test quality and regression pass rate combined to rank patches? Do all candidates use the same tests? How often does the selector discard a correct patch? Also, removing tests judged unrelated to the issue may miss regressions elsewhere. The authors should clarify the filtering procedure and distinguish the tests used for internal selection from those used for final evaluation. It would also help to compare the final generated patch with the best patch already available before this step.

Some table references are incorrect. For example, Section 4.5 refers to Table 3 for component ablations, while Table 4 mainly shows the results before and after integrating ReFix into existing agents.

The ‘Stability’ metric in Table 1 is calculated across different agent systems. It is unclear why this should indicate the stability or generalizability of ReFixAgent.

Questions for authors

Please see detailed comments in the above Weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.