acceptodds
Under review as a conference paper at ICLR 2027

When Repairs Stop Being Recipes: Re-Grounding Repair Memory for Coding Agents

Abstract

Recent coding-agent systems store patches and debugging strategies from resolved issues as repair memory, then retrieve these records to guide later repairs and avoid repeated investigation. However, reusing a past fix can misdirect the next repair: a record may identify useful code while recommending an edit that no longer resolves the current issue. This repair applicability gap raises a question: how can an agent benefit from an earlier investigation when its fix no longer works? To study this question, we introduce RepairMem, a benchmark that pairs 99 current repair tasks with related historical repairs, covering both controlled faults and real repository issues. We evaluate five coding models on these tasks and find that giving them historical records unchanged usually reduces repair success compared with providing no memory. To make these records useful for the current task, we propose ReGround. Before the agent begins its repair, ReGround checks the historical patch against the current tests, uses the outcome to revise the historical repair advice, and supplies relevant code from the current repository to guide further debugging. This simple procedure requires at most one replay of the historical patch and no additional model calls for preparation. Across five models and 99 repair tasks, ReGround improves aggregate repair success by 16.4 and 27.7 percentage points over memory-free repair and direct historical reuse, respectively, while using 20.2% and 22.5% fewer model calls.

Primary area:
foundation or frontier models, including LLMs
Keywords:
coding agents, program repair, agent memory, large language model agents, software engineering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VMzqOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how coding agents should reuse historical repair experience when an old repair is related to the current issue but its original patch is no longer a valid solution. It proposes a benchmark called RepairMem with controlled (or called constructed) and real "historical and current repair pairs". And it also proposes ReGround, which reassesses the old patch on the current task and provides current code evidence before the agent begins repair. Across 5 models, ReGround improves repair success and reduces model calls compared with no memory condition and direct reuse of historical repairs.

Critical strengths
  • The problem is practical and clearly motivated and the method is simple and intuitive.

  • The benchmark includes both controlled (or called constructed) and real constructions repair pairs, and the evaluation covers 5 coding models.

  • Most of the presentation of this paper is smooth and easy to follow, the content is well structured.

Critical weaknesses
  • In the Introduction, the paragraph "To evaluate this reuse directly, we introduce REPAIRMEM xxx" suddenly introduces two concepts, which makes the presentation very hard to follow. Controlled tasks are easy to understand, but particularly I can't get what "naturally sourced tasks" means, and I have to read the later sections to roughly understand it. Why not directly call them constructed tasks and real tasks, or use names that are easier to understand?

  • The main text does not clearly define what is a historical repair experience or how a related repair would be identified in a real memory system. In the benchmark, each task is directly provided with a selected historical repair record and patch. Controlled task's relation is clear since it is based on specific code relation construction, but for natural task, it is unclear what memories would be stored in practice and what method would retrieve or select the "related" repair for a current task.

  • The main experiment of the comparison does not isolate the effect of regrounding the historical repair. ReGround receives current execution feedback and relevant source excerpts, while No Memory and Raw History do not. According to my understanding, a normal coding agent should also execute code or obtain some relevant excerpts.

  • In addition, the claim that "raw historical repair can reduce repair success" is not very rigorous. All agents operate under a fixed 24 call budget (intuitively, I feel this may be too small, because using historical experience for testing is a very normal thing to do), and the matched-evidence results suggest that a major effect of ReGround is reducing interaction cost. It is therefore unclear whether Raw History is intrinsically harmful or mainly less efficient under the current budget.

  • The construction of the Controlled tasks is not clearly explained, including how the current faults are generated, what types of faults are included, how the final tasks are selected, and whether the constructed cases receive any independent or human validation.

If the authors can address the weaknesses and questions I mentioned, I would be happy to raise my rating to 3.

Questions for authors
  • How would a "related" historical repair be retrieved in a real memory system? And what is the definition of "related"?
  • Does Raw History still hurt with a larger interaction budget?
  • How are the Controlled-task faults generated and selected?
  • Are the constructed Controlled cases independently or manually validated? What are him/her/them background? SE expert? SE major Ph.D. student?
  • Also address the concerns I mentioned in Weaknesses part.
Reviewer_RdmcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies what the repair applicability gap: a historical repair record retrieved from memory may still point at relevant code while its stored patch no longer fixes the current issue. To study this, the paper introduces RepairMem, a 99-task executable benchmark with two tracks. The Controlled Hidden-Drift track branches a historical and a current fault from a shared passing checkout, keeping the public test fixed while making the historical patch fail on the current fault by construction. The Natural-Provenance track pairs earlier developer repairs with later SWE-bench issues linked by same-function or call/override relations. It then proposes ReGround, which prepares the agent’s initial context by running the current test, replaying the historical patch once, revising the historical advice based on the outcome, and supplying failure-anchored source excerpts, with no extra model calls. Across five backbones under a 24-call budget, ReGround improves success by 16.4 pp over no memory and 27.7 pp over raw history while using around 20% fewer calls.

Critical strengths

None

Critical weaknesses
  • ReGround’s stated contribution is to reassess a patch by replaying it. First, to replay the memory, the memory itself needs to contain a patch that is parsable from the content. The method therefore cannot be applied to memories in natural languge. Second, it is uninformative on real data. In the Natural-Provenance track, the historical patch comes from an earlier issue in the same repository. A PASS would require the two issues to share a root cause, which is not feasible in this setting. The only possible outcomes are PRESENT, FAIL, and UNAVAILABLE, and all three lead to the withdrawing the reuse recommendation. The replay therefore does not change a decision on real data. The observed gains may merely come from what ReGround does alongside it: pre-running the current tests, selecting failure-anchored source excerpts, and demoting the stored patch to a reference. None of these require a replay or a memory.
  • The evaluation is not reliable. Its benchmark only has 44 real samples (far below the standard of repository-level benchmarks) collected from SWE-bench (it has been reported to contain quality issues). Most importantly, the evaluation is with a 24-step budget, which is a small limit. I notice that, based on Table 1, the average calls for Opus on Raw History (Controlled) is exactly 24. It means all the tasks are truncated and thus leads to only 20% pass rate. Under such a budget, any intervention that saves early calls converts directly into accepted runs, so the reported success gains and call reductions are not independent evidence.
Questions for authors
  • Can you evaluation the method with substantially more real-world tasks, such as 200 tasks?
  • What is the contribution of the replay step itself? Since the historical patch never passes on the Natural track, an ablation that removes the patch replay but keeps the witness pre-run and the source frontier would clarify this.
Reviewer_xWksOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces REPAIRMEM, a benchmark pairing 99 current repair tasks with related historical repairs: 55 controlled faults and 44 naturally sourced repository issues. Across five coding models, the authors find that supplying historical records unchanged often reduces repair success. They propose REGROUND, which checks the historical patch against the current task, revises its advice based on the result, and supplies relevant current code to the agent. On REPAIRMEM, REGROUND improves aggregate repair success over both no memory and unchanged history while using fewer model calls.

Critical strengths
  1. REPAIRMEM separately measures whether a historical patch works on the current task and whether its record helps an agent produce a new repair.
  2. The evaluation covers five model backbones under a fixed agent harness and compares no memory, unchanged history, and REGROUND.
Critical weaknesses
  1. The evidence for generalization remains limited. The 55 controlled tasks derive from 18 repair relations across three repositories, while all main experiments use mini-SWE-agent. The 44 naturally sourced pairs broaden coverage, but the paper does not show whether the gains are stable across repositories and repair relations or persist under another agent framework. Task-level aggregate results alone cannot establish that broader conclusion.
  2. The headline comparisons do not fully isolate why REGROUND helps. Relative to no memory or unchanged history, REGROUND also supplies current execution evidence and code excerpts. The matched-evidence comparison addresses this concern for two models on Controlled-55, but not for the other models or Natural-44. The improvement of the complete system is clear on this benchmark; the separate contribution of revised historical advice is less certain across the full evaluation.
Questions for authors
  1. Do the gains over History + Evidence persist on Natural-44 and the remaining model backbones?
  2. How much do success rates vary by repository and repair relation? Do the gains persist with an agent harness other than mini-SWE-agent?
  3. Since final acceptance relies on disclosed task checks, what additional evidence rules out patches that pass those checks without adequately fixing the issue?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.