acceptodds
Desk rejected at ICLR 2027

Repair Before Regenerating: Scope-Controlled Recovery for LLM Agents

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

LLM agents increasingly solve programming tasks by writing, executing, and revising code. In these workflows, a failed intermediate program interrupts subsequent actions, making efficient recovery an essential part of task execution. Recovery begins with two useful resources: an existing candidate that encodes the current approach to the task, and execution feedback that reveals where its behavior departs from the specification. Existing pipelines often insert reflection or supervision before regenerating a complete module, adding model calls and repeatedly producing code beyond the correction itself. We introduce RepairAgent, an execution-grounded local-repair framework that turns concrete execution discrepancies into targeted changes to the current program. The framework separates two recovery decisions that existing pipelines typically couple: what evidence the model receives and how much of the artifact it regenerates. For supported public equality checks, RepairAgent reads the failed expression and its actual and expected values directly, then proposes exact-text replacements for selected spans. The interface validates replacement targets and the resulting module's syntax, preserving text outside the edited spans while concentrating generation on the correction. Public execution validates each candidate and supplies evidence for further repair, with complete-module generation available as a second-attempt fallback. On executable repair tasks from QuixBugs, MBPP, and HumanEval, RepairAgent reaches a 93.3% held-out pass rate at 14.14 seconds of dataset-macro latency, reducing complete-pipeline latency by 49.5–88.6% relative to adapted SupervisorAgent, Reflexion, and Self-Refine baselines. Matched controls isolate the scope effect: local edits reduce latency by 15.6–18.7% under fixed feedback.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM Agents, Automated Program Repair, Execution Feedback, Local Code Editing, Inference Efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_38SWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RepairAgent, an execution-grounded repair framework for LLM-generated Python programs. The method directly exposes actual–expected values from supported public equality checks and asks the model to produce validated exact-text replacements instead of regenerating the entire module. Full-module regeneration is retained as a fallback on the second attempt. Experiments on mutated tasks from QuixBugs, MBPP, and HumanEval report comparable held-out correctness to several feedback-based baselines while reducing latency and output tokens. However, the core design largely combines two familiar engineering choices—using concrete execution feedback and applying local patches—and the experimental setting is too narrow to establish a substantive research contribution.

Critical strengths

The paper identifies a real inefficiency in LLM-based code repair: regenerating an entire artifact when only a small span is incorrect. The controlled comparisons are also useful in showing that local edits reduce generation cost under matched feedback conditions, and the interface validates edit uniqueness and syntax before execution.

Critical weaknesses
  1. The main ideas—using execution feedback to guide repair and changing only the faulty code region—are intuitive extensions of existing execution-guided debugging, program repair, and patch-based editing methods, which has been already widely applied in many agent works and products.

  2. The evaluation uses synthetic or mutated defects selected from QuixBugs, MBPP, and HumanEval, with only publicly detectable failures, a fixed two-attempt budget, and a single Qwen3.5-27B model. These tasks do not adequately represent repository-level repair, multi-file dependencies, ambiguous failures, regressions, or realistic debugging workflows.

Questions for authors

see weakness

Reviewer_VAYjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces RepairAgent for code repair, focusing on the efficiency of the repair process. RepairAgent combines value feedback (expected vs. actual outputs) with a local editing tool, reducing repair latency by 49.5%-88.6% compared with the baseline methods. The paper also presents ablation studies on feedback type (raw, value, typed) and editing scope (full-module regeneration vs. local edits).

Critical strengths
  • The motivation and experimental design are clear.
  • The visualizations are clean and informative, and the paper is easy to follow.
Critical weaknesses
  • The datasets used in this study, including MBPP and HumanEval, were released in 2021 and are somewhat dated. This limits the timeliness and potential impact of the work. Evaluating on more recent coding benchmarks, such as SWE-bench verified, would strengthen the paper.

  • The finding that value feedback combined with local edits is the best-performing combination is informative. However, the evaluation is limited to a narrow setting: a single model (Qwen3.5-27B) and relatively simple code repair tasks. I am concerned about whether the contributions demonstrated within this scope are sufficient to warrant publication at ICLR. To expand the scope I recommend evaluating additional models and a broader range of repair tasks (e.g., constrained text generation).

Questions for authors
  • Qwen3.5-27B supports thinking mode. Was thinking mode enabled in your experiments? Does the 2,048-token generation budget include thinking tokens?
  • I found reports that Qwen3.5-27B achieves >95% on HumanEval in the direct code generation setting (https://huggingface.co/Jackrong/Qwopus3.5-27B-v3). This raises the question of how much the observed repair performance depends on the model’s underlying code generation capabilities: the model might solve the task without using the buggy code. Could the authors provide a direct code generation baseline using the same model, task specifications, and held-out tests?
Reviewer_qEzcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies efficient recovery from failing programs by separating execution feedback from repair output scope. RepairAgent combines actual–expected value feedback with validated text replacements, with full-module rewriting available on the second attempt. Experiments on repair suites derived from QuixBugs, MBPP, and HumanEval show lower latency than adapted feedback-based baselines, while matched comparisons demonstrate efficiency gains from local edits.

Critical strengths

This paper studies efficient recovery from failing programs by separating execution feedback from repair output scope. RepairAgent combines actual–expected value feedback with validated text replacements, with full-module rewriting available on the second attempt. Experiments on repair suites derived from QuixBugs, MBPP, and HumanEval show lower latency than adapted feedback-based baselines, while matched comparisons demonstrate efficiency gains from local edits.

Critical weaknesses
  1. The applicability of value feedback remains unclear. It requires supported checks that expose actual and expected values; other failures fall back to ordinary error feedback. The paper does not establish how frequently this information is available or when it is sufficient to guide repair without additional diagnosis. An output mismatch can describe a failure without explaining its cause, so the broader usefulness of this mechanism needs stronger evidence.

  2. The methodological contribution appears incremental. Execution feedback, iterative patching, and test-based validation are established ideas in program repair. The paper offers a reasonable combination and useful controlled comparisons, but does not clearly establish a substantial advance over existing approaches. Its strongest finding concerns the efficiency of local output, while the additional benefit of value feedback is less convincing.

  3. The evaluation scale is insufficiently documented. The numbers of tasks and unique source programs in each suite, their dataset breakdowns, and some dataset selection details are not explicitly reported. This makes the reliability and representativeness of the results difficult to assess. The independent ablation also lacks complete held-out success rates, leaving the efficiency–correctness trade-off unclear.

Questions for authors
  1. How often is actual–expected feedback available, and how do its benefits vary across failure types?
  2. What is the main technical advance over existing execution-guided, patch-based repair methods?
  3. What are the exact task and unique-source counts in both suites, and the held-out success counts for all ablation policies?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.