- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes RepairAgent, an execution-grounded repair framework for LLM-generated Python programs. The method directly exposes actual–expected values from supported public equality checks and asks the model to produce validated exact-text replacements instead of regenerating the entire module. Full-module regeneration is retained as a fallback on the second attempt. Experiments on mutated tasks from QuixBugs, MBPP, and HumanEval report comparable held-out correctness to several feedback-based baselines while reducing latency and output tokens. However, the core design largely combines two familiar engineering choices—using concrete execution feedback and applying local patches—and the experimental setting is too narrow to establish a substantive research contribution.
Critical strengths
The paper identifies a real inefficiency in LLM-based code repair: regenerating an entire artifact when only a small span is incorrect. The controlled comparisons are also useful in showing that local edits reduce generation cost under matched feedback conditions, and the interface validates edit uniqueness and syntax before execution.
Critical weaknesses
-
The main ideas—using execution feedback to guide repair and changing only the faulty code region—are intuitive extensions of existing execution-guided debugging, program repair, and patch-based editing methods, which has been already widely applied in many agent works and products.
-
The evaluation uses synthetic or mutated defects selected from QuixBugs, MBPP, and HumanEval, with only publicly detectable failures, a fixed two-attempt budget, and a single Qwen3.5-27B model. These tasks do not adequately represent repository-level repair, multi-file dependencies, ambiguous failures, regressions, or realistic debugging workflows.
Questions for authors
see weakness