- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies GRPO training for agentic code repair under weak execution feedback. It proposes a signal-reshaping framework combining layered semantic rewards, step-level process-score weighting, and failure-aware rollout handling. On an internal C/C++ compile-fix benchmark, the full method improves strict repair accuracy from 38.5% to 53.5%.
Critical strengths
The paper identifies an important mismatch in agentic RL: compilation is a useful but incomplete reward, and the decomposition into outcome ranking, intra-trajectory credit, and execution comparability provides a coherent framework for addressing this issue.
The empirical validation is relatively strong. Beyond the training reward model, Tables 6 and 7 use an independent LLM judge and blind human evaluation on the same 400 instances, both showing a statistically significant improvement of GRPO-full over GRPO-early.
Critical weaknesses
The layered reward relies on a questionable and insufficiently justified ordering assumption. It assigns every compiling but semantically incorrect patch a higher reward than every non-compiling patch (0.5 vs. 0), even though a nearly correct repair with a minor compilation error may be much closer to the target than a compilable patch that substantially changes or even reverses the intended behavior. Moreover, the specific choice of 0/0.5/1 is heuristic, and no sensitivity analysis is provided for alternative intermediate rewards.
The paper improves how semantic feedback is incorporated into GRPO, but does not substantially address the more fundamental problem of how reliable semantic supervision should be obtained in weak-feedback code repair. I do not consider the use of an LLM-based semantic judge itself to be a weakness; the paper provides reasonable human and independent-judge validation of Kimi-K2.5. My concern is instead about the scope of the claimed contribution. Recent work in code generation and automated program repair has already explored richer supervision beyond sparse compile/test outcomes. For example, BoostAPR learns sequence- and line-level reward models from execution-grounded repair outcomes [1], CODERL+ explicitly introduces execution-semantic signals into RL rather than relying only on binary correctness [2], and recent agentic program-repair systems combine static analysis, test execution, and LLM-based judging as complementary validation signals [3]. LLM-based semantic assessment of generated patches is itself also becoming an established tool in software-engineering evaluation [4]. Against this background, the central difficulty is not simply recognizing that compilation is semantically incomplete and replacing it with a stronger LLM-derived signal. The harder question is how to obtain semantic supervision that is grounded enough to go beyond the limitations of the evaluator itself. In this paper, semantic correctness is ultimately supplied by a strong offline LLM judge that is conditioned on a ground-truth repair, and the step-level process signal is produced by the same class of model-based supervision. The proposed signal reshaping therefore provides a better optimization interface for already-available semantic supervision, but it does not reduce the fundamental dependence on such supervision. This distinction is particularly important for the paper's weak-feedback framing: if a sufficiently strong GT-conditioned semantic oracle is assumed to be available during training, then a substantial part of the original semantic-feedback problem has already been outsourced to that oracle. I would therefore like the paper to more clearly distinguish its contribution to credit assignment and reward utilization from the broader problem of acquiring reliable semantic feedback, which, in my view, remains largely unresolved.
[1] Li, Yuanhao, Hongbo Wang, Xiaotang Shang, Xunzhu Tang, Yiming Cao, and Xuhong Chen. “BOOSTAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models.” In Proceedings of the 43rd International Conference on Machine Learning, 70897–70918. Proceedings of Machine Learning Research 306. PMLR, 2026.
[2] Jiang, Xue, Yihong Dong, Mengyang Liu, Hongyi Deng, Tian Wang, Yongding Tao, Zhi Jin, Wenpin Jiao, and Ge Li. “CODERL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment.” In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 3608–3622. Association for Computational Linguistics, 2026. ACL Anthology
[3] Maddila, Chandra, Adam Tait, Claire Chang, Daniel Cheng, Nauman Ahmad, Vijayaraghavan Murali, Marshall Roch, et al. “Agentic Program Repair from Test Failures at Scale: A Neuro-Symbolic Approach with Static Analysis and Test Execution Feedback.” IEEE Transactions on Software Engineering 52, no. 8 (2026): 2446–2462. https://doi.org/10.1109/TSE.2026.3696849
[4] Zhou, Xin, Kisub Kim, Ting Zhang, Martin Weyssow, Luís F. Gomes, Guang Yang, and David Lo. “An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks.” arXiv preprint arXiv:2505.20854, 2025.
Questions for authors
-
Reward ordering. The layered reward assumes that every compiling but semantically incorrect patch should receive a higher reward than every non-compiling patch (0.5 vs. 0). What justifies this ordering? For example, a nearly correct patch with a minor compilation error may be closer to the target than a compilable patch that substantially changes the intended behavior. Have you tested alternative orderings or intermediate reward values?
-
Dependence on a strong semantic oracle. The method relies on a GT-conditioned Kimi-K2.5 judge to provide both semantic outcome rewards and process-level supervision. To what extent is the proposed framework still applicable when such a strong semantic evaluator or ground-truth repair is unavailable? More generally, is the main contribution intended to address weak feedback itself, or to improve optimization given access to a strong offline semantic oracle?
-
Sensitivity to judge quality. The human audit shows that the semantic judge is reasonably accurate but still imperfect. Have you evaluated how training performance changes when the semantic judge is replaced by another model or when controlled noise is introduced into its labels? This would help clarify whether the reported gains depend on the particular quality of Kimi-K2.5.
2 comments
Thank you for the clarifications. However, my main concerns remain largely unresolved.
For Q1, the reported ablation supports the usefulness of an intermediate reward, but does not justify the specific reward ordering or value. For Q2, the response clarifies the distinction between training and deployment, but does not establish how the framework would perform without strong offline semantic supervision. For Q3, changing the evaluation judge does not address sensitivity to the judge used during training, which was the focus of my question. Furthermore, while the response clarifies that the method uses strong offline semantic supervision to improve optimization under weak rollout feedback, it does not sufficiently establish the methodological novelty beyond existing approaches that leverage richer semantic and process-level signals for code repair (which I mentioned in Critical Weaknesses of the original review).
I appreciate the authors' transparency in acknowledging these limitations. However, without additional experiments directly addressing these concerns, I do not find sufficient evidence to revise my assessment. I therefore maintain my original rating of Reject.
Response to Reviewer 5GJ1
Q1. Why give higher rewards to patches that compile but are incorrect, and why choose 0.5?
The reward reflects the checks a patch has passed. Passing compilation is necessary for semantic success and a successful repair, so it earns partial credit. Full credit requires passing both compilation and semantic checks.
The ablation results also show that adding the intermediate reward improves strict accuracy by 0.5pp, compilation success by 3.5pp, and single-edit success by 7.4pp. This shows that an intermediate reward helps the model make progress on repairs. We tested an intermediate reward of 0.5. We have not tested other values or reversed the reward order.
Q2. Does the method solve the weak-feedback problem or make better use of existing semantic supervision?
We thank the reviewer for pointing out the distinction between where supervision comes from and how it is used. The task originally provides only weak compiler feedback. We introduce semantic supervision during training through reference repairs and a judge. We then use outcome rewards to compare trajectories and process scores to adjust updates at each step, using this added supervision for both trajectory ranking and fine-grained learning. During rollout, the agent still receives only compilation feedback, and reference repairs are not needed at deployment. Our method therefore adds semantic supervision to the original weak feedback and uses it to guide both trajectory selection and step-level learning.
Q3. What happens if the training judge is replaced or label noise is added?
We changed the judge during evaluation: GLM-5.2 and blind human evaluation found gains of 5.75pp and 6.25pp for Full over Early, respectively. The improvement holds when the evaluator changes. We have not run experiments that replace the judge during training or add controlled noise to the labels.