acceptodds
Under review as a conference paper at ICLR 2027

Rewards Beyond Test Passing: Towards Generalizable Fidelity in LLM Debugging

Abstract

Reinforcement learning (RL) from unit tests is the common recipe for post-training code models, but a unit test only asks whether the final program passes. We analyze what this unit-test-only reward teaches a model during debugging and find that, while it raises pass rates, it also teaches the model to edit more and to hallucinate bugs in code that is already correct. We introduce PreciseCoder, a training algorithm that augments the unit-test reward with a signal for semantically verified, localized edits, which correlates with both smaller edits and higher pass rates. On 4B and 9B models, PreciseCoder reduces the average number of edited lines from over 13 to 3 at a pass rate on par with unit-test-only training. Trained only on single-line bugs, PreciseCoder can generalize to repository-level program repair, where it resolves more issues with smaller patches. Its precision also transfers from weaker to stronger models: handing the reasoning trace of a 9B PreciseCoder model to frontier models such as GPT-5.6 Sol raises edit precision by 4.6 points and pass rate by 1.2 points with no additional training. Linguistic analysis of the thinking traces shows that our reward encourages the model to reason about individual lines, name the buggy ones, and form consistent bug hypotheses.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Reinforcement Learning, Generalization, Coding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dWAQOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the results of training with with outcome rewards based on unit tests in competition-like and repository-level coding scenarios. They find that purely training on the outcome rewards from unit tests teaches several bad behaviors to the policy. They propose shaping the reward by rewarding the policy for making more localized edits, and show that policies trained with the shaped reward perform better while also making fewer edits (with the implication that the edits made by other policies were unnecessary).

Critical strengths

One major strength is that the authors show their method on Qwen3.5 rather than Qwen3. It is very easy to improve Qwen3 compared to Qwen3.5 and so showing results on Qwen3 is relatively strong evidence the proposed method is relevant going forward.

The experimental section in general feels well done, with +/- numbers on the PreciseCoder rows and multiple model scales. As this is a bit of an empirical paper, I think this should give readers more confidence the method works.

Critical weaknesses

The main weakness, to me, is that a core component of the method seems to not be described clearly. This is the alignment of faults needed for precise reward. I'm not sure how it's done. It's also not clear to me that this fault identification is general, what the limits are, and so on. Under what conditions can it be done? Under what conditions cannot it be done? Etc. It requires a reference solution, which is fine, but if so, this should be clearly stated. Or maybe doesn't? If so, it needs to be explained.

I suppose another weakness is that it's not clear to me that reducing the edit size matters. Let's say I reduce the edit size, but this says nothing about the quality of the resulting code. I can reduce the edit size just by writing very obfuscated, code-golfed code. I do think the authors need to do a bit more work to establish that the resulting code edits are, in fact, qualitatively better in some sense than code generated by more standard training (I don't believe LoC is a good metric here).

Questions for authors

Please see the weaknesses. I have asked two questions there. Both of the answers will help me raise my score. I am generally leaning towards acceptance if these points are improved.

4 comments

We thank the reviewer for acknowledging that PreciseCoder works against strong baselines, and that our results show strong statistical evidence that the PreciseCoder works well. We address the reviewer's concerns as below.

W1: fault identification

Ans: Yes, fault alignment uses reference repairs and unit tests. The alignment itself follows the public PDB evaluator [1]; we summarize it briefly below.

  1. The alignment compares the model's repair Δ^\hat\Delta with a reference repair Δ⋆\Delta^\star. Both repairs are split into blocks, which are runs of consecutive changed lines.

  2. There is a block matching algorithm that can match, e.g. a reference block to multiple predicted blocks. A heuristic-based elimination method is used to find the best matching: identical blocks first, then blocks in the same place (completely wrapping the reference blocks, or surrounded by the same unchanged lines), then blocks that make the same change at another line.

Once we find the matching, the unit-tests decide whether a match is a fix. The reference repair is applied with the matched reference blocks δi⋆\delta^\star_i swapped for the predicted blocks δ^i\hat\delta_i, that is, Δi=(Δ⋆∖δi⋆)∪δ^i\Delta_i = (\Delta^\star \setminus \delta^\star_i) \cup \hat\delta_i. Note that every other fault is fixed by the reference, so passing the tests means δ^i\hat\delta_i fixed its faults. The smallest subset of δ^i\hat\delta_i that still passes the test, i.e., essi\mathrm{ess}_i in Section 3 Eqn 1, counts as needed.

Recall is the fraction of faults fixed. Precision is the number of needed lines (allowing ϵ\epsilon extra per fault) divided by all edited lines, so unmatched edits count as unnecessary.

As for when it works, the matching is reliable when blocks are well separated (when faults are at least 5 lines apart as defined in [1]). It is unreliable when blocks are crowded together, because neighboring edits merge and too few unchanged lines remain between them to anchor the matching. In the [1]’s manual check of 240 patches, recall was accurate on over 97.5% of the data, and precision strongly agreed with expert ratings (ρ=0.72\rho = 0.72).

The fault alignment algorithm is not a contribution for this paper, instead, we (1) diagnose training trajectory and explain heuristically (Sec 3.2) and empirically (Sec 4) why the designed F1 measure as a reward is preferred for precise LLM debugging; (2) show reinforcement fine-tuning with our reward achieves state-of-the-art, on both in-domain and OOD complex coding benchmarks, compared to other fine-tuning approaches (Sec 4, 5.1); and (3) show how we can transfer fidelity from fine-tuned weaker models to non-fine-tuned stronger models, and analyze what information is transferred through language (Sec 5.2, 5.3). However, we agree that understanding the alignment algorithm is important for understanding the paper. We have clarified this in the revised Section 3 and added a detailed description of the algorithm to the appendix.

W2: Whether edit-line based metrics are representative of true repair quality and the metrics can be hacked just by writing very obfuscated, code-golfed code.

Ans: We thank the reviewer for raising this point, and we perform additional analysis and find that (1) the edit-level metrics agree with human experts on a high-quality repair; and (2) in the current setting, training on the metric does not lead to code-golfing, and extra safeguards are unnecessary. We provide the following analyses as evidence to support these two claims.

First, two human studies show that the edit-level metrics agree with human experts on a high-quality repair.

  • In the expert rating study of [1] (Sec. 5.5), annotators with more than five years of Python experience rated patches on a 1-5 scale for how targeted and intent-preserving each repair was. Edit-level precision correlates strongly with these ratings (Spearman ρ=0.72\rho = 0.72). Higher precision therefore reflects repairs that human experts judge to be of higher quality, rather than merely shorter patches.
  • We conducted the same study on PreciseCoder. We randomly sampled 100 tasks (50 BigCodeBench, 50 LiveCodeBench) on which both PreciseCoder-9B and the unit-test-only GRPO baseline pass all tests. Without seeing the model name and reference repair, three independent annotators rated each patch on the same 1-5 scale and indicated which of the two patches they would accept as a pull request.
PreciseCoderGRPO (unit-test only)
Mean rating4.653.45
Accepted as pull request (majority vote)6811

PreciseCoder's advantage in acceptance votes is highly significant (p<10−10p < 10^{-10}) and inter-annotator agreement is high (Krippendorff's α=0.91\alpha = 0.91). Also, the mean rating correlates with edit-level precision at Spearman ρ=0.88\rho = 0.88 on these 200 patches.

Next, we conduct a per-line-length analysis and code-quality reward training experiments to show that in the current setting, training on the metric does not lead to code-golfing, and extra safeguards are unnecessary.

(a) In the per-line-length analysis, we compare PreciseCoder’s edits with those of the base model, unit-test-only GRPO and the reference fix. Code-golfing would show up as fewer lines carrying more characters. In the table below, each cell reports mean / median / 95th percentile.

ModelEdited linesChar of added & modified linesChar change of modified lines
Reference fix2.2 / 2 / 429.7 / 25 / 68−2.8 / −1 / +10
Qwen3.5-4B, Base15.3 / 4 / 10141.7 / 32 / 94+10.7 / 0 / +68
Qwen3.5-4B, GRPO13.8 / 7 / 4527.0 / 23 / 65−2.0 / 0 / +23
Qwen3.5-4B, PreciseCoder3.1 / 3 / 730.4 / 25 / 68−2.1 / −1 / +17
Qwen3.5-9B, Base13.7 / 3 / 9945.6 / 38 / 103+12.1 / 0 / +75
Qwen3.5-9B, GRPO11.3 / 4 / 4327.6 / 23 / 66−1.2 / 0 / +19
Qwen3.5-9B, PreciseCoder3.0 / 3 / 631.1 / 25 / 68−2.1 / −1 / +16
Qwen3.6-27B, Base7.4 / 4 / 2828.7 / 24 / 66−0.8 / 0 / +18
Qwen3.6-27B, GRPO7.9 / 5 / 3027.1 / 23 / 65−1.7 / 0 / +20
Qwen3.6-27B, PreciseCoder2.6 / 2 / 530.5 / 26 / 68−2.0 / −1 / +14

We can see that

  • PreciseCoder’s fixes closely match the reference fixes in length. The mean is 30-31 vs. 30 characters, the median 25-26 vs. 25, and the 95th percentile 68 vs. 68.
  • Modified lines do not get longer on average either, their length changes by -2.0 to -2.1 characters, against -2.8 for the reference fix.

Overall, PreciseCoder's edits look like the reference fix, not like compressed code.

(b) In the code-quality reward experiments, we train Qwen3.5-4B with the PreciseCoder reward plus one of three extra safeguards:

  • (1) a Pylint code-quality reward [2, 3];
  • (2) an anti-compression penalty on compression that the edit introduces: syntactic patterns that pack several lines of logic into one (e.g., a = 1; b = 2, a lambda assigned to a name, nested comprehensions) and lines over 32 tokens (99th percentile of gt solution);
  • (3) statement scoring, which replaces edited line counts with statement counts when computing F1.
RewardPassPrecisionEdited linesPylint (0-10)
Reference fix––2.26.18
GRPO55.130.013.86.20
PreciseCoder55.364.93.06.35
PreciseCoder + 0.3·Pylint53.951.15.76.83
PreciseCoder + anti-compression penalty53.558.84.26.15
PreciseCoder + canonicalized scoring52.661.93.16.25

We can see that

  • Under canonicalized scoring the model still edits 3.1 lines (vs. 3.0), thus PreciseCoder's small edits do not come from merging statements.
  • PreciseCoder is already close to the reference fix on Pylint without any code-quality reward.
  • No safeguard improves PreciseCoder: none raises the pass rate (52.6-53.9 vs. 55.3), while the Pylint reward costs 14 precision points and the anti-compression penalty costs 6.

Overall, PreciseCoder's small edits do not come from code-golfing, and extra safeguards only cost precision. We added these results to the revised paper (Appendix B.11).

Our hypothesis is that, since the pretraining data and our fine-tuning data contain little golfed code that is also correct, this behavior is hard to trigger in the current setup. However, if golfed code were maliciously injected into the training data, such a scenario could occur in the future. We added this to the limitations section of the revised paper.

[1] Chai et al. Precise Debugging Benchmark: Is Your Model Debugging or Regenerating? NeurIPS 2026.

[2] Robeyns and Aitchison. Improving LLM-Generated Code Quality with GRPO. arXiv 2025.

[3] Ouyang et al. Beyond Execution: Static-Analysis Rewards and Hint-Conditioned Diffusion RL for Code Generation. arXiv 2026.

AuthorsOpenReview ↗

Did our response address your concerns?

Dear Reviewer dWAQ,

Thank you again for your thoughtful review. As the discussion phase ends in about a day, we wanted to check whether our response and the revised paper address your concerns on fault identification and code quality. We would be glad to clarify anything further in the remaining time.

Hello authors,

I have raised my score.

Reviewer_D38GOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper points out that LLM's often propose edits that have extra lines beyond the bug fix and can also hallucinate bugs where none exist. They propose a reward metric based on F1 score of recall and precision (based number of lines of fix compared with the gold edit) of the candidate bug fix, and fine-tune Qwen 3.5 4B, 9B, and 27B models with this reward. The resulting models produce much smaller edits without a reduced pass rate. The papers shows that seeding stronger models with the thinking trace from the fine-tuned 9B Qwen 3.5 model improves the localization of the stronger models as well, and that the fine-tuned Qwen 3.5 models hallucinate bugs when asked to fix already-correct programs less often.

Critical strengths

Strength: great method and evaluation. The proposed reward function is well motivated empirically and theoretically, and is shown to work well. The hallucination and weak-to-strong experiments are creative and insightful, and the main evaluation is thorough. The reward function is simple and principled.

Strength: this paper is tremendously well-written and put together. The paper is well-organized, the plots are easy to understand and informative, the technical content is detailed and clear, and all of the text is interesting to read and useful. This is one of the best written papers I've read in some time.

Critical weaknesses

My only concern with the paper is the novelty of the proposed fine-tuning method. This issue was raised to me by Claude Opus 5.5, who pointed out that the precision and recall metrics used to fine-tune Qwen models in this work are already introduced in the PDB benchmark paper. Reading the PDB paper, the abstract indeed says, "We define two novel metrics, edit-level precision and bug-level recall, which measure how many necessary edits are made and how many bugs are resolved". This paper adds a couple tricks to the reward function, such as gating recall to one if the edit passes, normalizing each score separately in GDPO, and then does the actual fine-tuning and does a lot of evaluation.

Questions for authors

The hallucination comparison uses the default "debug the given code" prompt. I think the comparison in Appendix 4, which uses the "edit or abstain" prompt (B.4, Table 8) is a much more insightful comparison. I would suggest replacing the GT debug column of table 3 with the numbers from Table 8.

2 comments

We thank the reviewer for finding the reward simple, principled, and well motivated; the hallucination and weak-to-strong experiments creative and insightful; and appreciating our paper as one of the best-written papers in some time.

We summarize our answers here and give the details for each question below.

W1: Precision and recall introduced in PDB.

Ans: Yes, precision and recall from PDB is not a contribution for this paper. Our contributions are to

  • (1) diagnose training trajectory and explain heuristically (Sec 3.2) and empirically (Sec 4) why the designed F1 measure as a reward is preferred for precise LLM debugging;
  • (2) show reinforcement fine-tuning with our reward achieves state-of-the-art, on both in-domain and OOD complex coding benchmarks, compared to other fine-tuning approaches (Sec 4, 5.1);
  • (3) show how we can transfer fidelity from fine-tuned weaker models to non-fine-tuned stronger models, and analyze what information is transferred through language (Sec 5.2, 5.3).

Q1: “Abstain and fix” prompt results for main table

Ans: Thank you for the constructive suggestion! We previously treated the original Table 8 as an ablation study and did not run every model from the main table. We agree that the abstain-or-fix results are insightful and should be included in the main table. We have now evaluated most of the remaining models with the abstain-or-fix prompt and report the results below.

ModelDefault prompt: no-op ↑ / damage ↓Abstain-or-fix: no-op ↑ / damage ↓
Qwen3.5-4B, Base30.6 / 19.368.3 / 11.5
Qwen3.5-4B, SFT34.9 / 19.464.1 / 10.9
Qwen3.5-4B, DPO30.6 / 17.763.3 / 12.3
Qwen3.5-4B, GRPO19.3 / 17.851.4 / 10.9
Qwen3.5-4B, EA-GRPO43.3 / 18.857.0 / 14.9
Qwen3.5-4B, PreciseCoder48.1/ 13.571.9 / 7.6
Qwen3.5-9B, Base36.6 / 18.771.0 / 9.6
Qwen3.5-9B, SFT41.0 / 15.165.3 / 11.6
Qwen3.5-9B, DPO41.3 / 13.466.6 / 9.1
Qwen3.5-9B, GRPO30.7 / 11.451.5 / 10.1
Qwen3.5-9B, EA-GRPO47.0 / 13.567.7 / 8.9
Qwen3.5-9B, PreciseCoder51.6 / 11.873.7 / 7.4
Qwen3.6-27B, Base31.1 / 15.575.5 / 7.1
Qwen3.6-27B, GRPO26.9 / 12.667.9 / 7.3
Qwen3.6-27B, PreciseCoder45.9 / 10.980.4 / 5.8
DeepSeek-V4.1-Flash39.7 / 21.574.0 / 10.0
Qwen3.8-Flash-Next22.1 / 17.948.7 / 14.6

The main conclusions of Appendix B.4 do not change: the abstain-or-fix prompt raises the no-op rate and lowers the damage rate of every model, and PreciseCoder remains the best on both at every model size. We have updated Table 7 (originally Table 8) with the version above, and we will merge these results into the main table in the camera-ready version once all models finish evaluation.

Thank you for your response.

Reviewer_49TwOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

PreciseCoder studies the distinction between passing tests and making only necessary repairs. It augments unit-test RL with reference-aligned edit precision and recall, using counterfactual execution checks and a gated F1 term combined through per-component reward normalization. The training tasks contain single-line bugs, while evaluation includes correct inputs, multi-line bugs, repository repair, tool-based editing, and trace transfer. On the 4B model, mean edited lines decrease from 13.8 to 3.0 with pass rates of 55.1% versus 55.3%; the 9B comparison is 63.9% versus 62.5% (Table 3). The supplement discloses scoring and reproduction caveats that require clarification.

Critical strengths
  1. The paper makes a practically relevant distinction between passing tests and editing only what is necessary. The training audit and reward ablations connect this distinction to measurable behavior. On the 4B model, PreciseCoder reduces mean edited lines from 13.8 to 3.0 while the reported pass rate remains close to the unit-test-only baseline, 55.3% versus 55.1% (Table 3). The separate no-op and damage measurements on correct inputs make the behavioral evaluation more informative than pass rate alone.

  2. The evaluation tests the editing behavior under several meaningful changes of setting. Clean-input training, abstain-or-fix prompting, all-donor trace transfer, and repository-level evaluation provide useful controls and transfer evidence (Tables 6, 8, and 9; Figure 4). The results support investigating precision-oriented repair rewards beyond the single-line training tasks, while leaving the implementation concerns below to be resolved.

Critical weaknesses
  1. The central necessary-edit scoring path has an unresolved implementation defect. The supplement assigns the whole match dictionary to gt_match_ids in the reduced-edit branch (PDB/src/evaluator.py, lines 509–519), so matched reference repairs can remain in programs used to test reduced predicted edits. This is a localized branch reached after a successful semantic check and with positive tolerance; it does not invalidate every score or determine the direction of any bias. However, the package reports no corrected historical rescoring, so the effect on the reported precision/F1 results and on training rewards remains unknown. The main metric requires a sensitivity check before I can rely on the corresponding conclusions.

  2. Two implementation details prevent the current evidence from cleanly supporting the stated attribution and agentic-training claims. The training scorer zeroes symbolic reward for edits above a reference-dependent size threshold, whereas offline scoring disables this guard and uses a different tolerance. The contribution of this additional size-based shaping is not isolated. Separately, the Figure 5 discussion claims multi-turn agentic RL, while the supplement describes single-turn edit-format training followed by multi-turn evaluation. These differences need to be reconciled with the reported experiments.

Questions for authors
  1. Did the historical training rewards and Table 3/4 evaluations use the gt_match_ids reduction branch in the supplied scorer? Please report how often it is reached and how corrected rescoring changes precision/F1 and the relevant comparisons. If training used the same defect, what evidence establishes the robustness of the trained result?

  2. How much of the smaller-edit behavior is attributable to the predicted-diff guard rather than the verified F1 term? Please clarify the guard/tolerance settings for each baseline and provide a matched guard/normalization control or sensitivity analysis.

  3. Was the Figure 5 Agent checkpoint trained on multi-turn tool trajectories, or on single-turn edit-format outputs before five-turn evaluation? Please identify the corresponding training entry point, rollout format, turn budget, and configuration, and reconcile them with the supplement README/RELEASE_NOTES.

3 comments

We thank the reviewer for recognizing that the distinction between passing tests and precise editing is practically important, and for pointing out that training audit, reward ablations, and no-op/damage measurements make the evaluation more informative than pass rate alone,

W1 & Q1: The central necessary-edit scoring path has an unresolved implementation defect

Ans: Yes, in the reduced-edit branch, gt_match_ids receives the match record instead of the matched reference-block ids, so the matched reference fix stays in the program that tests a reduced edit. This line is inherited unchanged from the public evaluator of [1], which we use as released; we have emailed an anonymous note with the fix to its maintainers.

Below we quantify its effect with a corrected re-scoring.

Was the branch used? Yes. Our training reward and all Table 2/3 evaluations use the public evaluator as released.

On the inference side, the branch affects only precision on PDB Test, and only lowers it. Because the reference fix stays in the program, a reduced edit passes too easily, so the scorer credits fewer of a large block's extra lines than specified.

We conducted a corrected re-scoring for 11 models with per-task outputs. The original construction reproduces the stored credit for all 2,726 affected blocks. The corrected one gives more credit to 663 blocks, the same to 2,029 and less to 34. Table 11 in Appendix B.8 lists each model's share of tasks reaching the branch and its precision and F1 before and after the correction.

All rankings are unchanged. The correction raises precision for every model by 0.4–2.2 points, most for unit-test-only GRPO (+1.8 to +2.2) and least for PreciseCoder (+0.4 to +0.9), which rarely writes oversized blocks:

  • PreciseCoder's precision gap over unit-test-only GRPO goes from 34.6 / 26.2 / 36.1 to 33.3 / 24.8 / 34.7 points (4B / 9B / 27B).
  • Its gap over the base model goes from 28.8 / 24.0 / 29.5 to 28.7 / 23.4 / 28.5 points.

On the training side, the effect on the reward is negligible. Under the training-time setting, the corrected edit term differs by at most 0.003, on a reward that ranges from 0 to 1.

Since the corrected re-scoring leaves every conclusion unchanged, we keep the results in the paper as computed with the public evaluator in this version, which keeps them comparable with the benchmark's published numbers. The correction also does not change our contributions on

  • (1) diagnosing the training trajectory and explaining heuristically (Sec 3.2) and empirically (Sec 4) why the designed F1 measure as a reward is preferred for precise LLM debugging;
  • (2) showing reinforcement fine-tuning with our reward achieves state-of-the-art, on both in-domain and OOD complex coding benchmarks, compared to other fine-tuning approaches (Sec 4, 5.1);
  • (3) showing how we can transfer fidelity from fine-tuned weaker models to non-fine-tuned stronger models, and analyze what information is transferred through language (Sec 5.2, 5.3).

Once the fix is acknowledged by the benchmark's maintainers, we will update the paper with the corrected scores in the camera-ready version.

W2 & Q2: The guard/tolerance settings for each baseline and provide a matched guard/normalization control

Ans: During PreciseCoder training, we use ε=1\varepsilon = 1 plus a size guard to increase training efficiency. Let ∣Δ⋆∣|\Delta^\star| be the number of edited lines in the reference fix. When a rollout edits more than 2∣Δ∗∣+32|\Delta^*| + 3 lines (i.e., more than double the reference fix plus the tolerance), we skip the semantic block scoring and set its precision, recall, and F1 to zero; the unit-test term is not affected. All evaluations use the full scorer (ε=2\varepsilon = 2 with the size guard off).

Scoring time is dominated by the semantic block check, which re-runs the unit tests on verification programs. An output with XX edited lines and KK faults needs up to 1+K+εKX1 + K + \varepsilon KX test runs, so the cost grows with how much the policy edits. The guard caps this at 1+K+εK(2∣Δ∗∣+3)1 + K + \varepsilon K(2|\Delta^*|+3), independent of XX, and ε=1\varepsilon = 1 roughly halves the sub-edit search. We scored the same PDB Test outputs with both scorers: the training-time scorer cuts the scoring time from 24.4 to 13.7 min, 1.8×1.8\times faster with 4.2×4.2\times fewer verification programs.

For models that edit precisely, even with the size guard, the training-time score closely matches the evaluation score. We re-scored the outputs of 11 models under the training-time setting (Table 15, Appendix C.3) and report the differences in precision / unit-gated F1. The differences are only 0-0.6 / 0-0.8 points for PreciseCoder.

Hence, the size guard provides a fast cold start, and the training-time scorer converges to the evaluation scorer later. In the per-rollout reward logs of PreciseCoder-4B, the guard fires on 10.4% of rollouts in the first 75 steps, while the policy still over-edits, on 2.6% in steps 75–328, and on 4.4% overall. It touches only 14% of the groups, so the edit signal in training still comes mainly from the gated F1 term.

W2 & Q3: Was the Figure 5 Agent checkpoint trained on multi-turn or on single-turn?

Ans: Figure 5’s Agent checkpoint was trained with single-turn RL in the agent's str_replace edit format and then evaluated in the multi-turn agent. Following the reviewer’s suggestion, we now train with multi-turn agentic RL inside the same five-turn agent used for evaluation (detailed setup in Appendix C.6). We updated Figure 5 with this experiment and moved the old single-turn, tool-call-format RL experiment to Figure 15 in Appendix B.12.

Multi-turn train and multi-turn evaluation

ModelTrainingUnitPrecisionF1Edited lines
Qwen3.5-9B–42.348.548.85.0
GRPOsingle-turn RL46.745.848.35.3
PreciseCoder-9Bsingle-turn RL57.968.768.22.6
GRPO-Agentmulti-turn agentic RL64.532.939.79.7
PreciseCoder-9B-Agentmulti-turn agentic RL68.572.073.12.9

PreciseCoder-9B-Agent reaches 72.0% precision at a 68.5% pass rate with 2.9 edited lines, the best of all 9B models on both pass rate and precision. Agentic RL with the unit-test reward alone raises the pass rate to 64.5% but drops precision to 32.9% (9.7 edited lines), the same pattern as in single-turn training.

[1] Chai et al. Precise Debugging Benchmark: Is Your Model Debugging or Regenerating? NeurIPS 2026.

AuthorsOpenReview ↗

Did our response address your concerns?

Dear Reviewer 49Tw,

Thank you again for your thoughtful review. As the discussion phase ends in about a day, we wanted to check whether our response and the revised paper address your concerns on implementations and their effects on performance. We would be glad to clarify anything further in the remaining time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank all the reviewers for recognizing our contributions on

  • diagnosing the training trajectory and explaining heuristically (Sec 3.2) and empirically (Sec 4) why the designed F1 measure as a reward is preferred for precise LLM debugging;
  • showing reinforcement fine-tuning with our reward achieves state-of-the-art, on both in-domain and OOD complex coding benchmarks, compared to other fine-tuning approaches (Sec 4, 5.1);
  • showing how we can transfer fidelity from fine-tuned weaker models to non-fine-tuned stronger models, and analyze what information is transferred through language (Sec 5.2, 5.3).

Your comments made the paper better. In the revision, we added a human study, experiments on code quality, multi-turn agentic RL training, and a re-scoring. None of these changed our conclusions.

Thank you to those who have already raised their score; if you have not, we would be grateful if you would consider it. If anything is still unclear, we are happy to answer in the time that remains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.