- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the results of training with with outcome rewards based on unit tests in competition-like and repository-level coding scenarios. They find that purely training on the outcome rewards from unit tests teaches several bad behaviors to the policy. They propose shaping the reward by rewarding the policy for making more localized edits, and show that policies trained with the shaped reward perform better while also making fewer edits (with the implication that the edits made by other policies were unnecessary).
Critical strengths
One major strength is that the authors show their method on Qwen3.5 rather than Qwen3. It is very easy to improve Qwen3 compared to Qwen3.5 and so showing results on Qwen3 is relatively strong evidence the proposed method is relevant going forward.
The experimental section in general feels well done, with +/- numbers on the PreciseCoder rows and multiple model scales. As this is a bit of an empirical paper, I think this should give readers more confidence the method works.
Critical weaknesses
The main weakness, to me, is that a core component of the method seems to not be described clearly. This is the alignment of faults needed for precise reward. I'm not sure how it's done. It's also not clear to me that this fault identification is general, what the limits are, and so on. Under what conditions can it be done? Under what conditions cannot it be done? Etc. It requires a reference solution, which is fine, but if so, this should be clearly stated. Or maybe doesn't? If so, it needs to be explained.
I suppose another weakness is that it's not clear to me that reducing the edit size matters. Let's say I reduce the edit size, but this says nothing about the quality of the resulting code. I can reduce the edit size just by writing very obfuscated, code-golfed code. I do think the authors need to do a bit more work to establish that the resulting code edits are, in fact, qualitatively better in some sense than code generated by more standard training (I don't believe LoC is a good metric here).
Questions for authors
Please see the weaknesses. I have asked two questions there. Both of the answers will help me raise my score. I am generally leaning towards acceptance if these points are improved.
4 comments
| PreciseCoder | GRPO (unit-test only) | |
|---|---|---|
| Mean rating | 4.65 | 3.45 |
| Accepted as pull request (majority vote) | 68 | 11 |
PreciseCoder's advantage in acceptance votes is highly significant () and inter-annotator agreement is high (Krippendorff's ). Also, the mean rating correlates with edit-level precision at Spearman on these 200 patches.
Next, we conduct a per-line-length analysis and code-quality reward training experiments to show that in the current setting, training on the metric does not lead to code-golfing, and extra safeguards are unnecessary.
(a) In the per-line-length analysis, we compare PreciseCoder’s edits with those of the base model, unit-test-only GRPO and the reference fix. Code-golfing would show up as fewer lines carrying more characters. In the table below, each cell reports mean / median / 95th percentile.
| Model | Edited lines | Char of added & modified lines | Char change of modified lines |
|---|---|---|---|
| Reference fix | 2.2 / 2 / 4 | 29.7 / 25 / 68 | −2.8 / −1 / +10 |
| Qwen3.5-4B, Base | 15.3 / 4 / 101 | 41.7 / 32 / 94 | +10.7 / 0 / +68 |
| Qwen3.5-4B, GRPO | 13.8 / 7 / 45 | 27.0 / 23 / 65 | −2.0 / 0 / +23 |
| Qwen3.5-4B, PreciseCoder | 3.1 / 3 / 7 | 30.4 / 25 / 68 | −2.1 / −1 / +17 |
| Qwen3.5-9B, Base | 13.7 / 3 / 99 | 45.6 / 38 / 103 | +12.1 / 0 / +75 |
| Qwen3.5-9B, GRPO | 11.3 / 4 / 43 | 27.6 / 23 / 66 | −1.2 / 0 / +19 |
| Qwen3.5-9B, PreciseCoder | 3.0 / 3 / 6 | 31.1 / 25 / 68 | −2.1 / −1 / +16 |
| Qwen3.6-27B, Base | 7.4 / 4 / 28 | 28.7 / 24 / 66 | −0.8 / 0 / +18 |
| Qwen3.6-27B, GRPO | 7.9 / 5 / 30 | 27.1 / 23 / 65 | −1.7 / 0 / +20 |
| Qwen3.6-27B, PreciseCoder | 2.6 / 2 / 5 | 30.5 / 26 / 68 | −2.0 / −1 / +14 |
We can see that
- PreciseCoder’s fixes closely match the reference fixes in length. The mean is 30-31 vs. 30 characters, the median 25-26 vs. 25, and the 95th percentile 68 vs. 68.
- Modified lines do not get longer on average either, their length changes by -2.0 to -2.1 characters, against -2.8 for the reference fix.
Overall, PreciseCoder's edits look like the reference fix, not like compressed code.
(b) In the code-quality reward experiments, we train Qwen3.5-4B with the PreciseCoder reward plus one of three extra safeguards:
- (1) a Pylint code-quality reward [2, 3];
- (2) an anti-compression penalty on compression that the edit introduces: syntactic patterns that pack several lines of logic into one (e.g.,
a = 1; b = 2, a lambda assigned to a name, nested comprehensions) and lines over 32 tokens (99th percentile of gt solution); - (3) statement scoring, which replaces edited line counts with statement counts when computing F1.
| Reward | Pass | Precision | Edited lines | Pylint (0-10) |
|---|---|---|---|---|
| Reference fix | – | – | 2.2 | 6.18 |
| GRPO | 55.1 | 30.0 | 13.8 | 6.20 |
| PreciseCoder | 55.3 | 64.9 | 3.0 | 6.35 |
| PreciseCoder + 0.3·Pylint | 53.9 | 51.1 | 5.7 | 6.83 |
| PreciseCoder + anti-compression penalty | 53.5 | 58.8 | 4.2 | 6.15 |
| PreciseCoder + canonicalized scoring | 52.6 | 61.9 | 3.1 | 6.25 |
We can see that
- Under canonicalized scoring the model still edits 3.1 lines (vs. 3.0), thus PreciseCoder's small edits do not come from merging statements.
- PreciseCoder is already close to the reference fix on Pylint without any code-quality reward.
- No safeguard improves PreciseCoder: none raises the pass rate (52.6-53.9 vs. 55.3), while the Pylint reward costs 14 precision points and the anti-compression penalty costs 6.
Overall, PreciseCoder's small edits do not come from code-golfing, and extra safeguards only cost precision. We added these results to the revised paper (Appendix B.11).
Our hypothesis is that, since the pretraining data and our fine-tuning data contain little golfed code that is also correct, this behavior is hard to trigger in the current setup. However, if golfed code were maliciously injected into the training data, such a scenario could occur in the future. We added this to the limitations section of the revised paper.
[1] Chai et al. Precise Debugging Benchmark: Is Your Model Debugging or Regenerating? NeurIPS 2026.
[2] Robeyns and Aitchison. Improving LLM-Generated Code Quality with GRPO. arXiv 2025.
[3] Ouyang et al. Beyond Execution: Static-Analysis Rewards and Hint-Conditioned Diffusion RL for Code Generation. arXiv 2026.
Did our response address your concerns?
Dear Reviewer dWAQ,
Thank you again for your thoughtful review. As the discussion phase ends in about a day, we wanted to check whether our response and the revised paper address your concerns on fault identification and code quality. We would be glad to clarify anything further in the remaining time.
Hello authors,
I have raised my score.
We thank the reviewer for acknowledging that PreciseCoder works against strong baselines, and that our results show strong statistical evidence that the PreciseCoder works well. We address the reviewer's concerns as below.
Ans: Yes, fault alignment uses reference repairs and unit tests. The alignment itself follows the public PDB evaluator [1]; we summarize it briefly below.
The alignment compares the model's repair Δ^ with a reference repair Δ⋆. Both repairs are split into blocks, which are runs of consecutive changed lines.
There is a block matching algorithm that can match, e.g. a reference block to multiple predicted blocks. A heuristic-based elimination method is used to find the best matching: identical blocks first, then blocks in the same place (completely wrapping the reference blocks, or surrounded by the same unchanged lines), then blocks that make the same change at another line.
Once we find the matching, the unit-tests decide whether a match is a fix. The reference repair is applied with the matched reference blocks δi⋆ swapped for the predicted blocks δ^i, that is, Δi=(Δ⋆∖δi⋆)∪δ^i. Note that every other fault is fixed by the reference, so passing the tests means δ^i fixed its faults. The smallest subset of δ^i that still passes the test, i.e., essi in Section 3 Eqn 1, counts as needed.
Recall is the fraction of faults fixed. Precision is the number of needed lines (allowing ϵ extra per fault) divided by all edited lines, so unmatched edits count as unnecessary.
As for when it works, the matching is reliable when blocks are well separated (when faults are at least 5 lines apart as defined in [1]). It is unreliable when blocks are crowded together, because neighboring edits merge and too few unchanged lines remain between them to anchor the matching. In the [1]’s manual check of 240 patches, recall was accurate on over 97.5% of the data, and precision strongly agreed with expert ratings (ρ=0.72).
The fault alignment algorithm is not a contribution for this paper, instead, we (1) diagnose training trajectory and explain heuristically (Sec 3.2) and empirically (Sec 4) why the designed F1 measure as a reward is preferred for precise LLM debugging; (2) show reinforcement fine-tuning with our reward achieves state-of-the-art, on both in-domain and OOD complex coding benchmarks, compared to other fine-tuning approaches (Sec 4, 5.1); and (3) show how we can transfer fidelity from fine-tuned weaker models to non-fine-tuned stronger models, and analyze what information is transferred through language (Sec 5.2, 5.3). However, we agree that understanding the alignment algorithm is important for understanding the paper. We have clarified this in the revised Section 3 and added a detailed description of the algorithm to the appendix.
Ans: We thank the reviewer for raising this point, and we perform additional analysis and find that (1) the edit-level metrics agree with human experts on a high-quality repair; and (2) in the current setting, training on the metric does not lead to code-golfing, and extra safeguards are unnecessary. We provide the following analyses as evidence to support these two claims.
First, two human studies show that the edit-level metrics agree with human experts on a high-quality repair.