acceptodds
Under review as a conference paper at ICLR 2027

Teaching Agents to Code Reliably

10 views

Abstract

Autonomous coding agents solve repository issues by reading code, running com- mands, editing files, and submitting patches. Extra inference-time compute yields gains only when it produces a useful repair and supplies reliable evidence for choosing one. Three behaviors decide both, and we argue they are teachable rather than byproducts of scale, so a policy can carry them instead of a scaffold. Location diversity remains narrow, since attempts return to the same site and extra samples add no coverage. Edit diversity is left unexploited, since methodologies that differ resolve complementary issues no single run reaches. Verification misleads, since a test the agent writes for its own patch accepts many incorrect ones. Directing search by execution feedback and scoring each patch against its own reverted tree resolves 52.8% of SWE-bench Verified using 48.1% of the agent-steps an eight- sample baseline spends. Training moves these behaviors into the policy. On the 270 issues held out from SFT and RL training, weighted supervised fine-tuning raises pass@1 from 31.9% to 35.2% and pass@8 from 46.7% to 51.1%. A reinforcement objective then trains the verifier against gold-labeled repairs and incorrect variants, crediting the assertions that detect them. It raises pass@1 to 43.0% and pass@8 to 60.7%, lifts verifier precision from 26.8% to 41.7%, and more than halves false acceptance. Resolution improves on two of three out-of-distribution suites and verifier precision on all three, and the gains hold at 7B, 14B, and 30B against published coder baselines.

Primary area:
reinforcement learning
Keywords:
Coding Agents, Test-Time Scaling, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_XQbiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies how to improve inference-time scaling for coding agents by focusing on two key factors: candidate coverage and selection quality. The authors first conduct a diagnostic study on SWE-bench Verified and show that simply increasing the number of attempts or repeatedly performing self-verification does not necessarily lead to effective scaling, due to limited exploration and unreliable verification. They propose Differential Verification to guide search using execution feedback, achieving higher resolution with substantially fewer agent steps than naive sample scaling. Based on these observations, the paper further trains these behaviors into the policy through weighted supervised fine-tuning over agent trajectories and reinforcement learning with mutation-based verification rewards and assertion-level credit assignment. The resulting policy improves both repair accuracy and verifier discrimination, with consistent gains across multiple benchmarks and model scales.

Critical strengths
  1. The paper studies an important and timely problem for coding agents. Rather than treating inference-time scaling as simply increasing the number of samples, the paper provides useful insights into why additional inference compute may fail to translate into better performance, and highlights the importance of both exploration diversity and reliable verification.

  2. The empirical improvements are substantial. On the held-out SWE-bench Verified split, the proposed SFT and RL stages improve pass@1 from 31.9% to 43.0% and pass@8 from 46.7% to 60.7%. The improvements also transfer to other benchmarks, suggesting that the learned behaviors are not limited to the training issues.

  3. The paper provides extensive analyses and ablations. The experiments separately examine candidate coverage, selection quality, verifier discrimination, trajectory behavior, reward components, different sources of incorrect repairs, and different model scales. These analyses make it easier to understand where the improvements come from rather than relying only on end-to-end benchmark results.

Critical weaknesses
  1. It remains unclear to what extent the learned policy actually reduces the need for test-time scaffolding. A central motivation of the paper is that the useful search and verification behaviors identified in the diagnostic study can be moved into the policy, such that “a policy can carry them instead of a scaffold.” However, the paper does not provide a direct, matched comparison between the trained policy and the test-time Differential Verification procedure in Section 2.1. In particular, the strongest trained-policy results still involve multi-sample candidate generation and verifier-based selection, while the Differential Verification results use a different search procedure, evaluation population, and inference budget. The strong pass@1 improvement does provide evidence that some of these behaviors are internalized, but it does not quantify how much of the test-time search and verification machinery can actually be removed after training. This makes the “policy instead of scaffold” framing somewhat stronger than what is directly established by the current experiments.

  2. Regarding presentation, Figure 1 appears to be AI-generated, as some of the text on the right side, such as “self-verification,” is unclear or distorted.

Questions for authors

To what extent can the trained policy replace the test-time search and verification scaffold introduced in Section 2.1? Could the authors evaluate the trained policy and the base policy under the same test-time scaffold and matched inference budget?

Reviewer_583pOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that three behaviors limit how much coding agents gain from extra inference compute: where they edit, how they vary edits, and how they verify a candidate patch. It claims these behaviors can be trained into the policy. The work has three parts:

  1. Test-time diagnostic on all 500 SWE-bench Verified issues.
    • Repeated self-verification rarely changes the patch.
    • Up-front (proactive) localization loses to blind retries.
    • Different problem-solving preambles solve complementary issues; their oracle union covers 61%.
    • Differential-verification search (DVS) runs the agent's own reproduction on the patched and the reverted tree. It reaches 52.8% vs. 46.8% for an 8-sample baseline, using 48.1% of the agent-steps.
  2. Weighted SFT. The model is fine-tuned on 5,061 trajectories from 230 Verified issues. Examples are weighted by quality tier, exploration and efficiency (Qwen3-Coder-30B-A3B, mini-SWE-agent).
  3. GRPO with a mutation-based verifier reward.
    • The policy writes both a patch and a test.
    • Tests are credited for failing on oracle-rejected mutants and sibling repairs. The mutants come from deterministic operators and from a frozen Claude Opus-4.8.
    • Credit is split across assertions using a Shapley-style rule and is gated.

Results on the 270 held-out Verified issues:

  • pass@1: 31.9% → 35.2% (SFT) → 43.0% (RL).
  • Oracle pass@8: 46.7% → 60.7%.
  • Verifier precision: 26.8% → 41.7%, with the false-positive rate more than halved.
  • Transfer: SWE-bench Pro 15.7% → 20.2% and SWE-PolyBench 24.2% → 31.8%. DeepSWE stays at 0% full resolution.
  • Similar trends at 7B and 14B.
Critical strengths
  1. It separates coverage from selection and evaluates the verifier carefully.

    • Oracle@8 is reported separately from Selected@8, with their ratio as selection efficiency.
    • A generator–verifier swap experiment attributes gains to each role.
    • The verifier is scored on one frozen bank shared across policies. The bank contains correct alternatives and natural incorrect repairs from independent generators.
    • Rejection rates are broken down by error source, from familiar mutation operators to natural wrong repairs.

    This shows the verifier gain is not just a stricter test, and the framework is reusable.

  2. The RL controls are thorough and the gains are consistent. All reward variants start from the same SFT checkpoint with matched budgets: correctness-only, scalar verification reward, uniform credit, shuffled credit, and a prior mutation-RL method. Each component adds something. The gain is also similar on the 90 test issues from repositories excluded from training (+7.8 points), on SWE-bench Pro and PolyBench, and at 7B and 14B. This makes it likely that the mutation-based verifier reward has a real effect.

Critical weaknesses
  1. Evaluation hygiene is weak, and the external evidence is modest.
    • Training (230 issues) and testing (270 issues) both use SWE-bench Verified.
    • Method development used all 500 issues, including the 270 test issues: the diagnostic, the DVS design, the weighting floors and the reward weights.
    • 180 of the 270 test issues come from repositories seen in training.
    • The transfer gains are modest, DeepSWE stays at 0, and SWE-bench Lite after RL is not reported for the 30B model.
    • The standard protocol would be to train on SWE-Gym, SWE-smith or R2E-Gym and evaluate on all of Verified.
    • The abstract claims gains "against published coder baselines", but Table 11 compares results on the 270-issue subset against published results on the full 500, with different scaffolds. The paper itself says this is not a matched comparison.
  2. The statistics and several protocols are under-specified.
    • No variance. The paper says "the configuration specifies three seeds" but reports no standard deviations.
    • Conclusions from tiny differences. Several ablation conclusions rest on 1–3 issues out of 270. For example, removing the exploration factor moves pass@1 from 35.2% to 34.8%, which is one issue; the signed-credit variant gains 0.3 points.
    • Numbers that cannot be per-issue counts. In Table 3, Selected@8 (33.5%, 50.5%), majority vote (35.0%, 44.0%) and random pick (26.0%, 36.0%) cannot be k/270 for any whole number k. Meanwhile, every number in Table 5a matches 230/180/90 denominators exactly.
    • A suspiciously narrow interval. The swap experiment reports an interaction of −0.1 with a 95% CI of [−2.6, 2.4]. That is a difference of differences over 270 binary outcomes, and the interval looks implausibly narrow.
    • An unclear swap protocol. The diagonal cells of the swap experiment equal greedy pass@1. It is not explained how swapping the verifier changes a single-attempt result.
    • The strongest number comes from a scaffold, not the trained policy. DVS reaches 52.8% without any training. It is never run on the 270 held-out issues or combined with the trained policy. This undercuts the claim that "a policy can carry them instead of a scaffold".
Questions for authors
  1. Can you train on an external corpus and evaluate on all 500 Verified issues? Alternatively, can you confirm which design choices (DVS scoring, the weighting floors, λ,β\lambda,\beta, the reward weights) were fixed without looking at the 270 test issues?
  2. Please report mean ± std over the three seeds. How were the Table 3 values that are not of the form k/270 aggregated? How was the interaction CI in Table 15 computed?
  3. In the generator–verifier swap, what exactly does "verifier" mean at a one-attempt budget, and how does it change end-to-end resolution?
  4. How does the 8-sample "Sample Scaling" baseline pick its final patch? Is it oracle selection? Please also run DVS on the 270 held-out issues, with both the base and the trained policy. Does training add to DVS, and does the trained policy beat DVS alone?
  5. What are the size and correct/incorrect ratio of the frozen verifier bank? The precision values in Tables 5b and 7 cannot be reproduced from Table 4's TPR and FPR under a single class ratio. Are they computed on different populations?
  6. Table 1 reports 37.8% for a single base attempt on the 500 issues. Table 5a implies 32.8% greedy pass@1 for the base model over the same 500 (164/500). What explains the 5-point gap?
Reviewer_W4ZZOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to improve the exploration and verification capabilities of repository-level coding agents. It first analyzes several test-time search strategies and introduces differential verification for evaluating candidate repairs. The proposed training pipeline then combines weighted supervised fine-tuning on agent trajectories with reinforcement learning that rewards patch correctness and the ability of generated tests to detect incorrect program variants. Experiments on SWE-bench Verified and several transfer benchmarks report improvements in repair accuracy, candidate coverage, and verifier discrimination.

Critical strengths
  • The paper addresses an important problem in coding agents. Distinguishing candidate coverage from the ability to select a correct repair is useful, especially when additional test-time sampling does not necessarily lead to better final decisions.

  • On the 270 SWE-bench Verified issues held out from training, pass@1 increases from 31.9% to 43.0%, while oracle pass@8 increases from 46.7% to 60.7%. The authors also report improvements on SWE-bench Pro and SWE-PolyBench, suggesting that some of the learned behaviors transfer beyond the main evaluation set.

  • The paper includes several useful controls. In particular, the comparison between correctness-only RL, scalar verification rewards, and assertion-level credit provides evidence that verification-oriented training contributes to performance. The frozen candidate bank and generator-verifier swap experiments also help separate repair generation from verification quality.

Critical weaknesses
  • The proposed method combines several established techniques, including weighted trajectory supervision, execution-guided repair, and mutation-based verification. While assertion-level credit assignment is an interesting addition, its distinction from existing approaches such as CURE and MIST-RL remains limited. The “Prior mutation-RL control” in Table 5(b) is also insufficiently described to establish the incremental contribution.

  • All 500 SWE-bench Verified issues are used in the initial diagnostic study before 230 are selected for training and 270 for evaluation. Although the latter are excluded from training, they have already informed method development, weakening the independence of the main evaluation.

  • Equation (5) assigns assertion credit to every token within an assertion, making its total contribution dependent on assertion length. Appendix A.4 acknowledges this issue without resolving it. Furthermore, Table 18 shows that signed credit outperforms the proposed unsigned variant on several metrics, leaving the choice of the main reward formulation insufficiently justified.

  • The efficiency comparison in Table 1 relies on agent-steps rather than actual computational cost. It excludes differences in token consumption and test-execution overhead, while the RL pipeline additionally requires generating and evaluating incorrect program variants. Without end-to-end compute measurements, the practical efficiency gains remain unclear.

Questions for authors
  • Could the authors clarify the methodological novelty relative to existing mutation-based verification and RL methods, particularly the implementation of the “Prior mutation-RL control” in Table 5(b)? 
  • How was the analysis of all 500 SWE-bench Verified issues used during method development? Could the authors also clarify the dataset overlap? 
  • How does length-normalized assertion credit compare with the proposed token-level reward? Why was the unsigned credit formulation retained when the signed variant performs better in Table 18? 
  • Could the authors provide end-to-end computational comparisons that account for LLM token usage, test execution, and mutation generation?
Reviewer_nwxSOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper separates coding-agent reliability into candidate coverage and correct-candidate selection. It studies differential verification search, weighted SFT, and mutation-based RL with per-assertion credit. On 270 Verified issues held out from SFT/RL, pass@1 improves from 31.9% to 43.0%, oracle@8 from 46.7% to 60.7%, and Selected@8 from 33.5% to 50.5% (Table 3). The paper reports frozen-candidate verification, matched reward/credit controls, role swaps, and transfer results. The issue split is explicitly within a benchmark used for method development, rather than a blind test of the full development process.

Critical strengths
  1. The paper distinguishes candidate coverage from selecting a correct candidate, and reports evidence on both. On the issue-held-out Verified split, oracle@8 improves from 46.7% to 60.7%, while Selected@8 improves from 33.5% to 50.5% and selection efficiency from 0.72 to 0.83 (Table 3). The frozen-candidate comparison also improves both incorrect-patch rejection and correct-patch acceptance, helping distinguish better discrimination from uniformly stricter rejection.

  2. The controls and transfer evaluations support a substantive training effect. The matched correctness-only, scalar-verification, and assertion-credit objectives, including uniform/shuffled credit controls, address several plausible alternatives (Table 5b). Improvements on SWE-bench Pro and SWE-PolyBench and pipeline gains at 7B/14B provide useful evidence beyond the 230 training issues. The paper clearly distinguishes these results from DeepSWE, where no full task is resolved.

Critical weaknesses
  1. The implementation description is insufficient to verify the paper's central per-assertion credit mechanism. Computing Eq. 4 requires knowing every assertion that detects each mutant, although ordinary execution stops at the first failure. Appendix A.4 notes the need for isolated executable nodes but does not explain how generated assertions with shared fixtures or state are handled. The worked example reports only per-variant verdicts (Table 12), and the main configuration's length normalization and assertion-bonus coefficient are not specified. This leaves an important gap between the stated attribution rule and the evidence needed to reproduce it.

  2. Some protocol and uncertainty details limit interpretation of the attribution results. The frozen verifier bank's size/composition and the generator–verifier swap's candidate and retry interface are not specified sufficiently to reconstruct their different acceptance/selection metrics. Most central gains are reported without per-seed variation despite a configuration specifying three seeds. These are addressable reporting gaps, but they matter for assessing the credit-ablation gains and the verifier's independent contribution. The all 500 diagnostic exposure also means the 270 issue-held-out evaluation is not a blind test of method development, as the paper itself acknowledges.

Questions for authors
  1. How was the full per-assertion kill matrix collected during RL, including assertion isolation and shared fixtures/state? Please provide a training example of the matrix and token mapping, specify whether the main runs use c_i or c_i/L_i and the bonus/class-weight settings, and explain whether Table 12's per-variant-only logging belongs to a different experimental stage.

  2. Please specify the frozen bank's per-issue candidate counts, label/source composition, and the exact denominators/procedures for acceptance, selection accuracy, and verifier precision. For Table 15, what candidate set and generator behavior are held fixed when only the verifier is swapped, and how do retries/revisions and the attempt budget relate to the reported greedy pass@1 values?

  3. Are the main and credit-ablation values means across the three configured seeds or results from one checkpoint/seed? Please provide per-seed results and paired uncertainty for the scalar-to-token-credit and generator/verifier-swap gains, together with the exact implementation of the “prior mutation-RL” control.

Comments on the paper

3 comments
Area_Chair_1nZEOpenReview ↗

Discussion

Dear Reviewers,

We are reaching the end of the discussion period! If the authors have responded to your review, please read their response, leave a comment, and update your rating if your assessment has changed.

Thank you,

AC.

Thank you for the detailed response and additional results. I appreciate the new experiments and clarifications, but I still have concerns about the methodological novelty and the independence of the main evaluation. I also find the presentation somewhat lacking, particularly Figure 1, where several labels and details are difficult to read. Overall, my concerns have not changed enough to warrant a score adjustment, so I will keep my original ratings.

AuthorsOpenReview ↗

Official Comment by Authors

Dear AC, SAC, and reviewers,

The discussion period closes soon, and this note summarizes it for the final assessment. We ran every clarification and experiment that the reviews request. The new results cover six items.

  1. The kill-matrix procedure, the credit settings, and a full training example.
  2. Three seeds and paired bootstrap intervals.
  3. DVS on the 270 held-out issues with the base and the trained policy at a matched budget.
  4. A length-normalized credit run and the signed rule on two transfer suites.
  5. End-to-end compute in tokens, A100 GPU-hours, and CPU-hours.
  6. The bank composition, the data overlap, and the data behind each design choice.

Reviewer nwxS (3). The review asks how we build the kill matrix, how the bank and the swap work, and for seed variance. We give the kill-matrix procedure with one full example, the bank and swap protocols, and per-seed results. Three seeds give 43.0 ± 0.7 pass@1.

Reviewer 583p (3). The review asks about evaluation hygiene, statistics, and DVS on the held-out issues. We give the data behind each design choice and SWE-bench Lite results without the 46 shared issues. DVS on the 270 issues rises from 45.2% with the base policy to 57.4% with the trained policy. One greedy attempt of the trained policy recovers 83% of the base DVS gain at 22% of the steps.

Reviewer XQbi (3). The review asks for a matched comparison of the trained policy and DVS. The trained policy without DVS resolves 47.4% at 4.32M input tokens per issue, against 45.2% for base DVS at 4.30M. The reviewer replies that the response addresses some concerns and keeps a positive score.

Reviewer W4ZZ (2). The review asks about novelty against CURE and MIST-RL, the 500-issue diagnostic, assertion length, signed credit, and compute. We check each prior method against its arXiv version and give the control, overlap, credit, and compute results. The length-normalized run gives 42.6 against 43.0 pass@1, inside the seed spread. The follow-up keeps the score, and our reply gives an ablation for each of four novel parts.

Overall, we thank all reviewers for their valuable feedback. The new results and clarifications will be added in the revised manuscript

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.