acceptodds
Under review as a conference paper at ICLR 2027

CodeLift: Your Test Suite Is Secretly a Dense SWE Curriculum

Abstract

The *pass-all-test* binary reward has become a standard recipe for training software engineering agents, but its sparsity eliminates difference of partially correct patches. The test *pass-rate* is a natural alternative. However, we find that it is not a reliable indicator of patch quality. A functionality being tested can be covered by either one challenging test or one hundred trivial tests, where the majority might be already satisfied pass-to-pass tests. We introduce **CodeLift**, a reinforcement learning method that dynamically calibrates pass-rate rewards, turning the test suite into an adaptive curriculum. At each iteration, CodeLift groups tests that kill the same patches and patches that pass the same tests, then gives more weight to test groups that fewer patch groups pass. This reduces the impact of majority simple tests and focuses more on what is still challenging. Within each rollout, CodeLift applies the same scoring rule to repository snapshots, assigning credit to useful actions even when later ones introduce regressions. Experiments on SWE benchmarks show that CodeLift consistently improves pass@1/2/4/8 over binary and pass-rate baselines. Analyses show that CodeLift provides dense, stable learning signals and unlocks the inherent curriculum in the test suite.

Primary area:
foundation or frontier models, including LLMs
Keywords:
SWE, Software Engineering, Agent, Reinforcement Learning with Verifiable Rewards, Agentic RL

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_wMifOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

CodeLift improves reward design for reinforcement learning of software engineering agents. It groups tests and patches by their execution outcomes, then dynamically weights test groups to reduce biases in raw pass-rate rewards. It also scores intermediate repository snapshots to provide credit within a rollout. Experiments on SWE benchmarks show improvements over binary and pass-rate-based rewards.

Critical strengths
  1. The paper studies a meaningful limitation of test-based rewards. Figure 2 provides empirical support for the test-suite imbalances motivating the method.
  2. The approach uses execution results to provide denser supervision, and the reported gains are supported by component ablations and training analyses. Rewarding useful intermediate work is particularly relevant to long-horizon SWE tasks.
Critical weaknesses
  1. The reliability of behavior-based grouping is unclear with only a small number of sampled rollouts. Tests covering different functionality may happen to produce identical outcomes and be merged. The paper does not establish how sensitive the resulting rewards are to the rollout count or sampling variation.
  2. The credit assignment in snapshot slicing needs stronger justification. Comparing an intermediate state's score with other rollouts' final scores measures its relative quality, but does not directly measure the contribution of the preceding actions. A regressive slice can still receive positive advantage, while a useful intermediate step can receive negative advantage. The slicing ablation shows an overall benefit, but does not resolve this concern about the chosen credit rule.
Questions for authors
  1. Why use other rollouts' final scores as the reference for intermediate snapshots? Have you compared this rule with a simple snapshot-to-snapshot progress signal, particularly on trajectories containing temporary regressions?
  2. How stable are the test groups and rewards across different rollout samples? A sensitivity analysis over the number of rollouts would help assess whether accidental grouping affects learning.
3 comments

Q1: Other rollouts' final scores as the reference for snapshots

We agree that a state better than the average final state is not necessarily better than the previous state. This is a deliberate choice: CodeLift does credit assignment, not process reward. It spreads the final reward back to earlier steps, instead of scoring each step on its own. We made this choice for two reasons:

  • If a state beats the group's average final outcome, it is worth learning at the current training step, even if it comes after a regression. The reverse also holds. The credit's order within a trajectory is still kept: a slice that regresses always gets a lower advantage than the better slice before it.
  • Credit based on the change between adjacent snapshots is a process reward. It compares each step only with the rollout's own previous state and ignores the other final outcome baselines. A step from 0.1 to 0.2 would get positive credit even if every other rollout reached 0.9. It also penalizes every temporary regression. Many useful SWE edits, such as multi-file refactors, break tests for a while before they are done. CodeLift penalizes such a state only when it is also below the other rollouts' final outcomes (as in the aiohttp case in Appendix D.3). CodeLift instead compares every snapshot with the same reference as the final outcome, and its last slice gets exactly the outcome advantage. So the whole trajectory is measured on one scale.

Given the reasons above, we select the outcome-baseline credit assignment as final method rather than the snapshot-to-snapshot process method.

Q2: Sensitivity to the number of rollouts

Thank you for raising this. Since retraining with different rollout (2-7) in SWE Agentic RL is heavy, we reuse the 8-rollout training data offline for analysis: For each N from 2 to 7, we take every subset of N rollouts (C(8, N) in total), recompute the groups and rewards, and average three measures:

  • (a) Held-out reward shift: the mean absolute reward change on the 8 − N patches left out of the subset (rewards in [0, 1]).
  • (b) Advantage sign change: on the same N patches, the share of final advantages whose sign (positive, zero, or negative) changes.
  • (c) Merged-test disagreement: for two tests in the same group under N rollouts, the share of the 8 − N unsampled rollouts on which their outcomes differ.
Rollouts (N)234567
(a) Held-out reward shift0.1760.1260.0880.0580.0340.012
(b) Advantage sign change2.21%2.04%1.68%1.38%0.98%0.58%
(c) Merged-test disagreement2.52%2.16%1.94%1.61%1.50%1.47%

Even with only 2 rollouts, two tests in the same group disagree on 2.52% of the unsampled rollouts. So accidental grouping rarely merges tests that behave differently on new patches. Across all N, the final advantage changes sign in at most 2.21% of cases. To check accidental grouping alone, we fix the same N patches and compare two options: regroup the tests, or keep the test groups from all 8 rollouts. The mean reward difference is only 0.0102 at N = 4 and 0.0027 at N = 7. So accidental grouping has little effect on rewards. We find this analysis useful and have added it to Appendix C.6.

Thank you for the clarification. Your response addressed some of my concerns. On balance, I will keep my original score.

Thank you for your support of our work! Your suggestions (e.g., sensitivity) are helpful for improving our work and are adopted in the manuscript! We really appreciate it!

Reviewer_zu2ZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to address the reward sparsity during reinforce learning on SWE tasks. Compared to the pass rate baseline, CodeLift introduces the behavior balancing method to re-assign weights to different test units and snapshot slicing method to achieve progressive reward. This paper does extensive experiments and analysis to demonstrate how CodeLift improves over it.

Critical strengths
  1. The topic is timely and the motivation is clear. The traditional binary reward in SWE RL is sparse while the pass rate is imbalanced. CodeLift tries to address these issues in a straightforward manner.
  2. The analysis is detailed. The authors does detailed experiments and analysis to try to explain how CodeLift works, which provides informative experience to the community.
Critical weaknesses

major:

  • The novelty framing in the introduction is somewhat misleading. The introduction presents test imbalance and dynamic test weighting as central motivations without acknowledging the substantial overlap with prior work such as VeRPO. The distinction is only discussed later in Related Work, while the introduction does not clearly state what is genuinely new.
  • The motivation and contribution of patch grouping are unclear. CodeLift estimates test difficulty over distinct patch groups rather than sampled rollouts, but it is not well justified why this choice is preferable, nor is its individual contribution isolated apart from test grouping.
  • The performance advantage is supported by only a small portion of the training trajectory. In Figure 3(a), CodeLift overlaps with the baselines for most training steps, with a clear separation only in a short final interval. Stronger evidence is needed to establish that this advantage is robust and sustained, e.g., through a longer training horizon.
  • The two components show very different effects across benchmarks. In Table 2, slicing contributes most of the gain on Verified, while scoring contributes most of the gain on Pro. The paper notes this difference but does not explain it.
  • The cost-effectiveness of snapshot slicing is unclear. Slicing evaluates 2.35 additional non-final patches per rollout, but no wall-clock or CPU overhead is reported. This is especially relevant given its very small gain on SWE-bench Pro.

minor

  • Figure 3 is difficult to read. Some colors are too similar, and the three panels use different evaluation protocols that are not visually obvious.
  • The vertical spacing between tables and their captions is inconsistent. For example, the spacing around Tables 2 and 3 differs noticeably.
Questions for authors

Please see the concerns in the weakness.

3 comments

Q1: Novelty framing in the introduction and overlap with VeRPO

Thank you for this point. We now cite VeRPO in the introduction where we discuss difficulty imbalance.

TL;DR: VeRPO handles difficulty imbalance, which is common in all coding tasks. CodeLift also handles more complicated challenges specific to real SWE tasks, such as multiplicity imbalance (Figure 2).

Take multiplicity imbalance: In real repositories, developers rarely test each feature evenly. In competitive programming, problem setters design the tests for scoring, so their number and coverage are better controlled. As a result, many SWE tests can check the same behavior (Figure 2).

For example: Consider an issue with 100 tests and 4 sampled patches. Test 1 checks feature X, and tests 2–100 check feature Y:

Test 1Tests 2–100
Patch 1PassFail
Patch 2PassFail
Patch 3FailPass
Patch 4FailPass
  • VeRPO: All 100 tests have the same pass rate (2/4), so its difficulty term and density correction treat them the same. Patches 3 and 4 pass 99 tests and get far higher rewards than patches 1 and 2, which pass only one.
  • CodeLift: Tests 2–100 always kill the same patches (1 and 2), so they form one group with the same weight as test 1. All four patches get the same reward.

In short, VeRPO's density correction shares weight among tests with similar pass rates, while CodeLift merges tests that kill the same patches. Tests with the same behavior always have the same pass rate, but not the reverse. In competitive programming, tests are usually designed to cover features more evenly, so this gap rarely shows. In SWE, it leads to multiplicity imbalance, which VeRPO is not designed to address but CodeLift is.

Our response to Reviewer aV7v reports a direct comparison, where CodeLift outperforms VeRPO.

Q2: Motivation and contribution of patch grouping

Patch grouping counts each behavior once, no matter how many patches repeat it.

For example: Consider a task with 2 tests, each checking one feature, and 8 patches:

Test 1Test 2
Patches 1–6PassFail
Patches 7–8FailPass
  • Counting rollouts: Test 2 kills 6 of 8 patches and test 1 kills 2, so test 2 gets three times the weight (0.75 vs. 0.25). Patches 7–8 then get three times the reward of patches 1–6, though the two tests are equally useful.
  • CodeLift: Patches 1–6 form one group, and patches 7–8 form another. Each test kills one group, so both tests get weight 0.5 and all patches get the same reward. This is the intended outcome: each patch group fixes one feature, so neither should be preferred.

Our offline analysis of the training data also shows that such repeats are common. Among all sampled 8-rollout groups with at least two distinct behaviors, 99% contain repeated behaviors, with 3.09 distinct behaviors on average. Counting patch groups instead of rollouts changes the rewards in 56% of them. Note that grouping does not discard how often the policy produces each behavior. That frequency enters through the advantage baseline, which counts every rollout (Section 3.3). The reward itself depends only on how good a patch is, not on how many other rollouts repeat it.

This is by design and follows mutation testing theory. Mutation testing judges tests by the share of mutants they kill, so the mutants must be semantically distinct. Duplicated mutants, which behave the same as each other, are treated as useless and removed before scoring [1]. Redundant mutants also inflate the score and can mislead test assessment [2, 3]. In CodeLift, patches play the role of mutants, so we merge patches with identical behavior in the same way. We therefore treat grouping as part of this score rather than as a separate component.

References

[1] Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple, Fast and Effective Equivalent Mutant Detection Technique, ICSE 2015

[2] Establishing Theoretical Minimal Sets of Mutants, ICST 2014

[3] Threats to the Validity of Mutation-Based Test Assessment, ISSTA 2016

Q3: Advantage over a small portion of the training trajectory in Figure 3(a)

  1. We agree that the clearest gap appears late in training, and this exactly matches how CodeLift works. The main gain of CodeLift comes from its dynamic curriculum. As analyzed in Section 5 (Figures 7–8), when training rewards rise, the count of all-pass tests increases, and pass-rate baselines will get less learning signal. CodeLift ignores those saturated all-pass tests and keeps finding harder tests and gives them more weight.
  2. CodeLift already leads at step 10, at about 32% versus about 27% for the baselines. So the lead is not limited to the end of training.
  3. Figure 3 only shows validation scores from a single greedy run (temperature 0). Table 1 reports test pass@k over 8 attempts. For example, CodeLift's best validation score is 36.2, but its test pass@1 is 38.3. The gains are also supported by 95% paired issue-bootstrap intervals (Appendix B).
  4. We fixed a budget of 200 steps (3 epochs) for all methods before training (Appendix B). Figure 3 also shows step 210, with no further gain after step 200. This budget matches the VeRL uni-agent reference setting .

Q4: Different effects of the two components across benchmarks

Thank you for raising this. Verified is closer to the training data (R2E-Gym) distribution. Pro tasks are much longer and more complex, so Pro is a stricter test of generalization. CodeLift also improves on Pro, mainly through behavior balancing. We observe two things:

  1. Behavior balancing may give a more general learning signal. On issues whose tests cover only a few features, CL-Outcome often succeeds where Binary fails. This may be because behavior balancing reduces multiplicity imbalance.
  2. On Pro, the gain of snapshot slicing may overlap with behavior balancing. Under harmonic scoring, slicing improves Pro pass@1 by 2.44 points (PR-Harmonic to PR-Process). So its small gain on Pro in CodeLift may be because behavior balancing already brings most of this gain.

We have added this analysis to Appendix C.5.

Q5: Cost-effectiveness of snapshot slicing

We now report the wall-clock overhead. CodeLift reuses the rollouts already generated in training. It needs no extra rollouts and no reward model. Its only extra cost is running tests, which is cheap and highly parallel. The table shows the time of each phase in one training step (512 rollouts, 512 parallel sandboxes).

PhaseWall timeShare
Rollout inference462.61 s40.63%
Old-policy forward126.29 s11.09%
Policy update442.01 s38.82%
Weight gather38.90 s3.42%
Outcome testing64.31 s5.65%
Snapshot testing (extra)4.40 s0.39%

The results show:

  1. Testing takes only a small share of each training step (about 6%), far less than the GPU phases.
  2. Although snapshot slicing tests 2.35 more states per rollout, the extra snapshot testing takes 0.39%, because of the straggler effect. In the parallel pool, fast or early-failing tests free their slots quickly, so the total time is still set by a few slow tests.

We have added this time breakdown to Appendix A.4.

Q6: Minor issues

We now use more distinct colors in Figure 3, label the evaluation setting of each panel, and make the spacing between tables and captions consistent.

Dear Reviewer zu2Z,

Thank you again for your thoughtful review. With only few hours left for discussion, we would like to ensure that our clarifications are helpful if you had a chance to review them. We deeply appreciate the effort that already put into review.

Best regards,

Authors

Reviewer_aV7vOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CodeLift converts unit-test outcomes into dense rewards for training software-engineering agents. It groups tests and sampled patches by identical outcome patterns, weights test groups by their failure frequency across distinct patch groups, and applies the resulting score to intermediate repository snapshots for slice-level credit assignment. Experiments on SWE-bench Verified and the Python subset of SWE-bench Pro show improvements over binary and pass-rate rewards, with additional results using a coding-specialized backbone and a different training dataset.

Critical strengths
  1. The paper addresses a concrete problem with pass-rate rewards: duplicated tests and numerous already-passing tests can distort the credit assigned to repairs and regressions. The proposed scoring rule provides a practical way to use this execution feedback without training a separate reward model.
  2. The improvements are supported by results on two SWE benchmarks and an additional model and training setting. The paired confidence intervals and the CL-Outcome ablation provide useful evidence for the overall gains and the contribution of snapshot credit.
Critical weaknesses
  1. The comparison with existing dynamic rewards is incomplete. VeRPO [1] already uses online test difficulty and density correction, whereas PR-Reweighted uses fixed weights. Although the paper explains its different grouping design, the experiments do not establish its advantage over this closely related approach.
  2. The contribution of the individual scoring components remains unclear. Table 2 evaluates behavior balancing as a whole, without separating test grouping, patch grouping, and adaptive weighting. It is therefore difficult to judge which design choices are responsible for the improvement.

[1] Beyond Binary: Turning Partial Success into Dense Verifiable Rewards for Reinforcement Learning in Code Generation.

Questions for authors
  1. How does CodeLift compare with an online weighting rule such as VeRPO under the same training setup? Keeping snapshot credit fixed would help isolate the benefit of the proposed scoring rule.
  2. How does performance change when test grouping or patch grouping is removed, or when test groups receive uniform weights? These comparisons would clarify which parts of behavior balancing are necessary.
  3. How much training time does snapshot evaluation add compared with final-only evaluation? Its additional benefit on SWE-bench Pro is small, so the cost would help assess when this component is worthwhile.
3 comments

Q1: Comparison with VeRPO under the same training setup

We adapt VeRPO to SWE (PR-VeRPO, Section 4) and train it with the same setup. Its results and training curves are now in Table 1 and Figure 3.

MethodV pass@1V pass@2V pass@4V pass@8P pass@1P pass@2P pass@4P pass@8
VeRPO34.4542.4149.4355.4019.5028.2436.6242.86
CL-Outcome35.8344.0951.3757.8019.8828.3736.6943.23
CodeLift38.3047.5355.8462.8020.1628.7136.8443.23

(V=Verified, P=Pro)

Note that keeping snapshot credit fixed (comparing VeRPO with CL-Outcome) is not a strictly controlled comparison, because VeRPO also assigns credit to intermediate steps. Even so, without any step-level credit, CL-Outcome still outperforms VeRPO on all metrics.

Our response to Reviewer zu2Z further explains how CodeLift differs from VeRPO in mechanism, with an example.

Q2: Contribution of the individual scoring components

Behavior balancing follows mutation testing as a whole, so it is one unit component, rather than a union of multiple components for ablation.

CodeLift is built from two units, behavior balancing and snapshot slicing, and Table 2 ablates at this level. Each part inside a unit, such as the grouping rule, the weighting rule, the slicing rule, or the credit rule, could be replaced by many possible alternatives, so we design and evaluate each unit as a whole. Here, for behavior balancing, this whole comes from mutation testing theory (line 76), with patches as mutants and tests as mutant killers. In mutation testing, removing duplicates is a required precondition, because a duplicated mutant is counted more than once and skews the mutation score. CodeLift's grouping of patches and tests performs exactly this deduplication. Weighting is the fundamental goal: each test group's weight is the share of patch groups it kills. As the policy improves, these weights shift to harder tests, which gives a dynamic curriculum (Section 5, Figures 7–8). Removing grouping would break the precondition, and uniform weights would remove the goal itself.

Still, we recognize that Table 2 only compares behavior balancing with the binary reward. It does not show what behavior balancing adds when snapshot slicing is kept. We therefore add PR-Process. It keeps snapshot slicing but replaces behavior balancing with the harmonic-mean score of PR-Harmonic. We hope this fills the gap in Table 2 by showing the value of behavior balancing as a whole.

MethodV pass@1V pass@2V pass@4V pass@8P pass@1P pass@2P pass@4P pass@8
PR-Process35.5544.1151.6058.4019.3627.4835.7942.48
PR-Harmonic34.4543.7651.6258.0016.9224.7032.5638.72
CL-Outcome35.8344.0951.3757.8019.8828.3736.6943.23
CodeLift38.3047.5355.8462.8020.1628.7136.8443.23

Our response to Reviewer zu2Z further explains why patch grouping is needed, with an example.

Q3: Training time added by snapshot evaluation

CodeLift reuses the rollouts already generated in training. Its only extra cost is running tests, which is cheap and highly parallel. The table shows the time of each phase in one training step (512 rollouts, 512 parallel sandboxes).

PhaseWall timeShare
Rollout inference462.61 s40.63%
Old-policy forward126.29 s11.09%
Policy update442.01 s38.82%
Weight gather38.90 s3.42%
Outcome testing64.31 s5.65%
Snapshot testing (extra)4.40 s0.39%

The results show that: compared with final-only testing (5.65%), snapshot testing adds 0.39% of each training step, and the total time cost of testing is much less than the cost on GPUs.

We have added this time breakdown to Appendix A.4.

Thank you for the clarification. The individual components were still not ablated separately. Nevertheless, I will maintain my original positive rating.

Thank you for your kind reply! Your suggestions are helpful and adopted in our revised manuscript (e.g., VeRPO comparison)!

Just to briefly clarify: we ablate behavior balancing as one module since its parts can always be split further and are interdependent. For example, whether to group/weight the tests/patches, and whether to apply these combinations to all snapshots / only final patches, already multiply into many variants which can be refined again. Thus, we follow mutation testing theory to take behavior balance a whole, and prioritized the limited computation resources to VeRPO and module-level variant (PR-Process) for reference.

Thank you again for your time and effort and we sincerely appreciate it!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal (FAQs)

Dear Reviewers and AC,

Thank you once again for your time and efforts. With one day left for the discussion, we provide a short summarization with clarifications for common concerns.

1. Comparison with VeRPO (aV7v Q1, zu2Z Q1)

  • CodeLift outperforms VeRPO on both SWE-bench Verified/Pro in the same training/testing setup.
  • CodeLift handles multiple complex SWE test imbalance issue (e.g., multiplicity imbalance) while VeRPO handles general difficulty imbalance in competitive programming.

2. Extra time from snapshot testing (aV7v Q3, zu2Z Q5)

  • Snapshot testing only add 0.39% of wall-clock time in each training step.
  • All testing together takes about 6% of a training step, far less than the GPU phases.

3. How behavior balancing works

  • Basic theory: Mutation testing theory, with patches as mutants and tests as mutant killers. It removes duplicate (1) patch behaviors (2) test semantics, then uses a dynamic mutation score that enables a curriculum.
  • Why it is one unit module (aV7v Q2, zu2Z Q2): Mutation testing needs both steps together. Removing duplicates is a required precondition for a valid mutation score. Below this level, the module could be always split in many ways, and each part split again, so we ablate behavior balancing as a whole. (Table2 and PR-Process (new)).
  • Why the strength grows later in training (zu2Z Q3, Figures 7–8): Early in training, learning signal is rich for every method. Later, many tests become all-pass tests, and pass-rate baselines get less signal. CodeLift ignores the saturated tests and shifts weight to harder ones, so it keeps getting useful signal while the baselines do not.

We hope this makes our responses easier to go through, and we are happy to clarify anything further.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.