acceptodds
Under review as a conference paper at ICLR 2027

Graded Execution Feedback for Co-Evolving Code and Test Generators

Abstract

Execution feedback for code generation is often represented as a binary pass/fail signal. This is natural for unit-test style correctness checking, but it discards useful information in tasks where multiple outputs are feasible and differ in solution quality. We study code generation for graded programming tasks, including combinatorial optimization and sequential decision-making, where programs are evaluated by normalized objective or rollout scores. We introduce Continuous Co-Evolution (CCE), a framework that trains a shared language model under both solution-program and test-generation prompts. Candidate programs are rewarded by their continuous performance on reference tests, while generated tests are rewarded when the program rankings they induce agree with rankings from reference tests. This ranking-based objective enables generated tests to provide informative feedback while preserving graded differences among candidate programs. Experiments on Traveling Salesman Problem (TSP), Set Cover, Parallel Machine Scheduling (PMS), and Game 2048 show that continuous test-feedback co-evolution improves over coder-only training and binary-feedback co-evolution baselines.

Primary area:
reinforcement learning
Keywords:
Code Generation, Execution Feedback, Reinforcement Learning, Test Generation, Continuous Feedback

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_goqSOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes Continuous Co-Evolution (CCE), a framework for jointly training code and test generators using continuous execution feedback rather than binary pass/fail signals. The key motivation is that, in graded programming tasks, valid programs can differ substantially in solution quality, which binary feedback fails to capture. CCE employs a shared LLM for both code and test generation, optimized through alternating GRPO-style updates. The coder is rewarded based on its average performance on reference tests, while the tester receives a reliability-based pairwise reward that encourages generated tests to preserve the program rankings induced by reference tests. The authors also provide a theoretical connection showing that the proposed tester reward recovers the CURE reward under a binary execution model. The framework is evaluated on four tasks: Traveling Salesman Problem (TSP), Set Cover, Parallel Machine Scheduling (PMS), and Game 2048. Experimental results show that CCE generally outperforms coder-only training, CURE-style binary-feedback co-evolution, and direct-inference baselines, with particularly notable improvements on Set Cover, PMS, and Game 2048.

Critical strengths
  • Well-motivated extension of coder–tester co-evolution to graded programming tasks. The paper addresses a meaningful limitation of binary execution feedback, which fails to distinguish between valid programs with different solution qualities. The proposed reliability-based pairwise reward provides a principled way to train test generators using continuous execution scores while encouraging consistency with reference-induced program rankings. The theoretical connection to CURE under binary execution feedback further supports the formulation. This extension is particularly relevant to optimization and sequential decision-making tasks where solution quality cannot be adequately characterized by pass/fail outcomes.

  • Consistent empirical improvements across diverse graded programming tasks. The experiments demonstrate that CCE generally outperforms both coder-only training and CURE-style binary-feedback co-evolution across four tasks, covering combinatorial optimization and sequential decision-making. The improvements are particularly notable on Set Cover, PMS, and Game 2048, with gains in both mean performance and Best-of-N metrics. These results provide empirical evidence that preserving continuous execution feedback can improve the quality of generated programs beyond what is achieved with binary feedback alone.

Critical weaknesses
  • Insufficient ablation of coder and tester reward designs. The proposed CCE framework simultaneously modifies the reward mechanisms for both the coder and tester, replacing binary execution feedback with continuous quality scores and introducing a reliability-based pairwise reward for test generation. However, the current experiments do not sufficiently disentangle the individual contributions of these two modifications. Although comparisons with CURE and Coder-Only training demonstrate the overall effectiveness of the framework, they do not clarify whether the improvements primarily originate from the continuous coder reward, the proposed tester reward, or their interaction. A more comprehensive ablation study that independently varies the coder and tester reward designs (e.g., binary versus continuous coder rewards and binary versus ranking-based tester rewards) would help identify the source of the performance gains and better substantiate the necessity of each component.

  • Lack of analysis of coder–tester co-evolution dynamics. Although the central motivation of the proposed framework is the co-evolution of code and test generation capabilities, the experimental analysis primarily focuses on final-checkpoint performance without examining how these capabilities evolve throughout training. In particular, the paper does not provide learning curves for coder performance or tester quality metrics, such as generated-test validity, discriminative rate, and ranking agreement. The generated-test diagnostics in Appendix H are reported only for the final training iteration. Consequently, it remains unclear whether the coder and tester progressively improve together, whether improvements in tester quality coincide with gains in coder performance, or whether alternating updates introduce interference between the two roles. Reporting the training trajectories of both roles across optimization iterations would provide more direct empirical evidence for the claimed co-evolutionary behavior.

Questions for authors
  • Could the authors provide a more detailed ablation study that independently examines the contributions of the continuous coder reward and the reliability-based tester reward? Specifically, how does performance change when only one of the two reward mechanisms is modified while keeping the other unchanged? Such results would help clarify whether the observed improvements primarily stem from the coder reward, the tester reward, or their interaction.

  • Could the authors provide training curves illustrating how the capabilities of both the coder and tester evolve throughout training? In particular, tracking coder performance alongside tester quality metrics (e.g., generated-test validity, discriminative rate, and ranking agreement) would help determine whether both roles progressively improve and whether alternating updates lead to mutual improvements or interference. This evidence would be important for assessing the effectiveness of the claimed co-evolutionary mechanism.

3 comments
AuthorsOpenReview ↗

Follow-Up on Author Response (Discussion Period Deadline: Nov. 18)

Dear Reviewer goqS,

Your comments on coder-tester training dynamics have been particularly helpful in improving the completeness of our experimental analysis. In response, we have added additional experimental results on complete training trajectories across all four tasks and the requested 2×2 reward ablation study.

As the author-reviewer discussion deadline approaches (Nov. 18), we would greatly appreciate it if you could consider our detailed rebuttal and additional results when finalizing your assessment.

Thank you again for your time and consideration.

Best regards,

Authors of CCE

Thank you for your response. I have updated my rating accordingly.

Thank you for your follow-up and for updating your itemized rating! We sincerely appreciate your consideration of our additional experiments and analyses; we are glad that they have helped address your questions.

If possible, we would be immensely grateful if you might also consider adjusting your overall Rating and/or Confidence to reflect the current assessment.

Reviewer_i5akOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Continuous Co-Evolution (CCE) uses a shared model to generate programs and test instances. It trains the coding role with continuous execution scores and rewards the testing role for preserving the program ordering induced by reference tests. The paper derives a linear-time expression for the pairwise reward and its correspondence to CURE's reward in the binary setting. On TSP, Set Cover, PMS, and 2048, CCE with Qwen2.5-7B achieves favorable mean scores and improves Best@B on three tasks. The central claim is that retaining graded feedback distinguishes program quality better than binary co-evolution.

Critical strengths
  1. The method addresses the stated problem directly. Training the testing role to preserve reference-score rankings fits graded tasks better than rewarding pass/fail alone. The binary reduction gives a unified interpretation, and the Set Cover programs and worked instance make quality differences among feasible solutions concrete.
  2. Implementation and evaluation are described in reasonable detail. Four task families share a 256-program × 256-hidden-instance framework, with Best@B, normalization, and training details. The paper also reports worse PMS parsing rates and qualifies the transfer matrix and post-hoc choices.
Critical weaknesses
  1. The behavior and independent effect of the central reward remain unclear. Reference reliability uses a product or log-sum; zeros receive a large penalty with δ = 10⁻⁸, so the ordering may be driven primarily by invalid-solution counts. Aggregate validity rates cannot establish per-program zero-score distributions or prove that the reward degenerates into a binary count. Direct analysis of existing rollouts is needed. Controls combining continuous coder rewards with binary tester rewards, binary coder rewards with ranking tester rewards, and random instance generation are also missing. The constructed CURE proxy changes both rewards, preventing attribution to the proposed testing reward. In most tasks, a "test" is an instance scored by an external evaluator and should be distinguished from a unit test that independently judges a program.
  2. Stability, diagnostic consistency, and external validity are insufficiently supported. There are no multiple training seeds or paired uncertainty estimates, some TSP differences are small, and identical medians on tasks such as PMS require interpretation alongside behavioral deduplication. Training instances are small, hidden-instance sizes are unclear, PMS uses inconsistent normalization across the two sides, and the 2048 rules differ from the standard setting. Table 9 evaluates ranking agreement against mean reference scores, while training uses product reliability. Appendix J.4's selection score of 0.966 versus hidden score of 0.170 also limits the learned-surrogate-evaluator interpretation. These issues are addressable, but currently leave the mechanism and deployment benefit insufficiently established, leading me to lean toward rejection.
Questions for authors
  1. Reward behavior: In existing rollouts, how much of the reliability ordering is determined by zero-score counts? Please report its relationship to rankings by zero counts and mean reference scores.
  2. Attribution and baselines: Do existing experiments separately replace the two rewards or use a random/frozen instance generator? How does the constructed CURE proxy differ from the original method? If these results are unavailable, please state the resulting limits on attribution.
  3. Evaluation distribution: What are the hidden-instance sizes in the main experiments? How many ties arise from PMS normalization and clipping, and how many behavioral patterns remain after score-vector deduplication of the 256 programs?
  4. Diagnostics: Why does Table 9 use mean scores rather than training reliability, and why are Valid and Discriminative identical? Do generated tests participate in selection at inference in the main experiments?
  5. Existing records: Please specify training and sampling seed counts, provide available per-problem differences, and explain the selection–hidden-score gap in Appendix J.4 and its relationship to the main method.
2 comments
AuthorsOpenReview ↗

Follow-Up on Author Response (Discussion Period Deadline: Nov. 18)

Dear Reviewer i5ak,

Your comments on evaluation protocols and task settings have been particularly helpful in improving the clarity and completeness of our paper. In response, we have clarified the evaluation settings and task-specific details, alongside additional experimental results with robustness analyses and three-seed evaluations.

As the author-reviewer discussion deadline approaches (Nov. 18), we would greatly appreciate it if you could consider our detailed rebuttal and revised manuscript when finalizing your assessment.

Thank you again for your time and consideration.

Best regards,

Authors of CCE

AuthorsOpenReview ↗

Follow-Up on Author Response (Discussion Period Deadline: Nov. 18)

Dear Reviewer i5ak,

With fewer than 8 hours left before the author-reviewer discussion deadline (Nov. 18, AoE), we would greatly appreciate it if you could consider our detailed rebuttal, additional experiments, and revised manuscript in your final assessment and reconsider your evaluation where appropriate.

Thank you again for your time and consideration.

Best regards,

Authors of CCE

Reviewer_rEsfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Continuous Co-Evolution (CCE), which jointly trains a shared LLM for program and test generation using graded execution feedback. The coder receives rewards based on reference-test performance, while the tester is rewarded for preserving a reference-induced ranking of candidate programs. The paper reports improvements over coder-only training and binary-feedback co-evolution on TSP, Set Cover, parallel-machine scheduling, and 2048.

Critical strengths

A clear objective for test generation. The proposed reward encourages generated tests to distinguish higher-quality programs from lower-quality ones according to reference-test performance. This provides a concrete training objective for tasks where correctness alone does not capture solution quality.

Promising results across multiple tasks. CCE achieves higher mean solution quality than coder-only training and the binary-feedback baseline across four tasks, with particularly noticeable gains on scheduling and 2048. These results suggest that jointly training program and test generation can be useful for optimization tasks.

Critical weaknesses

Missing related work and relevant baselines. The paper overlooks relevant studies that combine LLM-based evolutionary search with model training, particularly EvoTune [1] and CALM [2]. CALM already uses graded performance feedback for GRPO-based fine-tuning. Discussing these approaches would help establish the contribution beyond existing algorithm discovery methods.

Insufficiently established novelty. CCE closely follows the framework of CURE [3] for jointly training code and test generation within a shared model. Meanwhile, using continuous scores as feedback is already established in prior works. Therefore, using continuous scores alone does not establish a novel contribution. The potential methodological novelty appears to lie primarily in the reliability-based tester reward, but its distinction from and significance beyond existing approaches require stronger justification.

Insufficient isolation of the tester reward’s contribution. The current comparisons do not clearly separate the benefits of continuous coder rewards from those of the proposed tester reward. Keeping the coder reward continuous and comparing binary versus continuous tester rewards would clarify the tester’s contribution. Matching computational budgets would also help determine whether the gains come from the reward design or additional computation.

[1] Anja Surina, Amin Mansouri, Lars Quaedvlieg, Amal Seddas, Maryna Viazovska, Emmanuel Abbe, and Caglar Gulcehre. Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning. In Conference on Language Modeling (COLM), 2025.

[2] Ziyao Huang, Weiwei Wu, Kui Wu, Jianping Wang, and Wei-Bin Lee. CALM: Co-evolution of Algorithms and Language Model for Automatic Heuristic Design. In International Conference on Learning Representations (ICLR), 2026.

[3] Yinjie Wang, Ling Yang, Ye Tian, Ke Shen, and Mengdi Wang. Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning. In Advances in Neural Information Processing Systems (NeurIPS), 2025.

Questions for authors
  1. What is the specific methodological contribution beyond CURE’s joint training framework and existing approaches that train LLMs with performance feedback for algorithm discovery, such as EvoTune and CALM? How does CCE compare with these approaches under comparable settings?
  2. With the coder reward kept continuous, does replacing a binary tester reward with the proposed continuous tester reward improve performance?
  3. Are the total training and execution budgets matched between CCE and coder-only training? If not, how much of the improvement remains when these budgets are controlled?
3 comments
AuthorsOpenReview ↗

Follow-Up on Author Response (Discussion Period Deadline: Nov. 18)

Dear Reviewer rEsf,

Your comments on computational-budget analysis have been particularly helpful for the completeness of our paper. In response, we have added wall-clock comparisons and update-matched controls to better assess the computational efficiency and effectiveness of CCE.

As the author-reviewer discussion deadline approaches (Nov. 18), we would greatly appreciate it if you could consider our detailed rebuttal and additional experimental results when finalizing your assessment.

Thank you again for your time and consideration.

Best regards,

Authors of CCE

Thank you for the additional experiments and clarifications. The ablation study addresses my concerns regarding the coder rewards.

However, my concern about novelty remains unresolved. CCE extends CURE’s coder-tester co-evolution framework to graded feedback, which has already been used to train program generators in prior work such as CALM. While I acknowledge the proposed tester objective, I remain unconvinced that it constitutes a sufficient methodological advance beyond this combination.

Therefore, I maintain my current score.

AuthorsOpenReview ↗

Clarification on Novelty

Thank you for your follow-up and for acknowledging our additional experiments. We are sorry for any remaining confusion about the novelty of our CCE compared to CALM and CURE.

We would like to clarify our CCE's novelty compared to CALM:

  • Although CALM uses graded performance feedback, its "co-evolution" is fundamentally different from CCE's coder-tester co-evolution. CALM co-evolves the language model and the algorithms it generates, while the test instances and evaluation mechanism remain fixed. It does not train a test generator jointly with the program generator. In this sense, CALM is more like "self-evolving" than "co-evolving".

  • In contrast, CCE jointly trains both the program generator and the test generator under graded feedback. Our key methodological contribution is the reliability-based pairwise tester reward, which enables the test generator to learn to preserve reference-induced program rankings.

We would also like to clarify our CCE's novelty compared to CURE:

  • Although CURE jointly trains the program generator and the test generator:

    • Its coder-tester co-evolution is limited to binary feedback. CURE evaluates programs based on pass/fail outcomes, which cannot distinguish between feasible programs with different solution performances.
    • It does not provide a suitable training objective to train a tester LLM that accounts for graded (continuous) differences in program performance, thereby limiting its effectiveness.
    • For example, in complex combinatorial optimization (e.g., TSP) and sequential decision-making tasks (e.g., Game 2048), CURE with binary feedback can only distinguish feasible programs from infeasible programs. This is insufficient because even among feasible solutions, performance can vary substantially.
  • In contrast, CCE generalizes coder-tester co-evolution from binary to graded (continuous) feedback.

    • Our reliability-based pairwise tester reward enables the test generator to distinguish among programs with different solution qualities by preserving their reference-induced rankings.
    • Moreover, CURE's binary tester reward is a special case of our formulation, establishing CCE as a principled generalization of binary coder-tester co-evolution to graded settings, with a novel tester objective that explicitly captures graded (continuous) differences in program quality.

We hope this clarification helps distinguish the methodological contributions of these approaches. Thank you again for keeping the communication channel open, and please do not hesitate to let us know if you have any further questions.

Best regards,

Authors of CCE

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.