acceptodds
Under review as a conference paper at ICLR 2027

ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agents

Abstract

Coding agents are increasingly used to build programs from scratch as software engineers do rather than to resolve issues within a repository or to generate a repository from a given requirements document. However, scalable verifiable training environments that force agents to discover what is required and to decide how the system should be designed remain scarce. Moreover, existing agent work- flows also cannot sustain long-horizon development in which the agent explores the reference program before implementation. To address these challenges, we introduce ProgramGym, a framework consisting of Program-Env and Program- Agent for scalable training environments of long-horizon program reproduction agents. Specifically, Program-Env constructs verifiable training environments from repositories through a feedback-driven loop that repairs runtime sandbox builds and the quality of black-box tests. Using this novel pipeline, we obtain 1,648 training environments spanning C, C++, Go, and Rust. Program-Agent combines a role-isolated workflow of analysis, implementation, and review with revision management for compilation checks and failure recovery. Experiments on ProgramBench show that both Program-Env and Program-Agent improve performance on program reproduction. Fine-tuning on trajectories with these constructed environments raises the performance of Qwen3.8-27B from 34.87% to 42.84%. Program-Agent further raises the base and SFT models to 55.71% and 56.25%, both exceeding some flagship models. We will release the ProgramGym to support further research on coding agents that build programs from scratch.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models;Code Agents;

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3mztOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ProgramGym constructs 1,648 source-free program-reproduction environments and introduces an analysis–implementation–review agent. On ProgramBench, SFT raises Qwen3.8-27B's mean Tests Passed from 34.87% to 42.84%; Program-Agent reaches 55.71% and 56.25% with the base and SFT models.

Critical strengths
  1. The offline environments, reference executables, documentation, and 665,512 retained tests make ProgramGym a substantial resource for training program-reimplementation agents (Appendix B).
  2. Program-Agent substantially improves the same Qwen backbone (Table 2). The base-model improvement uses fewer output tokens than Claude Code (Appendix D.2).
Critical weaknesses
  1. I do not find sufficient evidence for reliable behavioral supervision. Reference agreement, coverage, empty-dummy rejection, and static checks do not establish whether the suites distinguish plausible faulty implementations (Eq. 3; Figure 17). The SFT gain demonstrates data usefulness, but does not validate this central property of the proposed verifier.
  2. The training evidence falls short of the goal of faithful reproduction. Under Claude Code, SFT improves the mean but reduces Almost Resolved from 3.5% to 1.5% and Resolved from 0.5% to 0%; with Program-Agent, SFT adds only 0.54 percentage points (Table 2). The authors acknowledge these trends, but the benefit of training for high-fidelity reproduction beyond the scaffold remains limited.
Questions for authors
  1. Could you validate the retained suites against representative faulty implementations to assess whether the proposed checks produce reliable behavioral supervision?
  2. What evidence supports improved high-fidelity reproduction from training, beyond the partial-reproduction gains already shown?
4 comments
AuthorsOpenReview ↗

W1/Q1: Verifier correctness, discrimination, and training utility(1/2)

Thanks a lot for your detailed feedback. We really appreciate your recognition of our work and also appreciate your time and effort in providing insightful suggestions that can help further polish our paper.

We address this concern from three angles.

1. Engineering practice. We define a reliable verifier by three properties: (1) Breadth. The suite exercises as many functional points as possible, so that it reflects the full black-box behavior of the program; (2) Faithfulness. Given broad coverage, every test passes on the golden program; (3) Discrimination. The suite fails on an empty implementation and cannot be passed unconditionally. Coverage feedback, the pass check, and the dummy check enforce these three properties respectively, while the double check and the weak-assert check further ensure stability and reduce weak oracles. Appendix B.4.1 details the design.

CheckPropertyPurpose
Coverage feedbackBreadthCover as much functionality as possible
Pass checkFaithfulnessCheck agreement with reference behavior
Double checkStabilityCheck repeatability across runs
Dummy checkDiscriminationNever accept every implementation
Weak-assert checkDiscriminationReduce weak oracles

Retained tests must pass on golden, fail on the empty implementation, pass weak-assert checks, and match across repeated reference runs. Tests violating these conditions are discarded. Figure 13 shows these metrics improving over iterations.

2. Effectiveness analysis. We evaluate real agent implementations sampled by DeepSeek-V4-Flash-0731 in Program-Env as plausible faulty implementations. For environment eie_i, P^i\hat{P}_i and B^i\hat{B}_i are the agent's source and executable; PiP_i and Bi⋆B_i^\star are their reference counterparts. The protocol is:

  1. We randomly sample 500 environments and, for every test t∈Tit \in \mathcal{T}_i of each sampled environment, construct a triple ⟨pt, p^t, t⟩\langle p_t,\ \hat{p}_t,\ t \rangle, where pt⊆Pip_t \subseteq P_i is the fragment of the reference source that implements the functionality exercised by tt, and p^t⊆P^i\hat{p}_t \subseteq \hat{P}_i is the agent's corresponding fragment. This yields 153,861 triples.
  2. Jt=Judge⁡(pt,p^t)∈{CORRECT,INCORRECT}J_t = \operatorname{Judge}(p_t, \hat{p}_t) \in \{\text{CORRECT}, \text{INCORRECT}\}. Claude Opus 5.5 judges whether p^t\hat{p}_t implements the same functionality as ptp_t. Four professional software engineers audited 4,000 of these judgments and found every label correct.
  3. Vt=t(B^i)∈{PASS,FAIL}V_t = t(\hat{B}_i) \in \{\text{PASS}, \text{FAIL}\} is the verifier's output, obtained by running tt against the executable B^i\hat{B}_i built from the agent's code.
  4. The suite is correct on tt when JtJ_t and VtV_t agree, that is, Vt=PASSV_t = \text{PASS} when Jt=CORRECTJ_t = \text{CORRECT} and Vt=FAILV_t = \text{FAIL} when Jt=INCORRECTJ_t = \text{INCORRECT}.

The verifier-effectiveness table below reports all 153,861 triples (Appendix B.5).

JtJ_tVt=PASSV_t = \text{PASS}Vt=FAILV_t = \text{FAIL}
CORRECTTP: 91,356 (correct accepted)FN: 116 (correct rejected)
INCORRECTFP: 1,726 (faulty missed)TN: 60,663 (faulty detected)

On these real, largely correct yet flawed agent outputs, the retained suites agree with the independent judgments on 98.80% of cases (TP + TN) and reject a correct implementation only 0.13% of the time (FN). The suites therefore provide reliable behavioral supervision that tells correct behavior from incorrect behavior, which is exactly the signal trajectory filtering relies on.

3. Training experiments

We use Program-Env's behavioral test suites to select 1,204 teacher rollouts with a test pass rate ≥50%, collected using DeepSeek-V4-Flash-0731 with Claude Code, and fine-tune Qwen3.8-27B on these rollouts to obtain Qwen3.8-27B-SFT-1204. The table below reports ProgramBench results under Claude Code. All Δ values are relative to the base Qwen3.8-27B model and are measured in percentage points.

ModelTests Passed (%)ΔAlmost Resolved (%)ΔResolved (%)Δ
Qwen3.8-27B34.87—3.50—0.50—
Qwen3.8-27B-SFT42.84+7.971.50−2.000.00−0.50
Qwen3.8-27B-SFT-120447.53+12.664.00+0.500.500.00

Compared with the original SFT model, the more strictly filtered SFT model improves Tests Passed by 4.69 percentage points, Almost Resolved by 2.50 points, and Resolved by 0.50 points. Relative to the base model, it improves Tests Passed by 12.66 points and Almost Resolved by 0.50 points, while matching the base model's 0.50% Resolved rate.

Revision: We have expanded Appendix B.5 with this effectiveness analysis, including the experimental protocol, Table 15 and Table16. All additions are highlighted in blue.

Thank you again for your feedback. Please let us know if any concerns remain unaddressed. We would be happy to discuss them further.

AuthorsOpenReview ↗

W2/Q2: Teacher supervision and high-fidelity reproduction(2/2)

Thanks a lot for your detailed feedback. We really appreciate your recognition of our work and also appreciate your time and effort in providing insightful suggestions that can help further polish our paper.

For W2 & Q2: Evidence of improved high-fidelity reproduction from training.

SFT outcomes depend on both the training tasks and the teacher trajectories [1]. Program-Env provides executable program-reproduction tasks and behavioral verifiers for selecting trajectories. Our original SFT used DeepSeek-V4-Flash-0731 with Claude Code as the teacher and retained any rollout with a non-zero test pass rate. This admitted many partially successful demonstrations, whose quality may limit the high-fidelity behavior learned through SFT.

The table below summarizes the teacher's score distribution on ProgramBench and among the retained Program-Env training rollouts.

Dataset[0.00, 0.25)[0.25, 0.50)[0.50, 0.75)[0.75, 1.00][0.95, 1.00]Total
ProgramBench31 (15.50%)37 (18.50%)76 (38.00%)56 (28.00%)9 (4.50%)200 (100.00%)
ProgramGym217 (13.17%)227 (13.77%)367 (22.27%)837 (50.79%)403 (24.45%)1648 (100.00%)

The [0.95, 1.00] column is a subset of [0.75, 1.00].

Only 4.50% of the teacher's ProgramBench solutions reach a 95% test pass rate. Among the retained training rollouts, 24.45% reach this threshold, while 75.55% remain below it. This supervision profile may help explain the limited high-fidelity gains of the original SFT model.

Using the same teacher, we selected 1,204 rollouts with a test pass rate ≥50% and repeated SFT (see the Training experiments table in our W1 & Q1 response). With Claude Code fixed, Almost Resolved rises to 4.00%, above the base model's 3.50%, while Resolved returns to 0.50%, matching the base model. This provides evidence of a training gain in near-complete reproduction and supports Program-Env's use for verifier-guided trajectory selection.

We will release the constructed environments and data to support further research on post-training for program reproduction.

[1] Negin Raoof et al. OpenThoughts-Agent: Data Recipes for Agentic Models. arXiv:2606.24855, 2026. https://arxiv.org/abs/2606.24855.

Revision: We have added the stricter-filtering SFT results to Table 2 and the corresponding analysis to Section 6.2, Effectiveness of ProgramGym (RQ1).

Thank you again for your feedback. Please let us know if any concerns remain unaddressed. We would be happy to discuss them further.

Dear Reviewer 3mzt,

We sincerely appreciate your time and effort in reviewing our work and providing valuable feedback.

We have carefully addressed your concerns through detailed responses and additional experiments, and have updated our manuscript accordingly. We would be grateful if you could take another look at our rebuttal at your earliest convenience and share any further feedback or remaining concerns. We would be happy to provide additional clarification if needed.

Thank you again for your time and consideration. We look forward to hearing from you soon.

Best, The Authors of Submission #4472

Dear Reviewer 3mzt,

We have made substantial efforts to address your concerns, including additional experiments and detailed responses.

As the author–reviewer discussion is about to end, we would greatly appreciate it if you could review our responses and reconsider your rating if you find that your concerns have been adequately addressed.

Thank you for your time and consideration.

Best,
The Authors of Submission #4472

Reviewer_iZwsOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces ProgramGym, a framework for scaling verifiable training environments (Program-Env) and improving agent workflows (Program-Agent) for long-horizon program reproduction. The authors construct 1,648 training environments and demonstrate improvements through SFT and structured agent workflows on ProgramBench.

Critical strengths
  • Important research direction. ProgramBench is a challenging long-horizon agent benchmark of growing interest to both academia and industry, making the scaling of training environments for this task a valuable research direction.

  • Well-designed pipeline and comprehensive evaluation. The environment construction pipeline is clearly described and thoughtfully designed to produce high-quality, verifiable tasks. The experiments are also extensive, covering a variety of open- and closed-source models.

Critical weaknesses
  • The 1,648 environments yield 64,505 training trajectory segments, which seems substantial. Could the authors provide the per-environment distribution and clarify how context compaction, session switching, or multi-agent execution contributes to this number?

  • Why are several closed-source models evaluated with mini-SWE-agent rather than Claude Code or Program-Agent? Moreover, Program-Agent yields only a marginal improvement for DS-V4-Flash-0731 (+1.29 points), while its improvement for Qwen3.8-27B (+20.84 points) substantially exceeds the SFT gain (+7.97 points). This raises questions about how much of the improvement comes from the generated training data versus the harness itself, and why the harness benefits vary so significantly across models.

Questions for authors

See Weaknesses

3 comments
AuthorsOpenReview ↗

W1: Training rollouts, context changes, and trajectory segmentation(1/2)

Thanks a lot for your detailed feedback. We really appreciate your recognition of our work and also appreciate your time and effort in providing insightful suggestions that can help further polish our paper.

We clarify how the training trajectories are constructed and provide additional analyses to explain Program-Agent's performance gains across models.

W1. How do 1,648 environments produce 64,505 training trajectory segments?

We use DS-V4-Flash-0731 with Claude Code to perform rejection sampling on 1,648 training environments. For each environment, we repeatedly sample rollouts until obtaining a trajectory with a non-zero test pass rate, and retain one such rollout.

We record all model requests and responses from each retained rollout. Claude Code may change the request context in two ways:

  1. System Prompt Update: Claude Code may update the system prompt, for example, to indicate that previous file observations are no longer valid after a tool modifies the file.
  2. Context compaction: When the context reaches a predefined threshold, Claude Code summarizes earlier interactions into a new system prompt and user message, then continues with the compressed context.

We split each rollout whenever the request context changes, ensuring that each training segment preserves the context seen by the model during rollout. Previously supervised assistant responses are loss-masked when reused as historical context. After context compaction, new segments are constructed from the compressed context.

The 1,648 retained rollouts yield 64,505 training segments, averaging 39.1 segments and 38.1 context changes per rollout. Each segment contains approximately 51 interactions, with about 4 providing new supervision. Neither session switching nor multi-agent execution is used during trajectory collection. Additional segments are created by these two context-change mechanisms.

Revision: We have expanded Appendix D.1 to clarify rejection sampling, context changes, and trajectory segmentation. We have also added Figure 19 to illustrate segmentation and loss masking. All additions are highlighted in blue.

Thank you again for your feedback. Please let us know if any concerns remain unaddressed. We would be happy to discuss them further.

AuthorsOpenReview ↗

W2: Harness comparisons, training effects, and model-dependent gains(2/2)

Thanks a lot for your detailed feedback. We really appreciate your recognition of our work and also appreciate your time and effort in providing insightful suggestions that can help further polish our paper.

W2-Q1. Why are closed-source models evaluated with mini-SWE-agent rather than Claude Code or Program-Agent?

The closed-source model results are taken directly from the official ProgramBench evaluation. We evaluate DS-V4-Flash-0731 and Qwen3.8-27B using Claude Code and Program-Agent.

Under our 256K context setting, mini-SWE-agent frequently encounters context overflow because it lacks automatic context compaction. We therefore use Claude Code as our baseline and compare it with Program-Agent under the same model, resource limits, and time budget.

W2-Q2 & Q3. How much of the improvement comes from training data versus the harness, and why do the gains differ across models?

  • Training Data: With Claude Code fixed, SFT improves Qwen3.8-27B from 34.87% to 42.84% in Tests Passed (+7.97 pp).
  • Program-Agent: With the base Qwen3.8-27B fixed, switching from Claude Code to Program-Agent improves Tests Passed from 34.87% to 55.71% (+20.84 pp).

Both training data and the harness independently improve performance. SFT provides an additional +0.54 pp under Program-Agent. This smaller gain may result from differences between the Claude Code training trajectories and the Program-Agent workflow, as well as overlapping benefits from SFT and the harness.

Additional evaluation on another model. To further assess Program-Agent across models, we evaluate Qwen3.8-Flash-Next on ProgramBench with Claude Code and Program-Agent. This comparison holds the model fixed and changes only the harness. All Δ values below are relative to Claude Code and are measured in percentage points.

Agent HarnessTests Passed (%)ΔAlmost Resolved (%)ΔResolved (%)Δ
Claude Code22.32—1.50—0.00—
Program-Agent47.44+25.124.50+3.000.50+0.50

Program-Agent improves all three metrics for Qwen3.8-Flash-Next, providing further evidence of its benefits across models and at higher reproduction thresholds.

Revision: We have added the Qwen3.8-Flash-Next results under Claude Code and Program-Agent to Table 2 and the corresponding analysis to Section 6.2, Effectiveness of ProgramGym (RQ1). The additions are highlighted in purple.

Program-Agent provides two key benefits:

  1. Role-Isolated Iteration: Separating Analysis, Implement, and Review enables systematic refinement of program functionality.
  2. Revision Management: Compilation checks, retries, and recovery of valid revisions improve delivery reliability.

To explain the different gains across models, we group tasks by how their Tests Passed scores change from Claude Code to Program-Agent:

Sample CategoryDS-V4-Flash-0731: Ratio (Contribution)Qwen3.8-27B: Ratio (Contribution)
Zero → Positive2.50% (+1.34 pp)36.00% (+22.84 pp)
Positive → Positive93.00% (+1.77 pp)55.50% (+0.19 pp)
Positive → Zero3.00% (−1.83 pp)3.50% (−2.19 pp)
Zero → Zero1.50% (0.00%)5.00% (0.00%)
All Samples100.00% (+1.29 pp)100.00% (+20.84 pp)

Contribution measures each category's contribution to the overall change in mean Tests Passed. Values are rounded.

For Qwen3.8-27B, Program-Agent increases the proportion of tasks with non-zero scores from 59.0% to 91.5%. The Zero → Positive category accounts for 36.0% of tasks and contributes +22.84 pp, explaining most of the overall +20.84 pp gain. This pattern suggests that Qwen3.8-27B may benefit substantially from Revision Management, which improves delivery reliability through compilation checks, retries, and revision recovery.

For DS-V4-Flash-0731, 96.0% of tasks already achieve non-zero scores under Claude Code. Zero → Positive tasks contribute +1.34 pp, while Positive → Zero tasks contribute −1.83 pp, largely offsetting each other. The Positive → Positive category contributes +1.77 pp to the overall score, suggesting that the gains for DS-V4-Flash-0731 are more closely associated with the functional refinement enabled by Role-Isolated Iteration.

Fan et al. [1] and Anthropic [2] likewise report that harness benefits vary with model capability. Our results suggest that Program-Agent's gains depend on the base model's capability.

[1] Run-Ze Fan et al. An Empirical Study of Harness Design for Coding Agents. arXiv:2609.20804, 2026. https://arxiv.org/abs/2609.20804 [2] Anthropic. Scaling Managed Agents: Decoupling the brain from the hands. April 8, 2026. https://www.anthropic.com/research/managed-agents

Thank you again for your feedback. Please let us know if any concerns remain unaddressed. We would be happy to discuss them further.

Dear Reviewer iZws,

We sincerely appreciate your time and effort in reviewing our work and providing valuable feedback.

We have carefully addressed your concerns through detailed responses and additional experiments, and have updated our manuscript accordingly. We would be grateful if you could take another look at our rebuttal at your earliest convenience and share any further feedback or remaining concerns. We would be happy to provide additional clarification if needed.

Thank you again for your time and consideration. We look forward to hearing from you soon.

Best, The Authors of Submission #4472

Reviewer_FRmkOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

ProgramGym introduces a framework for training and running agents that reproduce programs from reference executables and documentation without accessing original source code. It contains two components: Program-Env, which constructs verifiable training environments, and Program-Agent, which supports long-horizon development through separate analysis, implementation, and review sessions.

Program-Env combines automated build repair, parallel test generation, and static-analysis guidance to construct 1,648 environments across C, C++, Go, and Rust. The resulting corpus contains 665,512 validated tests with 72.99% mean coverage. Test validation checks reference-program consistency, stability, and assertion strength. Program-Agent preserves behavioral evidence through persistent handoff documents and uses independent compilation, checkpoints, and rollback to maintain buildable implementations.

Experiments on ProgramBench show that supervised fine-tuning on collected trajectories raises Qwen3.8-27B’s average test pass rate from 34.87% to 42.84%. Program-Agent further improves the base and fine-tuned models to 55.71% and 56.25%, respectively. Increasing implementation rounds also improves performance, supporting the value of iterative review and revision.

The study also analyzes construction costs, iteration budgets, and changes in agent behavior. However, improvements primarily reflect partial behavioral correctness: fine-tuning adds little under the proposed workflow, and full resolution remains at 0.50%. The evidence for improved complete program reproduction is therefore limited, despite the substantial average gains.

Critical strengths

1 - This paper contributed a meaningful amount program bench targeted training resources such as 1,648 functioning training environments. 2 - Clear gains from the structured agent workflow. The proposed workflow leads to a substantial improvement with the underlying model held fixed: Program-Agent raises Qwen3.8-27B’s average test pass rate from 34.87% to 55.71%. The improvement also extends to the fraction of tasks passing at least 75% of tests, which increases from 18% to 34%. I find the analysis of iteration budgets and token costs useful, as it provides additional evidence for the practical benefits of the workflow. Together, these results make a convincing case for the usefulness of structured analysis, implementation, and review in long-horizon program reproduction.

Critical weaknesses

Overall I think the technical contribution from this paper is limited and thus I suggest a rejection.

1 - The technical contribution over the original ProgramBench paper is limited. The central iterative test-generation and validation loop has been introduced in the ProgramBench paper already.

2 - The main metric that we should track for ProgramBench should be the almost solve rate instead of the mean test pass rate. With Claude Code, SFT adds 7.97 points to average tests passed, but almost-resolved performance falls from 3.50% to 1.50%, and full resolution falls from 0.50% to 0%. With Program-Agent, SFT adds only 0.54 points, while almost-resolved performance falls from 5.50% to 4.50%, and full resolution remains 0.50%.

3 - For training SFT traces, it's hard to say whether the gain is from the environment or it's from distillation.

Questions for authors

1 - Explain why SFT improved average test passing while reducing almost-resolved performance. 2 - What exactly is new relative to ProgramBench’s environment-construction pipeline?

4 comments
AuthorsOpenReview ↗

W1/Q2: Technical contributions beyond ProgramBench construction(1/2)

Thanks a lot for your detailed feedback. We really appreciate your recognition of our work and also appreciate your time and effort in providing insightful suggestions that can help further polish our paper.

For W1&Q2: Novelty over ProgramBench. The iterative test-generation and validation loop already appears in ProgramBench, leaving little technical contribution beyond it.

Constructing a program-reproduction task has two parts, the reference executable and the tests. ProgramBench’s coverage-guided iteration and validation checks are necessary components. In our engineering practice, however, they are not sufficient. Three problems arise, and each of our additions addresses one of them.

1. Coverage alone tells the agent that something is uncovered, not how to reach it.

(1) Coverage guidance alone is not enough. Black-box tests reach code only through the external interface, so the agent often cannot find an entry to an uncovered line and ends up generating duplicate tests. We therefore add queryable static analysis tools (Joern, CodeQL) that traverse the call graph backward from an uncovered function to a public entry. In Figure 12 (bottom), this revealed the lightweight entry rotation_averaging to a target the agent had abandoned as unreachable, raising module coverage from 36.41% to 45.47%.

(2) Query tools plus a coverage metric are still not enough. Raw coverage reports run to tens or hundreds of MB, and the raw outputs of static analysis and validation have low information density. We therefore distill all feedback into structured, rule-based documents. Validation output is summarized into a JSON of location, issue, and suggested fix. The agent can thus quickly locate which tests to repair, which functionality to cover, and how to reach it in every round. Figure 12 and Figure 13b (coverage 4.88% → 85.77%) show the effect of this design.

2. A single agent cannot stay focused on a whole repository. Source trees are large, and an agent given the entire repository tends to lose track of what remains. We introduce task decomposition, which splits the repository into n cohesive functional modules. Each module is assigned to a parallel sub-agent that writes tests for that module alone, which reduces exploration cost and keeps the agent focused (Appendix B.3.1).

3. Building the reference executable also needs feedback. Resolving build environments is hard. Compiling a GitHub repository from its README succeeds on the first attempt for only 65% of our environments (1,070 of 1,648, Figure 9). Discarding the failures would waste substantial sunk cost. Our build loop therefore returns structured diagnostics that extract the failure cause and attach a suggested fix, which recovered the remaining 35% within four attempts. Figure 10 shows one such repair.

Thank you again for your feedback. Please let us know if any concerns remain unaddressed. We would be happy to discuss them further.

AuthorsOpenReview ↗

W2&W3&Q1: Teacher supervision, trajectory selection, and SFT results(2/2)

Thanks a lot for your detailed feedback. We really appreciate your recognition of our work and also appreciate your time and effort in providing insightful suggestions that can help further polish our paper.

For W2 & W3& Q1: SFT improves mean Tests Passed but reduces Almost Resolved and Resolved. What explains this outcome, and how much of the training gain comes from the environments versus teacher distillation?

Effective distillation depends on the quality of both the training tasks and the teacher trajectories[1]. Program-Env provides executable program-reproduction tasks and behavioral verifiers for selecting trajectories. In our original SFT experiment, we used DeepSeek-V4-Flash-0731 with Claude Code as the teacher and retained rollouts with any non-zero test pass rate. Consequently, the training set included many partially successful demonstrations, whose quality may limit the high-fidelity behavior learned through SFT.

The table below reports the teacher's score distribution on ProgramBench and among the retained Program-Env training rollouts. The [0.95, 1.00] column reports the subset reaching Almost Resolved and overlaps with the [0.75, 1.00] column.

Dataset[0.00, 0.25)[0.25, 0.50)[0.50, 0.75)[0.75, 1.00][0.95, 1.00]Total
ProgramBench31 (15.50%)37 (18.50%)76 (38.00%)56 (28.00%)9 (4.50%)200 (100.00%)
ProgramGym217 (13.17%)227 (13.77%)367 (22.27%)837 (50.79%)403 (24.45%)1648 (100.00%)

Only 4.50% of the teacher's ProgramBench solutions reach a test pass rate of at least 95%. Among the retained training rollouts, 24.45% reach this threshold, while 75.55% remain below it. Thus, most of the original training demonstrations did not achieve near-complete reproduction according to their test suites. This supervision profile may help explain why the original SFT gains were concentrated in mean Tests Passed rather than high-fidelity reproduction.

We further used Program-Env's behavioral test suites to select 1,204 teacher rollouts with a test pass rate ≥50% and repeated SFT, obtaining Qwen3.8-27B-SFT-1204. All models below are evaluated on ProgramBench with Claude Code. All Δ values are relative to the base Qwen3.8-27B model and are measured in percentage points.

ModelTests Passed (%)ΔAlmost Resolved (%)ΔResolved (%)Δ
Qwen3.8-27B34.87—3.50—0.50—
Qwen3.8-27B-SFT42.84+7.971.50−2.000.00−0.50
Qwen3.8-27B-SFT-120447.53+12.664.00+0.500.500.00

With the same teacher, stricter filtering improves Tests Passed and Almost Resolved while restoring Resolved to the base model's level. This supports the value of Program-Env's verifiers for trajectory selection.

We will release the environments and data constructed through Program-Env so that the community can investigate other teachers, trajectory-selection strategies, and post-training methods for program reproduction.

[1] Negin Raoof et al. OpenThoughts-Agent: Data Recipes for Agentic Models. arXiv:2606.24855, 2026. https://arxiv.org/abs/2606.24855.

Revision: We have added the stricter-filtering SFT results to Table 2 and the corresponding analysis to Section 6.2, Effectiveness of ProgramGym (RQ1).

Thank you again for your feedback. Please let us know if any concerns remain unaddressed. We would be happy to discuss them further.

Dear Reviewer FRmk,

We sincerely appreciate your time and effort in reviewing our work and providing valuable feedback.

We have carefully addressed your concerns through detailed responses and additional experiments, and have updated our manuscript accordingly. We would be grateful if you could take another look at our rebuttal at your earliest convenience and share any further feedback or remaining concerns. We would be happy to provide additional clarification if needed.

Thank you again for your time and consideration. We look forward to hearing from you soon.

Best, The Authors of Submission #4472

Dear Reviewer FRmk,

We have made substantial efforts to address your concerns, including additional experiments and detailed responses.

As the author–reviewer discussion is about to end, we would greatly appreciate it if you could review our responses and reconsider your rating if you find that your concerns have been adequately addressed.

Thank you for your time and consideration.

Best,
The Authors of Submission #4472

Comments on the paper

1 comment
Area_Chair_ae3qOpenReview ↗

Please engage in the discussion

Dear Reviewers,

This paper receives diverse assessments (current ratings: 1,2,4). I would greatly appreciate if you can read the authors' responses and update your evaluations. In particular, please indicate what concerns have been addressed and whether you have remaining concerns.

As you know, we have only three reviewers in total. So each of your reviews may have a crucial impact on the final decision. Let's be professional.

Regards,

Your AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.