- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SEER connects multiple code-generation attempts through test feedback and reflection. It uses rewards from individual attempts and the overall search to give credit to failed attempts that help later ones succeed. It also trains on successful programs found during search, paired with the original problem, to improve direct generation without the search history. Experiments cover several code benchmarks and models from 3B to 14B, with additional model families evaluated in the appendix.
Critical strengths
-
The paper evaluates both whether search finds correct programs and whether these discoveries improve direct generation. This distinction is useful and makes the role of distillation easier to assess.
-
The experiments cover several model sizes and code benchmarks, with relevant baselines and useful ablations. They provide reasonable support for the method's effectiveness.
Critical weaknesses
-
The method appears to combine LaMer-style learning across attempts with supervised training on successful outputs. Both ideas have been studied before, and the paper does not clearly explain what is new beyond their combination.
-
The reward design relies on several heuristics without sufficient justification. The limited search analysis leaves it unclear how sensitive the results are to these choices or whether they generalize to more complex coding tasks.
Questions for authors
-
What accounts for SEER's additional gains over LaMer-style training followed by supervised learning on the same successful programs? Does the successful-sample replay baseline match the source of the training examples, how often they are reused, and the total training budget?
-
How does reintegration relate to On-Policy Self-Distillation (OPSD)? The objectives differ, but both aim to transfer abilities available with extra context to a policy given only the original problem. Why use supervised learning on successful programs, and what does this add relative to existing distillation methods?
4 comments
Thank you again for your constructive feedback. In our rebuttal, we added budget- and data-matched LaMer→SFT comparisons, controlled reintegration experiments, and reward/advantage ablations, further clarifying SEER’s contributions beyond combining existing techniques.
We also included harder-task hidden-test evaluations and direct comparisons with OPSD, including performance and computational costs. We would greatly appreciate your thoughts on whether these additional results adequately address your concerns.
Thanks for the authors' response and the extended experiments, I therefore raised my score.
We sincerely thank the reviewer for their thoughtful feedback and for taking the time to evaluate our revisions. We are glad that our additional experiments and clarifications addressed your concerns. We greatly appreciate your consideration and your decision to raise the score.
We thank the reviewer. We added reward/advantage ablations, replay, harder-task/hidden-test, and OPSD experiments to Appendices B, K, and L. Scores are %; H = hidden; runs differ.
W1/Q1. Contribution and Matched Replay
We agree cross-attempt learning and successful-output SFT are not new individually. SEER targets coding agents' first-answer ability and recovery after failure. SEER is motivated by two successive bottlenecks in the RLVR code.i) With limited rollouts, RL reinforces sampled solutions; ii) more rollouts increase search volume, not how to search. SEER links attempts via verifier feedback/reflection and credits productive failures. Search-discovered solutions may remain unlikely from the original prompt, so SEER reintegrates the first verified discovery under it.
On 12 matched HumanEval29 cases, SEER recovers 75.0% vs. 66.7% for StepCoder; resetting history or removing verifier feedback gives 66.7% (Fig. 7).
APPS-hard Direct@1 / Selected@1,@4
Long context, replay, or immediate credit alone cannot reproduce SEER:
Matched LaMer→SFT: same no-distill step-300 checkpoint; 247 identical successes/160 problems, 4 replays each (988 exposures), 247 updates, 140,232 target tokens. Evaluation uses only the original prompt.
With successes fixed, context matters:
Original-prompt SFT beats search-context SFT by 3.00 pp; on source-160 it recovers 9/84 failures, loses 0/76 successes (net +9/160). SFT/SEER replay: APPS 980/139,400 vs. 980/139,392; MBPP 1,158/131,202 vs. 1,158/131,198 (exposures/target tokens).
W2. Reward Sensitivity and Harder Tasks
Reward and advantage ablations are in Appendix K.
Advantage terms target first-answer quality, cross-attempt credit, and trajectory optimization:
On harder APPS (300 Competition), shaped rewards improve hidden-test Oracle over strict 0/1 rewards:
Normalization lowers length–token-weighted-credit correlation 0.646→0.556; long-first-attempt recovery (≥256 tokens) rises 4.6%→11.6%.
Q2. OPSD and Why SFT
SEER SFTs verifier-confirmed code under the original prompt; OPSD-KL uses a privileged-context teacher for on-policy outputs. Paired methods share fixed successes, timing, and replay; Full SEER retains dynamic discovery. Qwen2.5-Coder-3B-Instruct, 300 updates, 1 seed; APPS+: 4,063/300 train/val; MBPP: 374/90 train/val, 224 tests.
*MBPP90 is step 200. OPSD-KL gains 3.05 pp on HumanEval164 over SFT, ties on APPS300, and gains only 0.11/0.44 pp on MBPP/MBPP+. Full SEER performs best, including dynamic discovery gains.
Forward-token ratios exclude generation, backpropagation, and KL (not FLOPs); wall-clock is end-to-end. Verified code provides exact targets without extra student sampling/full-vocabulary KL. OPSD-KL offers no consistent gain at ~1.9× cost; SFT is simpler and cheaper.