acceptodds
Under review as a conference paper at ICLR 2027

LLM-as-an-Improver: Turning Verification into Better Candidates

Abstract

Verifier-based selection chooses the most promising solution from a pool of candidates generated at inference time. Because it only ranks a fixed candidate pool, however, it must fail when every initial candidate is incorrect, even if verification identifies useful failure evidence. We introduce LLM-as-an-Improver and propose Verify–Repair–Reselect (VRR), which reuses that evidence to expand the candidate set before final selection. VRR retains the initial winner while conditionally generating complementary alternatives, screens invalid and duplicate outputs, and then reselects from the retained winner and the surviving alternatives. Our experiments show that VRR can recover correct solutions even when all candidates in the initial pool are incorrect. These results demonstrate the feasibility of using verification feedback not only as a ranking signal but also as a generation signal for constructing correct candidates beyond the initial pool.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, test-time scaling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CGETOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Verify-Repair-Reselect (VRR), a test-time scaling method that reuses the diagnostic evidence produced during verification to generate new candidates. The author observe that fixed-pool selectors such as LLM-as-a-Verifier rank a fixed candidate pool using LLM judgements, and therefore fail whenever every initial candidate is incorrect. VRR tackles this problem by reusing the diagnostic evidence produced during verification to generate repaired and alternative candidates beyond the initial pool, and then reselecting the final answer from the initial winner and these new candidates under the same criteria. Across two models and eight benchmarks, VRR achieves better accuracy than LAV in 9 of 16 settings.

Critical strengths
  • The experimental configuration and the prompts are disclosed in detail, which supports reproducibility.

  • The paper is carefully written and easy to follow. Its claims are appropriately measured: the authors explicitly refrain from claiming statistical significance, and they report negative outcomes.

Critical weaknesses

Novelty is limited, and the discussion of closely related prior work is insufficient.

The central research question of the paper, “Can verification also guide the construction of better candidates?”, substantially overlaps with existing test-time scaling methods that construct new candidates beyond the initial pool using information from previous candidates. These methods are neither discussed nor compared against. For example, parallel-distill-refine [1] combines LLM-as-a-judge tournament selection with Parallel-Distill-Refine, conditioning new rollouts on the summaries selected from prior attempts. In addition, Recursive Self-Aggregation [2] aggregates subsets of a candidate population into improved solutions. Considering this research, formalizing the fixed-pool ceiling carries limited novelty, and the remaining contribution would need to be established through direct comparison with these methods.

Furthermore, the paper describes its question as complementary to feedback-guided refinement methods, such as Self-Debugging. However, it is not clear that the two are complementary rather than competing. Iterative refinement methods such as Self-Debugging may achieve similar accuracy gains by themselves, without reusing ranking-oriented verification evidence. The authors should evaluate which approach yields higher accuracy at matched compute. If the approaches are truly complementary, it would be better to quantify the gain from combining them. Overall, the coverage of prior work and the quantitative comparison against it are insufficient, and the paper’s contribution is not yet well established.

Generation and verification costs are not controlled, so the source of the gains cannot be identified.

The comparison with LAV in Table 2 shares only the initial pool. Beyond that, two methods differ in several ways, including the number of total candidates, additional costs for generating criteria, and selector costs. Consequently, it is difficult to tell whether VRR's accuracy gains come from (i) generating more candidates, (ii) the collected failure evidence, or (iii) the improved verification procedure. I recommend experiments that control at least one of these factors. Especially, I’m interested in a comparison with a matched number of candidates, which I believe is essential for demonstrating the effectiveness of a test-time scaling method.

[1] Kim et al., Scaling Test-Time Compute for Agentic Coding.

[2] Venkatraman et al., Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models.

Questions for authors
  • Figure 1 appears to show the reselection pool as an expanded set containing both the initial candidates (c1–c3) and all repair candidates (r1–r3). In contrast, Algorithm 1 retains only the initial winner and the repair candidates that pass screening. Which reflects the actual behavior?

  • In Table 2, the total number of candidates seen by LAV and VRR does not appear to be matched. Do the authors have results where the number of candidates is matched?

  • How does VRR perform on agentic tasks such as SWE-bench Verified and Terminal Bench, where candidates are long trajectories rather than short programs or answers?

3 comments

We thank the reviewer for recognizing the detailed disclosure of our experimental configuration and prompts, the clarity of the presentation, and our measured reporting of both positive and negative results.

W1: Novelty is limited, and the discussion and comparison with closely related methods such as Parallel-Distill-Refine, Recursive Self-Aggregation, and Self-Debugging are insufficient.

A-W1: Our intended novelty is not the general idea of using previous attempts to generate new ones. Rather, VRR operationalizes evidence originally produced for candidate ranking within a retain–repair–screen–reselect procedure: it retains the initial winner, generates complementary alternatives from the winner and runner-up, screens them, and reselects under the original candidate-blind criteria.

This differs from Recursive Self-Aggregation [2], which aggregates candidate subsets, and from single-trajectory refinement methods such as Self-Debugging, which iteratively revise one solution. VRR maintains the incumbent as an explicit option and requires generated alternatives to compete with it during reselection. Parallel-Distill-Refine [1] is more closely related because it combines tournament selection with conditioned generation; we will discuss this overlap directly and revise our novelty claim to focus on VRR’s specific integration of ranking-oriented verification, conditional multi-role generation, screening, and incumbent-preserving reselection.

We did not perform direct reproductions of PDR or RSA and will not claim empirical superiority over them. Instead, we added controlled comparisons with independent sampling and with the same role-based generation without criterion-level diagnostic information.

W2: Generation and verification costs are not controlled, so it is unclear whether the gains come from additional candidates, failure evidence, or the different verification procedure.

A-W2: We added controls using the exact same VRR initial selector, trigger set, generation count, 4,096-token output limit, screening procedure, and final selector:

MethodSeed 0Seed 1Mean backend callsMean prompt tokens
Exact VRR initial selector118/253 (46.64%)118/253 (46.64%)7,71032.58M
VRR with diagnostics127/253 (50.20%)128/253 (50.59%)9,36743.47M
Independent sampling124/253 (49.01%)126/253 (49.80%)8,38535.21M
Same roles without diagnostics128/253 (50.59%)129/253 (50.99%)9,27841.34M

These results establish that adding generation, screening, and reselection improves upon the exact same initial selector, so the gain is not solely caused by the difference between the LAV and VRR selectors. Both role-based conditions also outperform independent sampling with the same number of generation attempts, indicating that targeted, parent-conditioned role generation provides value beyond merely adding samples.

The difference between the two role-based variants is small in this setting. We will therefore distinguish the demonstrated benefit of role-based candidate expansion from the incremental effect of the structured diagnostic ledger, for which these experiments do not establish an additional gain. Because actual token usage differs with prompt length and the number of candidates passing screening, we report the measured calls and tokens rather than describing the experiment as exactly token-matched.

Q1: Figure 1 shows all initial and repair candidates in the reselection pool, whereas Algorithm 1 retains only the initial winner and screened repair candidates. Which reflects the actual behavior?

A1: Algorithm 1 reflects the actual implementation. The reselection pool is

Creselect={ci∗}∪Rpass,C_{\mathrm{reselect}}=\{c_{i^*}\}\cup R_{\mathrm{pass}},

where ci∗c_{i^*} is the initial winner and RpassR_{\mathrm{pass}} contains the generated candidates that pass screening. The other initial candidates are not included in reselection. Thus, the active pool contains between two and four candidates when at least one repair is admitted.

Q2: Do the authors have results where the number of candidates is matched?

A2: Yes. In the new controls above, every generation condition uses the same fixed 93-task trigger set and attempts exactly three additional candidates per triggered problem. All conditions also use the same 4,096-token output limit, public-test screening, and final selector.

The final admitted pool sizes may differ because candidates can fail the same screening procedure; this is an outcome of generation quality rather than a difference in the attempted candidate budget. Across both seeds, the role-based conditions achieved higher accuracy than independent sampling, while the difference between the two role-based variants was small.

Q3: How does VRR perform on agentic tasks such as SWE-bench Verified and Terminal-Bench?

A3: A direct VRR evaluation on long agentic trajectories requires an executable repair harness that can fork or continue a trajectory, execute the repaired actions in the task environment, and rerun the benchmark evaluator. Our current implementation does not yet provide this complete agentic repair interface, so we do not claim a direct VRR result on these benchmarks.

As a preliminary opportunity analysis, we treated existing frozen trajectories as a smaller initial pool plus held-out additional trajectories:

BenchmarkInitial + held-out trajectoriesInitial oracleExpanded oracleGain
SWE-bench Verified2 + 1408/500422/500+2.80 points
Terminal-Bench3 + 281/8982/89+1.12 points

The held-out trajectory recovered 14 of 92 initially all-wrong SWE-bench pools, while the two held-out trajectories recovered 1 of 8 initially all-wrong Terminal-Bench pools. These results show that the fixed-pool ceiling also occurs for long agentic trajectories, but they are an oracle opportunity analysis—not a VRR accuracy evaluation—because the held-out trajectories were not generated from VRR diagnostics.

Thank you for the detailed response and the new experiments. However, my concerns regarding weakness 1 and 2 are not fully addressed in the rebuttal.

W1: The authors acknowledge the overlap with PDR and narrow their novelty claim to the specific integration of components. However, given this overlap, the rebuttal does not sufficiently justify why a comparison with PDR or RSA is unnecessary, so the contribution relative to prior work remains unclear.

W2: The new controls show that the variant without diagnostics performs on par with full VRR, and that independent sampling recovers most of the gain over the initial selector. This suggests that the improvement mainly comes from additional candidate generation. This means that the benefit of reusing verification evidence is not observed, further narrowing the contribution of the paper.

Thus, I will maintain the current score.

Reviewer_K5QjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Verification based selection use a sampled set of candidate answers at inference-time to provide a ranking score. Existing approaches fail when the sampled answers are all incorrect. So this paper presents LLM-as-an-Improver which uses verification feedback to generate complementary alternatives, remove duplicate outputs, and reselects the initial winner. Results show that VRR can recover correct answer from an entirely incorrect initial pool, using verification feedback to guide generation as well as ranking.

Critical strengths

The methodology part is very comprehensive and easy to follow.

Critical weaknesses

The core motivation and novelty of the paper (ranking approaches fail when all candidate answers are incorrect is weak since we can increase the sample size, The problem is not that strong when it is compared to the cost of the proposed solution.

The performance results show no gain when the initial candidate pool has the correct answer. This limits the contribution of the proposed method.

Sampling itself is expensive, the proposed approach adds to this complexity. It is worth discussing the complexity of the proposed approach.

The manuscripts lacks sufficient comparisons using recent models e.g., Qwen3.8-27B and GLM-5.3-Flash which might show better initial pool of candidates.

Questions for authors

Failures cases analysis, why the improvement for the cases when the initial candidates are all incorrect are modest?

Complexity analysis and whether the extra cost outweigh the performance gains or not

1 comment

We thank the reviewer for highlighting the practical questions of whether targeted repair improves upon additional sampling and whether its additional cost is justified. We added a controlled sampling baseline, a failure-stage analysis, and detailed call/token accounting.

W1: The motivation is weak because an all-incorrect candidate pool can be addressed by increasing the sample size.

A-W1: We directly tested this alternative. Using the same trigger set, three additional generations, 4,096-token limit, screening, and final selector, the role-based conditions consistently outperformed three independent additional samples:

MethodSeed 0Seed 1
Exact VRR initial selector118/253 (46.64%)118/253 (46.64%)
VRR with diagnostics127/253 (50.20%)128/253 (50.59%)
Independent sampling124/253 (49.01%)126/253 (49.80%)
Same roles without diagnostics128/253 (50.59%)129/253 (50.99%)

Thus, the improvement is not explained solely by increasing the sample size. Targeted, parent-conditioned generation and role diversity recovered more problems under the same additional-candidate budget.

W2: The method shows no gain when the initial pool already contains a correct answer, which limits its contribution.

A-W2: VRR is intended primarily to address the fixed-pool ceiling rather than uniformly improve already-covered problems. Its repair stage is conditional: only 12 of the 136 mixed-pool problems triggered repair, and aggregate correctness on this subset was preserved. In contrast, VRR recovered 18 problems from the all-incorrect subset, where any fixed-pool selector has zero attainable accuracy. We will clarify that the contribution is targeted recovery on unreliable or uncovered pools, rather than uniform improvement across all pool types.

W3: Sampling is already expensive, and VRR adds further computational complexity.

A-W3: In the primary configuration, repair was triggered for 93 of the 253 analyzed problems rather than applied uniformly. Backend calls increased by 26.77%, prompt tokens by 42.14%, and completion tokens by 67.13%, while accuracy increased from 46.64% to 53.75% on these problems. We will present VRR as an explicit accuracy–compute trade-off and avoid claiming that it is compute-optimal for every setting.

W4: The evaluation lacks comparisons using more recent models that may produce stronger initial candidate pools.

A-W4: Stronger models may reduce the frequency of all-incorrect pools, but they do not remove the structural limitation of fixed-pool ranking: no selector can recover an answer absent from its finite candidate pool. VRR’s conditional trigger also reduces repair usage when the initial pool is judged reliable. Our experiments span two distinct model families, Gemma 4 31B IT and Qwen3-32B, across eight benchmarks. We agree that evaluation with additional recent models would strengthen the empirical coverage and will identify this as future work.

Q1: Why are the improvements on all-incorrect pools modest? Please provide a failure-case analysis.

A1: We analyzed all 279 repair attempts:

Candidate outcomeCount
No complete program generated22
Compilation failure4
Public-test failure155
Passed screening98

At least one correct candidate was generated for 30 triggered problems. The final selector missed a correct generated candidate in only one problem. On the all-incorrect subset specifically, all 18 problems for which a correct repair was generated were selected correctly.

Therefore, the primary bottleneck is generating and validating a correct candidate, rather than final reselection.

Q2: What is the computational complexity, and do the performance gains outweigh the additional cost?

A2: The exact VRR initial selector solved 118/253 problems (46.64%), whereas the full repair-and-reselection procedure solved 136/253 (53.75%), an improvement of 7.11 percentage points.

MeasureInitial selectorFull VRRIncrease
Backend calls7,7039,765+26.77%
Prompt tokens32.60M46.33M+42.14%
Completion tokens1.52M2.54M+67.13%

These measurements quantify the additional cost but do not establish universal compute optimality. Whether the gain justifies the cost depends on the deployment setting and the value assigned to recovering difficult problems.

Reviewer_fJ9uOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Verify-Repair-Reselect (VRR), an inference-time procedure that reuses verification feedback to construct additional candidate solutions. Starting from a fixed candidate pool, it ranks candidates under task-derived criteria and conditionally generates three alternatives: repairs of the selected candidate and the runner-up, and a solution using a new approach. After validity checks and deduplication, it reselects from the initial winner and the surviving alternatives. The same model performs generation, verification, repair, and reselection.

The central empirical claim is recovery beyond the initial candidate pool. On LiveCodeBench, VRR recovers 18 of 117 all-incorrect pools with Gemma 4 31B IT and 7 of 215 with Qwen3-32B. Across eight benchmarks and two models, it also has higher point accuracy than fixed-pool LAV selection in nine of sixteen settings, with four ties and three decreases.

Critical strengths
  1. The paper addresses a concrete limitation of fixed-pool selection and provides a complete, training-free procedure for reusing diagnostic feedback. Retaining the initial winner for reselection is a sensible design choice that keeps the original answer available while allowing new candidates to compete under the same evaluation criteria.

  2. The experiments share frozen initial candidate pools and withhold reference answers, correctness labels, and private tests during inference. Section 4.1 and Appendix C demonstrate recovery beyond the initial pool. Table 1 clearly separates these recoveries from final errors on initially covered problems, and the text correctly notes that covered errors are not necessarily repair-induced regressions.

Critical weaknesses

1 Limited demonstrated benefit for test time scaling

The reported recoveries support the paper's stated feasibility claim. My main concern is that the empirical benefit remains insufficiently compelling for the proposed test-time scaling method. In Table 2, VRR improves over LAV in nine of sixteen settings, ties in four, and performs worse in three. Most gains are modest: all improvements with Gemma are at most 1.33 percentage points, and the LiveCodeBench gains are 1.33 and 0.95 points for the two models. Qwen improves by 4.04 points on GPQA Diamond and 2.50 points on AMC, which are encouraging results, but it also declines on MBPP and MATH-500. Appendix A.2 states that each cell is a single end-to-end run and that the results do not establish statistical significance or consistency across seeds.

The paper appropriately distinguishes recovery from aggregate accuracy and treats the runtime measurements in Table 3 as operational records rather than a controlled efficiency comparison (Appendix A.8). These clarifications are useful, but they do not establish whether the gains provide a favorable accuracy-compute trade-off relative to simpler uses of the same additional inference budget. A controlled accuracy-versus-budget evaluation is still missing. Thus, even accepting the feasibility result, the limited and insufficiently characterized practical benefit remains my primary reason for recommending rejection.

2 The comparisons do not isolate the effectiveness of the proposed components

The aggregate comparison with LAV does not isolate the contribution of repair because the verification and selection procedures also differ. LAV uses expected scores over A-T tokens and a Probabilistic Pivot Tournament, whereas VRR uses SAT/VIOL/UNKNOWN judgments, evidence-weighted aggregation, adaptive verification, and a different comparison procedure (Section 3 and Appendices A.4, A.5, and B.7). Sharing the initial candidate pool does not remove this difference. The paper does not report the basic control of using the exact same VRR initial selector while disabling repair and reselection.

For the recovery analysis itself, a fixed-pool selector has zero attainable accuracy on all-incorrect pools by construction. This comparison therefore does not determine the value of diagnostic conditioning relative to other ways of generating additional candidates. Relevant controls would keep the trigger, generation budget, screening, and final selector fixed while comparing VRR with independent additional sampling and with the same role prompts without diagnostic feedback. Initial generation and repair also use different output limits: 4,096 versus 8,192 tokens (Appendices A.1 and A.4). The recoveries demonstrate an end-to-end capability of the complete pipeline, but do not isolate the incremental value of the verification feedback or the three-role design.

Questions for authors
  1. Can the authors report accuracy across comparable total inference budgets, accounting for generation, verification, and reselection? Repeated runs or paired uncertainty estimates, together with consistent batching and caching for any latency comparison, would help establish whether the modest gains in Table 2 translate into a reliable practical advantage.

  2. What is the accuracy of the exact same VRR initial selector before repair and reselection? With the trigger, generation budget, screening, final selector, and available information controlled, how do recovery and overall accuracy compare with independent additional sampling and with the same three role prompts without diagnostic feedback?

2 comments

We thank the reviewer for careful reading and comments.

Q1: Can the authors report accuracy across comparable inference budgets and provide repeated-run or paired uncertainty estimates?

A1: We compared the exact VRR initial selector with three conditions using the same 93-task trigger set, three additional generations per triggered task, public-test screening, and final selector. The table reports the two completed generation seeds:

MethodSeed 0Seed 1MeanMean backend callsMean prompt tokens
Initial VRR selector118/253 (46.64%)118/253 (46.64%)46.64%7,71032.58M
VRR with diagnostic feedback127/253 (50.20%)128/253 (50.59%)50.40%9,36743.47M
Independent additional sampling124/253 (49.01%)126/253 (49.80%)49.41%8,38535.21M
Same three roles without diagnostics128/253 (50.59%)129/253 (50.99%)50.79%9,27841.34M

Relative to the exact initial selector, diagnostic VRR improved accuracy by +3.56 and +3.95 percentage points across the two seeds. The task-bootstrap 95% CIs were +1.19 to +6.32 points for seed 0 and +1.58 to +6.32 points for seed 1.

Thus, the repair-and-reselection pipeline shows a consistent paired improvement over its own initial selector.


Q2: What is the accuracy of the exact VRR initial selector, and how does VRR compare with independent sampling and with the same roles without diagnostic feedback?

A2: The exact VRR initial selector achieved 118/253 (46.64%). The transition results were:

MethodSeedWrong → correctCorrect → wrong
VRR with diagnostics0101
VRR with diagnostics1100
Independent sampling060
Independent sampling180
Same roles without diagnostics0100
Same roles without diagnostics1110

These results show that adding generation, screening, and reselection to the exact same initial selector consistently improves end-to-end accuracy. Therefore, the improvement is not explained only by the difference between the LAV and VRR selectors. Across both seeds, structured role-based generation was more effective than independent sampling.

Reviewer_fJ9uOpenReview ↗

Response to the authors' rebuttal

Thank you to the authors for the detailed response and additional experiments. While the rebuttal helps clarify some technical details, my primary concern regarding the limited novelty and incremental contribution remains unresolved. I still feel that the overall contribution falls slightly short of the bar for ICLR. Therefore, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.