acceptodds
Withdrawn from ICLR 2027

THE WINNER’S CURSE IN LLM PIPELINE SELECTION: HOW CORRELATED CANDIDATE POOLS BREAK BEST-OF-N REPORTING, AND HOW TO CORRECT IT

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Best-of- pipeline selection, running candidate prompts on an -item dev set and reporting the winner's dev score, is the default way an LLM practitioner picks and reports a prompt. That dev score is optimistic: the winner was chosen in part because it got lucky on the dev items, and its true accuracy on fresh data is lower. We measure this winner's curse directly on four correctness matrices spanning three tasks (SST-2, Subj, AG News) and two models (Qwen2.5-1.5B-Instruct, Llama-3.2-3B-Instruct), sweeping from 1 to 100 and from 20 to 200. Bias grows with and shrinks roughly as , reaching 0.088 accuracy points at , on Subj. We then characterize why real candidate pools inflate this bias beyond the independent-candidates textbook case: two effective-candidate-count estimators, a spectral participation ratio and a moment-matching inversion of the observed selection gap, agree that a 105-to-111-candidate pool behaves like roughly 1.5 to 3.5 independent draws. Breaking that count down by candidate provenance turns up a result opposite our own working hypothesis: LLM-generated (APE-style) candidates carry far less effective diversity (-) than either hand-written seed prompts or paraphrase/mutation candidates (- for both), consistently across all three tasks. We compare six ways to report a best-of- winner (no correction, a naive dev split, two extreme-value union-bound corrections keyed to or to , cross-fitting, and a bootstrap plug-in) on bias, mean squared error, and 90% interval coverage; correcting against dominates the independent-candidates union bound on every axis and matches or beats every other method on mean squared error, while the naive -based correction overshoots badly enough to be worse than reporting nothing at all. Resampling the stored matrices at the exact of a real, live EvoPromptGA trajectory reproduces its measured bias on all three tasks, inside a 90% predictive band in every case, which supports treating a one-time correctness matrix plus free resampling as a faithful stand-in for repeated live search. Finally, we derive budget-allocation curves for a fixed evaluation budget : on tasks already near their accuracy ceiling, the optimal split spends almost the whole budget on ; on a task with real headroom left, it keeps paying to search wider even as stays in the low single digits.

Primary area:
datasets and benchmarks
Keywords:
winner’s curse, selection bias, prompt optimization, best-of-N selection, effective sample size, bias correction, large language models**

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 6 reviews, above 2% of the others.
Reviewer_TqKuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the winner's curse in best-of-N prompt selection, where reporting the best candidate's dev score overstates its true accuracy. The paper measures how this bias depends on the number of candidates and the size of the dev set, and argues that correlated candidate pools behave like only a few independent candidates. It proposes a correction based on the effective number of candidates. It also offers guidance on splitting a fixed evaluation budget between more candidates and larger dev sets.

Critical strengths

It is interesting to transfer the idea of the winner's curse to prompt selection domain.

Critical weaknesses
  1. Limited novelty and contribution. Optimistic bias from selecting a winner on its validation data is well-known. The setting is also narrow and out-of-date: single-token classification with two ≤3B models released in 2024. The contribution offers little new knowledge for current practice.

  2. A central claim contradicts the released results. The abstract and conclusion repeatedly claim that two estimators agree on approximately 1.5–3.5 effective candidates. But the paper reports no numerical results for the moment-matching estimator. In the linked repository’s tables/table4_neff.md, its exact estimates at m=100 are 6.3–15.6, which is approximately four to five times the spectral estimates.

Questions for authors
  1. Why did you study two small models released in 2024 and only single-token classification tasks? Do your main conclusions still hold on larger, more recent models and on more complex tasks, such as generation, reasoning or even agentic tasks?

  2. The abstract and conclusion say both estimators give ~1.5–3.5 effective candidates, but your repository's table4_neff.md reports 6.3–15.6 for the moment-matching one. Could you explain this mismatch?

Reviewer_Kj67OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper shows that selective bias on validation sets shrinks sublinearly at m\sqrt{m} (the size of the validation set) and grows linearly in the number of candidate prompts. Using 600 samples, the authors test 1.5-3B models on sentiment, topic and subjective-objective classification, demonstrating that the candidate prompts they generated all correlated with one another, and propose a fix using standard expectation sampling techniques from statistics to try to estimate a Winner's Curse bias term.

Critical strengths

This seems like an interesting problem, and I appreciated doing the background research on some of the selection effects that the Winner's Curse reveals in order to respond to this paper. But this is not intrinsic to the paper itself.

The different strategies for resampling on the dev set are interesting -- but poorly motivated.

Critical weaknesses
  • Novelty: this contribution is neither significant nor novel.

    • To my understanding, the SIREN paper does what this paper asks: find a corrective for adaptive benchmarking. The provided anonymous repo includes a NOVELTY_CHECK.md, which appears to surface results from a literature review with the interpretation that "[t]his is the highest-risk item in the plan and it blocks everything else." and that the risk "needs a human with a browser, and it should happen before any GPU time is spent."
    • The authors do not appear to disclose the fact that they delegated related-work search to AI, though based on this md file it appears that they did not do it themselves.
    • The remaining contribution is about (1) taking correlations between prompts (unsurprisingly, since most are AI-generated, they have a lower effective size) and (2) a bias correction step based on expectations under sampling. The comparison of different bias corrections is indeed an interesting method to estimate over-correction, but there's no evidence comparing this to SIREN.
    • For the record, the finding of low diversity is tautological when no effort was made by the team to make the prompts more diverse. Looking at them myself here it seems clear that they were AI-generated in separate, zero-shot settings from a low-temperature model.
    • The goal of the rest of the paper should be to differentiate against SIREN. I don't see it used as a baseline under the budget settings, or comparisons across established benchmarks for improving performance. What about DSPy optimization as they report? What about using their method for calculating bias? It seems like the work is referenced post-hoc -- that the authors discovered the existence of this paper only after conceiving of the idea themselves, and that, rather than reformulating their work as a response, they simply acknowledged that the scoop is there.
  • Scope is extremely limited: two small models, 600 items on a set of three extremely easy tasks, and no visibility into chosen candidate set KK.

    • The datasets used are sentiment/topic/tone classification based. This was arguably solved already, as shown by PromptGPT. I also find it suspicious that SST for both Qwen2.5-1.5B and Llama3.2-3B has a minimum accuracy of 0.5 (random chance) but median at 0.9. This makes me wonder which prompts are in play. Qwen2.5-1.5B apparently has a zero-shot score of 0.93 on SST-2.
    • How can a paper measuring sampling noise only have one run of 600 evaluation items at temperature 0 without resampling?
    • Why was only Qwen Qwen2.5-1.5B used? I understand compute can be a limitation, but I don't think results at such a scale are informative of anything. Colab offers higher-powered TPUs, so one could at least use a more modern model, like Qwen3.
    • There is arguably a construct validity problem in making a claim about a "Winner's Curse" where the "truth set" is in-distribution and the same size as the "dev set". Why one is ground truth and the other is overfitting is lost on me.
    • In fact, the size of the truth pool being 300 means that standard correctness error on any one would be roughly .02 to .03 (using a Bernoulli SE of 0.5). This is within the bias terms that are reported.
  • The writing is completely — egregiously — AI-generated, according to Pangram.

    • The writing is discontinuous, lacking a central narrative of the problem area and solution. I needed to read the SIREN paper in order to better understand the problem area. There are many lines that do not contribute to the paper. For instance, lines 145-148:

This is what makes an exhaustive sweep of the (N,m) grid, six correction methods, and thousands of bootstrap draws affordable on a free-tier GPU budget: the GPU cost is fixed at one pass per (task, model) pair, and every additional analysis is free.

  • Consider the following line, which is the only line describing the domain of this meta-evaluation:

The three core tasks, SST-2, Subj, and AG News, are the same task set EvoPrompt (Guo et al., 2024) benchmarks against, chosen so measured bias sits next to a published optimizer’s reported gains without an apples-to-oranges caveat (Section 9).

  • The codebase also appears to be AI generated, also according to Pangram. The authors do not disclose this use, and state: "We did not use generative AI tools for any other part of this research." Together, this is a violation of the AI use statement, which requires disclosure of "design or provide feedback on research methodology or experiments, implement methods." Personally, I find this to be independent grounds for rejection.
  • At the very least, this magnifies the lack of confidence I have in this paper being a meaningful contribution at ICLR. When planning to resubmit, I encourage the authors to be more transparent about the role AI plays in the results they share.

Given these considerations, I recommend that this paper be rejected from ICLR. I do not see a version of this paper which can undergo revisions that would not constitute functionally rewriting the paper from scratch. My counterfactual review would be higher if the paper had been:

  1. Written by the authors, and not by an AI (seems like Claude?), and acknowledging use of AI in a more transparent manner.
  2. Sharing results on a larger set of prompts, in harder-to-verify settings, with larger models.
  3. Isolating clearly what the contributions are, and in particular differentiating against the SIREN paper.
  4. Clearer experiments showing what the bias correction actually is, why it beats/is otherwise more interesting than SIREN.
Questions for authors
  • How is there an N_eff of 1.5-3.5 with 100+ candidate prompts? This suggests that the prompts were poorly chosen. We never see any examples of the candidate prompts at the center of this investigation.
    • The authors state: "[e]ach candidate pool mixes three provenance strata: 10 hand-written seed prompts, APE-style candidates generated by prompting an LLM to propose new instructions (Zhou et al., 2023), and paraphrase/mutation candidates produced by perturbing existing candidates." What is the proportion of each of these things? Which prompts correlate most strongly with each other? Are they semantically similar as well?
      • I don't find it surprising that the N_eff is low, since "paraphrase/mutation candidates produced by perturbing existing candidates" could comprise most of the sample set.
      • But how did the "10 hand-written seed prompts" all overlap with each other? If N_eff is 1.3, they must have, right?
    • This, to me, is a deal-breaker; given that all presented results are downstream of these prompts, and we have no validation that the authors made an effort to vary these in order to improve N_eff, I don't believe this is much of a contribution.
  • Why do you have sampling temperature at 0.0 if the point is to capture a "winner's curse"? This might be something I'm missing. I feel like this is not necessary, but would be quite informative. The SIREN paper uses temperature=1. Consider doing the same.
  • Where are the results on estimator B?
Reviewer_h3DaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines the Winner's Curse phenomenon in the selection of prompts and pipelines for LLMs: when researchers select the best-performing solution from a large set of candidate options based on limited validation set scores, the reported performance often overestimates the true performance. The behavioral correlation among candidate solutions may further contribute to this deviation, making the nominal number N of candidates unable to accurately reflect the actual selection risk.

The paper proposes an empirical research framework for addressing the practical issues in LLM pipeline evaluation. Its main contribution lies in integrating the selection bias, candidate pool correlation, and bias correction for a systematic analysis. However, the connection between the effective candidate size and extreme selection bias, the implementation details of some correction methods, and the external validity of the experimental results still require further clarification.

Critical strengths

The paper focuses on performance overestimation when selecting the optimal solution from a large set of prompts or pipelines. This issue directly affects the reliability of LLM system evaluations and the reproducibility of experimental results, and it has practical value.

The experiments report the biases, MSE and interval coverage rates of different methods. This provides a basis for understanding the trade-offs of different correction strategies.

This work reveals that there is a significant disparity between the nominal number of candidates and the estimated effective candidate size. This illustrates that the number of candidate options alone cannot be used to assess the risk of selection bias. This perspective offers certain insights for prompt search and the optimization of automated pipelines.

Critical weaknesses
  • The abstract and conclusion claim that candidate correlation increases selection bias, yet the results show that assuming independent candidates substantially overcorrects (line 314). With comparable means and variances, positive correlation generally reduces selection bias. The authors should reconcile this inconsistency and clarify the comparison underlying their claim.

  • It interprets the effective dimension of the candidate pool's spectrum as the number of effective candidates that determine the selection bias. However, this connection is not universally valid. Extreme selection bias depends on the joint distribution of candidate score errors, and a single effective dimension may not fully capture this distribution. Therefore, the core indicators of the paper may merely be a descriptive summary of the correlation of the candidate pool.

  • The description of cross-fitting does not clearly state whether the candidate selection is completely independent of the held-out fold. If argmax uses the full data for calculation, the held-out score may be contaminated by the selection process..

  • The static candidate pool and the adaptive search have different selection mechanisms. The candidate generation process of the latter may rely on historical evaluation noise, so the relevance of the final candidate pool may not be sufficient to explain the deviation of the entire search process.

Questions for authors

How do you reconcile the claim that correlated candidates increase bias beyond the independent-candidate case with Section 6’s finding that assuming independence substantially overcorrects? Does this comparison control for differences in candidate means and noise variances?

Reviewer_5g9AOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the winner's curse in best-of-N LLM prompt selection, where the highest score on a development set can overestimate the selected prompt's actual performance.

Critical strengths

The winner's curse is well known in statistics. Measuring its magnitude under different candidate counts and dev-set sizes is useful.

Critical weaknesses
  1. Almost all results come from randomly sampling candidates from a fixed correctness matrix.
  2. Is the spectral participation ratio the right quantity for correcting the winner's curse? It measures the effective dimension of a correlation matrix, but this doesn't equal the number of independent draws which will produce the same selection bias.
Questions for authors

Does the implementation select the winner using all dev items or only the training portion of each split?

Reviewer_aCEkOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper discuss the winner's curse in best-of-N LLM prompt selection. The authors propose to estimate the effective number of candidates based on their correlations, and use it to correct the selection bias. Experiments on three classification tasks compare different correction methods and study how to allocate the evaluation budget between candidate selection and evaluation.

Critical strengths
  1. The paper compares several bias correction methods and reports different metrics, including bias, MSE, and coverage. This provides more information than just reporting accuracy.
  2. The authors acknowledge some limitations and discuss closely related work. The provided code and correctness matrices are also helpful for reproducibility.
Critical weaknesses

Sorry that I mixed up the technical and writing weaknesses together below.

  1. The main estimator is not well justified. I don't understand why the spectral participation ratio can be directly used as the effective number of independent candidates. The paper only explains its behavior in some extreme cases, but does not prove its connection to the winner's curse. Using it directly for bias correction seems heuristic.

  2. The relationship between correlation and selection bias is unclear. The paper claims that correlated candidates make the winner's curse worse than the independent case. But positive correlation generally reduces the expected maximum of Gaussian noise with equal variance. I don't think the paper clearly explains this difference, or why replacing N with N_{eff} gives a valid correction.

  3. The candidate pool may heavily affect the results. Most candidates come from one APE-style generation method, while only 10 are handwritten and around 8–12 are mutations. I think this setup is too limited to support the general conclusion about candidate diversity. The very small N_{eff} could simply come from the way these prompts were generated.

  4. The static candidate pool does not represent real adaptive search well. Most experiments are based on resampling a fixed correctness matrix. However, in real prompt optimization, new prompts are generated based on previous evaluation results. The paper only validates this approximation with one live trajectory per task, which I don't think is enough.

  5. The experiments are too limited. The paper only uses three classification tasks and models up to 3B. Since the authors claim their experimental framework is very cheap after building the correctness matrix, I don't understand why more tasks, models, and generation settings were not tested. Even just use all benchmarks from EvoPrompt would be useful.

  6. The ground truth estimation is not sufficiently validated. The paper uses a fixed 300-item truth set to estimate the true accuracy, while the other 300 items are used for dev evaluation. The authors also observed a difficulty difference between these two splits. I don't understand why only one fixed split is used, and how much this difference affects the reported selection bias. Multiple random splits would make the results more convincing.

  7. The statistical evaluation details are not clear enough. The paper reports many confidence intervals, but I couldn't find a clear explanation of how many resampling runs and bootstrap repetitions were used for each experiment. It is also unclear what sources of randomness are included in these intervals.

  8. The correction method is not sufficiently validated. The authors compare six correction methods, but all of them are mainly evaluated using the same few correctness matrices. I don't think this is enough to show that the proposed correction works for different correlation structures or candidate distributions. The claimed agreement between Estimator A and B is also not clearly demonstrated.

  9. The budget allocation results are not very practical. The optimal allocation is determined using the true accuracy of the selected prompts, which is unknown in real applications. So I don't understand how practitioners can actually use this result to decide (N) and (m). The analysis also ignores the cost of generating new candidates.

  10. The comparison with existing prompt optimization work is weak. Section 9 compares the measured bias with published APE gains, but the models and experimental setups are different. I don't think this comparison tells us how much of the original gains actually come from selection bias. More direct experiments with existing optimizers would be more convincing.

  11. The paper is difficult to follow. I found the overall writing and organization not very clear. The abstract contains too many technical details but does not clearly explain the main contribution. Many definitions and notations are introduced without enough explanation, so I often needed to jump between sections to understand the method.

  12. The results section lacks deeper analysis, and some figures are confusing. A lot of the results discussion is simply repeating the numbers shown in figures and tables. I expected more explanation on why these results happen and when the proposed method may fail. Also, I don't understand why lots of figures uses a log scale axis when not necessary.

Questions for authors

Please see above. Please feel free to merge or pick the questions that matters the most in rebuttal if space is an issue.

Reviewer_EKXGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies selection bias in LLM prompt search: choosing the highest-scoring candidate on a development set can lead to an overly optimistic estimate of its performance. It examines how this bias depends on search size, evaluation size, and correlations between candidates. The paper compares several correction methods and explores how a fixed evaluation budget can be divided between testing more candidates and evaluating each candidate more thoroughly.

Critical strengths

The paper addresses a practically relevant evaluation problem and distinguishes inflated reported scores from the loss incurred by selecting a weaker candidate. The stored correctness-matrix approach enables systematic comparisons across search budgets and reporting methods without repeatedly running the models. Evaluating bias, estimation error, and interval coverage provides a useful picture of the trade-offs between correction methods. The live-search experiments offer an initial check beyond static pool resampling.

Critical weaknesses
  1. The connection between effective rank and the proposed correction is not sufficiently justified. The spectral statistic measures dependence in correctness patterns on a particular model and dataset. It does not directly establish how many independent candidates would produce the same selection bias. That bias can also depend on differences in candidates’ true accuracies and the structure of their errors. The paper needs stronger evidence that replacing the candidate count with this statistic gives an appropriate correction, rather than a heuristic that happens to work on the tested pools. The interpretation is also unclear: the abstract and conclusion attribute greater bias to candidate dependence, whereas Section 6 attributes overcorrection to treating dependent candidates as independent.
  2. The practical implications go beyond what the experiments establish. The provenance comparison confounds generation method with pool size: Table 2 compares 87–91 APE candidates against only 10 seeds and 8–12 mutation candidates using N_eff/K. Since this ratio can decrease as the pool grows, the observed differences do not establish whether APE generation itself produces less diverse candidates; comparisons at matched pool sizes are needed to support that interpretation. The budget-allocation study is also retrospective: it identifies favorable allocations using truth-pool performance, but does not establish how a practitioner would choose an allocation before observing those outcomes.
Questions for authors
  1. Can the authors test whether pools with similar spectral effective rank but different correlation structures or candidate-quality gaps require similar bias corrections? This would directly test the proposed interpretation of N_eff.
  2. How stable are the estimated N_eff and the correction’s performance across fresh item samples? Do the conclusions persist when the dev/truth partition changes?
  3. Does the provenance ranking remain after repeatedly subsampling equal numbers of candidates from each source? This would help separate differences in generation methods from the effect of pool size.
  4. Can the authors specify a budget-allocation rule using only information available before final evaluation, and test it on independent data? Otherwise, should Section 7 be framed as a descriptive analysis rather than actionable allocation guidance?
  5. Could the authors justify the choice of Datasets and Benchmarks as the primary area? The paper reads mainly as a statistical analysis of prompt-selection bias with a proposed correction, rather than a dataset or benchmark contribution. Evaluation methodology may fit this area, but what reusable evaluation resource or protocol does the paper establish, and how is it validated for use beyond the specific candidate pools studied here?
  6. Could the authors clarify the cross-fitting procedure in Section 6? The text states that held-out scores are averaged for the “full-data argmax.” If the winner is selected using all dev items and then held fixed across random splits, averaging its held-out scores should recover its original dev score in expectation, rather than remove selection bias. Is the winner instead reselected using only the training portion of each split? Please clarify the estimation target and reconcile the procedure with the bias reduction reported in Table 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.