acceptodds
Under review as a conference paper at ICLR 2027

PRESCALE: Exact Risk Accounting for Frozen-Bank Verifier Search

Abstract

Verifier-guided Best-of-N search can improve a proxy score while replacing an incumbent with a target-worse candidate. We study this risk through offline auditing of a frozen labeled candidate bank, with target labels hidden from the selector. PRESCALE exactly integrates harmful replacement over uniform subsets sampled without replacement, including competition between harmful and benign winners. A complete score-gap vector yields a conditional risk curve without a Gaussian assumption; independent complete vectors from a specified observation law additionally support finite-sample budget calibration. Candidate-wise verifier reliability does not identify this curve. Matched-margin quadratic-Gaussian controls isolate dependence effects. On one simulated PDE trajectory, aligned score gaps yield forecast MAE of 1.11 percentage points, versus 19.68 after column shuffling preserves empirical marginals; same-information subset Monte Carlo yields 1.15.In a controlled matched-data comparison, direct-loss calibration admits budget 16 versus 4 for a marginal union-bound reduction; known Gaussian marginals also admit 16. This comparison includes union-bound slack. On MATH-500, the rank interface quantifies harmful replacements alongside accuracy gains and separates cross-bank forecast discrepancies from within-bank sampling effects. This operational audit does not establish a distinct answer-selection advantage. PRESCALE makes frozen-bank search risk computable while separating exact conditional accounting, dependence-sensitive estimation, and statistical calibration.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
verifier-guided search, inference-time compute scaling, Best-of-N selection, reward overoptimization, reward hacking, joint verifier-error dependence, frozen candidate banks, finite-sample risk certification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_boRjOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies best of N search where a verifier scores alternatives, and where a selector, who selects based on the verifier's input, cannot access target labels; in particular, the selector is given a budget, and has to select from a uniformly random subset of the alternatives that fits the budget. As such, in this setting, one considers risk curves as a function of the budget, where the risk comes from the verifier mis-ranking the alternatives compared to the unseen target scores. Properly estimating this risk frontier demands care, due to the dependencies inherent in the estimands, and the paper proposes a way to do so in a practical way, as well as proves certain characterizations --- such as expressions for the expected risk in terms of relevant quantities --- and also provides a real-data empirical playground to test the risk estimation approach.

Critical strengths

I do not see any critical strengths of note that should be highlighted here. The paper is written in a way that complicates the task of reading and reviewing it as much as possible (read: slop), while not giving any strong contributions that would significantly enrich the literatures on best of N, or on risk control. In fact, just reading the list of the 4 contributions in the introduction may convince one of both above points. More on these below.

Critical weaknesses

The paper, as it is written, is, to a very substantial degree, AI slop. Reading the manuscript took inordinately long and was incredibly tedious, and consulting the appendices to try to clarify some of the chaotic terminology didn't help matters. This is the paper's biggest weakness (see below), for which reason I believe it's a clear reject at this point.

Secondly, I am not impressed with the theory, or indeed any of the contributions, proposed by the authors. For example, Proposition 2, declared in the Contributions overview as one of the 4 main contributions, is just a simple combinatorial observation; Proposition 1 is similarly unimpressive/trite. The "finite-sample calibration" contribution is completely mundane/rote too --- of course you need it to apply a method such as the proposed one, and of course you can do it. The "operational audits and limits" contribution, meanwhile, is too unfocused and hard to parse to count as good empirical evaluation. Zooming out into methodology, the literature on risk control, and on controlling multiple risks, is already quite rich and this paper is not contributing much of anything useful to it either.

Getting back to the AI slop discussion, already the abstract, and then the intro, show a near-complete disregard for proper academic writing --- or, indeed, for the basic goal of educating the reader about the substance of the paper. The reader is immediately inundated with obscure terminology such as "frozen bank", "matched-margin" "target-worse", "complete score-gap", "PDE trajectory" (just note this one for instance: how is the reader supposed to tell from reading the abstract ---- perhaps PDE means "partial differential equation" or perhaps "prompt driven exploration"?) . Immediately, unhelpful qualifiers such as "without a Gaussian assumption" spring up, as well as vague concepts like "candidate-wise verifier reliability". Then, instantly, the abstract switches to reporting numerical forecast errors and budget allocations (!) in some particular experiments that the paper runs.

Much the same story then continues in the introduction. Here, once again, instead of gently introducing the area of inquiry or at least concretizing the setting a little bit more, more of the vague language is put in that cannot be parsed by a reader who hasn't internalized the rest of the paper already. More vague terms like "candidate-wise crossing probabilities" and "harmful mask" are introduced, along with further useless and unreadable qualifications such as: "Quadratic-Gaussian geometry provides a controlled illustration, not an assumption of the rank functional". This is then followed by a generated Figure 1 that is almost as uninformative as it could get, with vague allusions to "aligned measurements", "upper envelopes", etc., and with graphics that make little sense on their own.

The same disregard for effective and proper writing continues on through the end of the paper. Now here is the surprise: "PRESCALE", which is in the title of the paper and features throughout the exposition, is actually never formally defined! Not as an algorithm environment, not as an enumerated recipe --- nothing at all. In fact, it is not even made clear to the reader what kind of object PRESCALE even is --- a method? an algorithm? a paradigm? something else entirely? I hope it is now clear where my evaluation of this paper comes from.

Questions for authors

The authors should feel free to respond to any of the above, but I do not currently foresee what could change my evaluation to more positive. This manuscript should have been de-slopped before submitting to a top-tier conference, and ICLR's built-in opportunities for the authors to revise their manuscript will not be appropriate here, as virtually the entirety of the paper needs to be revamped.

2 comments
AuthorsOpenReview ↗

Response to Reviewer boRj

Thank you for reading our paper. We will improve on our presentation, yet we disagree with the dismissive comment describing the work as "AI slop."

AI helped us refine the writing and assisted with methodology, derivations, code and analysis, as disclosed on p. 10. The authors take responsibility for the research decisions, experimental design and execution, and verification of the results.

  1. PRESCALE and the terminology. PRESCALE is an offline risk-auditing procedure. A "frozen bank" is a fixed candidate pool; the "incumbent" is the current answer. The question is how likely further search is to replace that answer with a higher-scoring but worse one. Section 3.1 (p. 3) now gives the procedure explicitly: use scores and offline target labels to count subsets in which a worse candidate wins, under uniform sampling without replacement. Labels are used for auditing, not by the selector.

  2. The contribution. The counting and concentration tools are classical. The contribution is their use in an audit that accounts for competition between better and worse candidates. A wrong answer may outscore the current answer yet lose to another correct answer. Proposition 2 computes the harmful-winner probability exactly over candidate subsets. Proposition 1 shows why individual crossing probabilities cannot determine it. Statistical calibration is a separate step, requiring independent complete scoring observations on the same pool. We have made this distinction explicit in the revised contribution paragraph.

  3. Why PDE appears. PDE means partial differential equation. The known equation provides structural correctness labels independently of the residual-based verifier. This lets us hold individual crossing probabilities fixed, change score dependence, and measure its effect on selection risk. MATH-500 checks the rank calculation using recorded PRM scores and tests transfer to new pools. It does not validate the PRM's latent error geometry. The revised opening and transitions in Sections 4 and 5 explain why these experiments are needed.

PRESCALE calculates harmful replacement risk exactly over candidate subsets and uses independent repeated complete scoring observations for statistical calibration. We have clarified this procedure and why each experiment is needed. The revisions are marked in blue.

Thank you for your responses to me and the other reviewers, clarifying certain aspects of the paper's setup/contribution, and for the revision. I have carefully read these. Major presentational issues remain (I don't think the revised manuscript would be clear enough if I was reading it for the first time), so further major revisions (that would be too substantial to handle within this ICLR's reviewing process) targeting clarity will be necessary to potentially turn this paper into a competitive submission.

Reviewer_J8PyOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

As I understood the paper, PRESCALE is introduced as a meta-auditor system, samples alternative labels to a predicted solution, calculating exact risk, without requiring normally distributed errors.

Critical strengths

The relative-risk is a refreshing measure because accuracy gains can coexist with harmful replacements, and I like that this was demonstrated adequately on the 276 MATH confirmation questions. The separation between conditional accounting and statistical certification is a big strength.

Critical weaknesses

Central experimental inputs are missing from the described package. The reproducibility statement says the original 703-by-64 residual bank is excluded, preventing regeneration of the central controlled results. The earlier calibration comparison also did not archive individual calibration vectors.

Questions for authors
  1. Can Table 2 include a variance-adaptive certificate for the marginal crossing mean on the same observations, to test whether the budget gap persists under a different marginal calibration bound?
  2. Can the anonymous package include the original residual matrix and sufficient calibration data to regenerate the central comparisons?
1 comment
AuthorsOpenReview ↗

Response to Reviewer J8Py

Thank you for these suggestions. We added the requested comparison to Table 2 (p. 8), using a one-sided predictable plug-in empirical Bernstein bound from Waudby-Smith and Ramdas (2024). Both methods used the same 32 original calibration replicates, with 256 complete score vectors per replicate. We kept the budget grid, 10% risk tolerance, and 95% confidence level the same.

The finite-data marginal method certified budget 4 in every replicate. Direct calibration certified 16 in 31 replicates and 32 in one. The comparator with known Gaussian marginals certified 16. The budget gap therefore persists with the variance-adaptive bound, and still includes the conservatism of the union bound. Appendix D.1 describes the calculation, and the companion archive contains the code and results for each replicate.

We have also included the original 703-by-64 numerical residual matrix, noise scale, and simulation seeds in the anonymous companion. We checked it from a fresh extraction in a new Python environment. The CPU verifier regenerated all 270 controlled geometry regimes, 3,168 calibration means, 864 historical budget decisions, and 33 reference grid cells. It also reproduced the new Table 2 comparison. After installing the pinned dependency, run

cd residual_replay
python scripts/verify_residual_replay_20261107.py

The calibration observations are regenerated from the original seeds. The reproducibility statement and Appendix J now describe this replay.

The updated supplementary ZIP retains the original artifact in original_artifact/ and adds this replay in residual_replay/. A blue-marked copy of the revised paper is also included.

Reviewer_Ca73OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary
  • Studied problem: When is it necessary to add more candidates during offline evaluation with best-of-N search under a proxy verifier? Specifically, how often will expanding verifier-guided search replace the current winner with a higher-scoring but target-worse candidate?
  • Submission approach: The paper proposes to use per-item harmful-swap rate rather than aggregate accuracy to evaluate a label-blind policy. Specifically, when does pushing harder repalces a correct answer/item for a confidently wrong one.
  • Approach realization: The authors support their formalziation with experiments and evaluations on MATH-500 with Qwen2.5-Math-7B-Instruct and Skywork-o1-Open-PRM-Qwen-2.5-7B.
Critical strengths
  • The paper takes a principled and theoretical approach to BoN search with a verifier proxy that could be impactful for the context of reward overoptimization. To be clear, I'm inferring this, as the authors do not make this explicit. Specifically, in RLHF or similar settings we optimize against a proxy and whether to continue training (and risking reward hacking/overoptimization) and its per-item mechanism behind it can be modeled by this work.
Critical weaknesses
  • The submission suffers from severe writing and clarity issues. It is too densely written and I found it extremely hard to follow. I recommend rejection solely on that basis. Specifically:
    • A lot of “why this matters” needs to be inferred by the reader, starting on page 1.
    • It reads like a dense list of purely technical details and individual results without connecting elements (Sections 4 and 5 in particular).
    • Section 3 (formalization) is hard to follow due to missing connectors and motivations.
    • Some terms like “PDE” are never defined. The paper also uses less common terminology (“frozen bank,” “reference-relative event,” “incumbent” for the known setting of best-of-N under a proxy verifier but it still cites the papers from the topic in its related work section)
    • It is unclear to me what the paper actually contributes to advance the studied problem.
    • As far as I can tell the math in the main body is correct, but I am not sure, given the density, lack of connections, and jump in concepts.
Questions for authors

None

3 comments
AuthorsOpenReview ↗

Response to Reviewer Ca73

Thank you for your suggestions. They helped us make the paper easier to read. We went back through the derivation to explain the steps connecting the formulas.

  1. In Section 3.1, we define the number of alternatives and explain that the budget includes the retained current answer. We also say which quantities are random. Target labels are assessed separately; this does not mean target quality and verifier scores must be statistically independent.

  2. Sections 3.2 to 3.4 now explain how a candidate can beat the current answer yet lose to another candidate, and how fixing the ranking turns the problem into counting subsets. We distinguish this exact calculation from statistical calibration over independent complete scoring observations on the same pool. The equations and counting proof are unchanged.

  3. Before Table 2, we identify the two row statistics: integrated harmful-winner risk and the fraction of all alternatives that are harmful and beat the current answer. The MATH section now states the 5% risk tolerance and distinguishes maximizing the admissible budget from maximizing predicted net gain.

We also explain what the PDE candidates, labels and scores represent, and define PRM and oracle availability at first use. The reported results and formal assumptions are unchanged. We kept the original wording where possible; only added or changed words are marked in blue.

Thank you for your clarifications. I do not see a single blue word indicating wording changes in the pdf. Are you sure you uploaded a revision?

Otherwise, my initial review recommending rejection due to presentation issues is unchanged and seem to be still relevant given a quick skim of the paper as I see it in the system.

AuthorsOpenReview ↗

Blue-marked revision

Thank you for checking. The PDF we uploaded on November 12 was the clean revised manuscript. The blue-marked copy was in the supplementary ZIP, and we should have said where to find it. We have now uploaded that copy as the main PDF. The blue edits in Sections 3.1–3.4 show where we added explanations connecting the setup, subset counting, and calibration.

Comments on the paper

1 comment

Dear Reviewers,

The authors have replied to your initial comments. Please have a look and engage in the discussion.

Thanks!

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.