acceptodds
Under review as a conference paper at ICLR 2027

Accuracy Is Not Stability: Answer Turnover in Test-Time Scaling

Abstract

More inference compute can repair errors but also discard answers that were already correct. We study these opposing flows, repair and corruption, under nested best-of- selection. Two populations can share identical accuracy and oracle-coverage curves at every width, yet lose correct answers at different rates. Exact score-record laws explain this gap: if correctness at extreme verifier scores tends to , each flow from to candidates is asymptotic to as grows. Adjacent turnover therefore vanishes, whereas multiplicative budget increases retain positive turnover even when accuracy rises monotonically. We derive unbiased finite-pool estimators of both flows with a Rao–Blackwell variance guarantee, and formulate answer preservation as a whole-policy risk constraint. Across five model–task–verifier settings, expanding from one to candidates improves accuracy, yet loses a previously correct answer in – of evaluations. At a mean budget of about candidates, a preservation policy attains similar observed accuracy to randomized widths while reducing conditional corruption by percentage points; an adaptive comparator attains higher accuracy with more corruption. These results separate accuracy improvement from answer preservation and quantify their trade-off.

Primary area:
learning theory
Keywords:
Test-Time Scaling, Best-of-N Selection, Answer Stability, Statistical Risk Control

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_T5ZkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how increasing test-time sampling can replace correct answers with wrong ones, even when overall accuracy improves. It provides theoretical analysis, estimators for answer turnover, and proposes "ComputeGuard" to reduce corruption.

Critical strengths
  1. Interesting problem. The paper studies answer stability beyond accuracy, which is a useful direction for evaluating test-time scaling.
  2. Simple method. ComputeGuard is easy to implement and shows lower corruption than random-width sampling with a similar number of candidates.
Critical weaknesses

I'm in general not satisfied with the quality of the manuscript. Here are my feedbacks:

  1. Missing experimental records. The authors state that the original evaluation records are missing, so the proposed bootstrap analysis was not performed. This makes the main experimental results and statistical comparisons difficult to verify.
  2. AI-like writing and limited novelty. The writing feels AI-like to me, with many complex words and different terms used for same concept. The paper makes simple observations unnecessarily complicated with many equations and proofs. For example, the main result in Section 4 can be explained with simple intuition about replacement probability and some curves. While formal proofs may have value, I do not see much additional insight from this complexity. Overall, the contributions seem limited, and I do not think the novelty is strong enough for an ICLR main conference paper.
  3. Weak connection between theory and method. It is unclear how the theoretical results in Sections 3 and 4 help design ComputeGuard. The method seems to be a separate confidence-based selection rule.
  4. Missing verifier analysis. The paper does not report how accurate the two verifiers are. It also does not study how verifier quality affects repair, corruption, or ComputeGuard.
  5. Limited improvement and baselines. ComputeGuard has almost the same accuracy as Random Width, while reducing both corruption and repair. The Adaptive Raw Score baseline achieves higher accuracy, and the paper lacks a full accuracy-corruption trade-off comparison.
  6. Unclear reliability of the predictions. ComputeGuard estimates correctness and future gains using 20 score bins from development data. The paper does not study how reliable these estimates are with limited data or across problems with different difficulty.
  7. Unfair compute comparison. The experiments mainly match the number of candidates, but not generated tokens, verifier cost, or inference time. Therefore, the actual compute cost may be different.
  8. Limited practical motivation. The paper does not clearly explain why reducing answer changes is more important than improving final accuracy in real applications.
Questions for authors
  1. How can the data splits and calibration assumptions be verified?
  2. How does ComputeGuard perform with different verifier quality levels and different amounts of development data?
  3. Can the authors compare the full accuracy-corruption trade-off with stronger baselines, using the same total inference cost?
  4. What additional insight do the theoretical results provide for the design of ComputeGuard?
Reviewer_pWMbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argue that in test-scaling some trial will change the correct answer into incorrect one. The paper prose ComputeGuard to prevent these type of mistake.

Critical strengths

The paper provide a new perspective about accuarcy that we should care the answer transfer.

The paper include some theory about the problems.

Critical weaknesses

Weakness 1: Unclear Utility Function and Questionable Optimization Objective

  • Divergence from Expected Utility: In conventional machine learning and inference-routing benchmarks, the primary objective is maximizing aggregate accuracy under a specified compute or latency budget. The authors place disproportionate weight on preventing "correct-to-incorrect" instance drift. However, if mitigating individual regressions comes at the expense of net aggregate accuracy, the framework effectively trades off overall model utility to enforce consistency on a specific subset of samples.
  • Lack of Real-World Justification: The paper fails to motivate the operational settings where this trade-off is advantageous. In non-critical general QA or standard reasoning tasks, a system that achieves 85%85\% overall accuracy is strictly preferred over one that scores 83%83\%, even if the latter exhibits fewer regressions relative to an earlier intermediate step.
  • Mismatch with Safety-Critical Constraints: If the intended deployment context involves safety-critical or zero-regression requirements, relying on a statistical, classifier-based routing policy—which itself is subject to classification error and false negatives—is insufficient compared to deterministic verification, formal methods, or strict fallback heuristics. The authors should provide a concrete downstream cost-benefit analysis justifying this objective over standard expected-utility optimization.

Weakness 2: High Operational Complexity, Poor Adaptability, and Retraining Overhead

  • Coupling to Specific Models and Benchmarks: The routing mechanism appears tightly coupled to the internal representations and trajectory characteristics of a specific model on a specific benchmark. ComputeGuard requires sampling trajectories, acquiring ground-truth labels, and fitting a dedicated classifier for each unique (model,benchmark)(model, benchmark) pair.
  • Offline Compute and Data Bottlenecks: Collecting rollouts and training auxiliary classifiers across diverse tasks introduces substantial offline engineering and computational overhead. This upfront cost severely undermines the practical efficiency gains achieved during inference.
  • Absence of Out-of-Distribution (OOD) Transferability: The evaluation lacks evidence demonstrating cross-domain or cross-model generalization. If a classifier trained on Model AA on benchmark XX cannot transfer to Model BB or task YY without retraining from scratch, ComputeGuard functions as an overfitted, post-hoc patch rather than a scalable, generalizable inference-time control mechanism.
Questions for authors

see weakness

I will change the score if the answer can convince me for the weakness 1.

Reviewer_VAaaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies changes in answer correctnesss as the sampling budget increase with best of N selection. It decomposes accuracy changes into repair and corruption, and shows that complete population accuracy do not fully determine these paired transitions. The paper derives the transition probabilities and characterizes when turnover persisits under multipilcative budget increases. It also proposes ComputeGuard, which combines conservative ansewrs replacement with adaptive stopping and whole policy risk calibration.

Critical strengths
  1. The paper provides a precise theoretical analysis of information missing from aggregated accuracy.
  2. The paper proposed a practical sampling algorithm, ComputeGuard, to reduce corruption.
Critical weaknesses
  1. The practical motivation for optimizing corruption instead of accuracy is not very clear.
  2. The proposed method, ComputeGuard, doesn't show a clearly better performance compared with baselines.
Questions for authors
  1. The introduction part mentions that corruption measures how sampling can discard a correct answer, could the authors clarify what motivate treating answer presercation as an independent objective ruthar than directly using accuracy?
  2. Am I correct in understanding that, in Table 2, ComputeGuard achieves lower accuracy while using slightly more candidates on average than the adaptive raw-score rule?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.