acceptodds
Under review as a conference paper at ICLR 2027

Active Testing of Large Language Models via Approximate Neyman Allocation

Abstract

Large language models (LLMs) are showing superb performance from general-QA to agentic tasks, mathematical frontier and even self-improvement in niche areas. Evaluating these models on domain-specific tasks often requires expert annotation, making it costly to label the entire evaluation pool. Active testing aims to alleviate this bottleneck by approximating the evaluation result from a small but informative subset of the evaluation pool. Prior active-testing approaches are either built for traditional deep learning models or restrict LLM answers to a single token. We remove this restriction and introduce a training-free active testing algorithm for modern reasoning LLMs on tasks with objectively verifiable answers. Our method leverages semantic entropy from surrogate models to stratify the evaluation pool and then conducts approximate Neyman allocation based on signals extracted from these surrogates. Across multiple language and multimodal scenarios, several surrogate-target pairs and different label budgets, our method attains an average 87.0% relative MSE against Uniform Sampling, translating to 15.0%–28.0% label savings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Active Testing, Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qxVGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method for reducing the cost of evaluating large language models by selecting a smaller subset of questions for annotation. The uncertainty is calculated based on the self-consistency of a smaller LLM based on the consistency of these responses. The questions are then grouped according to their self-consistency, and approximate Neyman allocation determines the number of samples that will be label from each group. The method aims to provide an unbiased estimate of a larger model's performance without requiring additional training. The authors evaluate this approach on many language and multimodal benchmarks and report a 13% average reduction in estimation error compared to uniform sampling, along while saving 15–28% of the labels .

Critical strengths

The problem is important, since reducing the cost of LLM evaluation when obtaining ground-truth labels requires expensive expert annotation. . The proposed approach is simple and practical, combining uncertainty estimates from a smaller LLM with self-consistency. The method is also training-free and can be applied to different language and multimodal models.

Critical weaknesses

The novelty appears limited, as it mainly combines existing ideas from active testing, semantic entropy, and stratified sampling. The proposed allocation strategy lacks a theoretical guarantees of improved efficiency. The method also assumes that uncertainty in a smaller LLM reflects the errors of a larger model, but this assumption is not sufficiently justified. The additional gains over proportional allocation are modest on MMLU-Pro (88.7% to 87.1% relative MSE) and almost negligible on MMBench (95.4% to 95.0%), raising questions about the practical benefit of the method. Finally, generating ten responses per question introduces additional computational costs, and the experiments on existing labeled benchmarks do not fully demonstrate the claimed benefits in realistic expert-annotation settings. Overall, the problem is important and the approach is practical, but I am not fully convinced that the methodological novelty and additional empirical gains are sufficient to justify the contribution.

Questions for authors

Could the authors provide a qualitative analysis of how uncertainty in the smaller surrogate LLM relates to the errors of the larger target model? The authors provide comparisons with adaptations of Berrada et al. (2025), which perform substantially worse than uniform sampling. Could the authors clarify whether this poor performance is an inherent limitation of the previous approach or a consequence of adapting it to multi-token reasoning? Could the authors report the absolute estimation error in accuracy percentage points, rather than only relative MSE? This would make it easier to understand how meaningful the reported improvements are in practice. The authors report computational overhead in Table 3, but the method uses a fixed number of ten surrogate generations per question. How sensitive are the estimation accuracy and computational cost to this choice?

2 comments

Dear Reviewer,

Thank you for the careful review. We are grateful for this opportunity to address and explain your thoughtful questions.

Response to Weaknesses (Contribution): We agree that several things in our method are classical. However, we view our contributions as follows in short: 1) We study the failure modes of prior art extensions. 2) We re-design the whole active testing pipeline, which is largely different from prior art (e.g. Berrada et al. 2025 [1]), as we list the differences in the reply to Reviewer KHMq. 3) We show practical results in the multi-token generation setting, which is essentially the dominant use case in the LLM era.

Response to Question 1 (Concern about smaller surrogate):

As we reply to Reviewer SGgD, we conduct additional experiments on two smaller surrogates (SmolVLM2 500M & Qwen3.5-0.8B). Combined with previous results, the table reports five different surrogates estimating GPT-5.4 as a target model on GPQA-Diamond:

SurrogateFull Pool Accuracy*Relative MSE, M=50*Relative MSE, M=100
SmolVLM2 500M30.3%103.5%101.2%
Qwen3.5-0.8B32.8%100.8%98.2%
Qwen3.5-2B58.1%101.2%101.1%
Qwen3.5-4B78.8%86.5%81.8%
GPT-5.4 Nano75.8%72.3%66.1%

*Relative MSE (% of Uniform Sampling, lower is better).

We can see that the performance struggles with Qwen3.5-2B or weaker as a surrogate, but tends to be helpful when Qwen3.5-4B or GPT-5.4 Nano is the surrogate. Although comprehensive experiments are deferred to the camera-ready version, we believe we have provided a preliminary conclusion that once the surrogate surpasses a certain capability threshold, our method becomes highly effective. It is also worth noting that, in our experiments even with incapable surrogates, our framework gracefully degrades to the uniform sampling baseline (100% relative MSE is equal to uniform sampling), instead of being harmful to the evaluation. We believe that this is quite useful in practical deployment.

Response to Question 2 (poor performance of prior art): The poor performance is a consequence of adapting it to multi-token reasoning. In the original paper of Berrada et al. 2025, their setting is to use an LLM as a classifier (which restricts the LLM to outputting only one token). Our setting is the multi-token generation setting without that restriction because that's the dominant use case in the LLM era. We first try to naturally extend Berrada et al. 2025 in this setting and then study its failure modes.

Response to Question 3 (absolute value of estimation error):

We report mean absolute error (MAE) in percentage points at two label budgets. For example, if full-pool accuracy is 80%, 6.97 in the table means the average absolute gap between estimation and oracle-80% is 6.97 percentage points. And please note that, as the label budget increases, the absolute value of estimation error decreases naturally.

Budget M=20M=20:

Benchmark and model pairUniformProportionalOurs
MMLU-Redux (Qwen3.5-2B → 9B)6.976.506.44
MMLU-Pro (Qwen3.5-2B → 9B)8.718.208.08
MMBench (Qwen-3-VL 2B → 8B)5.265.255.26
GPQA-Diamond (GPT-5.4 Nano → 5.4)5.865.124.90

Budget M=100M=100:

Benchmark and model pairUniformProportionalOurs
MMLU-Redux (Qwen3.5-2B → 9B)3.042.902.82
MMLU-Pro (Qwen3.5-2B → 9B)3.833.593.57
MMBench (Qwen-3-VL 2B → 8B)2.402.372.32
GPQA-Diamond (GPT-5.4 Nano → 5.4)1.901.761.55

Response to Question 4 (choice of k):

Regarding the choice of K, we conduct an additional ablation study on GPQA-Diamond as follows:

K=2K=3K=5K=10K=15K=20
GPQA-Diamond*96.34%90.61%87.27%81.8%80.55%80.06%

*Relative MSE (% of Uniform Sampling, lower is better). Target is GPT-5.4, surrogate is Qwen3.5-4B. Budget M is 100.

We view K=10 as a sweet spot between surrogate signal diversity and computational overhead. We plan to conduct comprehensive experiments over benchmarks in the camera-ready version to give a bigger picture of this question.

[1] Gabrielle Berrada et al. “Scaling Up Active Testing to Large Language Models.” NeurIPS 2025.

Dear Authors,

Thank you for trying to address my comments. I now appreciate the paper more. However: This type of pipelines are usually accompanied by some kind of theoretical guaranties. This would boost the contribution and soundness significantly. I am still sceptical about the paper soundness since the proposed method, as its effectiveness relies heavily on the assumption that uncertainty signals produced by a smaller surrogate model are sufficiently informative about the error distribution of a larger target model. thus it might provide unreliable signals and lead to suboptimal budget allocation that will hurt the performance. Finally while the problem is important and practically relevant. I still believe the contribution of the paper is incremental to the previous work that the authors very well identify.

Overall, I appreciate the authors' clarifications and efforts, but these concerns prevent me from substantially changing my assessment of the paper.

Reviewer_KHMqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free, unbiased active testing framework to evaluate modern LLMs/VLMs on tasks with verifiable answers under human-labeling budget. Unlike prior active testing methods (heavily rooted in the line of work from Yarin Gal’s group, see several cited publications in the paper) that rely on token-level entropy, single-token output restrictions, and LURE estimator, the authors show that naively extending LURE to multi-token fails. To address the issues, the key design replaces LURE and token-level PE with a stratified sampling pipeline: (1) generating CoT rollouts from a small surrogate model to compute answer-level semantic entropy and self-consistency (2) stratifying the evaluation pool by semantic entropy , (3) distributing the labeling budget via an approximate Neyman allocation using a smoothed Bernoulli proxy (4) estimating full-pool risk via a stratified Horvitz–Thompson estimator.

Critical strengths
  • Clear motivation from prior failures: The motivation of the design is very clear by analyzing why naively extending existing works from single token to multi-token won’t work.
  • Simple, training-free, and API-compatible design: By combining answer-level semantic entropy stratification with a surrogate Neyman allocation proxy, the method requires no additional training (compared with prior works), with un biased guarantee.
Critical weaknesses
  • Relatively small improvement over existing works: The paper reads largely as a direct synthesis and (incremental fix) over prior works (largely based on Yarin Gal’s group on active testing/LURE). Also, although the authors claim to extend beyond “single token” case, the problem they can tackle is essentially multiple choice? Which is still reduced to cases very similar to single token cases that the existing works can handle?
  • Experiment scope and "costly human expert annotation" motivation: The main evaluation covers 3 benchmark families (2 MMLU variants: MMLU-Redux and MMLU-Pro, MMBench, and GPQA-Diamond). It’s nice to extend it to more. Rather than naively scaling, it’s great to see cases that are really expensive for humans to annotate. For example, the adopted GPQA-Diamond, the annotation requires high-level education expertise. However, I’m not sure if MMBench and MMLU fit the criteria.
  • Marginal gains over baselines.
Questions for authors
  • Alignment with real-world human labeling costs: as aforementioned.
  • Pool size vs. budget on GPQA-Diamond: GPQA-Diamond is the most realistic proxy for costly expert annotation in the main text, yet its total pool size is only N=198 while Table 2 sweeps budgets up to M=190. How much of the reported gain on GPQA-Diamond comes from finite-population exhaustion?
  • Sensitivity to smoothing offset : Since \sqrt{p(1-p)} <= 0.5, adding =0.75 dominates the output. Would that be reasonable? Why is a small value not working?
4 comments

Dear Reviewer,

Thank you for the careful review and for recognizing the practical motivation. We are happy to take this chance to address several questions.

Response to Weakness 1(a) (contribution vs. prior work):

We credit Berrada et al. [1] for scaling active testing with LURE to LLM evaluation. Yes, our work starts from the natural extension of prior art to the multi-token generation setting, and therefore studies their failure modes as we present in the paper. Although our method is inspired by these failure studies, we don't view our work as an incremental effort for the following reasons:

  1. We study the multi-token generation scenario, which is the main use case nowadays compared to the single-token restriction in prior art (e.g. Berrada et al. 2025).
  2. Our method involves semantic entropy, stratification techniques, allocation policies, and Horvitz–Thompson (HT) estimation. None of them are in Berrada et al. 2025. Also the core design of Berrada et al. 2025 is LURE, which is not involved in our method.
  3. The only thing we share in common is using a surrogate model to provide signals. However, we use different signals (semantic entropy vs. predictive entropy).

Response to Weakness 1(b) (Extension beyond multiple choice): As we also show the results of IFEval and HMMT in the appendix, our method is applicable to any scenario where the answer is objectively verifiable, which means answers can be verifiably grouped into equivalence classes. (An example is math calculation, a counterexample is summarization). Therefore, the scope of our method is beyond multiple-choice questions. And given that most of the evaluation efforts of frontier LLMs are on this objectively verifiable scenario, we believe our method has strong practical value. We also present some agentic tasks' results in our reply to Reviewer SGgD, to further support our scope.

Response to Question 2 (Pool size on GPQA-Diamond):

To address the concern about pool size on GPQA-Diamond, we present the following results where the max label budget is 40 (compared to the full pool size of 198). Our method still shows meaningful performance gains over baselines.

Benchmark and budgetsMax M/N*Proportional*Ours
GPQA-Diamond (M=10, 20, 30, 40)20.2%92.6%89.7%

*Relative MSE (% of Uniform Sampling, lower is better).

Response to Question 3 (offset δ): When δ\delta is too small, unanimous strata (SC=1\text{SC}=1, p(1−p)=0\sqrt{p(1-p)}=0) receive almost no budget even though target errors still occur there, causing variance to spike. Adopting δ=0.75\delta=0.75 intentionally regularizes the allocation and Appendix B.3 shows broad results of strong performance across δ∈[0.5,2]\delta \in [0.5, 2].

Response to Weakness 2 & Question 1 (human labeling costs): Thank you for your advice. Yes, we plan to extend our method to advanced and agentic scenarios. And we already present preliminary results of agentic tasks in our reply to Reviewer SGgD.

[1] Gabrielle Berrada et al. “Scaling Up Active Testing to Large Language Models.” NeurIPS 2025.

Dear Reviewer KHMq,

Thank you again for your time and constructive feedback. We have posted our rebuttal and additional clarifications addressing your concerns. As the discussion period is approaching its end, we would greatly appreciate it if you could take another look at our responses and let us know whether they adequately address your questions.

If our responses have sufficiently addressed your concerns, we would also sincerely appreciate your consideration of updating your scores accordingly.

Thank you for your time and consideration!

Thanks, it resolves most of my questions. I'm still puzzled by why we need such large delta to regularize? as it would dominate, also, is it reasonable to have delta larger than 1?

Thank you for this great question. As we noted above, setting delta too low will undersample strata where the surrogate agrees but the target still makes errors. In our ablation, delta = 0.5 and delta = 0.75 performed similarly. So probably 0.5 is a good choice too. Given the limited time, we will include a finer sweep of delta from 0 to 0.5 in the revised version.

Values above 1 are valid, though the allocation will mathematically approach proportional sampling as delta further increases. From this point, I think we also show the performance difference between ours (delta = 0.75) and proportional in the paper, proving that the effectiveness of our choice of delta.

We are glad to hear that our previous reply resolved your concerns. We would also appreciate any score updates, thank you!

Reviewer_SGgDOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper is about estimating how well a model performs without having to run a large LLM over the entire set, but rather using surrogate models. The authors propose a training free active testing algorithm for modern LLMs.

Critical strengths

This is a timely result and they claim they need 15% to 28% fewer graded examples for comparable accuracy. Figure 1 showcases the main findings, their approach is better than uniform sampling and even if it is not the same as the oracle experiment, they show that they can approach it better.

The paper expands Berrada et al. 2025, which proposes active testing for LLMs for single token answer, to a multi-token reasoning settings.

They present results across multiple Qween and GPT models, and in 4 main datasets. They show improvemens and an approximation to the oracle in all experiments, by reducing the estimation error. They showcase the savings in gpu hours.

Critical weaknesses

In some datasets, the results seem close to the uniform sampling (for e.g., MMBench for Qwen3), would running over a different set of examples or datasets yield to a different conclusion?

What would happen if the surrogate models are incapable to performing more advanced tasks that only large LLMs can actually tackle? Providing clear results of the Surrogate vs the Target model and showing how the estimation is impacted by that would be useful.

Questions for authors

LLMs these days are used in multi-turn conversations, what would be a good mechanism to use this active testing strategy to evaluate an LLM that operate as an agent? I am not sure of the value of this work unless there is a path for this. if the authors provides convincing response with this, I am happy to change my score.

3 comments

Dear Reviewer,

Thank you for the constructive review and we appreciate this opportunity to address and answer your questions.

Response to Question 1 (Extension to multi-turn agent evaluation):

We believe that our framework is applicable to agentic tasks by small and natural extensions. We add some preliminary results on agentic tasks here, and the results show the same trend as previous results. Due to limited time, we will add comprehensive results in the camera-ready version.

For an agent task, we group surrogate rollouts by different state results, compute trajectory-level Semantic Entropy (SE) and Self-Consistency (SC), freeze strata and allocations before the target run as we did previously, then run the target agent under a certain budget and estimate the full-pool performance with the same Horvitz–Thompson (HT) estimator.

BenchmarkTarget modelSurrogate modelFull pool sizeLabel budget*Proportional*Ours*Oracle Neyman
TAU2-BenchQwen3.8-27BQwen3.5-9B278[10, 20, ..., 130]87.15%85.31%84.44%
WordleBenchQwen3.8-27BQwen3.5-9B100[10, 20, 30, 40, 50]95.09%90.08%88.11%
BFCL VehicleControlQwen3.5-9BQwen3.5-4B50[5, 10, 15, 20, 25]83.06%80.31%79.46%
Average————88.43%85.23%84.00%

*Relative MSE (% of Uniform Sampling, lower is better).

Response to Weakness 2 (When is a smaller surrogate too weak to help):

To answer this question, we conduct additional experiments on two smaller surrogates (SmolVLM2 500M & Qwen3.5-0.8B). Combined with the experiments in the paper, the table reports five different surrogates estimating GPT-5.4 as a target model on GPQA-Diamond:

SurrogateFull Pool Accuracy*Relative MSE, M=50*Relative MSE, M=100
SmolVLM2 500M30.3%103.5%101.2%
Qwen3.5-0.8B32.8%100.8%98.2%
Qwen3.5-2B58.1%101.2%101.1%
Qwen3.5-4B78.8%86.5%81.8%
GPT-5.4 Nano75.8%72.3%66.1%

*Relative MSE (% of Uniform Sampling, lower is better).

We can see that the performance struggles with Qwen3.5-2B or weaker as a surrogate, but tends to be helpful when Qwen3.5-4B or GPT-5.4 Nano is the surrogate. Although comprehensive experiments are deferred to the camera-ready version, we believe we have provided a preliminary conclusion that once the surrogate surpasses a certain capability threshold, our method becomes highly effective. It is also worth noting that, in our experiments even with incapable surrogates, our framework gracefully degrades to the uniform sampling baseline (100% relative MSE is equal to uniform sampling), instead of being harmful to the evaluation. We believe that this is quite useful in practical deployment.

Response to Weakness 1 (Poor performance on MMBench and stability across subsets/datasets):

To test sensitivity to the choice of examples on MMBench, we evaluated 20 overlapping random half-pools (2,438 of 4,876 items) across 2 budgets and several model pairs:

Pair and budget*Proportional mean*Ours mean
2B → 4B, M=5093.20%92.12%
2B → 4B, M=10094.16%91.73%
2B → 8B, M=5095.53%93.82%
2B → 8B, M=10096.53%93.46%
4B → 8B, M=5095.21%96.70%
4B → 8B, M=10095.89%95.53%

*Relative MSE (% of Uniform Sampling, lower is better).

Our method beats uniform (100%) in all 120/120 settings, and overall performs better than proportional baseline. Although the performance of our method varies across benchmarks and relies on the capacity of surrogate model as we discuss above, we generally show performance gain in our experiments and our method will gracefully degrade to random selection even in the very unfavorable case, which we view as a good property.

Dear Reviewer SGgD,

Thank you again for your time and constructive feedback. We have posted our rebuttal and additional clarifications addressing your concerns. As the discussion period is approaching its end, we would greatly appreciate it if you could take another look at our responses and let us know whether they adequately address your questions.

If our responses have sufficiently addressed your concerns, we would also sincerely appreciate your consideration of updating your scores accordingly.

Thank you for your time and consideration!

Reviewer_SGgDOpenReview ↗

Author response

I will increase my score as the authors are adding useful and convincing experiments on multi turn settings

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.