acceptodds
Withdrawn from ICLR 2027

BACON: Budgeted Human Calibration for Modeling and Evaluation with Multiple AI Judges

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

AI judges are increasingly used to reduce the cost of human evaluation. While they provide a scalable and inexpensive alternative, their outputs can be biased relative to human preferences and highly item-dependent, with substantial variation across judges, tasks, and domains. When practitioners rely on uncalibrated AI evaluations for model ranking, item scoring, or population-level quality reporting, systematic bias can propagate directly into downstream decisions and make the conclusions unreliable. We propose BACON, a four-stage pipeline that combines budgeted human calibration with multiple AI-judge outputs to obtain more accurate annotations. BACON first constructs full-coverage auxiliary features for every item in the evaluation pool, including multi-judge scores, token-level uncertainty statistics, and contextual embeddings. It then obtains human labels for a small sampled subset and trains a cross-fitted outcome model to produce calibrated item-level surrogate predictions. These predictions support two downstream modes: the first is population-level estimation of summary metrics, such as means, quantiles, or other estimands, using an augmented estimating-equation estimator with valid confidence intervals; and the second is individual-level surrogate scoring for item scoring and ranking. Throughout, BACON treats AI judges as auxiliary measurements rather than ground truth: human labels provide the calibration anchor, while AI-derived signals serve as surrogate predictions and improve efficiency. We validate BACON on a variety of tasks and domains. Across settings and labeling budgets, the cross-fitted outcome model improves predictive accuracy and ranking consistency, while the estimating-equation estimator reduces bias and variance relative to raw AI outputs and purely human-label-based methods. These results suggest that BACON provides a practical framework for scalable, statistically grounded evaluation with limited human annotation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM-as-a-Judge, Human Calibration, AI-assisted Evaluation, Prediction-Powered Inference, Multi-Judge Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_63MzOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework to calibrate LLM judges with human labels. In particular, it cross-fits a lightweight model that uses the scores, embeddings, and uncertainty features from multiple LLM judges to predict the human scores. Fitting is done a small subset of the dataset, so human labels are kept minimal. Experiment results show that the trained models achieve high item-level correlation with human scores and accurate population-level summary metrics. In addition, embedding features alone can achieve well-calibrated item-level and population-level results.

Critical strengths
  • The method is simple and practical. It does not involve training the LLM judges and only requires fitting lightweight models on a small dataset.
  • Ablation studies are extensive, covering single/multi-judges, with/without embeddings, and various human label budgets.
Critical weaknesses
  • The novelty is very limited. The proposed method simply combines various features to fit a lightweight neural network, and there is no methodological novelty or theoretical novelty. The choice of LLM scores, embedding features, and uncertainty features also seems arbitrary and lacks justification.
  • Important experimental results are missing in the main paper, and many results added in the Appendix are negative. Section 3.2 reports the item-level results on the PERSUADE dataset, while Section 3.3 reports the population-level results on the MQM dataset. Since the paper identifies both the item-level prediction and summary metric prediction as equally important use cases, it should report both of them on all datasets, instead of reporting each of them on a separate dataset. Although these results are reported in Appendix A.1 and Appendix A.2, the item-level results on the MQM dataset are much worse than on the PERSUADE dataset, with R2=0.369R^2=0.369 and Spearman correlation=0.657\text{Spearman correlation}=0.657.
Questions for authors
  • How does your method handle cases where disagreement among human raters is high? Current results seem to be based on a setting with a single human score per item.
  • How does your method compare with a training-based method that fine-tunes LLM judges on human ratings?
Reviewer_Dq77OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

BACON combines multiple AI judges with a small set of human labels to improve evaluation under a limited annotation budget. It learns calibrated predictions from judge scores, uncertainty measures, and contextual embeddings, then uses human-based corrections to estimate aggregate metrics with confidence intervals. Experiments on essay grading, translation evaluation, and website scoring report better item-level predictions and more accurate aggregate estimates than raw AI scores and human-only baselines.

Critical strengths

The paper focuses on a practical and important problem: how to obtain reliable evaluations from imperfect AI judges when human annotation is limited and expensive.

Critical weaknesses

What was the intention behind Figure 1? If it was to demonstrate disagreement among AI judges or between AI judges and human annotators, comparing score distributions alone does not clearly establish this. A judge could frequently disagree with humans at the item level while producing a very similar overall score distribution.

Moreover, agreement between a judge and humans or another judge can be assessed ordinally rather than cardinally. For example, if J(y1)>J(y2)J(y_1)>J(y_2) and the human scores satisfy J(y1)>J(y2)J^(y_1)>J^(y_2), the judge correctly ranks the two responses even if its score distribution differs substantially from the human distribution. In many applications, including BoN and other inference-time selection methods, relative rankings matter more than the exact scores. However, I recognize that cardinal calibration is relevant to the paper’s goal of estimating human-score means and other summary statistics.

Overall, although disagreement between AI judges and humans is a plausible motivation, Figure 1 primarily demonstrates differences in marginal score distributions. Adding item-level agreement and ranking metrics would make the nature of the mismatch, and its relevance to the paper’s two evaluation goals, clearer.

Another weakness worth pointing: the practicality of token-level uncertainty features needs further discussion. Length normalization alone does not make entropy comparable across judges, since it can depend on response format, explanation length, and tokenization. Computing it also requires access to token probabilities, which some APIs do not provide or expose only for the top-kk tokens, limiting both its availability and comparability. Table 13 also shows this so-called uncertainty features are not that helpful.

Lastly, with a small human-label budget, it is unclear whether the calibration sample sufficiently represents heterogeneous or rare cases, particularly for complex evaluation tasks. This could limit item-level scoring and ranking accuracy and reduce the efficiency gains of aggregate estimation. Additional experiments on more complex tasks, together with subgroup analyses and comparisons against more flexible predictors, would help establish the limits of the proposed calibration models.

Questions for authors

I have no major questions.

Reviewer_UihCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BACON combines multiple AI judges, embeddings, and uncertainty features with a small human-labeled sample. A cross-fitted outcome model supplies item-level surrogate scores and predictions for regression-adjusted population estimation. Experiments cover essay scoring, translation evaluation, and website perception. The paper acknowledges that its mean estimator is equivalent to existing PPI/regression-adjusted estimators.

Critical strengths
  1. Across three domains, the paper studies feature contributions, labeling budgets, panel size, cost, annotator disagreement, distribution shift, and quantiles. The results give useful guidance rather than simply favoring larger panels: embeddings dominate on PERSUADE, while MQM benefits more from multiple judges. On WebDesign, a panel costing 48% of GPT-4.1 achieves lower population-mean RMSE in 21 of 24 comparisons

  2. The paper connects human calibration, multiple judges, and population inference in a practical workflow. It distinguishes item-level surrogate scores from statistically justified aggregate estimates. The shift experiments make this distinction concrete: Table 11 shows surrogate R² falling to −0.86 on held-out human translations, demonstrating why fresh human calibration matters.

Critical weaknesses
  1. I am not fully convinced that BACON adds enough beyond combining supervised calibration with existing PPI methods. Some gains are modest: on PERSUADE, embeddings alone achieve R² = 0.674, compared with 0.699 for the full model, and uncertainty features add essentially nothing once embeddings are included (Tables 1 and 13). The cost-matched results in Table 3 are also close and mixed. These findings are useful, but the paper needs a clearer account of what the proposed combination contributes.

  2. Some claims need stronger support. Appendix D assumes differentiable estimating functions, but the quantile score contains an indicator, so the stated theorem does not directly justify that case. The conclusion that label noise affects efficiency rather than validity is also too broad: the experiments do not establish robustness to systematic annotator bias. Human calibration can correct disagreement with the observed labels, but it cannot ensure those labels are themselves unbiased.

Questions for authors
  1. Can you give a precise estimator and variance formula for the implemented combination of labeled out-of-fold predictions and unlabeled full-sample predictions, distinguishing finite-pool from superpopulation inference? What does n denote in Appendix D’s variance formula?

  2. What regularity conditions justify quantile inference, particularly for discrete MQM outcomes? How were quantile confidence intervals constructed?

  3. Can you report paired uncertainty for Table 3 and compare against PPI++ using the same features and calibration budget?

  4. Can you narrow the label-noise claim or test systematic rater bias explicitly?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.