acceptodds
Under review as a conference paper at ICLR 2027

A CaLLM Judge is a Better Judge: Mitigating Bias in LLM-as-a-judge

Abstract

Large language models (LLMs) are increasingly used as judges to predict which of two candidate answers a human evaluator would prefer, providing a scalable alternative when human evaluation is too costly. Unfortunately, LLM judgments can exhibit systematic biases. Prior work has documented position, verbosity or model family of the candidate answers influencing the judge decision rather than their quality, while additional biases may remain undocumented. Multicalibration provides a natural framework to address this problem: each bias corresponds to miscalibration on an identifiable group, e.g., position bias manifests as overconfidence in the answer shown first. The challenge is to fix miscalibration simultaneously across many, potentially unknown groups. To this end, we introduce CaLLM, a post hoc framework that multicalibrates an LLM judge, thereby bounding all such biases simultaneously. We prove that CaLLM bounds the bias of every group whose error structure is captured by an available representation, and that augmenting it with explicit bias groups further reduces the residual bias. We then derive theoretical conditions under which CaLLM improves the judge's accuracy, rather than trading it for calibration. Finally, we evaluate CaLLM across pairwise preference benchmarks, showing that it reduces the multicalibration error on both specified and unspecified bias groups, while improving predictive performance of the judge.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM-as-a-Judge, Calibration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VpTwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

CaLLM applies the MCGrad multicalibration algorithm to debias LLM-judge scores, with theoretical guarantees and gains in calibration and accuracy across five benchmarks.

Critical strengths

The empirical results are promising, improving both calibration and predictive accuracy. The paper also offers theoretical analysis showing how multicalibration in representation space controls group-wise errors.

Critical weaknesses
  1. Presentation is hard to follow. I was lost by page 2. Many definitions appear before the problem and method are explained, so their purpose is unclear. I suggest restructuring to first present the practical problem, available data, and the algorithm.

  2. Novelty and necessity are unclear. The method seems a straightforward application of an existing multicalibration algorithm to LLM scores and embeddings. With human labels available, why not train a supervised model on the same features? The experiments do not show why multicalibration is needed. Also, lower group-wise calibration error does not necessarily mean position or verbosity bias is removed.

Questions for authors
  1. Could the authors compare against standard supervised baselines (e.g., XGBoost or an MLP) trained on the same features and human labels?

  2. How does the group-wise calibration guarantee translate into mitigating actual LLM judge biases, especially under counterfactual changes such as swapping answer positions?

4 comments
AuthorsOpenReview ↗

Rebuttal to reviewer VpTw

We thank the reviewer. We are glad they found our results promising and acknowledges the value of our theoretical analysis. We address concerns in the following.

Clarity We understand the reviewer's concerns about clarity of the presentation. We followed the reviewer's suggestion and restructured the introduction and methodological section of our manuscript. We restated our introduction to more clearly identify the problem we tackle and the shortcomings of existing methodologies. We present our goal (lines 103-106) in the preliminary section of the manuscript, followed by the proposed methodology (lines 177-197), all before the theoretical section. We further improve the interpretability of the first bound (Theorem 1) and extend the discussion on its components and their meaning.

Novelty and Necessity. Our goal is to mitigate multiple sources of bias in LLM-as-a-judge within a unified framework, including biases that are not known in advance. Our main theoretical contribution is to address the unexplored problem of transferring multicalibration guarantees from a learned representation to groups defined in input space. Traditional multicalibration approaches operate on the same features that characterize groups. Attributes such as age can be provided as group indicators or used to discover miscalibration within ranges of their values. In contrast, LLM judges operate on unstructured textual inputs, where potentially relevant bias groups cannot always be directly identified or specified. While known biases can be represented through explicit group indicators, unknown biases must be addressed through information preserved in a generic representation, such as text embeddings. We bridge this gap by establishing conditions under which multicalibration in representation space controls the residual bias of groups defined over the original inputs, even when their membership cannot be recovered from the representation. The results go beyond existing multicalibration guarantees, which do not characterize this interaction between representation quality, prediction error, and input-space group membership. We additionally derive sufficient conditions under which multicalibration guarantees an improvement in predictive accuracy, even if it changes decisions. Unlike existing approaches, CaLLM provides a unified mechanism for mitigating multiple documented and potentially unknown biases simultaneously, without requiring a separate correction procedure for each. Thus, our contribution is not a new multicalibration algorithm, but the theoretical characterization and empirical validation of multicalibration as a general framework for mitigating biases in LLM judges operating on learned representations.

Supervised baselines. We add XGBoost and MLP models trained on the same representations and labels, considering both direct prediction and residual correction strategies (Table 12). Since MCGrad itself relies on gradient-boosted trees, these comparisons can help assess the benefits of the multicalibration procedure. Results show that simple supervised approaches can recover a portion of the improvements, highlighting the importance of the representation used by CaLLM. CaLLM achieves the lowest log loss on all five datasets, the highest accuracy (except for MTB where it is second best within a standard deviation), and consistently strong group-wise calibration. Notably, CaLLM achieves lower \mathrm{MCE}_{\\%} than both residual-correction baselines on all five datasets, while classifiers fitted on labels occasionally achieve competitive or lower multicalibration error, but consistently exhibit worse predictive performance. Overall, these findings support the choice of MCGrad as a well-rounded calibration strategy, offering a favorable balance between predictive accuracy, probabilistic quality, and multicalibration.

Group-wise calibration and counterfactual bias As noted in line 132, our framework adopts a distributional notion of bias, measuring systematic discrepancies between the judge's confidence and human preferences within groups associated with bias-inducing attributes. While this differs from counterfactual notions such as position-swap consistency, the two are related. In particular, assuming that human preferences are invariant to response ordering and that both orderings follow the same underlying distribution of comparisons, perfect calibration within the corresponding position groups implies E[f(X)+f(Xswap)−1]=0\mathbb{E}\left[f(X)+f(X^{\mathrm{swap}})-1\right]=0 where f(X)f(X) denotes the probability assigned to the first response and XswapX^{\mathrm{swap}} denotes the same comparison with reversed response order. This guarantee holds in expectation, rather than for individual comparisons: positive and negative counterfactual discrepancies may cancel out. Thus, our approach controls systematic position bias at the population level without necessarily enforcing counterfactual invariance for every question.

Comparison of CaLLM with supervised classification and residual-correction baselines in terms of multicalibration error and predictive performance.

DatasetMethodMCE_{\\%}MCEσACCLlogL_{\text{log}}
PKUf022.46 ± 0.2832.91 ± 0.470.630 ± 0.0020.730 ± 0.005
CaLLM5.55 ± 0.287.68 ± 0.380.692 ± 0.0010.572 ± 0.002
XGB$_{resid}$10.45 ± 0.1815.40 ± 0.260.679 ± 0.0010.607 ± 0.004
XGB$_{base}$5.87 ± 0.477.56 ± 0.610.658 ± 0.0010.618 ± 0.001
MLP$_{resid}$9.57 ± 0.2914.22 ± 0.420.683 ± 0.0010.599 ± 0.003
MLP$_{base}$5.47 ± 0.477.24 ± 0.650.663 ± 0.0010.612 ± 0.001
AEXf016.11 ± 0.3120.81 ± 0.380.633 ± 0.0010.760 ± 0.004
CaLLM5.09 ± 0.195.83 ± 0.210.677 ± 0.0020.598 ± 0.002
XGB$_{resid}$12.14 ± 0.2615.70 ± 0.310.664 ± 0.0020.649 ± 0.004
XGB$_{base}$6.26 ± 0.256.88 ± 0.280.640 ± 0.0010.625 ± 0.001
MLP$_{resid}$12.67 ± 0.2816.36 ± 0.340.664 ± 0.0020.650 ± 0.003
MLP$_{base}$6.33 ± 0.137.01 ± 0.130.638 ± 0.0020.626 ± 0.002
PPEf016.23 ± 0.449.03 ± 0.250.639 ± 0.0040.741 ± 0.008
CaLLM7.97 ± 0.433.87 ± 0.200.670 ± 0.0050.612 ± 0.002
XGB$_{resid}$13.97 ± 0.317.79 ± 0.180.663 ± 0.0040.664 ± 0.007
XGB$_{base}$9.02 ± 0.394.23 ± 0.180.636 ± 0.0030.640 ± 0.002
MLP$_{resid}$15.18 ± 0.548.49 ± 0.310.663 ± 0.0050.665 ± 0.006
MLP$_{base}$9.31 ± 0.484.40 ± 0.230.641 ± 0.0050.637 ± 0.001
RWBf013.07 ± 0.415.97 ± 0.140.860 ± 0.0090.357 ± 0.018
CaLLM7.29 ± 0.923.92 ± 0.470.930 ± 0.0020.208 ± 0.006
XGB$_{resid}$8.57 ± 1.113.80 ± 0.480.922 ± 0.0040.231 ± 0.010
XGB$_{base}$18.74 ± 1.006.21 ± 0.280.818 ± 0.0040.377 ± 0.011
MLP$_{resid}$9.63 ± 1.274.20 ± 0.530.925 ± 0.0050.232 ± 0.006
MLP$_{base}$24.91 ± 2.529.52 ± 1.230.814 ± 0.0070.398 ± 0.014
MTBf019.59 ± 0.563.50 ± 0.120.788 ± 0.0060.485 ± 0.021
CaLLM16.36 ± 0.842.70 ± 0.110.814 ± 0.0150.405 ± 0.021
XGB$_{resid}$18.36 ± 0.933.30 ± 0.170.817 ± 0.0060.429 ± 0.019
XGB$_{base}$16.04 ± 1.802.27 ± 0.250.757 ± 0.0130.560 ± 0.019
MLP$_{resid}$17.97 ± 0.933.12 ± 0.220.800 ± 0.0070.450 ± 0.011
MLP$_{base}$20.24 ± 1.972.94 ± 0.220.746 ± 0.0100.546 ± 0.006

The authors have addressed my concern about the missing supervised baselines by adding XGBoost and MLP experiments, which I appreciate.

However, my main concern about novelty remains. The method relies on the existing MCGrad algorithm, and the main theoretical contribution concerns transferring calibration guarantees from embeddings to the original input space. Since using embeddings is itself a design choice, it is unclear how much this result contributes to the original problem. For known biases such as position and verbosity, group indicators can be used directly. For unknown biases, the guarantee depends on whether the embeddings preserve the relevant information.

Overall, while the empirical results are promising, I still have concerns about the novelty and the broader claims of bias mitigation. I will maintain my rating.

We acknowledge the reviewer decision. With the goal to improve our work, we would kindly ask if it would be possible to better clarify which of the raised concerns was not addressed by our rebuttal.

Reviewer_dySeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies bias correction for LLM judges in pairwise response evaluation. The authors formulate bias as groupwise miscalibration. They propose Multi-Calibrated Large Language Models (CaLLM), which applies MCGrad (Tax et al., 2026) to confidence scores and response representations. The theory characterizes residual group bias and gives conditions for improved accuracy. Experiments on five benchmarks show improvements in calibration and accuracy.

Critical strengths

The problem is interesting and the proposed formulation is useful. It provides a common approach to several sources of systematic prediction error. The representation analysis also helps explain when group errors can be corrected without directly observing group membership. The empirical results are also positive, and the additional judge and independent embedder experiments further strengthen the findings.

Critical weaknesses
  1. The contribution beyond existing methods could be further clarified. CaLLM uses human labels, response representations, and an existing multicalibration algorithm. The other multicalibration methods in Appendix E.4 use different representations and grouping procedures. Thus, both the input information and the method change in these comparisons. It would be helpful to include a supervised predictor using the same features and labels (using the same splits and tuning budget as did). It would help clarify the added benefits of multicalibration.

  2. The temperature-scaling results also require clarification. Table 7 reports higher accuracy after temperature scaling on every dataset. Standard temperature scaling with a positive temperature preserves decisions at the 0.5 threshold. It therefore cannot produce this change from the same base scores. Please clarify the transformation and input predictions used for this baseline.

  3. The conclusions about removing miscalibration could be stated more carefully. The claim of no significant miscalibration uses a cutoff corresponding to a significance level of 10^{-5}. Failure to reject perfect calibration at this level does not establish that the remaining error is small. For example, the reported MCE% remains around 12 to 13.5 on MTBench. The reduction in calibration error should be distinguished from the absence of statistical significance. The percentage version of MCE should also be defined explicitly.

Questions for authors
  1. Theorem 2 assumes calibration on groups defined by the original judge's decisions and the final confidence intervals. Including the original score as a feature does not establish this property for the fitted model. Could the authors clarify how MCGrad satisfies this requirement? Is the resulting accuracy bound informative in the experiments?

  2. How much labelled calibration data is required for the gains? A calibration-size learning curve would help establish the practical benefit when human labels are limited. The present results support generalization to held-out prompts within a benchmark. Performance on unseen response-model families or under changes in the prompt distribution remains less clear.

6 comments
AuthorsOpenReview ↗

Rebuttal to reviewer dySe

We thank the reviewer and we are glad they consider our proposed formulation useful and the empirical findings positive. We now address the remaining concerns.

Supervised baselines We add XGBoost and MLP models, considering both direct prediction and residual correction strategies (Table 12). Results show that simple supervised approaches can recover a portion of the improvements, highlighting the importance of the representation used by CaLLM. Nevertheless, CaLLM with MCGrad achieves the lowest log loss on all five datasets, the highest accuracy (except for MTB where it is second best within a standard deviation), and consistently strong group-wise calibration. CaLLM achieves lower \text{MCE}_{\\%} than both residual-correction baselines on all five datasets, while classifiers fitted on labels occasionally achieve competitive or lower multicalibration error, but consistently exhibit worse predictive performance.

LOGR and IGLB with CaLLM representations We run LOGR and IGLB on the same representations fed to McGrad (Table 13). We stress that those approaches require groups to be explicitly defined; therefore, we apply the same procedure of [1] to the representation provided to McGrad. Results are mostly consistent across representation types, with the new method approach slightly outperforming the original one. Still, CaLLM with McGrad is consistently the best method.

Temperature Scaling implementation We mistakenly reported temperature scaling on the probabilistic output of the baseline BPC. The resulting accuracies exactly match. We updated the paper to present temperature scaling applied to the base model f0f_0 (Table 3 and Tables 11 and 20). Not surprisingly, results from applying temperature scaling to BPC are better than those obtained by applying it on top of the base model, since the output of BPC is of higher quality so CaLLM outperforms it.

Conclusions on Multicalibration Error We stress that we always claim, whenever appropriate, to remove evidence of miscalibration, never to zero the error: we refer to the absence of evidence of miscalibration only when MCEσ is below the relevant threshold. To clarify this point, we revised line 461 of the edited manuscript, removing the word “completely”, which could have invited a stronger interpretation than intended. We also explicitly distinguish between \text{MCE}_{\\%} and MCEσ in the metric definition (line 160) and in Appendix D.

Question on Theorem 2 The guarantee is conditional on the multicalibrator achieving sufficiently small calibration error on the groups induced by the original predictions, within the confidence intervals of the calibrated predictor. MCGrad does not explicitly guarantee that these groups will be identified. However, since the original confidence is included among the input features, when the relevan groups exhibit sufficiently strong and detectable miscalibration, MCGrad can identify and correct the corresponding violations. We emphasize that the purpose of Theorem 2 is to characterize sufficient conditions under which multicalibration improves predictive accuracy, rather than to establish an unconditional accuracy-improvement guarantee for MCGrad. We clarify this point and explicitly mention it in the limitations section.

We report (see comment below) the values of Γ\Gamma and ζ\zeta i.e., the two components of the bound in Theorem 2, to evaluate its informativeness. The bound is positive on PKU, our largest dataset, where the empirical lower bound indicates an improvement of approximately 2 percentage points, compared with the observed improvement of 5.5 points. It is approximately zero on AEX and RWB, and negative on PPE and MTB. These results illustrate both the practical relevance and the conservative nature of the bound: its sufficient condition can be satisfied empirically, but a negative lower bound does not preclude an actual improvement in accuracy. The larger residual calibration terms on smaller datasets are also consistent with increased estimation variability, although the tightness of the bound depends on both the calibration error and the magnitude of the decision changes. These results show that Theorem 2 can provide a nontrivial empirical lower bound on accuracy improvement, while its sufficient condition remains conservative, particularly when residual calibration errors are large relative to the gains from corrected decisions.

Calibration set size We report in Figure 10 the performance (\mathrm{MCE}_{\\%} and Log-loss) for 5 different calibration set sizes for the largest datasets (PKU). At 10k samples, CaLLM achieves bias-mitigation performance similar to that obtained with the full calibration set, while smaller sets lead to more pronounced degradation. A similar trend emerges for log-loss, although CaLLM exhibits greater robustness in low-data regimes.

References

[1] Detommaso, Gianluca, et al. "Multicalibration for confidence scoring in llms, 2024.".

Comparison of CaLLM with supervised classification and residual-correction baselines in terms of multicalibration error and predictive performance.

DatasetMethodMCE_{\\%}MCEACCLlogL_{\text{log}}
PKUf022.46 ± 0.2832.91 ± 0.470.630 ± 0.0020.730 ± 0.005
CaLLM5.55 ± 0.287.68 ± 0.380.692 ± 0.0010.572 ± 0.002
XGB$_{resid}$10.45 ± 0.1815.40 ± 0.260.679 ± 0.0010.607 ± 0.004
XGB$_{base}$5.87 ± 0.477.56 ± 0.610.658 ± 0.0010.618 ± 0.001
MLP$_{resid}$9.57 ± 0.2914.22 ± 0.420.683 ± 0.0010.599 ± 0.003
MLP$_{base}$5.47 ± 0.477.24 ± 0.650.663 ± 0.0010.612 ± 0.001
AEXf016.11 ± 0.3120.81 ± 0.380.633 ± 0.0010.760 ± 0.004
CaLLM5.09 ± 0.195.83 ± 0.210.677 ± 0.0020.598 ± 0.002
XGB$_{resid}$12.14 ± 0.2615.70 ± 0.310.664 ± 0.0020.649 ± 0.004
XGB$_{base}$6.26 ± 0.256.88 ± 0.280.640 ± 0.0010.625 ± 0.001
MLP$_{resid}$12.67 ± 0.2816.36 ± 0.340.664 ± 0.0020.650 ± 0.003
MLP$_{base}$6.33 ± 0.137.01 ± 0.130.638 ± 0.0020.626 ± 0.002
PPEf016.23 ± 0.449.03 ± 0.250.639 ± 0.0040.741 ± 0.008
CaLLM7.97 ± 0.433.87 ± 0.200.670 ± 0.0050.612 ± 0.002
XGB$_{resid}$13.97 ± 0.317.79 ± 0.180.663 ± 0.0040.664 ± 0.007
XGB$_{base}$9.02 ± 0.394.23 ± 0.180.636 ± 0.0030.640 ± 0.002
MLP$_{resid}$15.18 ± 0.548.49 ± 0.310.663 ± 0.0050.665 ± 0.006
MLP$_{base}$9.31 ± 0.484.40 ± 0.230.641 ± 0.0050.637 ± 0.001
RWBf013.07 ± 0.415.97 ± 0.140.860 ± 0.0090.357 ± 0.018
CaLLM7.29 ± 0.923.92 ± 0.470.930 ± 0.0020.208 ± 0.006
XGB$_{resid}$8.57 ± 1.113.80 ± 0.480.922 ± 0.0040.231 ± 0.010
XGB$_{base}$18.74 ± 1.006.21 ± 0.280.818 ± 0.0040.377 ± 0.011
MLP$_{resid}$9.63 ± 1.274.20 ± 0.530.925 ± 0.0050.232 ± 0.006
MLP$_{base}$24.91 ± 2.529.52 ± 1.230.814 ± 0.0070.398 ± 0.014
MTBf019.59 ± 0.563.50 ± 0.120.788 ± 0.0060.485 ± 0.021
CaLLM16.36 ± 0.842.70 ± 0.110.814 ± 0.0150.405 ± 0.021
XGB$_{resid}$18.36 ± 0.933.30 ± 0.170.817 ± 0.0060.429 ± 0.019
XGB$_{base}$16.04 ± 1.802.27 ± 0.250.757 ± 0.0130.560 ± 0.019
MLP$_{resid}$17.97 ± 0.933.12 ± 0.220.800 ± 0.0070.450 ± 0.011
MLP$_{base}$20.24 ± 1.972.94 ± 0.220.746 ± 0.0100.546 ± 0.006

LOGR and IGLB fitted on the same representations of CaLLM.

DatasetMethodACCLlogL_{\text{log}}MCE_{\\%}MCEσ
PKUCaLLM0.692 ± 0.0010.572 ± 0.0025.55 ± 0.287.68 ± 0.38
LOGR (orig)0.645 ± 0.0030.622 ± 0.00117.70 ± 0.4722.72 ± 0.60
IGLB (orig)0.644 ± 0.0020.617 ± 0.00116.85 ± 0.4721.77 ± 0.60
LOGR (CaLLM feat.)0.645 ± 0.0020.622 ± 0.00117.86 ± 0.4722.93 ± 0.60
IGLB (CaLLM feat.)0.647 ± 0.0020.615 ± 0.00116.60 ± 0.4521.45 ± 0.58
AEXCaLLM0.677 ± 0.0020.598 ± 0.0025.09 ± 0.195.83 ± 0.21
LOGR (orig)0.645 ± 0.0020.633 ± 0.0019.26 ± 0.2010.08 ± 0.21
IGLB (orig)0.645 ± 0.0020.632 ± 0.0029.20 ± 0.2210.06 ± 0.24
LOGR (CaLLM feat.)0.645 ± 0.0020.632 ± 0.0029.27 ± 0.2310.09 ± 0.24
IGLB (CaLLM feat.)0.647 ± 0.0020.630 ± 0.0029.26 ± 0.2610.14 ± 0.28
PPECaLLM0.670 ± 0.0050.612 ± 0.0027.97 ± 0.433.87 ± 0.20
LOGR (orig)0.653 ± 0.0030.630 ± 0.00211.72 ± 0.245.53 ± 0.11
IGLB (orig)0.649 ± 0.0040.633 ± 0.00111.97 ± 0.275.69 ± 0.11
LOGR (CaLLM feat.)0.656 ± 0.0050.627 ± 0.00311.55 ± 0.415.46 ± 0.19
IGLB (CaLLM feat.)0.655 ± 0.0060.630 ± 0.00311.88 ± 0.705.65 ± 0.34
RWBCaLLM0.930 ± 0.0020.208 ± 0.0067.29 ± 0.923.92 ± 0.47
LOGR (orig)0.884 ± 0.0060.302 ± 0.00712.59 ± 0.474.81 ± 0.17
IGLB (orig)0.882 ± 0.0080.297 ± 0.00811.40 ± 0.624.31 ± 0.21
LOGR (CaLLM feat.)0.883 ± 0.0070.301 ± 0.00712.38 ± 0.504.71 ± 0.18
IGLB (CaLLM feat.)0.880 ± 0.0070.289 ± 0.00911.34 ± 0.914.28 ± 0.27
MTBCaLLM0.814 ± 0.0150.406 ± 0.02116.36 ± 0.842.70 ± 0.11
LOGR (orig)0.805 ± 0.0070.432 ± 0.00820.13 ± 1.313.09 ± 0.20
IGLB (orig)0.810 ± 0.0090.436 ± 0.01521.94 ± 0.843.67 ± 0.14
LOGR (CaLLM feat.)0.801 ± 0.0060.434 ± 0.01420.04 ± 0.863.05 ± 0.12
IGLB (CaLLM feat.)0.801 ± 0.0050.436 ± 0.01019.98 ± 0.613.32 ± 0.13

Empirical estimate of terms appearing in the bound of Theorem 2.

DatasetΓ\Gammaζ\zetaα\alphaΓ−ζ\Gamma- \zetaΔ\Delta Acc
PKU0.061 ± 0.0020.041 ± 0.0040.034 ± 0.0030.020 ± 0.0050.055 ± 0.002
AEX0.034 ± 0.0020.035 ± 0.0030.028 ± 0.003-0.002 ± 0.0020.033 ± 0.001
PPE0.023 ± 0.0040.045 ± 0.0020.035 ± 0.002-0.022 ± 0.0040.020 ± 0.002
RWB0.055 ± 0.0030.054 ± 0.0100.087 ± 0.0160.001 ± 0.0080.055 ± 0.004
MTB0.020 ± 0.0040.105 ± 0.0220.108 ± 0.023-0.084 ± 0.0250.001 ± 0.019
Reviewer_bcxQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CaLLM, a post-hoc calibration method for mitigating bias in LLM-as-a-judge. It combines verbalized confidence score with contrastive response embeddings, and applies multicalibration method, MCGrad, using human preference labels. The paper provides theoretical results relating bias mitigation to representation quality, and characterizing conditions for accuracy improvement. Experiments on five datasets demonstrate the effectiveness of CaLLM across groups defined by position, verbosity, and model family, showing improvements in calibration and predictive performance.

Critical strengths
  • Clear writing and organization. The paper is well written and easy to follow.
  • Clearly presented theoretical results. The theoretical results have explicit assumptions and interpretable conclusions, helping clarify the role of the input representation in bias mitigation and the conditions under which calibration can improve accuracy.
  • Promising empirical results. The experiments cover multiple datasets and bias types, and the reported improvements demonstrate the effectiveness of the proposed approach.
Critical weaknesses
  • Insufficiently controlled comparisons with supervised baselines. The current experiments do not sufficiently disentangle the contributions of the input representation and the multicalibration procedure. Appendix E.4 provides relevant comparisons against calibration methods fitted on the same labeled calibration data. However, LOGR and IGLB use a different representation and group construction pipeline from CaLLM, leaving the source of the performance gap unclear. A particularly informative baseline would be a standard supervised probabilistic classifier, such as a GBDT trained with binary cross-entropy, that predicts human preferences from exactly the same input ϕ(X)\phi(X). Comparisons with LOGR and IGLB using groups constructed from this same representation would also help isolate the contribution of MCGrad. Moreover, the supervision-matched comparisons in Appendix E.4 are more reasonable baselines than the ones in Section 4.2, in terms of fairness in using calibration data. Hence, I recommend to move them in the main text.

  • Limited evaluation across confidence extraction methods and judge settings. The evaluated base judges are open-weight models, yet only the verbalized confidence is utilized. This is a reasonable design choice given the black-box motivation, but the experiments do not establish how sensitive CaLLM’s improvements are to this choice. An ablation using token-based preference probabilities, such as probabilities normalized over the A/B decision tokens [1], would clarify whether the method remains effective with an alternative confidence signal available from the same judges. Also, the reported experiments do not establish whether the improvements transfer to an actual closed, API-only judge. Such an evaluation would strengthen the empirical support for the intended black-box application setting.

  • Unexplained accuracy changes after temperature scaling. Table 7 reports accuracy improvements after temperature scaling. However, standard positive scalar temperature scaling preserves binary predictions at the 0.5 threshold used in the paper. Its accuracy should therefore remain unchanged when evaluated on the same examples. Could the authors clarify the exact transformation and prediction rule used for this baseline? Resolving this discrepancy is important for assessing the reliability of the calibration baseline comparisons in Appendix E.4.

[1] Badshah et al., SCOPE: Selective Conformal Optimized Pairwise LLM Judging, ICML 2026

Questions for authors

Please address the weaknesses above.

4 comments
AuthorsOpenReview ↗

Rebuttal to reviewer bcxQ

We thank the reviewer for acknowledging that our paper is well written, that our theory leads to interpretable conclusions, and that our experimental evaluation is broad. We now address the remaining concerns

Supervised baselines We include the comparison with multicalibration methods and supervised baselines in the main text. We add a RQ4 where we disentangle the effect of multicalibration from label supervision. We now present a comparison between CaLLM, global calibration methods, IGLB and LOGR, and two supervised XGboost classifiers: one fitted on labels and one fitted on the base judge residuals. We report in Table 12 results for two MLPs fitted on labels and residuals as well. Classifiers fitted on residuals provide a more direct comparison with CaLLM, as they preserve the post-hoc correction setting. CaLLM achieves a lower \text{MCE}_{\\%} and MCEσ than both residual-correction baselines. Classifiers fitted on labels occasionally achieve competitive or lower multicalibration error, but consistently exhibit worse predictive performance.

LOGR and IGLB with CaLLM representations We run LOGR and IGLB on the same representations fed to McGrad (Table 13). Since these approaches require groups to be explicitly defined we run the same procedure of [1] on the representation provided to McGrad. Results for LOGR and IGLB are mostly consistent across representation types, with the new approach occasionally slightly outperforming the original one. Importantly, CaLLM with McGrad is consistently the best method across datasets.

Token probabilities We evaluate with model confidence extracted as the normalized token probabilities of the judge as the reviewer suggests. We report results (Figure 7, Table 8, Table 9, Figure 8 and Table 10) for all four research questions (the Portia baseline is computationally expensive so we will report its results once the experiments are complete). Findings for RQ1 with token probabilities are largely consistent with results with verbalized confidence: CaLLM achieves the best \text{MCE}_{\\%} on 10 out of 14 dataset-bias pairs and second best in the four exceptions. The answer to RQ2 is consistent across probability extraction modes. CaLLM-G improves with respect to not providing bias groups on all dataset-bias pairs. Consistent with the results with verbalized confidence, CaLLM is the best-performing approach according to both accuracy and logloss across all datasets. Similarly with our findings on verbalized confidence, CaLLM achieves consistently the lowest or second-best error in all evaluations outmatching supervised baselines in terms of its stability of results across datasets and groups. Our analysis of accuracy and Log-loss further consolidates CaLLM as the best approach: it achieves the highest accuracy and lowest Log-loss across all datasets. The only exception is MTB, where CaLLM matches the accuracy of several baselines.

Evaluation on closed API-only judges CaLLM is is perfectly compatible with black-box operation: it requires only verbalized confidence scores and text embeddings, which can be obtained independently of the judge. In our experiments we evaluate two different judges and additionally experiment with embeddings extracted from an independent model (Appendix E3), demonstrating that CaLLM does not rely on judge-specific internal representations and achieves comparable performance in both bias mitigation and accuracy. While an additional proprietary judge would be an addition to the empirical evaluation, we do not identify any methodological dependency on access to model weights that would make closed, API-only judges fundamentally different in this setting. Differences in systematic biases and calibration quality may arise across individual judges, irrespective of whether their weights are publicly available. Moreover, we believe that using closed-source LLMs raises two fundamental issues:

  • Reproducibility: the model behind an endpoint can change or be deprecated without notice, so our results cannot be reproduced over time.
  • Accessibility: using these models requires a paid subscription, whereas our evaluation should be accessible to the entire research community.

Temperature Scaling We thank the reviewer for mentioning this. The reviewer correctly notices that temperature scaling should not change a classifier prediction. Indeed, we reported temperature scaling on the probabilistic output of the baseline BPC. The resulting accuracies exactly match. We updated the paper to present temperature scaling applied to the base model f0f_0 (Table 3 in main text and Table 11 and 20 in Appendix). Not surprisingly, results when applying temperature scaling to BPC are better than the ones obtained applying it on top of the base model since the output of BPC is of higher quality, so CaLLM still outperforms it.

References

[1] Detommaso, Gianluca, et al. "Multicalibration for confidence scoring in llms, 2024.".

Comparison of CaLLM with supervised classification and residual-correction baselines in terms of multicalibration error and predictive performance.

DatasetMethodMCE_{\\%}MCEACCLlogL_{\text{log}}
PKUf022.46 ± 0.2832.91 ± 0.470.630 ± 0.0020.730 ± 0.005
CaLLM5.55 ± 0.287.68 ± 0.380.692 ± 0.0010.572 ± 0.002
XGB$_{resid}$10.45 ± 0.1815.40 ± 0.260.679 ± 0.0010.607 ± 0.004
XGB$_{base}$5.87 ± 0.477.56 ± 0.610.658 ± 0.0010.618 ± 0.001
MLP$_{resid}$9.57 ± 0.2914.22 ± 0.420.683 ± 0.0010.599 ± 0.003
MLP$_{base}$5.47 ± 0.477.24 ± 0.650.663 ± 0.0010.612 ± 0.001
AEXf016.11 ± 0.3120.81 ± 0.380.633 ± 0.0010.760 ± 0.004
CaLLM5.09 ± 0.195.83 ± 0.210.677 ± 0.0020.598 ± 0.002
XGB$_{resid}$12.14 ± 0.2615.70 ± 0.310.664 ± 0.0020.649 ± 0.004
XGB$_{base}$6.26 ± 0.256.88 ± 0.280.640 ± 0.0010.625 ± 0.001
MLP$_{resid}$12.67 ± 0.2816.36 ± 0.340.664 ± 0.0020.650 ± 0.003
MLP$_{base}$6.33 ± 0.137.01 ± 0.130.638 ± 0.0020.626 ± 0.002
PPEf016.23 ± 0.449.03 ± 0.250.639 ± 0.0040.741 ± 0.008
CaLLM7.97 ± 0.433.87 ± 0.200.670 ± 0.0050.612 ± 0.002
XGB$_{resid}$13.97 ± 0.317.79 ± 0.180.663 ± 0.0040.664 ± 0.007
XGB$_{base}$9.02 ± 0.394.23 ± 0.180.636 ± 0.0030.640 ± 0.002
MLP$_{resid}$15.18 ± 0.548.49 ± 0.310.663 ± 0.0050.665 ± 0.006
MLP$_{base}$9.31 ± 0.484.40 ± 0.230.641 ± 0.0050.637 ± 0.001
RWBf013.07 ± 0.415.97 ± 0.140.860 ± 0.0090.357 ± 0.018
CaLLM7.29 ± 0.923.92 ± 0.470.930 ± 0.0020.208 ± 0.006
XGB$_{resid}$8.57 ± 1.113.80 ± 0.480.922 ± 0.0040.231 ± 0.010
XGB$_{base}$18.74 ± 1.006.21 ± 0.280.818 ± 0.0040.377 ± 0.011
MLP$_{resid}$9.63 ± 1.274.20 ± 0.530.925 ± 0.0050.232 ± 0.006
MLP$_{base}$24.91 ± 2.529.52 ± 1.230.814 ± 0.0070.398 ± 0.014
MTBf019.59 ± 0.563.50 ± 0.120.788 ± 0.0060.485 ± 0.021
CaLLM16.36 ± 0.842.70 ± 0.110.814 ± 0.0150.405 ± 0.021
XGB$_{resid}$18.36 ± 0.933.30 ± 0.170.817 ± 0.0060.429 ± 0.019
XGB$_{base}$16.04 ± 1.802.27 ± 0.250.757 ± 0.0130.560 ± 0.019
MLP$_{resid}$17.97 ± 0.933.12 ± 0.220.800 ± 0.0070.450 ± 0.011
MLP$_{base}$20.24 ± 1.972.94 ± 0.220.746 ± 0.0100.546 ± 0.006

LOGR and IGLB fitted on the same representations of CaLLM.

DatasetMethodACCLlogL_{\text{log}}MCE_{\\%}MCEσ
PKUCaLLM0.692 ± 0.0010.572 ± 0.0025.55 ± 0.287.68 ± 0.38
LOGR (orig)0.645 ± 0.0030.622 ± 0.00117.70 ± 0.4722.72 ± 0.60
IGLB (orig)0.644 ± 0.0020.617 ± 0.00116.85 ± 0.4721.77 ± 0.60
LOGR (CaLLM feat.)0.645 ± 0.0020.622 ± 0.00117.86 ± 0.4722.93 ± 0.60
IGLB (CaLLM feat.)0.647 ± 0.0020.615 ± 0.00116.60 ± 0.4521.45 ± 0.58
AEXCaLLM0.677 ± 0.0020.598 ± 0.0025.09 ± 0.195.83 ± 0.21
LOGR (orig)0.645 ± 0.0020.633 ± 0.0019.26 ± 0.2010.08 ± 0.21
IGLB (orig)0.645 ± 0.0020.632 ± 0.0029.20 ± 0.2210.06 ± 0.24
LOGR (CaLLM feat.)0.645 ± 0.0020.632 ± 0.0029.27 ± 0.2310.09 ± 0.24
IGLB (CaLLM feat.)0.647 ± 0.0020.630 ± 0.0029.26 ± 0.2610.14 ± 0.28
PPECaLLM0.670 ± 0.0050.612 ± 0.0027.97 ± 0.433.87 ± 0.20
LOGR (orig)0.653 ± 0.0030.630 ± 0.00211.72 ± 0.245.53 ± 0.11
IGLB (orig)0.649 ± 0.0040.633 ± 0.00111.97 ± 0.275.69 ± 0.11
LOGR (CaLLM feat.)0.656 ± 0.0050.627 ± 0.00311.55 ± 0.415.46 ± 0.19
IGLB (CaLLM feat.)0.655 ± 0.0060.630 ± 0.00311.88 ± 0.705.65 ± 0.34
RWBCaLLM0.930 ± 0.0020.208 ± 0.0067.29 ± 0.923.92 ± 0.47
LOGR (orig)0.884 ± 0.0060.302 ± 0.00712.59 ± 0.474.81 ± 0.17
IGLB (orig)0.882 ± 0.0080.297 ± 0.00811.40 ± 0.624.31 ± 0.21
LOGR (CaLLM feat.)0.883 ± 0.0070.301 ± 0.00712.38 ± 0.504.71 ± 0.18
IGLB (CaLLM feat.)0.880 ± 0.0070.289 ± 0.00911.34 ± 0.914.28 ± 0.27
MTBCaLLM0.814 ± 0.0150.406 ± 0.02116.36 ± 0.842.70 ± 0.11
LOGR (orig)0.805 ± 0.0070.432 ± 0.00820.13 ± 1.313.09 ± 0.20
IGLB (orig)0.810 ± 0.0090.436 ± 0.01521.94 ± 0.843.67 ± 0.14
LOGR (CaLLM feat.)0.801 ± 0.0060.434 ± 0.01420.04 ± 0.863.05 ± 0.12
IGLB (CaLLM feat.)0.801 ± 0.0050.436 ± 0.01019.98 ± 0.613.32 ± 0.13

Dear authors,

Thank you for the detailed explanation with additional experiments. While the reduced gap with new fairly compared baselines is quite worried, I believe the consistent improvement along with theoretical results outweigh this concern. Therefore, I raised my rating accordingly. Good luck!

Dear Reviewer bcxQ,

We appreciate your recognition of our contributions and your willingness to reconsider your assessment. Thank you for your constructive feedback, which has helped us substantially strengthen the empirical evaluation and improve the paper.

Cheers,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal summary

We thank the reviewers for their feedback. We are glad that the majority of reviewers agree that the problem we tackle is interesting, our theoretical analysis is informative and our empirical evaluation is extensive with promising results.

Request for additional experiments. We are glad reviewer bcxQ recognizes that: “experiments cover multiple datasets and bias types” and that reviewer dySe recognizes that: “the additional judge and independent embedder experiments further strengthen the findings”. We answer their request for supervised baselines adding XGBoost and MLP models trained on either labels or residuals. We also run multicalibration baselines on the same representation provided to our proposal and test sensitivity to calibration-set size. Finally, we run our evaluation on token probabilities instead of verbalized confidence. CaLLM achieves the lowest log-loss on all five datasets, competitive or best accuracy, and consistently lower MCE_{\\%} than both supervised residual-correction methods. We run all requested experiments except one: evaluation with a closed API-only judge. We stress that we evaluate CaLLM with multiple judges and follow a black-box compatible protocol as we additionally experiment with embeddings extracted from an independent model (Appendix E3). We do not identify any methodological dependency on access to model weights that would make API-only judges fundamentally different in this setting. Moreover, we believe that using closed-source LLMs raises two issues:

  • Reproducibility: the model can change or be deprecated without notice, so our results cannot be reproduced over time.
  • Accessibility: these models require a paid subscription, whereas our evaluation should be accessible to the entire research community.

Questions about experiments We clarify that we reported temperature scaling implemented on the confidence output of BPC, a bias-mitigation baseline. We fix this and present the implementation fitted on the base confidence of the judge, which performs worse than our proposal. We additionally fixed our wording around the claims for lack of evidence of miscalibration. Following the suggestion of reviewer dySe we updated the metrics definition to explicitly state both \text{MCE}_{\\%} and MCEσ .

Questions about theory We are glad reviewer bcxQ considers that our theory is useful in “helping clarify the role of the input representation in bias mitigation and the conditions under which calibration can improve accuracy” and that according to reviewer dySe “The representation analysis also helps explain when group errors can be corrected without directly observing group membership”. Reviewer dySe asks whether the bound of Theorem 2 is informative. We compute it, showing that it can provide a non-trivial lower bound on predictive accuracy and illustrating both its practical relevance and its conservative nature. Moreover, our theory is conditional on the multicalibrator achieving sufficiently small calibration error on the groups induced by the original predictions, within confidence intervals. MCGrad does not explicitly guarantee this, however when these groups exhibit sufficiently strong and detectable miscalibration, it can identify and correct the corresponding violations. Still, the purpose of Theorem 2 is to characterize sufficient accuracy improvement conditions for multicalibration, rather than to establish an unconditional guarantee for MCGrad. Since our framework adopts a distributional notion of bias, reviewer VpTw asks about the connection with counterfactual notions such as swap consistency. Under order-invariant human preferences and a swap-balanced evaluation distribution, perfect calibration on the corresponding position groups implies that the average signed counterfactual discrepancy vanishes, although individual comparisons may remain inconsistent.

Clarity We are glad both reviewers bcxQ and dySe consider our work clear. Reviewer bxcQ notices that: “The paper is well written and easy to follow” as a critical strength of our manuscript together with “Clearly presented theoretical results” and follows: “The theoretical results have explicit assumptions and interpretable conclusions, helping clarify the role of the input representation in bias mitigation and the conditions under which calibration can improve accuracy”. Also reviewer sySe regards the paper as generally understandable and underscores that “The representation analysis also helps explain when group errors can be corrected without directly observing group membership”. Reviewer VpTw instead mentions being “lost by page 2” because “Many definitions appear before the problem and method are explained”. We followed his suggestion to restructure the introduction and preliminaries: we now introduce the methodological component of our contribution before our theoretical analysis and further explain the bias-mitigation problem and set-up in the preliminaries.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.