acceptodds
Under review as a conference paper at ICLR 2027

UtilityBench: Measuring Language Model Utility When Errors Have Different Costs

Abstract

As language models are increasingly deployed in real-world scenarios with important outcomes, the impact of their mistakes often extends far beyond what correctness can capture. In domains like medicine, a single error can lead to serious consequences and put patient safety at risk. However, mainstream evaluation protocols still treat all errors as equal, overlooking the spectrum of real-world costs. To address this, we propose a utility-based evaluation framework that explicitly incorporates the varying consequences of errors. We develop UtilityBench, a large-scale benchmark of 15,000 instances with verifiable answers, quantifying error severity using Best–Worst Scaling to elicit meaningful, data-driven cost estimates. These quantified utilities shape when a model should answer or abstain, guiding it to make decisions that align with the true importance of each scenario. As another merit, we propose UtilityAUC, a metric that measures the area under the utility curve determined by a model’s response ordering, thereby enabling utility preference alignment beyond error frequency. Our theoretical analysis demonstrates that UtilityAUC can distinguish between models where standard accuracy and AURC fall short. Empirically, evaluating 20 state-of-the-art models, we find that greater accuracy or more advanced architectures do not reliably translate to higher UtilityAUC, highlighting a gap between problem-solving ability and responsible decision-making in language model evaluation.

Primary area:
datasets and benchmarks
Keywords:
Large Language Model Evaluation, Utility-aware Evaluation, Responsible AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_eNAnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces UtilityBench, a 15,000-instance benchmark for evaluating language models when errors have different consequences. It combines answerable and refusal-required queries, assigns sample-specific error costs through multi-model Best–Worst Scaling judgments and Bradley–Terry aggregation, and assesses the severity rankings using human annotations. The proposed UtilityAUC integrates utility across response coverage, accounting for correct answers, costly errors, appropriate refusals, and unnecessary refusals. Experiments across 20 models examine accuracy, reasoning strength, prompting, and response-scoring strategies. The results suggest that higher accuracy or stronger reasoning does not consistently translate into higher utility under the benchmark’s cost assumptions.

Critical strengths
  1. The paper addresses an important limitation of accuracy: errors with substantially different consequences should not necessarily receive equal penalties.
  2. The benchmark includes substantial annotation work and human validation, with additional analysis showing agreement beyond broad domain differences.
Critical weaknesses
  1. The validity of the cardinal costs remains insufficiently established. Human–LLM ranking agreement supports relative severity, but does not validate the cost ratios obtained through exponentiating Bradley–Terry scores and choosing a median anchor. These numerical assumptions directly influence model rankings. Sensitivity to the refusal penalty alone does not establish robustness to uncertainty in the error costs.
  2. The main experiments provide limited evidence of utility-ranking ability. The headline comparison assigns only binary answer/refusal scores, averaging over all within-group orderings. It therefore primarily measures cost-weighted response outcomes and answering rates, rather than the ability to prioritize beneficial answers over costly mistakes within attempted responses.
Questions for authors

What evidence justifies interpreting the exponentiated Bradley–Terry scores as cardinal, additive harm costs comparable to the fixed reward for correctness and refusal penalty? Without validating this interpretation, what supports treating UtilityAUC as a measure of utility rather than alignment with the judges’ perceived-severity preferences?

4 comments

Thank you for your thoughtful review. We address your concerns below.

W1/Q1\color{blue}{\textbf{W1/Q1}}: What justifies using exponentiated Bradley–Terry scores and a median anchor as numerical error costs?

A: Fitting a Bradley–Terry model to Best–Worst Scaling (BWS) annotations via maximum likelihood estimation (MLE) is an established approach (relevant literature listed below) that preserves the numerical ratio across samples.

  • Bradley et al., Rank Analysis of Incomplete Block Designs: The Method of Paired Comparisons, 1952.
  • Finn et al., Determining the Appropriate Response to Evidence of Public Concern: The Case of Food Safety, 1992.
  • Marley et al., Some Probabilistic Models of Best, Worst, and Best–Worst Choices, 2005.
  • Geržinić et al., Estimating Decision Rule Differences Between “Best” and “Worst” Choices in a Sequential Best Worst Discrete Choice Experiment, 2021.

Specifically, the cost ratio Li/LjL_i/L_j is given by eη^i−η^je^{\widehat\eta_i-\widehat\eta_j}, with η^=(η^1,…,η^n)\widehat{\boldsymbol\eta}=(\widehat\eta_1,\dots,\widehat\eta_n) denoting the optimal MLE solution over all BWS comparisons. Formally, let C\mathcal{C} denote all annotated comparison choices across the benchmark:

\widehat{\boldsymbol\eta}=\arg\max_{\boldsymbol\eta}\sum_{c\in\mathcal{C}}\log P(c\mid\boldsymbol\eta) \quad\Longrightarrow\quad \widehat P(i\succ j)=\frac{e^{\widehat\eta_i}}{e^{\widehat\eta_i}+e^{\widehat\eta_j}} \quad\text{(Pairwise probability)},$$ $$\frac{\widehat P(i\succ j)}{\widehat P(j\succ i)} =e^{\widehat\eta_i-\widehat\eta_j} =\frac{2e^{\widehat\eta_i-\operatorname{median}_k\widehat\eta_k}} {2e^{\widehat\eta_j-\operatorname{median}_k\widehat\eta_k}} =\frac{L_i}{L_j}\quad\text{(Cost ratio)}.$$ This mapping preserves the pairwise ratios inferred from BWS. We express these costs relative to the $+1$ correctness reward and set the median cost to $2$ based on prospect theory's finding that losses weigh roughly twice as much as comparable gains (Kahneman et al., *Prospect Theory: An Analysis of Decision under Risk*, 1979). We also evaluate sensitivity to such transformations across all 29 models while keeping model outputs and $C_R$ fixed: $$\widetilde L_i(\gamma) =\operatorname{median}_j(L_j)\, \frac{L_i^\gamma}{\operatorname{median}_j(L_j^\gamma)}.$$ For $0<\gamma<1$, the power function $L_i^\gamma$ is monotonically increasing, thereby preserving the ordering of samples. Meanwhile, the cost ratio is compressed from $L_i/L_j$ to $(L_i/L_j)^\gamma$. Under different compression factors, the Spearman correlations of model rankings and the numbers of model pairs that reverse order are shown below. ||Ranking Spearman $\rho$|Reversed model pairs| |---|---:|---:| |$\gamma=0.75$|0.9906|11/406 (2.7%)| |$\gamma=0.5$|0.9823|18/406 (4.4%)| |$\gamma=0.25$|0.9591|31/406 (7.6%)| **The results show that model rankings remain largely robust to the transformation of cost ratios.** However, we preserve the ratio inferred from BWS, as further compression or expansion introduces an additional preference, such as limiting the influence of high-cost samples in the tail. > $\color{blue}{\textbf{W2}}$: UtilityAUC with binary answer/refusal scores mainly measure response utility and answering rates. They provide limited evidence that models can rank beneficial answers ahead of costly mistakes within attempted responses. **A:** **We additionally evaluate five models that provide token log probabilities and find consistent UtilityAUC improvements over binary scoring, driven primarily by ranking correct answers ahead of incorrect ones.** The table below reports UtilityAUC (with changes relative to binary scoring) and the Spearman correlations $\rho(s_i, c_i)$ and $\rho(s_i, -L_i)$, where $s_i$ is the mean token log-probability, $c_i$ is correctness, and $-L_i$ is the negative error cost: |Model|UtilityAUC|Spearman $\rho(s_i, c_i)$|Spearman $\rho(s_i, -L_i)$| |:---|:---:|:---:|:---:| |GPT-5.4 Mini|0.7287 ($\color{blue}{+0.0691}$)|0.467|-0.099| |GPT-5.6 Luna|0.7746 ($\color{blue}{+0.0604}$)|0.391|-0.055| |DeepSeek V4.1 Flash|0.7917 ($\color{blue}{+0.0497}$)|0.432|0.023| |GPT-6.1 Terra|0.8148 ($\color{blue}{+0.0301}$)|0.354|-0.085| |Qwen Plus|0.7814 ($\color{blue}{+0.0586}$)|0.143|-0.064| **As the table shows, using mean token log probability as the response score improves UtilityAUC by better distinguishing correct answers from incorrect ones, yet it does not rank lower-risk errors ahead of higher-risk ones.** How to elicit such risk-aware ordering is an important direction for future work. UtilityAUC is already designed to evaluate this preference, though it remains difficult to obtain ranking signals that align with this design.

Q2\color{blue}{\textbf{Q2}}: What supports interpreting UtilityAUC as a measure of utility rather than alignment with the judges' subjective preferences?

A: Estimating error costs from aggregated BWS judgments is a practical way to approximate real-world consequences, as directly estimating harm ratios across domains can be unreliable. For example:

How many one-hour train delays equal the harm of a medication error that causes permanent injury?

Reaching consensus on this ratio is nearly impossible, but people can generally agree that permanent injury is more severe than a one-hour delay. In BWS, we group samples into overlapping four-sample tuples and ask multiple judges to select the greatest- and least-consequence errors in each tuple (Appendix A.3). The resulting choices reflect numerical cost ratios, as more consistent selection of one error over others indicates a larger severity gap in real-world consequences. Using BWS to extract implicit ratio information from large-scale qualitative comparisons is therefore a reliable and feasible way to approximate real-world error consequences.

To further verify these numerical cost ratios against human judgments, we supplement the ranking analysis in RQ6 by measuring how much two judges disagree on the scale of relative costs across sample pairs P\mathcal P using the root-mean-square error (RMSE):

=\sqrt{\frac{1}{|\mathcal P|}\sum_{(i,j)\in\mathcal P} \left(\log\frac{L_i^{(1)}}{L_j^{(1)}}-\log\frac{L_i^{(2)}}{L_j^{(2)}}\right)^2},$$ where $L^{(1)}$ and $L^{(2)}$ are the costs estimated from each judge, which can be an individual human, an individual LLM, or an aggregation of multiple judges: |Comparison|RMSE| |:---|:---:| |Human vs. Human|1.53| |LLM vs. LLM|1.67| |Human vs. LLM|1.76| |**Human vs. LLM (aggregated)**|**1.10**| **As the table shows, LLM and human cost ratios in our annotations are similar to the variation already present among humans.** Moreover, aggregating the choices on both the human and LLM sides brings the difference well below the disagreement between individual humans. Taking the subjective nature of risk judgments into account, these results further support our aggregated BWS pipeline as a practical estimate of real-world cost differences.

Dear Reviewer eNAn,

Thank you again for the time and effort you have dedicated to reviewing our paper. As the discussion deadline on Nov. 18 (AoE) is approaching, we wanted to kindly follow up and see if you have had a chance to read our response, in which we:

  • Clarified the theoretical basis for deriving numerical cost ratios from BWS comparisons;
  • Verified human–LLM agreement on these numerical cost ratios;
  • Confirmed that model rankings remain stable under cost-ratio compression;
  • Added experiments on models with confidence estimation using token log probabilities.

We would love to hear your thoughts and are more than happy to provide any further clarifications if needed. We sincerely hope that our response fully addresses your remaining concerns and enables you to support the acceptance of our paper.

Best regards,
The Authors of Paper 18333

Thank you again for your time and thoughtful feedback on our paper. As the discussion deadline on Nov. 18 (AoE) is approaching, we wanted to follow up and let you know that we have uploaded the revised manuscript, which incorporates all the new experiments, analyses, and clarifications from our rebuttal:

  • Section 2.2 & Appendix A.4: Expanded the discussion of eliciting implicit cost ratios via BWS comparisons and preserving pairwise ratios Li/Lj=exp⁡(ηi−ηj)L_i/L_j = \exp(\eta_i-\eta_j) through Bradley–Terry maximum likelihood estimation (W1/Q1, Q2\color{blue}{\text{W1/Q1, Q2}}).
  • Appendix A.5: Added the pairwise log-cost ratio RMSE analysis comparing human and LLM annotations to validate the numerical cost scale (Q2\color{blue}{\text{Q2}}).
  • Section 4.4 & Appendix E.5: Added the sensitivity analysis compressing cost ratios via L~i(γ)\widetilde L_i(\gamma) across all 29 evaluated models (W1/Q1\color{blue}{\text{W1/Q1}}).
  • Section 4.3 & Appendix E.4: Added the full-benchmark evaluation of five models using mean token log probability to estimate model confidence, along with the correlation analysis with correctness and error cost (W2\color{blue}{\text{W2}}).
  • Section 4.1: Expanded the main benchmark evaluation from 20 to 29 models (Table 1).

We invite you to check the updated PDF, where all revised parts are highlighted in blue\color{#0066FF}{\textbf{blue}}. Please let us know if any further clarifications or adjustments are needed. We sincerely hope that our response and these revisions resolve your concerns, allowing you to support the acceptance of our paper.

Reviewer_gByXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper identifies a drawback of using accuracy to evaluate model performance, and proposes a novel metric, UtilityAUC, to assess models' practical utility. The authors offer mathematical analyses on the discrepancies between accuracy and UtilityAUC, while perform experiments on 20 LLMs using this metric and a dedicated benchmark (UtilityBench) to showcase the advantages and caveats of said metrics.

Critical strengths
    1. The paper identifies an interesting drawbacks of using accuracy to assess performance of LLMs, and propose a meaningful metric and benchmark dataset for investigation. The mathematical arguments, based on the authors' proposed construct, are sound while the dataset requires a considerable efforts.
    1. The experiments are commendable at scale (20 LLMs) and attempt to address interesting research questions, which reveal interesing insights.
Critical weaknesses
    1. While mathematically solid, the proposed metric is lacking in construct validity, particularly on its correlation to practical significance/cost of a mistake. More specifically, while the paper shows that humans and LLMs generally agree which mistakes are more serious, it does not establish how much more serious those mistakes care. For instance, human raters may agree that a medical error is more serious than an arithmetic error, it is unclear whether assigning a 10x greater cost reflects the actual difference in real-world harm.
    1. The paper offers insufficient emperical evidence that the findings reflect consequence-aware decision making rather than generic refusal. That is, models achieving higher UtilityAUC may just be more refusing more, rather that actually realizing and avoiding mistakes with higher consequences. I suggest the authors to investigate whether high scoring models abstain on high-cost questions, controlling for overall refusal rates.
Questions for authors
    1. While the writing is generally understandable, key empirical details on benchmark constructions are missing from the main text. As an instrumental part of the paper, it should be featured more prominently, preferably with examples.
    1. Can the authors provide more evidence on how well differences in numerical costs (L) reflect differences in real world differences? Currently, human validation establishes agreement on relative severity, but more validation on practical mapping would be welcomed.
    1. How robust are the main UtilityAUC results and model rankings to alternative transformations of L? For example, if we compress the cost scores by a factor yet still preserve the relative ranking, would the main findings and rankings still remain the same?
    1. Since the experiments use binary answers (refuse scores), can we distinguish between models that avoid high-consequence mistakes selectively vs those that just refuse more often?
    1. Related to 4, since half of the dataset is refusal, more evidence is needed to examine the sensitivity of the findings to the proportion unanswerable questions.

These questions concern the central validity of the proposed metrics. While the RQs presented in the current paper are interesting, I would consider addressing these more prominently in the main text.

5 comments

Thank you for your thoughtful review. We address your concerns below.

W1/Q2\color{blue}{\textbf{W1/Q2}}: Human–LLM agreement supports the ordering of error severity, but how well do the numerical ratios between consequence costs capture real-world differences?

A: Human–LLM differences in estimated cost ratios are comparable to the differences among human annotators, and aggregating judgments on both sides substantially improves their agreement. To supplement the ranking analysis in RQ6, we measure how much two judges disagree on the scale of relative costs across sample pairs P\mathcal P using the root-mean-square error (RMSE):

=\sqrt{\frac{1}{|\mathcal P|}\sum_{(i,j)\in\mathcal P} \left(\log\frac{L_i^{(1)}}{L_j^{(1)}}-\log\frac{L_i^{(2)}}{L_j^{(2)}}\right)^2},$$ where $L^{(1)}$ and $L^{(2)}$ are the costs estimated from each judge, which can be an individual human, an individual LLM, or an aggregation of multiple judges: |Comparison|RMSE| |:---|:---:| |Human vs. Human|1.53| |LLM vs. LLM|1.67| |Human vs. LLM|1.76| |**Human vs. LLM (aggregated)**|**1.10**| **As the table shows, LLM and human cost ratios in our annotations are similar to the variation already present among humans.** Moreover, aggregating the choices on both the human and LLM sides brings the difference well below the disagreement between individual humans. Taking the subjective nature of risk judgments into account, these results further support our aggregated BWS pipeline as a practical estimate of real-world cost differences. > $\color{blue}{\textbf{W2/Q4}}$: Does a higher UtilityAUC imply selective avoidance of high-consequence errors, or simply a lower answer rate? Do high-scoring models answer fewer high-cost questions when controlling for overall answer rates? **A:** **UtilityAUC is the uniform average of utility $v_i(\tau)$ across different answer rates. Therefore at any given answer rate $\tau$, avoiding high-consequence errors naturally improves UtilityAUC.** We demonstrate this both theoretically and empirically. In our paper, UtilityAUC is defined as: $$\mathrm{UtilityAUC} =\frac{\mathcal A-\mathcal A^-}{\mathcal A ^+-\mathcal A^-}, \qquad \mathcal A=\int_0^1 \frac{1}{n}\sum_{i=1}^{n}v_i(\tau)\,d\tau =\mathbb{E}_{\tau\sim\mathcal U(0,1)}\left[\frac{1}{n}\sum _{i=1}^{n}v _i(\tau)\right],$$ where $v_i(\tau)$ is the response **utility** of sample $i$ at answer rate $\tau$, and $\mathcal A^+, \mathcal A^-$ are the theoretical maximum and minimum areas. Thus, UtilityAUC computes the uniform expectation of utility over answer rates $\tau\in[0,1]$. If we restrict two models $S$ and $T$ to a specific answer rate $\tau_0$ using a Dirac delta distribution $\delta(\tau-\tau_0)$, the difference between their UtilityAUC is: $$\mathrm{UtilityAUC}(S)-\mathrm{UtilityAUC}(T) =\frac{1}{\mathcal A ^+-\mathcal A ^-} \left( \mathbb{E} _{\tau\sim\delta(\tau-\tau_0)}\left[\frac{1}{n}\sum _{i=1}^{n}v _i^S(\tau)\right] -\mathbb{E} _{\tau\sim\delta(\tau-\tau_0)}\left[\frac{1}{n}\sum _{i=1}^{n}v _i^T(\tau)\right] \right)$$ $$=\frac{1}{\mathcal A^+-\mathcal A^-} \underbrace{\left( \frac{1}{n}\sum_{i=1}^{n} v_i^S(\tau_0)- \frac{1}{n}\sum_{i=1}^{n} v_i^T(\tau_0) \right)}_{\text{difference in utility}}.$$ Since the coefficient $\frac{1}{\mathcal A^+-\mathcal A^-}$ is a positive constant, the model with higher utility achieves a higher UtilityAUC. To examine this empirically, we compare GPT-6.1 Terra, GPT-5.1, and Claude Haiku 4.5. The tables below report their answer rates (AR) across the five risk bands alongside their overall AR and UtilityAUC. |Error Cost $L_i$|**GPT-6.1 Terra**|**GPT-5.1**|**Claude Haiku 4.5**| |---|:---|:---|:---| ||Answer Rate|Answer Rate|Answer Rate| |**Very low**|$\color{#6A428A}{\mathbf{10.25\%}}$|$\color{#8965A3}{\mathbf{7.60\%}}$|$\color{#AA89BF}{\mathbf{6.12\%}}$| |**Low**|$\color{#6A428A}{\mathbf{33.71\%}}$|$\color{#8965A3}{\mathbf{33.60\%}}$|$\color{#AA89BF}{\mathbf{28.73\%}}$| |**Medium**|$\color{#6A428A}{\mathbf{77.12\%}}$|$\color{#8965A3}{\mathbf{74.78\%}}$|$\color{#AA89BF}{\mathbf{74.04\%}}$| |**High**|$\color{#AA89BF}{\mathbf{75.40\%}}$|$\color{#8965A3}{\mathbf{80.36\%}}$|$\color{#6A428A}{\mathbf{86.22\%}}$| |**Very high**|$\color{#AA89BF}{\mathbf{86.76\%}}$|$\color{#8965A3}{\mathbf{89.64\%}}$|$\color{#6A428A}{\mathbf{94.06\%}}$| |**Metric**|**GPT-6.1 Terra**|**GPT-5.1**|**Claude Haiku 4.5**| |---|:---|:---|:---| |**Overall AR**|$\mathbf{60.83\%}$|$\mathbf{60.73\%}$|$\mathbf{60.07\%}$| |**UtilityAUC**|$\color{#6A428A}{\mathbf{0.7853}}$|$\color{#8965A3}{\mathbf{0.7167}}$|$\color{#AA89BF}{\mathbf{0.6676}}$| **The three models have similar overall answer rates, yet they achieve substantially different UtilityAUC.** Specifically, higher-scoring models answer fewer questions in the higher-risk bands and more questions in the lower-risk bands. This pattern demonstrates that higher UtilityAUC reflects selective avoidance of costly errors rather than a lower overall answer rate.

Q1\color{blue}{\textbf{Q1}}: The main text should describe the key steps of benchmark construction more prominently, preferably with examples.

A: In the revision, we will expand the discussion to include the key construction steps and representative examples, making the benchmark easier to understand without relying on the appendix.

Q3\color{blue}{\textbf{Q3}}: How sensitive are model rankings to transformations of LiL_i, such as compressing cost scores by a factor while preserving their relative ordering?

A: Model rankings are largely insensitive to the tested transformations of the LiL_i scale. We recompute UtilityAUC for all 29 models, including 9 newly evaluated models, with the following transformation of LiL_i:

=\operatorname{median}_j(L_j)\, \frac{L_i^\gamma}{\operatorname{median}_j(L_j^\gamma)}.$$ For $0<\gamma<1$, the power function $L_i^\gamma$ is monotonically increasing, thereby preserving the ordering of samples. Meanwhile, the cost ratio is compressed from $L_i/L_j$ to $(L_i/L_j)^\gamma$. We also keep the median cost at $2$ based on prospect theory's finding that losses weigh roughly twice as much as comparable gains (Kahneman et al., *Prospect Theory: An Analysis of Decision under Risk*, 1979). Under different compression factors, the Spearman correlations of model rankings and the numbers of model pairs that reverse order are shown below. ||Ranking Spearman $\rho$|Reversed model pairs| |---|:---|:---| |$\gamma=0.75$|0.9906|11/406 (2.7%)| |$\gamma=0.5$|0.9823|18/406 (4.4%)| |$\gamma=0.25$|0.9591|31/406 (7.6%)| These results show that model rankings remain largely stable under cost compression. However, we preserve the ratio inferred from BWS in the paper, as further compression or expansion introduces an additional preference, such as limiting the influence of high-cost samples in the tail. > $\color{blue}{\textbf{Q5}}$: Half of the benchmark consists of refusal-required queries. How sensitive are the findings to this proportion? **A:** **Our 50/50 split provides a balanced default for evaluating both answering and refusal. As for the sensitivity analysis, we also find model rankings remain relatively stable as the proportion of refusal-required samples changes.** We randomly sample subsets at each proportion and recompute UtilityAUC for all 29 models. The table reports mean ± standard deviation. |Refusal-required proportion|Ranking Spearman $\rho$|Reversed model pairs| |---|:---|:---| |10%|0.7908 ± 0.0032|19.7 ± 0.4%| |25%|0.8769 ± 0.0103|13.7 ± 0.8%| |40%|0.9634 ± 0.0111|6.5 ± 1.2%| |60%|0.9854 ± 0.0051|3.8 ± 0.9%| |75%|0.8682 ± 0.0151|15.0 ± 1.1%| |90%|0.7647 ± 0.0112|21.0 ± 0.6%| The resulting model rankings remain largely robust to moderate distribution shifts. However, when the proportion of refusal-required samples can be estimated for a specific application, evaluators can sample a subset to better match their desired usage.

Dear Reviewer gByX,

Thank you again for the time and effort you have dedicated to reviewing our paper. As the discussion deadline on Nov. 18 (AoE) is approaching, we wanted to kindly follow up and see if you have had a chance to read our response, in which we:

  • Verified human–LLM agreement on the numerical cost ratios;
  • Demonstrated both theoretically and empirically that higher UtilityAUC reflects selective avoidance of high-consequence errors at equal answer rates;
  • Confirmed that model rankings remain stable when compressing the error costs LiL_i or varying the proportion of refusal-required queries;
  • Committed to including key benchmark construction steps and illustrative examples in the main text.

We would love to hear your thoughts and are more than happy to provide any further clarifications if needed. We sincerely hope that our response fully addresses your remaining concerns and enables you to support the acceptance of our paper.

Best regards,
The Authors of Paper 18333

Reviewer_pWuyOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that mainstream evaluation, penalizes alll refusals error equally. Paper adds per-question cost of being wrong and propose utilitybench attaches a data-driven severity estimate. They propose UtilityAUC scores the area under the utility curve induced by a model’s response ordering under these costs, where accuracy and AUROC cannot separate errors by consequence. Across 20 models, the paper reports that higher accuracy or more advanced architectures do not always give higher UtilityAUC score. This gap is interpreted as gap between problem-solving ability and responsible decision-making.

Critical strengths

The key problem of models being calibrated to refusal is still a relevant topic. The human annotations could be useful for this caliberation.

Critical weaknesses

The score is not strong as a refuse all baseline post normalization works well on the metric. If my understanding is right, a refuse all baseline could score higher than some of reported models, and the paper reports no trivial baselines that way.

Reported mean cost, this baseline scores at least 0.67 UtilityAUC? This is already above GPT-5.4 Mini and Haiku 4.5. These are strong models scoring below a reject all doesn't make sense if the number readouts are right.

Models are not really given the true advantage of knowing the objective. The main prompt says only “answer if you are confident, otherwise say ‘I cannot answer’.” This has no info on what is being evaluated, no costs or threshold or price for refusing. So its good to see how models behave when the metric reward used for scoring is known (its easy responsibility control).

The previous point is more important as the prompt with cues alone moves scores by 0.07–0.13 I guess (Table 4), about the whole spread between models. The claimed “gap between problem-solving ability and responsible decision-making” may largely reflect how each model reads the word “confident.”

Questions for authors

Not sure why the correct answers were all chosen +1. Wrong answers carry a question-specific cost; correct answers, correct refusals and unnecessary refusals all get fixed values. That asymmetry could explain “higher-stakes questions need higher confidence”. Could it be the case that, if a correct high stake answer were also worth more than a correct trivia answer, so that every outcome scaled with stakes, the stakes would cancel and the confidence threshold would be the same for every question.

The closest LLM work is not cited, and some of it anticipates the paper’s findings. The classical cost-sensitive evaluation literature is also ignored. Not listing all but the takeaway from previous work I feel is not correctly represented. AgentAbstain (NeurIPS 2026) already shows that the ability to abstain is unrelated to general capability and AA-Omniscience and SimpleQA already penalize confident errors relative to abstaining.

6 comments

Thank you for your thoughtful review. We address your concerns below.

W1\color{blue}{\textbf{W1}}: The all-refusal baseline performs strongly, potentially exceeding capable models.

A: The all-refusal baseline's ranking depends on the refusal-cost hyperparameter CRC_R. Our default CR=2C_R=2 favors caution, while larger values favor more active assistance and substantially lower the baseline's ranking. The table below shows the UtilityAUC rankings at different CRC_R values, with model responses held fixed and selected ranks displayed.

RankCR=0C_R=0CR=1C_R=1CR=2C_R=2CR=3C_R=3CR=4C_R=4CR=8C_R=8
#1All Refuse\color{red}{\textbf{All Refuse}}Gemini 4 ArgonGemini 4 ArgonGemini 4 ArgonGemini 4 ArgonGemini 4 Argon
#2Gemini 4 ArgonAll Refuse\color{red}{\textbf{All Refuse}}Gemini 3.6 FlashGemini 3.6 FlashGemini 3 FlashGemini 3 Flash
…………………
#9Claude Sonnet 5Claude Sonnet 5All Refuse\color{red}{\textbf{All Refuse}}Kimi K2.6Qwen3.7 PlusClaude Sonnet 5
…………………
#26GPT-5.1GPT-5.1GPT-5.1All Refuse\color{red}{\textbf{All Refuse}}GPT-5.6 LunaGPT-5.6 Luna
#27GPT-5.6 LunaGPT-5.6 LunaGPT-5.6 LunaGPT-5.6 LunaClaude Haiku 4.5Claude Haiku 4.5
#28Claude Haiku 4.5Claude Haiku 4.5Claude Haiku 4.5Claude Haiku 4.5All Refuse\color{red}{\textbf{All Refuse}}MiMo V2.6 Flash
#29MiMo V2.6 FlashMiMo V2.6 FlashMiMo V2.6 FlashMiMo V2.6 FlashMiMo V2.6 FlashGPT-5.4 Mini
#30GPT-5.4 MiniGPT-5.4 MiniGPT-5.4 MiniGPT-5.4 MiniGPT-5.4 MiniAll Refuse\color{red}{\textbf{All Refuse}}

This comparison includes 9 newly evaluated models.

The strong all-refusal ranking at CR=2C_R=2 reflects the practical value of avoiding costly mistakes. Leaving a task unresolved can be preferable to completing it incorrectly. In fact, AA-Omniscience also reports that an all-abstention baseline would rank #4 out of 37. We will report the all-refusal baseline and its dependence on application preferences in the revision.

W2\color{blue}{\textbf{W2}}: Models are not told the scoring objective. The utility differences may reflect different interpretations of “answer if confident” rather than differences in risk-sensitive decision-making.

A: As a Datasets and Benchmarks\color{blue}{\textbf{Datasets and Benchmarks}} track submission, our work focuses on evaluating existing models\color{blue}{\textbf{evaluating existing models}} instead of training new ones. In real applications, models rarely receive exact risk values and must estimate them from context. Therefore, it is reasonable that the models are not provided with the scoring objective in the main experiments.

To test whether encouraging the model to evaluate potential consequence improves model decisions, we include a Consequence-Guided prompt in our prompting experiments (Table 4; exact prompt in Appendix E.1). This prompt explicitly informs models of the behavior expected by the evaluation, asking them to first estimate the risks and then answer more actively in low-risk situations and exercise greater caution in high-risk ones.

To further address your suggestion, we supplement our work with a condition that supplies numerical LiL_i alongside behavior instructions. The table reports changes relative to the Refusal Hint used in the main evaluation. (answer rate in percentage points and UtilityAUC in score units)

Error Cost LiL_iDeepSeek V4 Flash Con. GuidedDeepSeek V4 Flash Input LiL_iMiMo V2.5 Con. GuidedMiMo V2.5 Input LiL_iQwen3.7 Plus Con. GuidedQwen3.7 Plus Input LiL_i
Answer RateAnswer RateAnswer RateAnswer RateAnswer RateAnswer Rate
Very low+6.28{+6.28}+21.49{+21.49}+4.63{+4.63}+13.22{+13.22}+13.72{+13.72}+13.88{+13.88}
Low+13.36{+13.36}+17.07{+17.07}+6.72{+6.72}+13.40{+13.40}+13.36{+13.36}+14.73{+14.73}
Medium+9.16{+9.16}+5.74{+5.74}+0.98{+0.98}+2.65{+2.65}+11.85{+11.85}+9.36{+9.36}
High−1.17\color{red}{-1.17}−21.28\color{red}{-21.28}+1.42{+1.42}−34.48\color{red}{-34.48}+21.94{+21.94}+2.08{+2.08}
Very high−2.38\color{red}{-2.38}−23.94\color{red}{-23.94}+3.06{+3.06}−66.72\color{red}{-66.72}+10.70{+10.70}−4.41\color{red}{-4.41}
MetricDeepSeek V4 Flash Con. GuidedDeepSeek V4 Flash Input LiL_iMiMo V2.5 Con. GuidedMiMo V2.5 Input LiL_iQwen3.7 Plus Con. GuidedQwen3.7 Plus Input LiL_i
Δ\DeltaUtilityAUC+0.0220{+0.0220}+0.0496{+0.0496}−0.0239\color{red}{-0.0239}+0.0784{+0.0784}−0.1015\color{red}{-0.1015}−0.0178\color{red}{-0.0178}

The Consequence-Guided prompt produces inconsistent behavioral changes across models, while supplying LiL_i leads all three models towards more refusals on high-risk queries. However, the effectiveness in improving utility differs across models, as UtilityAUC also considers correctness. In other words, even with explicit risk estimates and a stated behavioral objective, it remains challenging to utilize such information effectively to achieve consistent utility improvement. We hope our work can inspire further exploration of how models assess risks and translate them into effective decisions, which could be a promising direction for responsible AI.

Q1\color{blue}{\textbf{Q1}}: Why assign +1+1 to all correct answers, and what happens if the success reward is different?

A: Utility is relative, and UtilityAUC remains unchanged when all utilities are multiplied by a positive factor (proof below). We choose +1+1 as the success reference for convenience in computation and analysis.

Suppose instead that we assign R>0R>0 to correct answers, ℓi\ell_i to error costs, and cc to unnecessary refusal costs. The outcome utilities, decision threshold, and UtilityAUC then become the following, where AR\mathcal A_R and AR±\mathcal A_R^{\pm} denote the observed and reference areas under these utilities:

\boldsymbol u_i=(R,-\ell_i,R,-c) \iff \frac{\boldsymbol u_i}{R} =\left(1,-{\color{blue}{\frac{\ell_i}{R}}},1,-{\color{blue}{\frac{c}{R}}}\right),$$ $$\text{Decision threshold:}\quad p_iR-(1-p_i)\ell_i\geq-c \iff p_i\geq\frac{\ell_i-c}{R+\ell_i} =\frac{{\color{blue}{\ell_i/R}}-{\color{blue}{c/R}}}{1+{\color{blue}{\ell_i/R}}},$$ $$\text{UtilityAUC:}\quad \frac{\mathcal A_R-\mathcal A_R^-}{\mathcal A_R^+-\mathcal A_R^-} =\frac{\mathcal A_R/R-\mathcal A_R^-/R}{\mathcal A_R^+/R-\mathcal A_R^-/R},$$ $$\frac{\mathcal A_R}{R} =\frac{1}{R}\int_0^1\frac{1}{n}\sum_{i=1}^{n}v_i^R(\tau)\,d\tau =\int_0^1\frac{1}{n}\sum_{i=1}^{n}{\color{blue}{\frac{v_i^R(\tau)}{R}}}\,d\tau.$$ Now write $L_i={\color{blue}{\ell_i/R}}$ and $C_R={\color{blue}{c/R}}$, which naturally gives the equivalence $v_i(\tau)={\color{blue}{v_i^R(\tau)/R}}$. This recovers our original formulation: successful completion receives $+1$, the decision threshold is $(L_i-C_R)/(1+L_i)$, and normalized UtilityAUC is unchanged. As a result, for a question with error cost $L_i$, answering correctly rather than incorrectly improves utility by $1+L_i$. This suggests correctly answering a higher-risk question yields a greater improvement relative to getting it wrong, even though successful completion shares the same reference value of $+1$. > $\color{blue}{\textbf{Q2}}$: The discussion of related work is incomplete. **A: We will expand the related-work discussion to acknowledge prior findings and clarify our contribution.** Specifically, we will add or strengthen the following comparisons: - **LLM abstention.** AgentAbstain already shows that abstention capability is largely independent of general task-solving capability; our contribution builds on this distinction by evaluating the utility of answering and refusing under heterogeneous error consequences. - **Factual reliability.** SimpleQA distinguishes correct, incorrect, and unattempted responses, and AA-Omniscience explicitly penalizes errors relative to abstention, while our evaluation additionally differentiates errors by their question-specific consequences and distinguishes appropriate from unnecessary refusals. - **Cost-sensitive decision-making.** Classical cost-sensitive learning and learning to defer already discussed in our appendix, provide the decision-theoretic foundation; our work develops a benchmark with estimated per-question consequence costs and jointly evaluates the four answer/refusal outcomes.

Dear Reviewer pWuy,

Thank you again for the time and effort you have dedicated to reviewing our paper. As the discussion deadline on Nov. 18 (AoE) is approaching, we wanted to kindly follow up and see if you have had a chance to read our response, in which we:

  • Analyzed how the all-refusal baseline's ranking changes across different refusal penalties;
  • Tested providing the numerical error cost LiL_i directly in the prompt;
  • Proved that UtilityAUC is invariant to rescaling the +1+1 correctness reward;
  • Expanded our related-work discussion.

We would love to hear your thoughts and are more than happy to provide any further clarifications if needed. We sincerely hope that our response fully addresses your remaining concerns and enables you to support the acceptance of our paper.

Best regards,
The Authors of Paper 18333

Comments on the paper

2 comments

Dear All,

We sincerely thank all the reviewers for your thoughtful feedback. Within the limited rebuttal period, we have made our best effort to address every concern with new experiments, new theoretical analyses, and detailed clarifications.


Our main efforts are summarized as follows:

  • Validated the numerical error costs LiL_i and ranking robustness (gByX W1/Q2, Q3, Q5\color{blue}{\text{gByX W1/Q2, Q3, Q5}}, eNAn W1/Q1, Q2\color{blue}{\text{eNAn W1/Q1, Q2}}). In UtilityBench, each question is assigned a sample-specific error cost LiL_i derived from Best–Worst Scaling (BWS) comparisons to capture the real-world severity of making a mistake. We validated these numerical costs and their impact on model rankings from three perspectives:

    • Justification of BWS cost ratios (eNAn W1/Q1, Q2\color{blue}{\text{eNAn W1/Q1, Q2}}): Directly estimating numerical harm ratios across diverse domains is unreliable, so we use Best–Worst Scaling (BWS) to elicit implicit cost ratios from large-scale qualitative comparisons. Furthermore, we showed that fitting a Bradley–Terry model to BWS annotations via maximum likelihood estimation is an established approach that preserves the pairwise cost ratios Li/LjL_i/L_j implied by the BWS choices.
    • Human–LLM agreement on numerical cost ratios (gByX W1/Q2\color{blue}{\text{gByX W1/Q2}}, eNAn Q2\color{blue}{\text{eNAn Q2}}): To validate the resulting numerical scale against human judgments, we computed the root-mean-square error (RMSE) of pairwise log-cost ratios. The results show that human–LLM differences (RMSE = 1.76) are comparable to the natural variation among individual humans (1.53). Moreover, aggregating judgments on both sides reduces the human–LLM RMSE to 1.10, well below individual human disagreement.
    • Robustness to cost-ratio compression and dataset composition (gByX Q3, Q5\color{blue}{\text{gByX Q3, Q5}}, eNAn W1/Q1\color{blue}{\text{eNAn W1/Q1}}): To address concerns about sensitivity to cost ratios and dataset composition, we compressed the cost ratios via monotonic power-law transformations LiγL_i^\gamma. Across all 29 models, the rankings remain highly stable with Spearman ρ=0.96–0.99\rho = 0.96\text{–}0.99. Furthermore, varying the proportion of refusal-required queries across the benchmark also maintains stable model rankings (ρ=0.87–0.99\rho = 0.87\text{–}0.99 across 25−7525-75%).
  • Clarified the theoretical properties and behavioral interpretation of UtilityAUC (pWuy W1, Q1\color{blue}{\text{pWuy W1, Q1}}, gByX W2/Q4\color{blue}{\text{gByX W2/Q4}}). We provided theoretical proofs and empirical analyses to address reviewer concerns regarding how UtilityAUC behaves across baselines, answer rates, and utility designs:

    • Included an all-refusal baseline and analyzed its ranking (pWuy W1\color{blue}{\text{pWuy W1}}): Reviewer pWuy noted that the all-refusal baseline, which refuses every query, achieves a high ranking under our default setting. We showed that this ranking depends on the refusal-cost hyperparameter CRC_R and our default CR=2C_R=2 favors caution, while larger values favor active assistance and substantially lower the all-refusal baseline's ranking. This reflects the practical trade-off between avoiding costly mistakes and providing active assistance.
    • Demonstrated selective avoidance of high-consequence errors at equal answer rates (gByX W2/Q4\color{blue}{\text{gByX W2/Q4}}): Reviewer gByX asked whether higher UtilityAUC reflects selective avoidance of high-consequence errors or simply a lower answer rate. Theoretically, because UtilityAUC averages utility uniformly across answer rates, avoiding high-consequence errors at any given answer rate directly improves UtilityAUC. Empirically, comparing three models with nearly identical answer rates shows substantial differences in UtilityAUC, as higher-scoring models answer fewer high-risk questions and more low-risk ones.
    • Proved the invariance of UtilityAUC to reward scaling (pWuy Q1\color{blue}{\text{pWuy Q1}}): We proved mathematically that utility is relative, so multiplying the +1+1 correctness reward by any positive factor leaves both the optimal decision threshold and UtilityAUC unchanged. Under this formulation, answering a question with error cost LiL_i correctly instead of incorrectly improves utility by 1+Li1+L_i, meaning correctly answering a higher-risk question still yields a greater utility improvement.
  • Expanded model evaluations and conducted new experiments on prompting and model confidence (pWuy W2\color{blue}{\text{pWuy W2}}, eNAn W2\color{blue}{\text{eNAn W2}}). We broadened our empirical evaluation on UtilityBench to include the latest models and to address reviewer questions on explicit risk prompting and token-level confidence:

    • Expanded the benchmark to include the latest models: We evaluated newly released models on UtilityBench, including flagship models such as Gemini 4 Argon, GPT-6 Astra, and Claude Opus 5.5. This brings the total number of models on our leaderboard to 29.
    • Evaluated prompting with explicit error costs LiL_i (pWuy W2\color{blue}{\text{pWuy W2}}): Our main evaluation does not provide scoring parameters in the prompt, as our Datasets and Benchmarks submission focuses on evaluating existing models in realistic settings where exact risk values are rarely given. To address Reviewer pWuy's suggestion, we additionally tested providing the error cost LiL_i directly in the prompt. While supplying LiL_i leads all tested models to refuse high-risk queries more often, we found translating this explicit risk information into higher overall UtilityAUC remains challenging for these LLMs.
    • Incorporated token log probabilities to estimate model confidence (eNAn W2\color{blue}{\text{eNAn W2}}): In our main evaluation, UtilityAUC is computed from binary answer-or-refuse decisions because many closed API models do not expose token probabilities. To address Reviewer eNAn's question on whether continuous confidence signals can prioritize beneficial answers over costly mistakes, we used mean token log probability to estimate model confidence across 5 models on UtilityBench. Doing so consistently improves UtilityAUC by assigning higher confidence to correct answers than to incorrect ones, yet it still fails to assign lower confidence to higher-risk errors than to lower-risk ones.
  • Detailed clarifications and revision plans (pWuy Q2\color{blue}{\text{pWuy Q2}}, gByX Q1\color{blue}{\text{gByX Q1}}). In the revision, we will expand our related-work discussion on LLM abstention, factual reliability, and cost-sensitive learning (pWuy Q2\color{blue}{\text{pWuy Q2}}), and include key benchmark construction steps and examples in the main text (gByX Q1\color{blue}{\text{gByX Q1}}).


We truly understand and respect the reviewers' schedule. However, we believe that meaningful reviewer engagement during the discussion phase is essential to a fair and thorough peer review process. We sincerely appreciate your time and support, and we are kindly encouraging all the reviewers to engage in the discussion and let us know whether your concerns have been adequately addressed.

Best regards,

The Authors of Paper 18333

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.