acceptodds
Under review as a conference paper at ICLR 2027

Beyond Single-Threshold Fairness: Residual Distributions and Group-Level Error Rates

Abstract

Fairness audits often compare groups at a single decision threshold, even though deployed thresholds can vary across settings and over time. We study how false-negative and false-positive differences between groups change over the full threshold range. Using the signed residual, we show that the Wasserstein distance between two groups' residual distributions equals the total area between their group FN and FP curves as the threshold varies. The two components show which error type drives the difference and how prevalence connects these group rates to FNR and FPR. We construct exactly calibrated populations that agree on the examined accuracy, ranking, calibration, average-error, and single-threshold fairness summaries while having different residual distributions and error gaps at other thresholds. Experiments on Adult, ACSIncome, HateXplain, and Measuring Hate Speech show that calibration can move group mean residuals closer to zero while residual distance increases. They also show that different prevalences can enlarge or reduce group FN/FP gaps. Across ACSIncome model configurations, validation FN and FP components preserve rankings of related test-time error differences more closely than fixed-threshold equalized odds or a mean-residual summary for the evaluated targets. Together, residual distributions provide a direct way to audit group error differences across decision thresholds.

Primary area:
interpretability and explainable AI
Keywords:
Fair AI, Fairness auditing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Q7DYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a fairness metric involving the (signed) residual distribution of a binary probabilistic classifier. Their main contribution is showing the Wasserstein distance between the protected attribute conditional residual distributions is related to averaged absolute gaps of familiar aggregate statistics of (group) FPR and FNR. In their experiments, they investigate several related questions concerning their introduced metric, such as the effect of model calibration and group prevalence, as well as whether their metric ranks models between the validation and test set more accurately than alternatives such as equalized odds at a fixed threshold.

Critical strengths
  1. Authors introduce a creative and intuitive object, the residual distribution, and connect it with the familiar classification metrics of FPR and FNR via prevalence rates. This distributional audit has a nice geometric and intuitive interpretation, which the authors investigate thoroughly.
  2. The experiments explore natural follow up questions, and they provide analyses with a diverse range of datasets including tabular and text classification.
Critical weaknesses
  1. The authors do not sufficiently justify the use of their introduced metric. It is unclear why their approach is preferable to other distributional fairness metrics, e.g., comparing a models' EO and FPR and FNR across (non fixed) thresholds, or other methods mentioned in their related work section-- justification for their method is argued by comparing against a classifier with a fixed threshold, but this is not the natural baseline to compare against, as distributional metrics naturally aggregate information across thresholds.

  2. Beyond the identity in Equation (3), the overall contribution seems limited. The authors do point out that existing literature for distribution based metrics exist (Becker et al., 2024), but do not directly compare against it (e.g., for a task with model selection). They do provide experiments for model selection, but these primarily concern the stability of their introduced metric (measured via ranking and Spearman correlation), and provide limited evidence that this metric is preferable relative to existing alternatives.

Questions for authors
  1. What additional practical benefit does your introduced residual formulation provide over existing distributional aware fairness measures (e.g., Becker et al. (2024))? For example, can this benefit be demonstrated in a concrete model selection experiment?

  2. On a related note, could the authors add empirical results that rank the models according to overall classification accuracy for various thresholds, rather than the FN cost gap between groups?

  3. (Future work) Could the metric be extended to multiple classes? And can your residual distribution formulation motivate a post-processing algorithm that adjusts the scores to improve (group) fairness?

1 comment

We thank the reviewer for recognizing the intuitive interpretation of residual distributions and the connection between our formulation and familiar classification error measures.

1. Additional practical value over existing distributional fairness measures

Our formulation provides a complementary auditing perspective by explicitly incorporating group prevalence into the distributional analysis of classification errors.

While outcome-conditional Wasserstein measures characterize differences in score distributions within each label class, residual Wasserstein distance captures the resulting prevalence-aware group-level FN/FP burdens.

This distinction becomes important when group prevalences differ, because similar conditional error disparities can correspond to different proportions of individuals experiencing classification errors.

We demonstrate this through two complementary analyses.

First, in a controlled synthetic construction, two models have the same conditional Wasserstein disparity (E=0.20E=0.20) but different residual distances (0.16 and 0.04) at prevalence 0.20. The residual formulation distinguishes how positive- and negative-label disparities contribute to group-level error burden.

Second, our ACSIncome experiments reveal differences in demographic group-pair rankings. For instance, Black–White changes from rank 2 under conditional Wasserstein distance to rank 5 under residual distance, while Asian–Two or More Races changes from rank 6 to rank 3 (ascending disparity rankings).

These findings demonstrate that residual-based auditing provides a different prioritization of group disparities when the auditing objective incorporates group prevalence.

2. Model selection and predictive utility

We additionally evaluated model selection under different fairness objectives while monitoring predictive accuracy.

On Adult, residual-based selection produces a prevalence-weighted false-negative gap of 0.057, compared with 0.064 under fixed-threshold EO selection, with the selected models remaining within a comparable predictive accuracy range of approximately 0.871.

This illustrates how the choice of fairness measure influences model selection according to the type of error disparity being prioritized.

The FN/FP decomposition further allows these contributions to be examined separately, making the relationship between fairness objectives and predictive utility more interpretable.

3. Extensions to multiclass classification and post-processing

Our current formulation considers binary probabilistic classification, where signed residuals connect naturally to threshold-dependent false-negative and false-positive errors.

A multiclass extension could examine class-specific residual distributions and their corresponding prevalence-weighted error contributions.

Likewise, the decomposition provides a foundation for investigating post-processing procedures that target specific group-level error burdens.

In summary, the residual formulation contributes an interpretable, prevalence-aware perspective on distributional fairness, connecting classification scores, group prevalence, and the FN/FP burdens experienced by different demographic groups across decision thresholds.

Reviewer_kYAzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes using the Wasserstein distance between the group residual distributions as a measure of fairness across decision thresholds. Their main result is that this distance is equal to the sum of the integrated group false negative and false positive disparities (the group level disparities in the misclassification rates), which are in turn a function of the group label prevalence. Experiments on Adult, ACSIncome and two text classification datasets look at the effect of calibration, prevalence and threshold choice. The authors also perform an experiment on 432 different ACSIncome model configurations and find that their measures are more closely correlated with the corresponding test set error disparities than a number of other existing fairness metrics.

Critical strengths

Eq 3 shows that the distance between the group residual distributions is equal to the sum of the integrated FN and FP differences across thresholds. This decomposition allows us to see which type of error (FN or FP) is the dominant contributor to the overall disparity. It also takes into account differences in within-group label prevalence which the conditional FNR/FPR measures do not directly.

Critical weaknesses
  1. Limited methodological novelty.

Becker et al. (2024) had already established Wasserstein based measures of conditional FNR/FPR disparity across classification thresholds. The key conceptual difference in this paper is the inclusion of group prevalence, which results in a measure of unconditional group error disparity rather than conditional error rate disparity. It's a different fairness measure, the residual Wasserstein decomposition is a direct consequence of the standard one-dimensional Wasserstein identity. The theoretical development is therefore a straightforward extension of existing work. The real value of the paper is in the empirical evidence it provides to demonstrate the added practical value of the new measure and it's here that we need to see much stronger evidence.

  1. Weak empirical evidence (Table 4).

I am not convinced that the experiments establish the additional value of the proposed measure over existing fairness measures.

The high correlations of 0.941 (Avg. FN) and 0.797 (Avg. FP) are validation-to-test self-correlations. They show that the proposed metrics are reasonably stable, but do not establish an advantage over other fairness measures.

Eq 7 shows that the FN-cost gap is bounded above by BFN(i,j)B_\mathrm{FN}(i,j). So the FN cost gap is structurally related to the proposed FN component. Although Table 4 uses an average FN gap over a restricted threshold interval, both quantities are based on the same prevalence-weighted FN disparity curve.

The last target value in Table 4, High Confidence FP (0.7), is not defined in the paper. Inferring from line 257-259, it appears to be ∣GFPRi(0.7)−GFPRj(0.7)∣|\mathrm{GFPR}_i(0.7) - \mathrm{GFPR}_j(0.7)|, a point evaluation of the curve underlying the proposed FP component.

Therefore, by choice, all four test targets structurally favor the proposed metrics over other fairness compared measures compared. The experiments show that the proposed metrics are consistent with structurally related quantities, but provide limited independent evidence of their additional practical value. This is a significant weakness given the relatively limited methodological novelty.

Questions for authors
  1. Becker et al. (2024) already use the Wasserstein distance to measure the difference in conditional error rates across thresholds. The main novelty in this paper seems to be the incorporation of label prevalence via the residual formulation. Can the authors be more specific as to what is theoretically novel beyond that extension and the standard one dimensional Wasserstein identity?

  2. The correlations reported in Table 4 are with respect to test targets closely related to the proposed metrics. The Avg. FN and Avg. FP comparisons include validation-to-test self-correlations. The FN cost gap is bounded by BFNB_{\mathrm{FN}} as per Eq 7, the high confidence FP gap is correlated with the group FP rate at a given threshold (if my understanding is correct). What value do these comparisons add over other fairness measures? Are the better performance of the proposed metrics simply a reflection of the choice of test targets?

  3. Table 4 uses "High confidence FP (0.7)" as its test target but I can't actually find a precise mathematical definition of it. Why was 0.7 selected and was that threshold pre-specified?

1 comment

We thank the reviewer for highlighting the need to clarify the high-confidence FP evaluation and its relationship to the proposed FP component.

Our formulation distinguishes the group-level false-positive disparity at a given threshold from its integrated counterpart:

(1-\pi_i)\mathrm{FPR}_i(t) -(1-\pi_j)\mathrm{FPR}_j(t),$$ $$B_{\mathrm{FP}}=\int_0^1 |g^{\mathrm{FP}}_{i,j}(t)|\,dt.$$ The pointwise quantity $|g^{\mathrm{FP}}_{i,j}(0.7)|$ measures the difference in the proportion of each group receiving a false-positive prediction at threshold 0.7. This is distinct from the label-conditional FPR gap, whose denominator consists only of negative-labeled examples. The threshold of 0.7 represents a higher-confidence positive-decision operating point than the conventional threshold of 0.5. We do not have a record establishing that this threshold was pre-specified, and therefore interpret it as an illustrative operating point rather than a pre-specified independent evaluation target. To examine threshold sensitivity, we additionally evaluated thresholds from 0.5 to 0.9. On ACSIncome, the Spearman correlation between the integrated residual FP component and the corresponding pointwise group-level FP gap is 0.925, 0.835, 0.835, 0.801, and 0.670, respectively. These results clarify the role of the integrated formulation: it characterizes group-level FP disparity across the threshold range rather than relying on any particular operating point. The pointwise analysis provides an operating-threshold interpretation, while the integrated component captures the broader distributional behavior. We will clarify this distinction and the interpretation of the 0.7 threshold in the revised manuscript.
Reviewer_3tVKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses residual distributions to audit group error differences across decision thresholds. Its main result shows that residual Wasserstein distance equals the combined areas between group false-negative and false-positive curves. A calibrated counterexample and experiments on tabular and text datasets illustrate what single-threshold summaries can miss.

Critical strengths
  • The FN/FP decomposition is simple and interpretable. It explains which type of error drives a residual disparity and carefully distinguishes group error fractions from label-conditional rates.

  • The calibrated counterexample is convincing: groups can match on accuracy, AUC, calibration, and fairness at threshold 0.5 while differing elsewhere.

  • I appreciate the treatment of prevalence and uncertainty. The authors also acknowledge that the ACS calibration result becomes less convincing when calibration is refitted in the permutation test.

Critical weaknesses
  • The theoretical advance seems incremental relative to existing Wasserstein score audits, such as Becker et al. (2024). With equal prevalence. The additional practical insight needs a stronger demonstration.

  • The strongest ranking results compare the same FN/FP statistic on validation and test data. This establishes stability, but the comparison with equalized odds favors metrics aligned with the chosen target. The cost-gap results help, although they do not yet establish a clear advantage for model selection in a deployment setting.

  • The text experiments are difficult to reproduce. The appendix lists model checkpoints but leaves important choices unclear, particularly the MHS group definitions and conversion of dataset labels into binary outcomes.

Questions for authors
  • Can you show a concrete auditing decision that improves when residual components are added to label-conditional Wasserstein measures?
  • Does selecting models using these components improve a separately specified deployment cost, compared with selecting directly on that cost?
  • How sensitive are the model rankings to the chosen threshold interval and its weighting?
  • Could you provide the text-data group definitions, label mappings, preprocessing, and evaluation code?
1 comment

We thank the reviewer for recognizing the interpretability of the FN/FP decomposition and the value of our calibrated counterexample. We appreciate the opportunity to clarify the practical contribution of our work.

1. Practical contribution beyond existing distributional fairness measures

The central distinction is the auditing objective. Existing outcome-conditional distributional measures, including Becker et al. (2024), characterize differences in predictive behavior conditional on the true label. Our residual formulation instead quantifies how these errors translate into prevalence-aware group-level error burdens across decision thresholds.

Specifically,

\mathrm{GFP}_g(t)=(1-\pi_g)\mathrm{FPR}_g(t),$$ where $\pi_g=P(Y=1\mid A=g)$ denotes group prevalence. This distinction matters because comparable conditional error rates do not necessarily imply comparable proportions of individuals affected by prediction errors within each group. Our controlled analysis makes this difference explicit. Holding conditional score distributions fixed while varying prevalence leaves conditional Wasserstein measures unchanged, whereas residual distance captures the corresponding changes in group-level error burden. In an exactly controlled construction with identical conditional score distributions, conditional disparity is zero across thresholds while residual distance captures the prevalence-dependent difference. **2. Concrete auditing implications** The distinction also appears in our ACSIncome group comparisons. With group pairs ranked in ascending order of disparity: - **Black–White:** Rank 2 under conditional Wasserstein distance ($E=0.065$), compared with rank 5 under residual distance ($W_{\mathrm{res}}=0.042$). - **Asian–Two or More Races:** Rank 6 under conditional Wasserstein distance ($E=0.098$), compared with rank 3 under residual distance ($W_{\mathrm{res}}=0.034$). These differences demonstrate that incorporating prevalence changes which group comparisons receive attention under a group-level error-burden auditing objective. The residual formulation therefore provides additional information for auditing decisions concerned with the fraction of each group experiencing classification errors, complementing conditional error-rate comparisons. **3. Threshold sensitivity and reproducibility** We further evaluated four threshold intervals, $[0,1]$, $[0.1,0.9]$, $[0.2,0.8]$, and $[0.3,0.7]$, using uniform and Beta(2,2) weighting. On Adult, ranking agreement with the full uniform integral remains high (Kendall correlation 0.84–0.92). On ACSIncome, the narrowest interval yields a Kendall correlation of 0.71, illustrating how operating-range assumptions can influence model rankings. Regarding reproducibility, we will release the complete implementation and evaluation code, including text-data preprocessing, group definitions, label mappings, and experimental configurations, upon acceptance as part of the camera-ready release. **Overall, our contribution is an interpretable, prevalence-aware distributional audit that explicitly connects conditional predictive behavior with group-level FN/FP burdens across decision thresholds.**

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.