acceptodds
Under review as a conference paper at ICLR 2027

A-IEOF: Anchored Interventional Equalized-Odds Fairness for Feature Addition

Abstract

Machine learning systems in high-stakes domains must balance utility with group fairness. Conventional debiasing optimizes fairness on the training distribution and does not address disparities that appear under distributional shift. We introduce Anchored Interventional Equalized-Odds Fairness (A-IEOF), an in-processing framework for feature addition, where a newly available feature block B can improve utility but may cause fairness regressions. A-IEOF trains a student model with a minimax objective that (i) enforces worst-case multi-group equalized odds under adversarially generated counterfactual replacements of B, (ii) anchors predictions to a no-B teacher to preserve behavior on shared inputs, and (iii) stabilizes calibrated scores against variation in B. On a multi- institutional clinical corpus, A-IEOF attains the highest AUC (0.976) and lowest equalized-odds gap (0.085–0.126) across global, per-group, and Hardt-EO post-processing policies, with better calibration (Brier 0.0495–0.0507; ECE 0.0262–0.0343) than adversarial gradient-reversal (GRL) debiasing. To test the robustness claim directly we introduce a controlled tabular feature-addition benchmark in which P (B |S) can be shifted by a prescribed total-variation amount: A-IEOF holds its equalized-odds gap essentially flat as the shift grows (degradation < 0.01 up to ∆T V = 0.5), consistent with our transfer bound, while an empirical-risk baseline is four times more sensitive to B. Over ten seeds A-IEOF matches a constrained-optimization (reductions) baseline in distribution and surpasses invariant-learning baselines (IRM, CORAL) and GRL. On three real tabular datasets (UCI Adult, COMPAS, CT metadata) it attains the lowest equalized-odds gap on Adult and COMPAS and the lowest sensitivity to B on all three. A-IEOF thus offers a principled, auditable route to feature addition that aligns utility, calibration, and fairness under realistic deployment conditions. Our implementation and benchmark are available at https://anonymous.4open.science/r/AIEOF-778E.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Interventional Fairnes, Equalized Odds, Feature Addition, Distributional Robustness, Counterfactual Stability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VmGEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes A-IEOF, a fairness-aware learning framework for binary classification when new features are added to an existing model. The main goal is to maintain equalized odds across sensitive groups, even when the distribution of the newly added features changes after deployment.

The method combines three components: adversarially generated feature replacements to control worst-case fairness disparities, teacher anchoring to preserve predictions from the original model, and a stability regularizer to reduce sensitivity to the new features. The authors also provide a theoretical bound on fairness degradation under a restricted form of distribution shift.

The method is evaluated on clinical data, Adult, COMPAS, and a controlled synthetic benchmark. Results show improved fairness and robustness compared with several baselines, although predictive accuracy sometimes decreases. Ablation studies suggest that standard fairness regularization primarily improves fairness on the original distribution, while the proposed intervention and stability terms improve robustness to feature shifts.

Critical strengths
  1. Practically motivated problem setting. The paper addresses fairness under distribution shift when new features are introduced into an existing model. This is a relevant deployment scenario that goes beyond enforcing fairness on a fixed training distribution.

  2. Theoretical motivation and targeted empirical validation. The paper provides a bound connecting fairness degradation to feature sensitivity and the magnitude of distribution shift. The controlled shift experiments and ablation studies support this connection, showing that the proposed intervention and stability terms improve robustness rather than merely reducing fairness gaps on the training distribution.

Critical weaknesses
  1. Discussion of closely related work is somewhat insufficient. The paper does not adequately discuss closely related fairness methods such as "FairICP: Encouraging Equalized Odds via Inverse Conditional Permutation" and "Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness". GerryFair considers fairness across rich collections of subgroups while FairICP addresses conditional independence across multiple sensitive attributes through adversarial learning. These approaches are relevant to the paper's multi-group fairness objective and adversarial optimization framework. A more careful comparison is needed to clarify what is fundamentally new about the proposed method, beyond combining existing techniques in a feature-addition setting.

  2. Limited scope of the theoretical guarantee. The fairness-transfer bound assumes that the newly added feature provides no additional information about the outcome conditional on the existing features and sensitive group. This appears restrictive given the motivation of adding informative features to improve prediction. It is unclear how well the theoretical guarantees apply to realistic settings where this assumption is violated.

Questions for authors

See weaknesses.

2 comments

Part 1 of 2.

W1. Related work omits FairICP and GerryFair. What is new beyond combining existing techniques?

Thank you for pointing this out. The revision adds a related-work paragraph titled "Multi-group and equalized-odds in-processing". It covers the following methods.

  • Prejudice Remover (Kamishima et al.) penalizes the mutual information between the prediction and the sensitive attribute.
  • The meta-algorithm of Celis et al. handles a family of group-fairness constraints within one framework.
  • ERMI (Lowy et al.) gives a stochastic optimization route to independence-type constraints.
  • Reductions (Agarwal et al.) are implemented in the Fairlearn and AIF360 toolkits.
  • GerryFair (Kearns et al.) audits and learns against a rich class of subgroups.
  • FairICP (Lai et al.) enforces equalized odds for multiple sensitive attributes. It trains an adversary to separate the observed (prediction, attributes, label) triple from one whose attributes are resampled by inverse conditional permutation.

The paragraph states that A-IEOF shares ingredients with an adversary, a soft EO penalty and multi-group gaps. We do not claim these ingredients as new. The difference lies in what is resampled and what is guaranteed.

  • FairICP resamples the sensitive attributes given the label. It tests and enforces equalized odds under the training distribution. A-IEOF instead resamples a designated non-sensitive block B given (x_{-B}, s). An adversarial generator searches for replacement values of B that break equalized odds, and the model is trained to withstand them. Theorem 1 then bounds how much the EO gap can change when the deployment distribution of B moves:

    Theorem 1. If the deployment distribution P' differs from the training distribution P only in the conditional law of B, then Δ_EO^{soft, P'} ≤ Δ_EO^{soft, P} + 12 · ε_stab · Δ^Y_TV(P, P'). Here ε_stab is the largest change in the predicted probability when only B changes, and Δ^Y_TV is the size of the shift in the label-conditional law of B.

    No result of this kind exists for FairICP, because FairICP does not single out a feature block whose distribution may shift.

  • GerryFair widens the set of groups to a rich subgroup class. A-IEOF keeps a small, fixed set of intersectional groups and widens the set of distributions instead.

The two directions are complementary. A subgroup auditor such as GerryFair could replace our max-min over groups. FairICP's permutation sampler could replace our generator when B is continuous and high-dimensional.

The new experiments support this distinction for the in-distribution methods we could run. Reductions-EO, Fairlearn's exponentiated gradient and Prejudice Remover all control EO on the training distribution. Under a shift of 0.5 in P(B | S), all three lose 0.24 to 0.28 in EO, while A-IEOF changes by 0.002 (Table 4, shown under Reviewer 1, W1c). We did not run FairICP or GerryFair, and Appendix H says so.

W2. The bound assumes the new feature adds no information about the outcome given the existing features and group. This seems restrictive.

Thank you for pointing this out. The assumption was stronger than the result needs, and the revised theorem no longer requires it.

Where the assumption entered. In the original proof, conditional independence (B independent of Y given X_{-B} and S) was used at one step only. It replaced P(B | X_{-B}, S, Y) by P(B | X_{-B}, S). This allowed a shift in P(B | X_{-B}, S) to be carried over to the label-conditional rates that define equalized odds.

The relaxed result. We measure the shift directly on the label-conditional law of B. The revised theorem (Appendix G) states:

Δ_EO^{soft, P'} ≤ Δ_EO^{soft, P} + C · ε_stab · Δ^Y_TV, with Δ^Y_TV ≤ Δ_TV + η_P + η_P'.

Here Δ_TV is the shift in P(B | X_{-B}, S). η_P measures, in total variation, how far P(B | X_{-B}, S, Y) is from P(B | X_{-B}, S) under P. No independence assumption is needed. Conditional independence is the special case η_P = η_P' = 0, which recovers the original bound.

The cost of a violation. When B carries extra label information, the guarantee weakens additively by C · ε_stab · (η_P + η_P'). It does not fail abruptly when η is above zero. It loosens in proportion to how much label information B carries and to how sensitive the model is to B. Training with a small ε_stab therefore keeps the bound informative under mild violations.

Part 2 of 2.

How restrictive it is in practice. We estimated η coarsely, conditioning on the group only. On the controlled benchmark, where independence holds by construction, the estimate is below 0.10, which is sampling noise around zero. On the CT metadata, where the institution identifier strongly predicts the label, it is 0.25 to 0.42. With A-IEOF's sensitivity there (ε̂_stab = 0.571), the relaxed bound is vacuous, and the experiments agree. At Δ_TV = 0.5, the EO gap of A-IEOF rises from 0.403 to 1.213. This is less than ERM (0.356 to 1.394). It is more than the reductions learner (0.384 to 0.990) and IRM (0.635 to 0.737). We report this case as a limitation of the method in the abstract, the real-data section, the conclusion and the ethics statement.

Why the setting still matters. There is a tension, which the Scope paragraph now states. A model that extracts label signal from B must be sensitive to B. Utility from B and robustness to shifts in B therefore pull in opposite directions. The bound does not remove this tension. It quantifies it, and the training weights λ_interv and λ_stab choose a point on it.

Reviewer_DHxkOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose an in-processing training framework: Anchored Interventional Equalized-Odds Fairness (A-IEOF), which was designed to for model-fairness-robusteness when a new, potentially bias-rich feature is added to a machine learning system. The authors claim that conventional debiasing methods often fail when the distribution of the new feature shifts in real-world deployment - because the fairness is computed only on the training data. Core ideas and contributions: Usage of a Minimax Fairness Objective to ensures the model remains fair even if the feature's distribution shifts. Anchoring: "student" model's predictions are anchored to a baseline teacher model that does not have access to the new feature, helping to preserve the original accuracy. Transfer Bound Theory: the paper propose a mathematical bound that expliciting the limits of how much the equalized-odds fairness gap can degrade during deployment shifts. Empirical Validation: tested on several data sets (clinical COVID-19 CT, Adult, COMPAS, CT metadata). Authors claim their method achieved higher accuracy and lower equalized-odds gaps vs. baselines (adversarial gradient-reversal (GRL) and empirical risk minimization (ERM)).

Critical strengths

What I particularly liked in this paper was the Theoretical Transfer Bound. The authors provide a formal transfer bound that shows that if a model maintains low counterfactual sensitivity (stability) to a newly added feature block, the degradation of its equalized-odds gap grows at most linearly with the magnitude of the distribution shift. I also (minor) appreciate the experimental Empirical Stress Testing, although the scope of the baseline seems too much restricted.

Critical weaknesses

Main concern of the paper is the the baseline methods (GRL et ERM) - which are then outperformed - are from the same family and thus only shows an improvement over a family of optimisation methods rather a disruption (or even improvement) of the state of the art. Techniques such as: Adversarial Debiasing (Zhang et al.), Constrained Optimization (Agarwal et al.), Prejudice Remover (Kamishima et al.), Meta-Fairness Algorithms (Celis et al.) or Information-Theoretic Regularization (e.g., ERMI Minimization) should be considered - KNOWING that most or many are available in toolkits like Fairlearn or IBM AIF360. Second concern/criticism is that in many scenarios the theorem holds only with conditional-independence, which is not only ways the case / not to say never 100%. This is even experimentally noted by the authors when the newly added feature block is highly predictive of the label - which is indeed a complicated/contradictory situation.

Questions for authors

How often do you think conditional-independence holds, so that your the theorem holds ? How much is something true what the premise is not 100% true ?

3 comments

Part 1 of 2.

W1. The baselines (GRL, ERM) are from one family. Compare against Zhang et al., Agarwal et al., Kamishima et al., Celis et al., ERMI, as available in Fairlearn or AIF360.

Thank you for the suggestion. The tabular studies compare against five methods, not two: ERM, gradient reversal (GRL), invariant risk minimization (IRM), Deep CORAL, and a constrained-optimization learner that solves the saddle-point problem of Agarwal et al. GRL is the gradient-reversal form of the adversarial debiasing method of Zhang et al.

For this revision we added two baselines.

  • Fairlearn EG-EO. This is Fairlearn's ExponentiatedGradient with an EqualizedOdds constraint. It is the reference implementation of Agarwal et al. Our earlier "Reductions-EO" was our own Lagrangian implementation of the same method, and the paper now says so.
  • Prejudice Remover (Kamishima et al.). Its regularizer weight is chosen on validation data.

Controlled benchmark, mean ± standard deviation over ten seeds. "EO in distribution" uses the observed test values of B. "EO at Δ_TV = 0.5" is the thresholded gap after a shift of 0.5 in P(B | S).

MethodAUCEO in distributionEO at Δ_TV = 0.5
Reductions-EO (our implementation of Agarwal et al.)0.841 ± 0.0760.244 ± 0.0640.501
Fairlearn EG-EO (Agarwal et al.)0.811 ± 0.0780.371 ± 0.1240.603
Prejudice Remover (Kamishima et al.)0.856 ± 0.0800.324 ± 0.0930.613
Reductions-EO without B0.823 ± 0.0810.261 ± 0.0880.261
A-IEOF0.825 ± 0.0840.305 ± 0.1070.294

In distribution, A-IEOF does not outperform these methods, and we do not claim that it does. Reductions-EO has a lower gap. Prejudice Remover has a higher AUC at a similar gap. The paper's claim concerns what happens when the distribution of the added block shifts. Under a shift of 0.5, all three baselines lose 0.24 to 0.28 in EO, while A-IEOF is unchanged.

We did not run the meta-algorithm of Celis et al. or ERMI. Both are now discussed in a new related-work paragraph, titled "Multi-group and equalized-odds in-processing". It describes Prejudice Remover as penalizing the mutual information between the prediction and the sensitive attribute. It describes the meta-algorithm of Celis et al. as handling a whole family of group-fairness constraints within one framework. It describes ERMI as a stochastic optimization route to independence-type constraints such as equalized odds. It notes that reductions are implemented in the Fairlearn and AIF360 toolkits. Appendix H states that the Celis et al. and ERMI methods were not run. We expect them to behave like the three methods above, because they also constrain fairness on the training distribution only. We have no data for this, however, and the paper does not claim it.

W2 and question. The theorem needs conditional independence, which rarely holds exactly. How often does it hold, and how much remains true when it does not?

Thank you for raising this. The assumption rarely holds exactly, and the paper now says so. Our answer has four parts.

The assumption is not needed for the bound. The original proof used conditional independence at one step only. It replaced P(B | X_{-B}, S, Y) by P(B | X_{-B}, S). If the shift is measured on the label-conditional law of B, the bound holds with no independence assumption. The revised Theorem 1 states:

Theorem 1 (Fairness transfer, revised). Let P' differ from P only in the conditional law of B, with P(X_{-B}, Y, S) unchanged. Then

Δ_EO^{soft, P'} ≤ Δ_EO^{soft, P} + C · ε_stab · Δ^Y_TV(P, P'), with C = 12, and Δ^Y_TV ≤ Δ_TV + η_P + η_P'.

Here η measures, in total variation, how much label information B carries beyond (X_{-B}, S). It is zero exactly under conditional independence. In that case Δ^Y_TV = Δ_TV, and the original bound is recovered.

When the premise holds only partly, the guarantee weakens additively. The cost of a violation is C · ε_stab · (η_P + η_P'). Nothing breaks when η is above zero. The bound loosens in proportion to the size of the violation and to the model's sensitivity to B.

How often it holds. It holds by construction in our controlled benchmark. It is a reasonable approximation when B is a proxy whose link to the label runs through the base features and the group. It fails when B is a strong predictor in its own right. We estimated η coarsely, conditioning on the group only. The estimate is below 0.10 on the benchmark, which is sampling noise around zero. It is 0.25 to 0.42 on the CT metadata, where the institution identifier strongly predicts the label.

Part 2 of 2.

What failure looks like. The CT metadata shows it. With ε̂_stab = 0.571 and η of about 0.3, the relaxed bound is vacuous, and the data agree. At Δ_TV = 0.5, A-IEOF's gap rises from 0.403 to 1.213. This is less than ERM (0.356 to 1.394). It is more than Reductions-EO (0.384 to 0.990) and IRM (0.635 to 0.737).

We report this case as a limitation in four places in the paper:

  • Abstract: "Where B carries label information beyond the base features (CT metadata), the premise of the bound fails and A-IEOF does not protect equalized odds under shift. We report this case as a limit of the method."
  • Section 5.6 (real datasets): "The CT metadata is a boundary case. Institution strongly predicts the label, so conditional independence fails … A-IEOF has no robustness advantage over a strong fairness baseline in this regime."
  • Conclusion: "When B is a strong label predictor (CT metadata), no method that uses B is protected, A-IEOF included."
  • Ethics statement: "The transfer bound is not a deployment certificate. It is loose by two orders of magnitude on our own benchmark, its premise fails on the CT metadata, and there A-IEOF's equalized-odds gap triples under a shift of 0.5. Citing the bound to justify adding a site identifier to a clinical model without shift monitoring would be a misuse of this paper."

You also point to a tension between adding a feature for utility and assuming that it carries no extra label information. This tension is real. A model that extracts label signal from B must be sensitive to B. Utility from B and robustness to shifts in B therefore pull in opposite directions. Theorem 1 does not remove this tension. It quantifies it through the product ε_stab · (Δ_TV + η_P + η_P'). The weights λ_interv and λ_stab in the training objective choose a point on this trade-off. The Scope paragraph after Theorem 1 now explains this.

The no-B comparison (Table 6) shows both ends of the trade-off.

ModelBenchmark AUCBenchmark EOBenchmark EO at 0.5CT AUCCT EOCT EO at 0.5
Teacher (no B)0.892 ± 0.0640.871 ± 0.3030.8710.690 ± 0.0050.384 ± 0.0750.384
Reductions-EO (no B)0.823 ± 0.0810.261 ± 0.0880.2610.705 ± 0.0060.399 ± 0.0980.399
Reductions-EO (B)0.841 ± 0.0760.244 ± 0.0640.5010.798 ± 0.0120.387 ± 0.0640.990
A-IEOF (B)0.825 ± 0.0840.305 ± 0.1070.2940.810 ± 0.0120.403 ± 0.0601.213

On the benchmark, B adds nothing, and a fair no-B model is as good as A-IEOF. On the CT metadata, B adds 0.105 AUC and costs robustness.

Ethics flag.

Thank you for raising this. We expanded the ethics statement with three points that follow from the new results.

  • The transfer bound is not a deployment certificate. It is loose on our own benchmark. Its premise fails on the CT metadata, where the EO gap triples under shift.
  • The clinical fairness results come from one split whose smallest stratum has 10 positive cases. They should not be read as evidence that the model is fair for any small group.
  • COMPAS is used only as a public benchmark. Its use does not endorse recidivism prediction.

Thank you for your comments and the effort you put into the rebuttal. I really appreciate the effort : unfortunately, I stick to my rating.

Reviewer_WBcKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper looks at feature addition, where a new block B is added to a deployed classifier. This may improve utility but can also break group fairness. A-IEOF trains a student that uses B, combining anchoring to a no-B teacher, worst-case soft EO under adversarial counterfactual replacement of B, observational soft EO, and a stability penalty. The authors derive a bound on soft EO degradation under shifts confined to B, subject to their stability assumptions, with the bound depending on sensitivity to B. The data include clinical CT, a synthetic tabular set with controllable shift, and Adult, COMPAS and CT metadata. The baselines are ERM, GRL, IRM, CORAL and a reductions learner.

Critical strengths

(1) The feature-addition framing is quite practical, and at least I have not seen it much before. The authors are also fairly candid about the scope and about the cases where they cannot beat Reductions-EO or CORAL.

(2) Using data with controllable shift to match the transfer bound is the right idea, and A-IEOF's sensitivity to B is indeed clearly lower than ERM's.

Critical weaknesses

(1) The connection between the transfer bound and the reported robustness needs clarification. Section 3.3 bounds soft rates, whereas the experiments report thresholded EO; Appendix B also uses an empirical sensitivity percentile, whose connection to the required supremum is not clear to me despite the finite-sample argument. Table 4 shows lower sensitivity and a stable EO gap for A-IEOF, but ERM's gap does not worsen under the tested shifts either, and Reductions-EO is absent. This leaves the relative fairness-transfer advantage over a strong fairness baseline unresolved.

(2) The benefit of adding B is difficult to separate from the fairness constraint. In Tables 3 and 5, A-IEOF has lower AUC than the no-B teacher on the synthetic tabular data, Adult and COMPAS, while improving EO. A fairness-constrained no-B baseline would help establish whether using B offers a better utility-fairness tradeoff than omitting it.

(3) The clinical results in Section 5.2 rely on one split with small group supports, so their stability needs further assessment. The paper mentions bootstrap intervals in the released code; reporting those intervals would help interpret Tables 1 and 2. Brier and ECE are identical in the GRL rows of Table 1 and should be checked. Appendix A.7 also retains an absolute claim that all components are necessary before qualifying it using the tabular ablation; the qualification is helpful, but the earlier wording should be narrowed to the tested setting.

Questions for authors

See Weaknesses.

6 comments

Part 1 of 5.

W1a. Section 3.3 bounds soft rates, but the experiments report thresholded EO.

Thank you for pointing this out. We have made the distinction explicit in three ways.

First, Theorem 1 states that it bounds the soft EO gap. The revised statement is:

Theorem 1 (Fairness transfer). Let the deployment distribution P' differ from the training distribution P only in the conditional law of B, with P(X_{-B}, Y, S) unchanged. Then the soft equalized-odds gap satisfies

Δ_EO^{soft, P'} ≤ Δ_EO^{soft, P} + C · ε_stab · Δ^Y_TV(P, P'), with C = 4(w₊ + w₋) = 12.

If B is conditionally independent of Y given (X_{-B}, S) under both P and P', then Δ^Y_TV equals Δ_TV, which recovers the bound as originally stated. Without conditional independence, Δ^Y_TV ≤ Δ_TV + η_P + η_P'.

Second, we report the soft gap under shift for every method. For A-IEOF, the soft gap moves from 0.157 to 0.161 as the shift grows from 0 to 0.5. The thresholded gap moves from 0.292 to 0.294.

Third, a new corollary (Corollary 1, Appendix G) gives the bound for thresholded decisions. The theorem cannot be applied to a hard decision directly, because the worst-case sensitivity of a 0/1 indicator is 1. The relevant quantity is the flip fraction ρ. It is the largest probability, over all (group, label) strata, that a context's decision changes for some value of B. The corollary states:

Corollary 1 (Thresholded equalized odds). For decisions ŷ = 1[p(x) ≥ t_s] with thresholds fixed on validation data,

Δ_EO^{P'} ≤ Δ_EO^{P} + C · ρ · Δ^Y_TV(P, P'), and ρ ≤ max over (g, y) of Pr(|p(X) − t_S| ≤ ε_stab | Y = y, S = g).

In words, ρ is at most the probability mass lying within ε_stab of the decision threshold. A small ε_stab therefore protects thresholded EO through the margin. The flip fraction on the controlled benchmark is 0.112 for A-IEOF and 0.34 to 0.51 for the three fairness baselines.

W1b. Appendix B uses an empirical percentile, whose link to the supremum is unclear.

Thank you for pointing this out. The quantities involved are as follows:

  • A context c is an input with B removed, written as x_{-B}. For each context, x^(b) is the same input with B set to the value b.
  • osc(c) = max over b, b' of |p(x^(b)) − p(x^(b'))| is the sensitivity of that context. It is the largest change in the predicted probability when only B changes and every other feature stays fixed.
  • ε_stab = sup over contexts of osc(c) is the worst-case sensitivity of the model. Theorem 1 uses this population supremum.
  • ε̂ is an estimate of ε_stab from data. In the submission it was the 99th percentile of the prediction change, computed from n sampled validation pairs. The submission reported ε̂ = 0.154 for ERM and ε̂ = 0.040 for A-IEOF.
  • n is the number of validation samples used to compute ε̂.

The previous finite-sample corollary stated that ε_stab ≤ ε̂ + O(n^{-1/2}). In words, the true worst-case sensitivity exceeds the empirical 99th percentile by at most a term that shrinks like one over the square root of n. This step does not follow. The Dvoretzky-Kiefer-Wolfowitz (DKW) inequality controls how much probability lies above a percentile. It does not control how large the sensitivity can be in that upper tail.

We replaced that corollary with the following result (Proposition 1, Appendix G):

Proposition 1 (Bound from a quantile of the sensitivity). Fix q in (0, 1). Suppose that, in every (group, label) stratum, the probability that osc(c) exceeds ε_q is at most 1 − q. Then

Δ_EO^{soft, P'} ≤ Δ_EO^{soft, P} + C · (ε_q + 1 − q) · Δ^Y_TV(P, P').

If ε̂_q is the largest empirical q-quantile over the 2G strata, computed from at least n_min held-out contexts per stratum, then with probability at least 1 − δ the same bound holds with ε_q + 1 − q replaced by ε̂_q + 1 − q + sqrt(log(4G/δ) / (2 n_min)).

The proposition pays for the unobserved tail with the explicit term 1 − q, plus the DKW deviation. This price is real on small strata. The smallest test stratum of the controlled benchmark has about 31 contexts. The deviation term is then 0.30.

We also removed a sentence from the appendix. It claimed that minimizing the expected sensitivity bounds its supremum "by Jensen's inequality". This is not valid, because an expectation is a lower bound on a supremum, not an upper bound.

Two facts reduce the reliance on a percentile in our experiments. First, B is categorical in all our datasets. The maximum over b and b' is therefore computed exactly for each context by enumerating all K values of B. Only the supremum over contexts has to be estimated. Second, the empirical supremum and the 99th percentile are now reported side by side. For the linear models they agree to three decimals.

Part 2 of 5.

Table 12. Sensitivity to B on test contexts, calibrated probabilities, mean over ten seeds. "sup" is the empirical supremum of osc(c). "q.99" is the largest per-stratum 99th percentile. "mean" is the average of osc(c). ρ̂ is the flip fraction under the global threshold.

MethodBenchmark supq.99meanρ̂CT metadata supq.99meanρ̂
ERM0.2180.2180.1020.2340.8480.8480.8001.000
GRL0.5440.5080.1860.3580.9430.9430.8361.000
IRM0.4000.4000.2120.4180.3350.3350.2810.716
CORAL0.4830.4230.1500.3030.9790.9790.9041.000
Reductions-EO0.4440.4440.2580.5050.8560.8550.7770.981
Fairlearn EG-EO0.6720.6590.2140.3420.7710.7710.6260.993
Prejudice Remover0.3940.3940.2260.4350.8450.8450.8031.000
A-IEOF0.0880.0880.0470.1120.5710.5710.5200.958

For a linear logit model with one-hot B and temperature T, the supremum has a closed form: osc(c) ≤ tanh(R / 4T), where R is the range of the B coefficients. No sampling argument is needed in that case.

With the empirical supremum of 0.088, Theorem 1 allows A-IEOF's soft gap to increase by at most 12 × 0.088 × 0.5 = 0.53 at Δ_TV = 0.5. Corollary 1, with ρ = 0.112, allows 0.67 for the thresholded gap. The observed increases are 0.004. The bound is therefore valid but loose by about two orders of magnitude. Its role is qualitative. It identifies sensitivity to B as the quantity a learner should control. Because it is an upper bound, it does not rank methods. The revised paper states this in Section 5.4.

W1c. ERM's gap does not worsen under shift either, and Reductions-EO is absent from Table 4.

Thank you for pointing this out. The revised Table 4 now includes all eight methods that use B, plus the fairness-constrained model without B.

Table 4. Controlled benchmark, thresholded EO gap under a shift of size Δ_TV in P(B | S), global-threshold policy, mean over ten seeds. ε̂_stab is the empirical supremum of the sensitivity on calibrated probabilities. Rise is the gap at 0.5 minus the gap at 0.

Methodε̂_stab (sup)Δ_TV = 00.10.20.30.40.5Rise
ERM0.2180.9320.9250.9120.8910.8720.879−0.053
GRL0.5440.6960.7020.6810.7230.7440.760+0.064
IRM0.4000.8730.8610.8790.9100.9050.915+0.042
CORAL0.4830.8210.8120.8060.8200.8280.821+0.000
Reductions-EO0.4440.2250.2600.3390.4230.4780.501+0.276
Fairlearn EG-EO0.6720.3640.3910.4760.5510.5840.603+0.239
Prejudice Remover0.3940.3430.3400.4270.5200.5900.613+0.270
A-IEOF0.0880.2920.2920.2970.2870.2930.294+0.002
Reductions-EO (no B)00.2610.2610.2610.2610.2610.2610

At Δ_TV = 0 the test values of B are resampled from the training law P(B | S). These entries can therefore differ slightly from the in-distribution gaps in the main results table, which use the observed values of B.

ERM's gap does not worsen because it already starts at 0.93. Its disparity comes from group base rates that the base features already encode. Moving B therefore has little left to break. The informative comparison is with methods that reach a low gap in distribution. Reductions-EO starts below A-IEOF (0.225 against 0.292). It ends well above it (0.501 against 0.294). The two curves cross between Δ_TV = 0.1 and 0.2. The rise at 0.5 exceeds 0.05 in 9 of 10 seeds for Reductions-EO and in 1 of 10 seeds for A-IEOF. Fairlearn's implementation and Prejudice Remover behave like our Lagrangian reductions learner. All three equalize error rates on the training law of B. They are also four to eight times more sensitive to B than A-IEOF.

One caveat has been added. The nominal shift is realized exactly up to 0.3. At nominal values of 0.4 and 0.5, the largest per-group shift averages 0.39 and 0.44. This happens because the shift generator moves probability mass only from each group's most likely site.

This advantage holds when the premise of Theorem 1 holds, that is, when B carries little label information beyond (X_{-B}, S). It does not hold on the CT metadata. There, the institution identifier strongly predicts the label. A coarse estimate of η_P is 0.25 to 0.42, against below 0.10 on the controlled benchmark. Under a shift of 0.5, A-IEOF's thresholded gap rises from 0.403 to 1.213. This is less than ERM (0.356 to 1.394). It is more than Reductions-EO (0.384 to 0.990) and IRM (0.635 to 0.737). We report this case in the paper as a limitation of the method.

Part 3 of 5.

W2. A-IEOF has lower AUC than the no-B teacher on three datasets. A fairness-constrained no-B baseline is needed.

Thank you for the suggestion. We added this baseline. It is the same reductions learner, trained on X_{-B} only.

Table 6. Using B against omitting it under the same fairness constraint. Global-threshold policy, mean ± standard deviation over ten seeds. "EO at 0.5" is the thresholded gap after a shift of nominal Δ_TV = 0.5 in P(B | S).

ModelBenchmark AUCBenchmark EOBenchmark EO at 0.5CT AUCCT EOCT EO at 0.5
Teacher (no B)0.892 ± 0.0640.871 ± 0.3030.8710.690 ± 0.0050.384 ± 0.0750.384
Reductions-EO (no B)0.823 ± 0.0810.261 ± 0.0880.2610.705 ± 0.0060.399 ± 0.0980.399
Reductions-EO (B)0.841 ± 0.0760.244 ± 0.0640.5010.798 ± 0.0120.387 ± 0.0640.990
A-IEOF (B)0.825 ± 0.0840.305 ± 0.1070.2940.810 ± 0.0120.403 ± 0.0601.213

The answer depends on whether B carries label information beyond the base features.

  • Controlled benchmark. Here B is conditionally uninformative by construction. The constrained no-B model matches A-IEOF within one standard deviation on both AUC and EO. A shift in B cannot affect it. Adding B brings no benefit in this setting, and the paper now says that omitting B is the better choice. What A-IEOF offers here is safe inclusion. It is the only method using B whose low gap survives the shift.
  • CT metadata. Here B carries real signal. A-IEOF gains 0.105 AUC over the constrained no-B model, at a statistically indistinguishable in-distribution gap. Under a shift, however, its gap grows to 1.213, while the no-B model stays at 0.399.

We repeated the comparison on two public datasets: UCI Adult, where B is occupation, and COMPAS, where B is charge degree. These experiments use the pure in-processing policy. It applies no temperature scaling and no threshold tuning, so differences between rows reflect only the training objective. The main real-data table of the paper (Table 5) uses the same policy, so the two tables are directly comparable.

Table 14. Adult and COMPAS: using B against omitting it under the same fairness constraint. Pure in-processing policy, mean ± standard deviation over ten seeds.

ModelAdult AUCAdult EOCOMPAS AUCCOMPAS EO
Teacher (no B)0.796 ± 0.0100.187 ± 0.0660.728 ± 0.0121.163 ± 0.157
Reductions-EO (no B)0.787 ± 0.0110.146 ± 0.0510.690 ± 0.0381.112 ± 0.269
Reductions-EO (B)0.807 ± 0.0090.151 ± 0.0490.633 ± 0.0610.865 ± 0.208
Prejudice Remover (B)0.809 ± 0.0120.186 ± 0.0790.722 ± 0.0191.062 ± 0.221
Fairlearn EG-EO (B)0.689 ± 0.0240.184 ± 0.0750.641 ± 0.0451.025 ± 0.187
A-IEOF (B)0.792 ± 0.0100.149 ± 0.0620.625 ± 0.0630.311 ± 0.513
  • Adult. The constrained no-B model matches A-IEOF on both AUC (0.787 against 0.792) and EO (0.146 against 0.149). Under the fairness constraint, occupation adds no information that either model can retain.
  • COMPAS. Charge degree has little influence on any model. A-IEOF reaches a much lower gap than the constrained no-B model (0.311 against 1.112), but at lower AUC (0.625 against 0.690). This is a different operating point caused by a stronger fairness penalty, not a gain from B. It is also unstable across seeds. The standard deviation is 0.513, and the smallest test stratum has about six cases.

A fair no-B model is therefore a strong reference point. In none of the four datasets does adding B under A-IEOF improve both AUC and EO over it. The CT metadata is the only dataset where A-IEOF improves AUC at equal EO. When B has label signal, A-IEOF turns it into utility at equal in-distribution fairness. The cost is exposure to shifts in B, and that exposure grows with the signal.

The revised paper adds this finding in two places. Section 5.6 (real datasets) has a new paragraph titled "Is B worth adding? A fairness-constrained no-B baseline". It reports the results above and ends with the sentence: "In none of the four datasets does adding B under A-IEOF improve both AUC and EO over the constrained no-B model." The conclusion now states: "When B adds no label information (the benchmark, Adult, COMPAS), a fairness-constrained model without B does as well and is immune to the shift. When B is a strong label predictor (CT metadata), no method that uses B is protected, A-IEOF included."

W3a. Clinical results rely on one split with small group supports. Report the intervals.

Thank you for pointing this out. The clinical test split has 403 images (161 positive, 242 negative) in four intersectional groups formed by gender and age: f·A40+ (females aged 40 or older), f·A<40, m·A40+ and m·A<40. Table 2 of the paper reports the per-group true-positive and false-positive rates.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.