acceptodds
Under review as a conference paper at ICLR 2027

Counterfactual Fairness and Utility Trade-offs: Characterizing the Pareto Frontier

Abstract

Artificial intelligence is increasingly utilized in high-stakes decision-making applications, and this trend is expected to continue growing, resulting in the fairness concerns becoming increasingly significant. While achieving fairness is an idealized goal, it often comes at a cost. From a machine learning perspective, fairness constraints can increase the prediction error and reduce the predictive utility of a model. Such losses in predictive accuracy may also harm individuals or society in some real-world applications. Therefore, it is necessary to consider the balance between fairness and utility by allowing some degree of fairness relaxation in such settings. The main challenge lies in how to achieve optimal trade-offs and how to define the trade-offs between fairness and utility. We study this problem under the framework of counterfactual fairness (CF). We formulate a quantifiable and interpretable measurement of fairness relaxation and characterize the Pareto-optimal predictor at each fairness level. With incomplete causal knowledge, we provide a theoretical analysis of the decomposition and variation of the excess risk bound. Synthetic experiments validate our theoretical results and semi-synthetic experiments based on real-world datasets further demonstrate the effectiveness of our method in more practical settings.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
counterfactual fairness, fairness-utility trade-off, pareto optimal, causal inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oFViOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the trade-off between counterfactual fairness and predictive accuracy. The main idea is that enforcing exact counterfactual fairness can reduce predictive performance, especially when the sensitive attribute, or some of its causal descendants, contains useful information for the prediction. To address this, the authors propose a relaxed version of counterfactual fairness controlled by a parameter (ϵ\epsilon), which determines how much of the counterfactual prediction gap of the Bayes-optimal predictor is allowed to remain. The paper first formulates the problem as a constrained optimization problem and derives the optimal predictors for regression under squared loss and for binary classification under cross-entropy loss. In the regression case, they obtain an explicit solution that can be seen as an interpolation between the fully counterfactually fair predictor and the unconstrained Bayes-optimal predictor. They also derive the corresponding excess risk analytically. For cross-entropy loss, the optimal predictor does not have the same closed-form expression and is instead characterized through a nonlinear equation. The authors then consider the more realistic case where neither the Bayes-optimal predictor nor the counterfactual generator is known and both need to be estimated from data. They derive upper bounds showing how estimation errors affect the predictive excess risk. Finally, the theoretical results are validated using synthetic experiments, while semi-synthetic experiments on the Law School and Adult Income datasets compare the proposed approach with other methods.

Critical strengths

In my opinion, here are the strong aspects of the paper:

  • The paper addresses an important concern related to the use of causality in AI fairness, namely its practical applicability. Exact counterfactual fairness can come at the cost of predictive accuracy, while the underlying causal mechanisms are generally not known in practice. Studying relaxed fairness guarantees under imperfect causal knowledge is therefore a relevant research direction.

  • The proposed parameter (ϵ\epsilon) is also easy to interpret, as it controls the fraction of the original counterfactual prediction gap that is allowed to remain. This provides a clear connection between exact counterfactual fairness and unconstrained prediction.

  • The authors characterize the optimal fairness-constrained predictors for both squared loss and cross-entropy loss. In particular, the explicit interpolation result and the excess-risk expression obtained for squared loss are interesting and provide a clear interpretation of the fairness-accuracy trade-off.

  • The extension to the case where the Bayes-optimal predictor and the counterfactual generator need to be estimated is also valuable. The derived excess-risk bounds help understand how estimation errors and imperfect causal knowledge affect predictive performance. The synthetic and semi-synthetic experiments provide initial empirical support for these theoretical results.

Critical weaknesses

The paper is unnecessarily difficult to follow, and understanding the objectives of the different sections requires considerable effort. The technical introduction introduces several mathematical quantities before clearly establishing the overall framework and motivation. The objectives of Sections 3–6, as well as the connections between them, should be explained more explicitly. Several editorial issues also affect readability. Equations are not numbered, some variables, such as (q), are insufficiently defined, and the notation is not always introduced in a systematic way. A high-level overview of the methodology at the beginning of the paper would considerably improve the presentation.

The counterfactual generator (G⋆G^\star) should also be more explicitly connected to Pearl's abduction–action–prediction procedure. This would help clarify the role of the latent state and the invertibility assumption. The invertibility assumption is particularly strong, and its practical implications deserve further discussion. Moreover, invertibility alone does not guarantee that the true counterfactual mechanism can be identified from observational data, which is an important limitation in practice.

Section 5 derives upper bounds on the predictive excess risk resulting from errors in the estimated Bayes predictor and counterfactual generator. However, the paper does not explicitly characterize how these estimation errors affect the actual fairness level of the resulting predictor. The fairness constraint is defined with respect to the true counterfactual prediction gap, while the practical construction relies on estimated quantities. An analysis of the discrepancy between the intended fairness constraint and the fairness actually achieved under estimation errors would strengthen the practical contribution.

The experimental evaluation could also be stronger. The synthetic experiments mainly confirm the qualitative theoretical behavior under a relatively simple causal model. It would be useful to directly evaluate how tight the proposed excess-risk bounds are in practice. The semi-synthetic experiments are limited to two datasets and a relatively small set of baselines. In addition, the proposed method largely overlaps with L2 regularization on the Law School dataset, while the improvements on Adult Income are not consistent across all settings. A broader evaluation, including more diverse datasets, stronger baselines, and uncertainty estimates, would provide stronger empirical support for the claimed benefits of the method.

Questions for authors

Q1. In the practical setting, the fairness-constrained predictor is constructed using an estimated Bayes predictor (\hat\mu) and an estimated counterfactual generator (G^\hat G). Can the authors derive a theoretical bound on the deviation between the achieved counterfactual gap and the prescribed relaxation level, for example, ∣Δϕ^ϵ(u)−ϵδ(u)∣?\left|\Delta_{\hat\phi^\epsilon}(u)-\epsilon\delta(u)\right|? Such a result would complement the excess-risk analysis and provide a stronger guarantee concerning the practical control of counterfactual fairness.

Q2. Section 5 derives excess-risk bounds involving errors in the counterfactual generator and Bayes predictor. Can the authors evaluate the tightness of these bounds experimentally, rather than only validating their qualitative behavior? In particular, how informative are the bounds when both components are estimated simultaneously, and how does the prescribed relaxation parameter (\epsilon) relate to the achieved RTE under increasing estimation errors?

3 comments

Thanks the reviewer for the detailed feedback and we address the concerns below.

Weakness — Presentation

We will improve the presentation by adding a brief roadmap clarifying the connections between Sections: Section 2 introduces the problem setup, Section 3 characterizes the oracle optimal predictors, Section 4 presents the practical estimation procedure, Section 5 establishes the excess-risk bounds, and Section 6 provides empirical evaluations. We will also clarify the notation and number key equations.

Weakness — Counterfactual generator and invertibility

We thank the reviewer for the suggestion. We will make the connection between G⋆G^\star and Pearl's abduction–action–prediction procedure more explicit. Specifically, given (x,a)(x,a), invertibility allows us to recover the latent state uu (abduction). We then intervene on AA while keeping uu fixed (action), and obtain the counterfactual G⋆(x,a,1−a)=x1−a(u)G^\star(x,a,1-a)=x_{1-a}(u) (prediction).

The adoptation of invertibility assumption is mainly to firstly establish the fundamental theory in a relatively simplified setting, without simultaneously dealing with the additional uncertainty arising from non-unique counterfactuals. In fact, even under this simplified setting, characterizing the optimal predictors and deriving the risk analysis already involve nontrivial theoretical challenges. While the theoretical results are established under invertibility, Section 6 deliberately considers experiments based on real data without imposing this assumption.

Regarding identification. The invertibility only ensures the uniqueness, which is distinct from identifying the counterfactual mechanism from observational data. We will clarify this distinction explicitly.

Weakness and Question 1 — Deviation from the prescribed fairness level

Based on the assumptions in section 5, we have:

For regression:

\leq(1-\epsilon)L\eta_G+2\eta_\phi.$$ For classification: $$|\Delta_{\hat\phi^\epsilon}(u)-\epsilon\delta(u)| \leq 2C_\rho(1-\epsilon)(L\eta_G+2\eta_\phi) +2\epsilon\eta_\phi.$$ We will include these results and their proofs in the revised manuscript to provide a more complete theoretical characterization of the achieved counterfactual fairness guarantees.

Weakness and Question 2 — Bound tightness and achieved RTE

Thank the reviewer for suggesting the quantitative evaluation. We conduct complementary experiments examining how closely the excess-risk bounds can be approached and how the prescribed relaxation level relates to achieved RTE. Given the rebuttal space limit, the tables below present selected results from the broader set of experiments across error budgets and relaxation levels.

To assess worst-case tightness, we constructed simultaneous perturbations of the counterfactual generator and Bayes predictor within error budgets ηG\eta_G and ηϕ\eta_\phi, respectively, seeking configurations that approach the upper bounds. We report the largest attained excess risk as a percentage of the theoretical bound.

For regression, the bound is attained exactly (100% tightness), consistent with its proof, where equality conditions can hold simultaneously. For classification, a selected setting in Table 1 achieves 98.43%–99.77% tightness across ϵ=0,0.1,…,1\epsilon=0,0.1,\ldots,1. These results demonstrate that the bounds can be attained or closely approached under simultaneous errors in both components. The remaining gap is consistent with the its proof derivation, which combines several intermediate upper bounds whose equality conditions need not hold simultaneously.

Table 1.

ϵ\epsilonTightness (%)
0.098.4305
0.198.5617
0.298.6935
0.398.8261
0.498.9593
0.599.0932
0.699.2277
0.799.3630
0.899.4989
0.999.6356
1.099.7733

For achieved RTE, we retain the original DGP and estimation settings. For brevity, Tables 2–3 present regression results at n=50n=50 and n=1000n=1000, the two endpoints of our original training sample size. The tables report mean (SD) over 30 repetitions, evaluated with different generator error η\eta and sample size nn.

With true GG, achieved RTE equals the prescribed ϵ\epsilon, despite estimation error in μ\mu. With misspecified GG, RTE initially decreases as ϵ\epsilon decreases from 1, but turns upward at small ϵ\epsilon. This reflects stronger enforcement using inaccurate counterfactuals, which can overcorrect the prediction difference between true counterfactual pairs. Smaller generator errors improve overall agreement with the prescribed level, while larger training samples generally reduce variability.

Table 2. Regression RTE, n=50n=50.

ϵ\epsilonTrue GGη=0.25\eta=0.25η=0.50\eta=0.50
0.00.0000 (0.0000)0.4136 (0.0205)0.7964 (0.0412)
0.10.1000 (0.0000)0.2825 (0.0155)0.6271 (0.0339)
0.20.2000 (0.0000)0.1560 (0.0110)0.4577 (0.0271)
0.30.3000 (0.0000)0.1746 (0.0172)0.2953 (0.0193)
0.40.4000 (0.0000)0.2604 (0.0188)0.2796 (0.0240)
0.50.5000 (0.0000)0.3683 (0.0172)0.3409 (0.0267)
0.60.6000 (0.0000)0.4878 (0.0143)0.4393 (0.0261)
0.70.7000 (0.0000)0.6124 (0.0109)0.5606 (0.0218)
0.80.8000 (0.0000)0.7401 (0.0073)0.6986 (0.0155)
0.90.9000 (0.0000)0.8695 (0.0037)0.8463 (0.0080)
1.01.0000 (0.0000)1.0000 (0.0000)1.0000 (0.0000)

Table 3. Regression RTE, n=1000n=1000.

ϵ\epsilonTrue GGη=0.25\eta=0.25η=0.50\eta=0.50
0.00.0000 (0.0000)0.4039 (0.0092)0.7808 (0.0180)
0.10.1000 (0.0000)0.2771 (0.0063)0.6163 (0.0144)
0.20.2000 (0.0000)0.1553 (0.0075)0.4518 (0.0110)
0.30.3000 (0.0000)0.1731 (0.0112)0.2980 (0.0149)
0.40.4000 (0.0000)0.2606 (0.0065)0.2826 (0.0225)
0.50.5000 (0.0000)0.3703 (0.0043)0.3425 (0.0163)
0.60.6000 (0.0000)0.4900 (0.0033)0.4428 (0.0101)
0.70.7000 (0.0000)0.6147 (0.0026)0.5646 (0.0060)
0.80.8000 (0.0000)0.7417 (0.0019)0.7020 (0.0038)
0.90.9000 (0.0000)0.8703 (0.0009)0.8482 (0.0021)
1.01.0000 (0.0000)1.0000 (0.0000)1.0000 (0.0000)

I appreciate the effort made to address my concerns, particularly the new bounds on achieved counterfactual fairness and the experiments evaluating the tightness of the excess-risk bounds. These additions strengthen the paper and provide useful clarification. However, while the rebuttal addresses several of my technical questions, I do not consider the additional evidence sufficient to justify a change in my overall assessment.

Reviewer_R2nQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the trade-off between predictive accuracy and counterfactual fairness within the Structural Causal Model framework. The authors propose a pointwise relaxation of counterfactual fairness where the individual counterfactual prediction gap is constrained to be at most an epsilon-fraction of the unconstrained Bayes-optimal counterfactual gap, with epsilon taking values between zero and one. By showing that the constrained risk minimization decomposes into independent pointwise subproblems over the latent background variable, the paper characterizes the Pareto-optimal predictors:

  1. Under squared loss, the optimal predictor is an exact linear interpolation between the hard counterfactually fair predictor and the unconstrained Bayes predictor, with excess risk decaying quadratically in one minus epsilon;
  2. Under cross-entropy loss, the optimal predictor is uniquely characterized as the root of a one-dimensional strictly convex equation;
  3. For plug-in estimation with limited causal knowledge, the authors derive excess risk bounds that decouple the error of the estimated counterfactual generator from the error of the estimated Bayes predictor, showing that the generator error term vanishes as epsilon approaches one;
  4. The theoretical findings are validated on a one-dimensional synthetic causal model and evaluated on two semi-synthetic benchmarks, namely Law School and Adult Income.
Critical strengths
  1. Clean analytical characterization of the Pareto frontier: the paper provides an elegant and transparent treatment of the fairness-utility trade-off. In particular, the linear interpolation under squared loss offers a clear geometric path between exact counterfactual fairness and unconstrained Bayes prediction, while the cross-entropy formulation reduces neatly to a tractable one-dimensional convex problem.
  2. Structural separation of estimation errors: the theoretical analysis effectively decouples the errors arising from counterfactual generation and outcome prediction. The bounds provide useful insight by proving that the counterfactual reconstruction error scales with one minus epsilon and vanishes at the unconstrained Bayes endpoint, whereas predictor estimation error persists uniformly.
Critical weaknesses
  1. Problematic normative implications of the relaxation metric: bounding the counterfactual gap proportionally to the unconstrained Bayes gap means that individuals facing larger baseline disparities are granted larger allowable counterfactual discrepancies. Conversely, individuals for whom the Bayes predictor is already fair are strictly kept at zero disparity. The authors provide no normative or ethical justification for permitting greater unfairness precisely where historical bias in the status-quo predictor is highest, rather than adopting a uniform absolute budget.
  2. Restrictive assumptions and empirical deficiencies: the theory relies heavily on exact invertibility between observed covariates and latent background variables, which is rarely met in realistic observational data with unobserved confounding. Additionally, the error bounds depend on strong uniform supremum conditions rather than standard integrated L2 bounds. Empirically, the comparison with L2 regularization on Law School is an algebraic identity proven by the authors rather than a genuine empirical finding, while the Adult benchmark shows marginal gains and completely lacks error bars or standard deviations across repetitions to demonstrate statistical significance.
Questions for authors
  1. What is the normative justification for allowing larger counterfactual disparities for individuals with higher Bayes gaps compared to enforcing an absolute uniform bound across all individuals?
  2. How do your theoretical excess risk bounds degrade when the invertibility assumption is violated and the posterior distribution over latent variables given observed features is non-degenerate?
  3. Could you provide standard deviations or confidence intervals for the semi-synthetic experiments on Law School and Adult to verify whether the performance differences over baselines are statistically significant?
  4. Is it possible to relax the uniform error bounds in Section 5 to integrated L2 population errors, which more realistically reflect the guarantees of standard empirical risk minimization?
1 comment

We thank the reviewer for the thoughtful reviews and address the concerns below.

Weakness 1 and Question 1 — relaxation metric

We consider that fairness is inherently context-dependent, and there is no universally accepted metric that is appropriate for every setting. Our relative relaxation gap reflects one reasonable choice. Although the reviewer's concern is valid, however, absolute relaxation gap also has its limitations. That is, a uniform absolute gap may leave individuals with small original gaps entirely unaffected by fairness adjustments.

For example, consider two loan applicants, A and B. For A, the predicted repayment amount is 1100, but would be 1000 if only their sensitive attribute were changed. For B, the corresponding amounts are 110000 and 100000. Suppose an absolute fairness relaxation is used, which allows a maximum counterfactual gap of 5000. Then A's gap of 100 requires no correction. Thus, A's disparity remains entirely unaddressed.

This is an example of absolute gap failure. However, in this case, our relative gap remains effective.

Weakness 2 and Question 2 — Invertibility concern

The non-invertible setting raised by the reviewer is a more general and challenging problem. Currently, we are not able to quantify how the excess risk bounds degrade when invertibility is violated. This is an important direction for future work.

The adoptation of invertibility assumption is mainly to firstly establish the fundamental theory in a relatively simplified setting, without simultaneously dealing with the additional uncertainty arising from non-unique counterfactuals. In fact, even under this simplified setting, characterizing the optimal predictors and deriving the risk analysis already involve nontrivial theoretical challenges.

We acknowledge that invertibility is a restrictive assumption in realistic settings. However, theoretical analysis and empirical evaluation serve complementary purposes: theory provides guarantees under specific assumptions, while experiments examine whether a method remains effective when these assumptions do not fully hold. For this reason, Section 6 includes semi-synthetic experiments based on real data without imposing invertibility, demonstrating the practical effectiveness beyond the assumption.

Regarding some intuitions, even when counterfactuals follow a non-degenerate distribution, practical prediction still typically requires a single point estimate. We therefore conjecture that non-invertibility may not fundamentally change the form of the estimation procedure and, consequently, may have limited impact on the effectiveness of the resulting predictor.

Weakness 2 and Question 3 — experiment evaluation

We report the mean and sd over the 5 repetitions in the tables below. Given the magnitude of the standard deviations, the results reasonably show that our method outperforms the baselines.

Note that, since the evaluated points of those methods do not necessarily attain the specified RTE levels exactly, for each run we interpolate the risk–RTE curve of each method to obtain the corresponding risk values at the target RTE levels. Thus, the reported results are interpolated values rather than directly observed risk values at those exact RTE levels.

Law School
RTEoursL1L2CGS
0.830.0143(0.0054)0.0221(0.0073)0.0143(0.0054)0.0164(0.0070)
0.850.0086(0.0033)0.0128(0.0052)0.0086(0.0033)0.0098(0.0041)
0.880.0057(0.0042)0.0087(0.0063)0.0057(0.0042)0.0064(0.0046)
0.900.0032(0.0027)0.0050(0.0039)0.0032(0.0027)0.0036(0.0030)
0.920.0015(0.0018)0.0027(0.0024)0.0015(0.0018)0.0018(0.0020)
0.950.0002(0.0009)0.0006(0.0011)0.0002(0.0009)0.0003(0.0010)
Adult
RTEoursL1L2CGS
0.600.0223(0.0023)0.0261(0.0019)0.0254(0.0016)0.0299(0.0023)
0.650.0173(0.0020)0.0195(0.0019)0.0195(0.0017)0.0210(0.0018)
0.700.0122(0.0018)0.0140(0.0018)0.0145(0.0016)0.0158(0.0015)
0.750.0079(0.0015)0.0096(0.0015)0.0104(0.0014)0.0117(0.0013)
0.800.0052(0.0013)0.0064(0.0014)0.0073(0.0012)0.0084(0.0011)
0.850.0022(0.0010)0.0043(0.0013)0.0050(0.0011)0.0059(0.0010)
0.900.0014(0.0006)0.0031(0.0012)0.0035(0.0010)0.0041(0.0010)

Weakness 2 and Question 4 — Integrated L2 error assumptions

We also once considered the MSE assumption as the reviewer mentioned, but we found that this assumption require us to control both the factual input distribution and the counterfactual input distribution induced by counterfactual generators. We consider it is better to only impose assumptions on factual data, and not rely much on the unknown counterfacutal generators. Thus, we do not adopt this type of assumption.

Reviewer_gbKhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the trade-off between counterfactual fairness and predictive utility under a binary sensitive attribute. It introduces a pointwise relaxation that permits each individual’s prediction gap to retain at most an ϵ\epsilon-fraction of the corresponding Bayes-optimal gap. Assuming independence between the sensitive attribute and latent background variables, together with invertibility of the feature-generating mechanism, the paper derives the risk-minimizing predictor at each relaxation level. Under squared loss, the solution interpolates between the optimal exactly fair predictor and the Bayes predictor; under cross-entropy, it is characterized by a one-dimensional convex optimization problem. The paper also proposes a plug-in implementation, analyzes excess risk under estimation errors, and evaluates the construction using synthetic and VAE-based semi-synthetic data.

Critical strengths
  • The paper provides an explicit characterization of a counterfactual fairness relaxation. Rather than introducing only a regularization parameter, it specifies how much of each individual’s Bayes prediction gap may remain. The squared-loss solution and its exact excess-risk expression make the implications of this particular relaxation transparent.

  • The analysis distinguishes counterfactual-generator error from prediction-function error. This separation explains why generator error becomes less influential near the unconstrained endpoint, whereas prediction estimation error remains. The synthetic experiments isolate these effects and help interpret the oracle-to-estimation transition.

Critical weaknesses
  • The practical construction lacks corresponding control of achieved fairness, and one of its risk bounds is incorrect as stated. Section 5 bounds predictive risk but does not bound violation of the true counterfactual fairness constraint when the generator is estimated. This matters because the nominal relaxation level need not describe the achieved fairness; Figure 2 only reaches approximately RTE 0.82 on Law School and 0.5 on Adult. Moreover, Proposition 5.4 fails even with an exact generator: at ϵ\epsilon = 1, take true probabilities identically 0.5 and estimated probabilities identically 0.6. Then ηϕ\eta_{\phi} = 0.1 and the stated bound reduces to 0.01, while the actual excess risk is KL(Ber(0.5)∣∣Ber(0.6))≈0.02041KL(Ber(0.5) || Ber(0.6)) ≈ 0.02041. This example satisfies the interiority assumption. A boundary-dependent coefficient appears necessary on the quadratic estimation-error term. Although this correction seems feasible, the current results do not fully support the claimed controllability of the practical fairness–utility trade-off.
Questions for authors
  1. Can you address the counterexample to Proposition 5.4 and provide the corrected coefficient on the ηϕ2\eta_{\phi}^2 term, including the corresponding correction to Appendix A.7?

  2. Under the estimation assumptions in Section 5, what bound can be given on the achieved true counterfactual gap or violation of the prescribed pointwise cap? In particular, what remains guaranteed at ϵ\epsilon = 0?

3 comments

We thank the reviewer for the thoughtful reviews and address the concerns below.

Weakness and Question 1 — Proposition 5.4 and Appendix A.7

We thank the reviewer for carefully examining Proposition 5.4. We would like to clarify that, in the reviewer's example, the stated excess-risk bound reduces to Cρηϕ2C_\rho\eta_\phi^2, rather than ηϕ2\eta_\phi^2. Therefore, the relevant comparison should be between Cρηϕ2C_\rho\eta_\phi^2 and 0.020410.02041.

Since the example considers a very special case, we can clarify this directly from the Taylor expansion used in the proof of Proposition 5.4, without revisiting the full proof.

Specifically, in Appendix A.7 (line 1152), we apply Taylor's theorem similar to the proof of Proposition 5.3 (Appendix A.6, line 1089). To make the omitted intermediate steps explicit:

\ell_{\mathrm{CE}}(\widehat\phi^\epsilon_a,\mu_a) -\ell_{\mathrm{CE}}(\widehat\phi^\epsilon_{G,a},\mu_a) \leq \left| \frac{\widehat\phi^\epsilon_{G,a}-\mu_a} {\widehat\phi^\epsilon_{G,a}(1-\widehat\phi^\epsilon_{G,a})} \right||d_a| +\frac{1}{2\rho^2}d_a^2, \end{aligned}$$ where $d_a=\widehat\phi^\epsilon_a-\widehat\phi^\epsilon_{G,a}$. The coefficient $1/(2\rho^2)$ follows from bounding the second derivative of the cross-entropy loss under Assumption 2: $$\ell_{\mathrm{CE}}''(q,\mu_a) =\frac{\mu_a}{q^2}+\frac{1-\mu_a}{(1-q)^2} \leq\frac{1}{\rho^2},$$ where $q\in[\rho,1-\rho]$. The boundary-dependent constant $C_\rho$ in Proposition 5.4 absorbs the coefficients arising in these bounds, including $1/(2\rho^2)$, and therefore $C_\rho\geq 1/(2\rho^2)$. In the reviewer's example, $\epsilon=1$ implies $\widehat\phi^1_{G,a}=\mu_a=0.5$ and $\widehat\phi^1_a=0.6$, $d_a=\eta_\phi=0.1$, $\rho\leq 0.4$. Hence, the first-order term vanishes since $\widehat\phi^1_{G,a}=\mu_a$. Taking $\rho=0.4$, which satisfies Assumption 2, we obtain $$C_\rho\eta_\phi^2 \geq\frac{\eta_\phi^2}{2\rho^2} =\frac{0.01}{2(0.4)^2} =0.03125>0.02041.$$ Therefore, the example does not violate the bound in Proposition 5.4. **Weakness and Question 2 — Achieved fairness** Denote the achieved counterfactual gap as $$\Delta_{\hat\phi^\epsilon}(u) =\hat\phi^\epsilon_1(u)-\hat\phi^\epsilon_0(u) =\hat\phi_1^\epsilon(x_1(u),1)-\hat\phi_0^\epsilon(x_0(u),0),$$ The violation between achieved counterfactual gap and prescribed cap is $|\Delta_{\hat\phi^\epsilon}(u)-\epsilon\delta(u)|$. Under the assumptions in section 5, we have: For regression: $$|\Delta_{\hat\phi^\epsilon}(u)-\epsilon\delta(u)| \leq(1-\epsilon)L\eta_G+2\eta_\phi.$$ For classification: $$|\Delta_{\hat\phi^\epsilon}(u)-\epsilon\delta(u)| \leq 2C_\rho(1-\epsilon)(L\eta_G+2\eta_\phi) +2\epsilon\eta_\phi.$$ We will include these results and their proofs to provide a more complete theoretical characterization of the achieved counterfactual fairness guarantees.

Thank you for the clarification. I had misread the scope of CρC_\rho, so I withdraw my objection to Proposition 5.4. The additional fairness-deviation bounds also address my original question.

After reading the full discussion, I still have some concern about the motivation and empirical contribution. The loan example helps explain how relative and absolute budgets differ, but I would have liked a clearer application-based justification for choosing proportional gap reduction. I also appreciate the additional means and standard deviations. However, the empirical evidence remains limited to two semi-synthetic datasets. On Law School, the method coincides with L2 regularization, as expected from the theoretical equivalence. The Adult results are encouraging, but I am not yet convinced that the evaluation establishes a sufficiently strong practical benefit.

While my original technical concerns have been addressed, these remaining concerns leave me keeping the original score.

Thank you for confirming that your objection to Proposition 5.4 has been resolved and that the additional fairness-deviation bounds address your original question.

Regarding the application-based motivation, our loan example illustrates a practical objective: ensuring that fairness adjustments apply to every applicant, rather than leaving smaller disparities entirely unaddressed. Proportional gap reduction provides a direct way to achieve this objective. Different fairness criteria inevitably reflect different priorities. We do not expect any single criterion to be universally preferable; we adopt one that is well-defined and interpretable.

Regarding the empirical results, the equivalence with L2 is a general theoretical result under squared-loss regression, rather than a dataset-specific result on Law School. Hence, their agreement is theoretically expected. In contrast, this equivalence does not generally hold for classification, where the Adult results demonstrate improvements over L2 method.

More importantly, the central contribution of this work is the theoretical characterization of the optimal fairness–utility frontier and its estimation guarantees, with the experiments providing complementary evidence. We hope these contributions, together with the resolution of the original technical concerns, will be considered in the final assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Highlights of Our Rebuttal

We thank the reviewers for their valuable comments and suggestions. We would like to briefly highlight the main additions provided in our rebuttal:

  • Achieved fairness under estimation error. We provide bounds quantifying the deviation of the achieved counterfactual gap from its prescribed target, for both regression and classification.

  • Bound tightness and achieved RTE. We report controlled perturbation experiments with simultaneous errors in the counterfactual generator and Bayes predictor, attaining the regression excess-risk bound and closely approaching the classification bound in the reported settings. Additional experiments examine how the relaxation parameter ϵ\epsilon relates to achieved RTE under estimation error.

  • Variability in benchmark comparisons. We report mean and standard deviation across repeated runs for Law School and Adult at specified RTE levels. These results provide further evidence that the performance improvements are not merely due to run-to-run variability.

We kindly invite the reviewers to take a look at these additions in our detailed responses and would be happy to discuss any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.