- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies mean demographic parity in regression when the sensitive attribute is continuous and potentially multidimensional. It uses DPVar, defined as the variance of the conditional mean across the sensitive attribute , to measure fairness. The authors formulate the problem as functional bilevel optimization, where an inner neural network estimates the conditional mean and an outer network minimizes the prediction loss together with a DPVar penalty. Two optimization approaches are proposed: FBO, which uses an explicit hypergradient, and ITD, which differentiates through the unrolled inner updates. The experiments include a controlled simulation, a 21-task semi-synthetic benchmark based on OpenML datasets, and the Communities & Crime (C&C) dataset.
Critical strengths
-
Mean demographic parity for continuous and multidimensional sensitive attributes is a relevant problem. The paper also makes a clear distinction between mean parity and full statistical independence, which helps motivate the proposed fairness criterion.
-
The hypergradient derivation appears correct, and the resulting FBO update is relatively simple and computationally efficient.
-
The controlled simulation includes an analytic oracle, which is useful for evaluating the method. The authors have also released their code, making the experiments easier to reproduce.
Critical weaknesses
-
The novelty is limited, and the paper does not sufficiently discuss closely related work. Mean-parity regression, fairness with continuous sensitive attributes, and bilevel optimization have all been studied in previous work. Several papers that seem particularly relevant are missing from the discussion, making it difficult to identify the main methodological contribution.
For mean parity and conditional-mean fairness in regression, relevant papers include:
- Komiyama et al., "Nonconvex Optimization for Regression with Fairness Constraints" (ICML 2018).
- Fitzsimons et al., "A General Framework for Fair Regression" (Entropy 2019).
- Scutari et al., "Achieving Fairness with a Simple Ridge Penalty" (Statistics and Computing 2022).
- Wei et al., "Mean Parity Fair Regression in RKHS" (AISTATS 2023).
- Giuliani et al., "Generalized Disparate Impact for Configurable Fairness Solutions in ML" (ICML 2023). In particular, this work already considers conditional-mean fairness for continuous protected attributes.
More importantly, the DPVar objective can be written as
�0�
.$$
This gives a min-max formulation in which an adversary tries to predict the model output from the sensitive attribute. From this perspective, FBO appears closely related to alternating gradient descent-ascent. Similar ideas have already been explored in Zhang et al., "Mitigating Unwanted Biases with Adversarial Learning" (AIES 2018), and in neural HGR-based fairness methods such as Grari et al., "Fairness-Aware Neural Rényi Minimization for Continuous Features" (IJCAI 2020).. I think the authors need to explain more carefully what is new beyond applying these existing techniques to the DPVar objective.
-
The baselines are weak, and some comparisons are not fair. I have several concerns about the experimental comparisons.
-
GDP uses a restricted representation of the sensitive attribute. The baseline is applied to a two-dimensional PCA projection of . In the controlled experiment, this projection removes the only direction responsible for the disparity, putting GDP at a clear disadvantage.
-
The HSIC baseline does not use the same model capacity. It is implemented with a linear predictor and a linear kernel, whereas FBO and ITD use neural networks. As a result, the comparison mixes differences in model expressiveness with differences in fairness regularization. It is difficult to determine whether the reported improvements actually come from the proposed fairness method.
-
Several important competitors are missing, which include HGR-NN (cited but not evaluated), nonlinear-kernel HSIC and MMD penalties, GeDI, and reduction-based approaches for fair regression. These methods would provide much stronger comparisons, particularly if all methods used the same downstream predictor architecture.
-
The proposed methods do not consistently outperform even the weaker baselines. On the C&C dataset, the linear HSIC baseline matches or outperforms FBO and ITD at most DPVar budgets, according to the authors' released
aggregate_frontiers.csv. This makes the claimed empirical advantage less convincing.
-
-
The DPVar estimator itself is not adequately validated. This is one of my main concerns. In both the tabular benchmark and the C&C experiments, fairness is evaluated using a neural DPVar estimator from the same model family as the proposed method. However, it is unclear whether this estimator can reliably detect nonlinear dependence between the predictions and sensitive attributes.
The authors' released controlled simulation results (
summary.csv) illustrate this problem. In one setting, the estimated DPVar is only , while the true value is . Along the FBO optimization path, the true DPVar decreases by approximately 80-fold, whereas the estimated DPVar decreases by only about threefold.This is a substantial discrepancy. It suggests that the estimator may miss much of the nonlinear disparity that the proposed method is intended to address. Since the same type of estimator is used to evaluate fairness in the real-data experiments, I am concerned that the reported fairness improvements may not accurately reflect the true level of disparity.
I think an independent fairness evaluation, preferably using a stronger and more flexible estimator, is needed before drawing conclusions about the effectiveness of the proposed methods.
Questions for authors
-
Beyond applying functional bilevel optimization to DPVar, what are the main methodological contributions relative to HGR-NN, GeDI, and Wei et al., "Mean Parity Fair Regression in RKHS" (AISTATS 2023)? Also, do the authors agree that the proposed FBO method can be interpreted as alternating gradient descent-ascent on the equivalent min-max problem? If so, what is the advantage of the bilevel formulation?
-
How do FBO and ITD compare with HGR-NN, nonlinear-kernel HSIC/MMD penalties, and full-dimensional GDP when all methods use the same neural predictor architecture and comparable training budgets?
-
Could the authors validate the DPVar estimates using an independent, high-capacity estimator, particularly in the controlled setting where the true DPVar is available? I would also like to see ablation studies on the number of inner steps, unrolling depth, and inner-network capacity to understand how these choices affect the results.