- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes to use the concept of Envelope Regression for fair regression by (under a given condition) decomposing the space into 4 different components containing qualitatively different information about the regression task. The fairness regularization if then confined only to the subspace relevant for predictive-sensitive associated information where as the sensitive only component can be discarded.
Critical strengths
-
The paper addresses a central gap in the literature that previous fair regression models usually do not explicitly take the overlap of the sensitive and target variable(s) into account. For this they extend the idea of envelope regression to further separate the information into different subspaces: the idea is well motivated and the paper well written. In particular, this paper allows to make the fairness-utility tradeoff explicit by isolating the subspace relevant for this.
-
The theory support central claims of the paper: from variance reduction to how the penalized regressor interpolates between the extremes these statements support the derived methodology.
Critical weaknesses
-
The whole premise of the paper is built on the fact that the coefficient is a sensible proxy for an actual fairness score. While I know that this is in line with previous works, in the mean-time several extensions of fairness scores where developed that are also usable for regression and also continuous sensitive attributes, e.g. [3-4] (which arguably was not the case yet for the previous works). However, the empirical evaluation completely relies on measuring fairness with the same metric it optimizes itself (which is not great) so the results don't actually indicate an achieved level of "fairness".
-
The empirical evaluation does not (yet) support the paper claims. Next to the missing fairness scores (see point above), an indirect but central claim is that the decomposition is useful and more intelligent than other naive approaches not taking the predictive component into account; whilst true from the motivation there is only limited empirical evidence of this advantage as the FRRM baseline is also based on envelope regression - a comparison to fair kernel learning [2] with a linear kernel or any other baseline would be helpful to actually support this claim. Also one of the real world datasets is confined to the appendix which is an odd choice given only a single real-world dataset is considered in the main text. Visually, the plots are also difficult to read: the text is too small and does not match the paper font, and the labels/ticks seem not to be vector graphics at least they render blurry, for ICLR I would expect better quality figures
-
Rather minor: The extensions to the non-linear kernel or DL setting are discussed in the appendix: this is a bit odd as these are discussed as straight-forward extensions so one either expects results supporting that this is indeed works or by shortening that outlook and placing it in the main text as clear future work, currently it is not fitting to the main text.
Overall I believe this work to be very well fit for ICLR but mainly the empirical section needs significant improvements to support the paper claims, I am open to increase my score if these issues can be adequately addressed.
[2] Pérez-Suay, et al. Fair Kernel Learning (MLKDD 2017)
[3] Jiang et al., Generalized Demographic Parity for Group Fairness (ICLR 2022)
[4] Narasimhan, H.,et al, Pairwise fairness for ranking and regression. AAAI Conference (2020)
Questions for authors
-
The acronym "FERM" is already used by at least one other approach in fairness (see [1]) so I suggest to change the acronym here to reduce the possibility of confusion
-
Can you clarify how you find ? I understand that it is computed based on as the smallest lambda satisfying the required level, but can you compute that analytically?
-
You assume what you call the "compatibility condition" on the projection matrices and to commute: how sensible is that assumption ? Does the algorithm work even if this is violated?
[1] Donini, M., Oneto, L., Ben-David, S., Shawe-Taylor, J. S., & Pontil, M. (2018). Empirical Risk Minimization Under Fairness Constraints. In Advances in Neural Information Processing Systems
5 comments
Rebuttal 2
Q1. Method acronym: We appreciate the reviewer for identifying the naming overlap with Donini et al. (2018). To avoid confusion, we will rename our method Fair Subspace Envelope Regression (FSER) throughout the revised manuscript, better reflecting its predictive–sensitive subspace decomposition.
Q2. Selection of : We determine through a one-dimensional numerical search, selecting the smallest nonnegative satisfying the prescribed fairness constraint. Although the fairness measure can be expressed analytically as a function of , inverting this relationship generally does not yield a simple closed-form solution.
Q3. Compatibility condition:
We characterize the commutativity condition as a sufficient condition for the exact four-way orthogonal decomposition, rather than a requirement for implementing the algorithm. Without this condition, predictive and sensitive directions may be closely related without overlapping exactly, leaving some directions unaccounted for in the decomposition. The predictive envelope remains well-defined, and the regression algorithm can still be implemented, although the exact identification of shared and predictive-only directions and the associated theoretical guarantees may no longer hold.
To assess the assumption in practice, we tracked the commutator norm in our experiments. Across all MEPS experiments, -, indicating compatibility up to numerical precision. In Communities and Crime, the mean decreased from 0.609 to 0.329 with increasing sample size. Despite these departures, the algorithm remained operational and achieved favourable fairness-utility performance in selected settings. Thus, compatibility ensures the exact theoretical interpretation, while our experiments demonstrate that exact commutativity is not necessary for practical implementation. We will clarify this distinction and include the diagnostics in the revision.
We thank the reviewer again for their constructive suggestions, which have helped us strengthen the empirical evaluation, improve the presentation, and clarify the scope of our theoretical contributions. We believe these additions directly address the reviewer's concerns and substantially improve the manuscript.
Thank you for the detailed response and extensive set of additional evaluations. My main concerns regarding the choice of the fairness criterion as well as those regarding the scope of the empirical evaluation are now resolved. The same is true for my questions raised above.
To improve readability, we have also redesigned the figures to present test MSE–unfairness trade-off frontiers, with clear labels, consistent typography, and standard-error bars, using vector graphics for improved visual quality.
As the revised document is not uploaded to OpenReview I cannot yet confirm that these (and other) points are adequately incorporated in the revision. Once you have uploaded the revised document, please reply to let me know so I can review the updated PDF and raise my score accordingly.
Rebuttal 1
We thank the reviewer for recognizing the paper's conceptual and theoretical contributions and its relevance to ICLR. We have substantially expanded the empirical evaluation to address the concerns raised. Representative results, including larger-sample and non-healthcare experiments, additional methodological comparisons, and nonlinear dependence diagnostics, are reported in our response to Reviewer maSX. Below, we address the reviewer's specific comments and questions.
W1. Fairness criterion and independent evaluation:
We respectfully clarify that the central premise of our paper is not the choice of RS2 as a fairness measure, but the interpretable four-way decomposition of predictive and sensitive information. This decomposition is established independently of the subsequent fairness regularization. We adopt the established multivariate RS2 criterion in the second stage because it yields a tractable fairness-utility trade-off, characterized analytically in Corollary 5.6. Thus, the structural decomposition is not tied to this particular fairness criterion, and extending the regularization framework to nonlinear measures, including kernel-based approaches suggested by the reviewer, is a natural direction for future work.
We agree that controlling RS2 does not guarantee nonlinear independence, and appreciate the reviewer's suggestion to evaluate fairness beyond the optimized criterion. Accordingly, we added held-out HGR and normalized HSIC fairness scores, which assess nonlinear dependence between predictions and sensitive attributes. The new results show that FERM achieves lower nonlinear dependence than the references in several settings, though not uniformly across sample sizes and fairness budgets. Detailed results are provided in our response to Reviewer maSX.
W2. Additional comparisons and empirical validation:
We respectfully clarify that FRRM is not an envelope-regression-based method. We selected FRRM because it employs the same multivariate RS2 fairness criterion and ridge regularization, allowing us to isolate the contribution of our proposed structural decomposition. Beyond providing interpretable, subspace-level auditing of fairness regularization, our controlled simulations demonstrate statistical efficiency gains in small-to-moderate samples with relatively high-dimensional predictors.
To further strengthen this evidence, we added Komiyama et al. (2018) as an additional non-envelope reference and substantially expanded the real-data experiments. In the new demographic MEPS experiments (Section 4 of our response to Reviewer maSX), at n=1,000 and r=0.20, FERM achieves lower mean test MSE (0.905 versus 0.933 for both references) and HGR (0.341 versus 0.368). In the non-healthcare Communities and Crime experiments (Section 5 of our response to Reviewer maSX), at (n=150,p=70,r=0.10), FERM achieves lower MSE (1.344 versus 1.542 and 1.977), held-out RS2 (0.096 versus 0.146), and normalized HSIC (0.128 versus 0.152 and 0.151). These findings provide additional evidence of the decomposition's benefits in data-limited, higher-dimensional settings. At larger sample sizes, the references can achieve lower MSE, highlighting that the advantage is setting-dependent rather than universal.
Finally, due to space constraints, we originally prioritized theory-validating simulations in the main text. With additional space in the revision, we will move the second real-data experiment into the main text and report the expanded results in Appendices F.3–F.5. To improve readability, we have also redesigned the figures to present test MSE–unfairness trade-off frontiers, with clear labels, consistent typography, and standard-error bars, using vector graphics for improved visual quality.
W3. Nonlinear extensions: We agree that the kernel and deep-learning extensions should be presented more clearly as future directions. The appendix currently outlines how the proposed structural ideas might extend beyond linear regression, but does not establish corresponding theoretical or empirical guarantees. We will revise this discussion accordingly, emphasizing that developing such guarantees requires further research.