- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents a Fairness-Aware Additive Network (FAN) for mortgage lending. The aim is to reduce group disparities while still maintaining predictive performance and interpretability. It's an additive neural network that takes univariate features and sparse pairwise interactions. The training loss is a weighted sum of binary cross-entropy, a soft equalized odds (EO) penalty, and a 1-norm sparsity penalty. The authors also provide a theoretical analysis of the relationship between the soft EO penalty and the threshold-averaged EO penalty and derive a uniform convergence bound for the empirical EO penalty. The method is evaluated on the HMDA dataset.
Critical strengths
The paper is a fairly comprehensive empirical analysis of fairness-aware mortgage lending using large-scale HMDA data. In addition to standard predictive and fairness metrics, they examine within-lender variation, institutional underwriting thresholds, and robustness over time. Together, these findings provide useful evidence of the practical relevance of fairness regulation.
Critical weaknesses
- The methodological and theoretical novelty is very limited.
The proposed FAN architecture is essentially a re-implementation of GAMI-Net (Yang et al., 2021) with the same additive neural architecture, the same sparse pairwise interactions and the same three stage training process. The main methodological difference is the inclusion of a differentiable equalized odds (EO) penalty which has been studied before in the literature (e.g. Pfohl et al., 2021). The theoretical contributions are also very limited. Prop 1 is a standard integral identity. Prop 2 does not actually demonstrate that minimizing the EO penalty will control the threshold based fairness gap. Prop 3 is a standard uniform convergence bound for the soft EO penalty. There's essentially no new methodological or theoretical contribution here.
- The baselines in the comparisons are weak.
The evaluation is limited to logistic regression and unconstrained FAN with no comparison made to GAMI-Net or any of the other well known fairness aware learning methods. The experiments do show that adding the EO regularization to FAN does result in better fairness metrics, but they don't actually show that it is better than any of the other methods. That's a significant methodological gap.
Reference
-
Yang, Z., Zhang, A., & Sudjianto, A. (2021). GAMI-Net: An explainable neural network based on generalized additive models with structured interactions. Pattern Recognition, 120, 108192.
-
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021). An empirical characterization of fair machine learning for clinical risk prediction. Journal of Biomedical Informatics, 113, 103621.
Questions for authors
- Methodological novelty.
The FAN architecture is very similar to GAMI-Net (Yang et al. 2021) in that it has an additive neural structure, sparse pairwise interactions, and a 3 stage training process. Can the authors be more specific about the architectural or algorithmic differences between FAN and GAMI-Net? Pfohl et al. (2021, eq 9-12) already introduce a differentiable EO regularizer based on the squared differences in group-conditional mean. For two protected groups, their formulation reduces to a weighted version of the penalty in FAN. Could the author clarify the novelty of their EO regularizer beyond the normalization and the use of predicted probabilities instead of log-probabilities?
- Theoretical contributions.
Prop 2 establishes , which is simply a consequence of the non-negativity of variance. For two protected groups we have . Based on the uniform variance across groups assumption (line 1422-1426), it follows that and The factor changes if the group weights are unequal but the variance decomposition still hold.
Since the training objective is to minimize and the variance term isn't directly under control then Prop 2 doesn't establish that minimizing the EO penalty will lead to reduced threshold averaged EO disparities. Can the authors clarify how this result supports their stated fairness guarantee? Are further assumptions required to control the variance term? More generally what are the non-trivial theoretical contributions of Props 1-3 beyond the standard integral identities, variance inequalities and uniform convergence arguments?
- Experimental comparisons.
The evaluation looks at FAN against logistic regression and unconstrained FAN. Are there any comparisons against GAMI-Net and other established fairness-aware learning methods? Given the substantial architectural similarities between FAN and GAMI-Net, FAN with appears to approximate an unconstrained GAMI-Net. Could the authors clarify which differences between these models affect the performance? In particular, what evidence shows that FAN with EO regularization offers advantages over GAMI-Net with the EO regularization? Such comparisons would help establish the incremental value of the proposed method.
Responses
Contribution: FAN builds on additive neural modeling, as described in Section 3.1, p. 3, and Appendix D, pp. 18–22, but builds a custom framework for the empirical question of interest. Its principal modification is fairness-aware estimation while preserving exact additive score decompositions. Pfohl et al. provide a relevant antecedent for outcome-conditional mean-matching penalties. However, normalization and the choice of probabilities versus log-probabilities are not the basis of our contribution. The contribution is therefore the integrated framework and the economic evidence it enables. Sections 6–7, pp. 5–9, show how fairness adjustments reduce reliance on geographic proxies while preserving core underwriting signals, and how disparities decline within lenders even when approval capacity is fixed. The existence of the individual methodological ingredients does not establish these findings or explain how fairness interventions change model-implied mortgage allocation. Finally, the paper’s contribution should also be assessed in the financial-economic and regulatory context stated in Section 1, pp. 1–2: can mortgage decision rules reduce group disparities while retaining nonlinear predictive performance and providing transparent explanations of individual decisions?
Theoretical contributions: Proposition 2 establishes a lower bound, not a guarantee that minimizing the soft penalty reduces threshold-averaged disparity. Appendix F, p. 26 already excludes a pointwise fairness guarantee. The role of Proposition 1 is to give the conditional mean probabilities a threshold-average interpretation and Proposition 3 concerns generalization of the soft penalty under the stated assumptions. A stronger guarantee for averaged disparity would require additional control of the variance remainder or direct regularization of the averaged disparity. Our empirical conclusions are supported by direct evaluation: Appendix H, Table 8, p. 35, reports that test-set threshold-weighted disparity falls from 0.00114 to 0.00046. Appendix I, Table 9, p. 37, reports smaller hard-decision error-rate gaps at matched approval rates, with additional threshold sweeps in Tables 10–11, pp. 38–39. These are empirical improvements in this application, rather than consequences of reversing Proposition 2.
Experimental comparisons: The existing comparisons already address whether incorporating fairness into an interpretable mortgage model produces less disparate decisions while preserving predictive fit. Section 5, Table 1, p. 4, evaluates this within the FAN model family. Appendix I, Table 9, p. 37, matches overall approval rates and Section 7, Table 3, p. 9, fixes approval capacity within lenders. These results support the paper’s economic finding that candidate less discriminatory decision rules exist within a common public-information set. The manuscript does not report a separate comparison against other models trained with the identical EO penalty, and therefore does not intend to establish an architectural advantage over such models. Such comparisons would be required to claim superiority over alternative fairness algorithms, but they do not negate the documented fairness–fit trade-off or its economic interpretation.