acceptodds
Under review as a conference paper at ICLR 2027

Fair Representation Learning Beyond Euclidean Geometry

Abstract

Fair representation learning enforces that different demographic groups follow similar distributions in representation space, with similarity measured by a distributional discrepancy computed in Euclidean coordinates. This implicitly assumes the representation space is flat, which is an assumption violated whenever representations are embedded in curved spaces such as hyperbolic space, where the ambient Euclidean distance can differ arbitrarily from the geodesic distance the model actually uses. We show the mismatch changes the conclusion rather than the constant: two group distributions can be nearly matched under a Euclidean discrepancy while remaining well separated on the underlying manifold, so the resulting fairness guarantees do not transfer. We therefore measure discrepancy intrinsically, on Hadamard manifolds, non-positively curved spaces that include Euclidean and hyperbolic geometry as special cases and whose unique geodesics make projection well defined. Our Cartan–Hadamard Sliced-Wasserstein (CHSW) distance aggregates one-dimensional Wasserstein distances along geodesic projections, avoiding full optimal transport on the manifold while respecting its curvature, and is differentiable and cheap enough to serve as a training regularizer. We prove that minimizing it bounds demographic parity violation, and show on standard benchmarks that it improves fairness substantially at competitive accuracy.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Fair Representation Learning, Cartan–Hadamard Sliced-Wasserstein

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_6xK4OpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces the Cartan-Hadamard Sliced-Wasserstein (CHSW) manifold distance for fair representation learning that takes into account that geodesic distances differ from the usually chosen euclidean assumption. It is theoretically shown that minimizing this distance actually bounds fairness scores (i.e. demographic parity) and demonstrates empirical advantages on selected (theoretical and real-world) fairness benchmarks.

Critical strengths
  • The work is well motivated, well written and theoretical strong. The idea of extending fair representation learning to take the manifold structure into account is relevant and interesting. In particular, the paper addresses a well thought-of research question throughout.

  • The theory shows that the proposed distance can be used to bound demographic parity for pullback euclidean manifolds and also introduces a soft DP violation for more general Hadamard manifolds also including a bound by the proposed manifold distance.

Critical weaknesses

First

On such a manifold, Euclidean distance is the wrong ruler: it measures separation along straight lines through the ambient space, whereas a downstream predictor, or an adversary trying to recover the protected attribute, operates in the distance along the manifold

But is this really the case in Fair Representation Learning? Usually the manifold hypothesis directly affects the original input space of the data, but it is also known that Deep Learning usually disentangles this structure (e.g. [1]) so this provides evidence for the contrary, e.g. that assuming an euclidean representation of the embedding space and relying on an euclidean penalty should be fine. While Figure 6 and 7 only test the network size for the proposed approach it would be interesting to see if the same conclusion hold for any other baseline (i.e. [1] suggests that manifold distances matter less the bigger the network), i.e. currently large networks seem to not perform well for the proposed approach but it would be relevant to see if larger networks are more beneficial for the baselines.

Currently the main motivational figures are in the appendix, it would be further helpful to include these in the main text and also show how the proposed CHSW is computed in contrast to the euclidean approach.

Second

Whilst the results in Table 1 favor the proposed approach, it is not discussed that in several cases the "best performance" comes with only marginal improvements yet much higher standard deviation, which is also not discussed in section 5.2.1 (for example on SPD and Pullback for change in demographic parity).

Further, Figure 3 does not report standard deviation and is missing a "dummy" baseline, i.e. a constant predictor to anchor the results (i.e. the results in the bottom right probably just correspond to a constant model that predicts the trainings data mean). Both changes are highly helpful in actually interpreting the results.

Also the authors report to test coefficients up to λ=1014\lambda=10^{14} which is super confusing as such extreme penalties should not be useful and numerically extremely unstable (and this should probably be evident from the standard deviation).


To maintain my already positive score it would be required to incorporate the changes suggested in point 2 where for point 1 it is sufficient to adapt the framing accordingly or add convincing arguments why manifold distances are still required (e.g. based on the empirical results later).

[1] Why Deep Learning Works: A Manifold Disentanglement Perspective, Brahma et. al. (IEEE TNNLS, 2016)

Questions for authors
  • Is there a reason you choose λ=1014\lambda=10^{14} or expect such values to be useful?

  • Could you add a notation section ? This would make reading the paper much more easy, even though many seem to follow standard convention it takes extra effort to confirm, e.g. I assume that ToMT_o\mathcal{M} indicates the tangential plane touching origin and that P(M)\mathcal{P}(\mathcal{M}) is the set of all probability measures on M\mathcal{M}?

  • I assume that the approach works for any discrete groups and arbitrary targets (i.e. also for regression) but not for continuous sensitive attributes right?

6 comments

We sincerely thank the reviewer for the thoughtful review and encouraging assessment of our work. We particularly appreciate your recognition of the relevance of manifold-aware fair representation learning, the clarity of our presentation, and the strength of our theoretical analysis. Your positive assessment of both the demographic-parity bound on pullback Euclidean manifolds and the soft demographic-parity bound on general Hadamard manifolds is especially encouraging. Below, we address your comments and suggestions to further improve the paper. All changes in the revised manuscript are highlighted in red.

Q1: Representation geometry, baseline capacity comparisons, main-text motivation, and computational illustrations.

A: Thank you for this thoughtful comment. We agree that the quoted sentence is stated too strongly, and we have revised it in Section 1 (pages 1-2). Our premise concerns the representation space on which the fairness regularizer acts, not the input space. Below, we clarify this motivation through related work and provide additional experiments examining network capacity and learned representation geometry.

1. Deep networks do not necessarily flatten their representations. Deep networks can disentangle features and improve class separability without necessarily making the representation geometry flat. Ansuini et al. [1] find that learned representations can remain curved even in the last hidden layer, suggesting that flattening is not a general outcome of deep representation learning. Kaufman and Azencot [2] directly study latent image manifolds and observe substantial curvature in the final layers of trained networks across architectures. Khrulkov et al. [3] report high hyperbolicity of image datasets measured through features extracted by pretrained networks, motivating the use of hyperbolic image representations. Together, these studies support examining representation geometry empirically rather than assuming that increasing network capacity necessarily makes Euclidean distances sufficient.

[1] Ansuini A et al. Intrinsic dimension of data representations in deep neural networks. NeurIPS, 2019.

[2] Kaufman I et al. Data representations’ study of latent image manifolds. ICML, 2023.

[3] Khrulkov V et al. Hyperbolic image embeddings. CVPR, 2020.

2. Group information can persist despite apparent invariance under a restricted criterion. Elazar and Goldberg [4] report that, in their experiments, an adversarial component can reach chance-level accuracy during training while a post-hoc classifier still recovers demographic information above chance. Ravfogel et al. [5] show that guardedness against log-linear prediction does not generally prevent a multiclass downstream log-linear model from indirectly recovering the erased concept. These findings motivate distinguishing successful task prediction or restricted invariance from full removal of group information. They complement the geometric motivation, rather than directly establishing that Euclidean penalties are inadequate.

Our framework accommodates both flat and curved representations. For manifold-valued encoders, such as hyperbolic embeddings or SPD matrices, CHSW measures distributional discrepancies using the specified representation geometry. For pullback Euclidean manifolds, CHSW reduces to ordinary sliced Wasserstein distance in the transformed coordinates (Appendix F.3.4). Consistent with this flexibility, manifold selection yields pullback Euclidean geometry for Adult and hyperbolic geometry for LFWA. We therefore treat geometry as a modeling choice evaluated through predictive and fairness performance, rather than assuming that Euclidean regularization always fails.

[4] Elazar Y et al. Adversarial removal of demographic attributes from text data. EMNLP, 2018.

[5] Ravfogel S et al. Log-linear guardedness and its implications. ACL, 2023.

3. Capacity comparisons with all five baselines. Following the suggestion, we extended the LFWA network-size analysis to all five baselines, using the same data splits and the full grid dz,h∈d_z,h\in {16,32,64,128 }. We explored regularization coefficients separately for different methods and architecture settings, since a single coefficient need not be appropriate for every network size. The supplementary figure4(a) in page 10 report ACC, ΔDP\Delta_{\mathrm{DP}}, and ΔDPsoft\Delta_{\mathrm{DP}}^{\mathrm{soft}}.

Larger networks can benefit the baselines, but CHSW also performs well at large capacities. For example, at dz=h=128d_z=h=128, CHSW achieves ACC of 91.29%91.29\%, hard DP of 5.68%5.68\%, and soft DP of 5.15%5.15\%, compared with 91.06%91.06\%, 6.98%6.98\%, and 7.07%7.07\%, respectively, for SW. Thus, the observed behavior is not that CHSW necessarily deteriorates as the network becomes larger.

4. Distance diagnostics across network sizes. We additionally conducted a separate diagnostic using CHSW models trained with a fixed λ\lambda across the same 16 architecture settings. For each model, we sampled 2000 distinct test pairs and compared hyperbolic geodesic distances between mapped representations with two Euclidean distances: the ambient distances between the mapped manifold points, and the distances between the raw encoder outputs. The result is shown in figure 4(b) in page 10.

For the largest network, dz=h=128d_z=h=128, the corresponding Spearman correlations are 0.9987±0.00040.9987\pm0.0004 and 0.3422±0.04260.3422\pm0.0426, respectively. Thus, even for this large network, Euclidean distances in the raw encoder coordinates do not closely preserve the distance rankings induced by the specified hyperbolic geometry. Meanwhile, ambient Euclidean distances between mapped points closely preserve these rankings.

5. Main-text motivation and computational comparison. We have moved the motivating example (former Figures 1 and 4) into Section 1 of the main text (Figure 1, page 2), with additional Euclidean SW, MMD, and Sinkhorn values. The example shows that small ambient Euclidean discrepancies can coexist with substantial separation under the hyperbolic geometry, clarifying why Euclidean coordinate distances may not faithfully reflect intrinsic geometry when representations are modeled on curved manifolds.

The computational distinction is as follows. Euclidean SW uses linear projections of encoder outputs, whereas CHSW uses geometry-aware one-dimensional projections on the representation manifold. Both methods then compute Wasserstein barycenter discrepancies using empirical quantiles and average over slicing directions. We have added this computational comparison to Appendix E.1 (pages 26–28).

Q2: Discussion of improvement magnitudes and variability on Pullback and SPD.

A: Thank you for highlighting this. We agree that the original Section 5.2.1 insufficiently discussed effect sizes and variability. On Pullback and SPD, mean ΔDP\Delta_{\mathrm{DP}} improves from 0.1920.192 to 0.1870.187 and from 0.2150.215 to 0.2100.210, while the corresponding standard deviations increase from 0.0770.077 to 0.1220.122 and from 0.0900.090 to 0.1450.145. These are modest mean improvements relative to the across-seed variability. We have revised Section 5.2.1 (pages 7-8) to explicitly discuss these numerical comparisons and interpret the mean improvements alongside their variability.

Fairness should also be assessed jointly with accuracy. For example, on SPD, CHSW achieves mean accuracy of 0.7950.795, compared with 0.7840.784 for MAVGM_{\mathrm{AVG}}, the second-best method in mean hard DP, while also reducing mean DP. Figure 3 provides complementary trade-off evidence, particularly for soft DP and on SPD. The revised discussion therefore emphasizes the favorable fairness–accuracy balance while acknowledging the modest improvements and higher variability in these cases. All changes are highlighted in red.

Q3: Variability estimates and a constant-predictor reference for Figure 3.

A: Thank you for these helpful suggestions. The original Figure 3 showed single-seed curves without variability estimates. We have revised Figure 3 on page 9 to report mean curves with error bars indicating ±1\pm 1 standard deviation across seeds, and added the suggested dummy baseline—a constant predictor that outputs the mean training label. The revisions are highlighted in red.

Q4: Is there a reason you choose λ=1014\lambda = 10^{14} or expect such values to be useful?

A: Thank you for pointing out this typo. The intended value is λ=104\lambda=10^{4}, not 101410^{14}. We have corrected it in Section 5.4 on page 9 and highlighted the correction in red.

Q5: Notation and symbol definitions.

A: Thank you for this helpful suggestion. We have added a notation table (Table 5, “Main notation used in this paper”) in Appendix B (pages 15–17). In particular, ToMT_o\mathcal{M} denotes the tangent space at the reference point oo, and P(M)\mathcal{P}(\mathcal{M}) denotes the set of Borel probability measures on M\mathcal{M}.

Q6: Applicability to discrete groups, regression, and continuous sensitive attributes.

A: Our regularizer supports finitely many discrete sensitive groups and is independent of the prediction target. It can therefore be combined with a regression loss, although our stated fairness guarantees concern classification scores and decisions.

Continuous sensitive attributes are not directly covered by the current formulation. Extending the approach would require adapting the conditional-distribution estimation and regularization, together with further theoretical analysis. We focus on the widely adopted discrete-group setting and regard continuous attributes as an interesting extension.

report mean curves with error bars indicating standard deviation across seeds

Thank you, visualizing them with a shaded region would be preferable for readability.

Our regularizer supports finitely many discrete sensitive groups and is independent of the prediction target.

Can you make that more clear with a short sentence in the beginning of the paper ? This is otherwise a common point of confusion in the literature thus it would be helpful for the community.

Thank you for pointing out this typo. The intended value is , not . We have corrected it in Section 5.4 on page 9 and highlighted the correction in red.

This is a bit unexpected: your provided code quite clearly states "lambdas=[0,1,10,100,1000,10000,100000,10000000000,100000000000000]", are your results now run with these lambdas or your original values and which values do the paper results correspond to?

Reviewer_sdgsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper studies fair representation learning in non-Euclidean latent spaces. Its central motivation is that aligning demographic groups in ambient Euclidean coordinates may not adequately control their discrepancy under the geometry used by the model. The authors propose a fairness regularizer based on Cartan–Hadamard Sliced-Wasserstein (CHSW) distances. They derive a hard demographic-parity bound for pullback Euclidean manifolds under additional regularity assumptions, and a soft demographic-parity bound on general Hadamard manifolds for a restricted class of score functions. The practical objective uses independently computed one-dimensional barycenters. Experiments on synthetic datasets, Adult, and LFWA evaluate predictive accuracy, demographic-parity gaps, and computational cost.

Critical strengths
  1. The paper makes the choice of geometry explicit in fairness regularization.
    When a model uses a manifold metric to organize its representations, it is reasonable to ask whether group alignment should use the same geometry. The paper turns this observation into a concrete regularization framework rather than leaving it as a conceptual argument.
  2. The proposed objective is computationally practical. The slice-wise construction avoids solving full optimal transport problems on the manifold. The reported runtime comparisons suggest that this approach can be used during training without the cost of a full manifold transport solver.
  3. The analysis addresses the relaxation used in practice. Appendix D explicitly recognizes that independently optimized slice barycenters need not correspond to a single manifold distribution, and provides pairwise discrepancy control for the relaxed population objective. This is an important justification of the implemented approach.
Critical weaknesses
  1. The motivating example does not directly test the claimed weakness of Euclidean distributional alignment. Figures 1 and 4 compare Frobenius differences between covariance parameters with CHSW distances between the resulting distributions. Small differences between covariance parameters do not establish that the distributions are close under the Euclidean MMD, Sinkhorn, or SW objectives used by the baselines. Measuring these discrepancies on the same samples would make the argument more convincing. The paper should also distinguish approximate discrepancy control from exact equality of group distributions: the latter is not invalidated simply by changing the geometry.
  2. The general-manifold guarantee is not established for the classifier used in the experiments.
    Theorem 4.5 requires the score function to admit the sliced representation in Equation (8), with uniformly controlled slice-wise Lipschitz constants. The implementation uses an MLP head instead. Appendix C.5 introduces an approximation error, but does not bound or measure it for the trained models. The result is therefore a conditional guarantee for a particular function class, rather than a verified guarantee for the reported implementation. This matters because the hard-DP result is restricted to the flat pullback Euclidean setting.
  3. The empirical results do not yet support a consistent advantage across fairness criteria.
    In Table 1, the proposed method has a larger hard-DP gap than WB on the hyperbolic dataset, despite a much smaller soft-DP gap. On Adult and LFWA, the hard-DP improvements over SW are modest relative to the reported variability. These observations do not establish that the gains are absent, but they warrant a more qualified interpretation. Matched-accuracy comparisons and a precise description of the validation-based operating-point selection are needed to assess the trade-off fairly.
  4. The contribution of geometry is not sufficiently isolated. For Adult, the method uses a pullback Euclidean geometry, where CHSW reduces to ordinary SW after the coordinate transformation described in Appendix E.3.4. A key control is therefore SW applied to exactly those transformed coordinates. More generally, the comparisons should hold the representation mapping fixed while changing only the discrepancy. The manifold-selection criterion also needs clarification, since the main text describes selecting the best manifold per dataset, while the appendix states that geometry is not tuned using fairness metrics.
Questions for authors
  1. For the distributions in Figures 1 and 4, what are the Euclidean SW, MMD, or Sinkhorn discrepancies computed directly on the samples? Do these measurements support the claimed mismatch?
  2. Can you bound or empirically assess the sliced approximation error η for the MLP heads used in the experiments, together with the corresponding Lslice? Alternatively, does an explicitly sliced classifier retain the reported performance?
  3. How does CHSW compare with Euclidean discrepancies when the representation mapping is held fixed? In particular, how does the Adult result compare with ordinary SW applied to the same pullback coordinates?
  4. What exact validation criterion is used to select the regularization coefficient and the manifold? How many runs underlie the reported results, and do the real-data improvements persist at matched accuracy?
5 comments

We thank the reviewer for the careful review and constructive feedback. We appreciate your recognition of our explicit treatment of geometry in fairness regularization and the computational practicality of the slice-wise objective. We are particularly grateful for your acknowledgment of our analysis of the relaxed barycenter objective in Appendix D, which justifies the implemented approach through pairwise discrepancy control. Below, we address your concerns point by point. All changes in the revised manuscript are highlighted in red.

Q1: Sample-based validation of the motivation and clarification of approximate versus exact alignment.

A: Thank you for the suggestion. We agree that covariance differences alone are insufficient. We computed the Euclidean discrepancies directly on the same samples used in Figure 1, corresponding to the distribution pairs in Figure 5:

PairEuclidean SW$_1$MMD (σ=1\sigma=1)Sinkhorn divergenceCHSW$_1$
A2.85×10−42.85\times10^{-4}2.05×10−42.05\times10^{-4}2.16×10−52.16\times10^{-5}3.283.28
B4.64×10−54.64\times10^{-5}2.98×10−52.98\times10^{-5}1.09×10−91.09\times10^{-9}1.891.89

These results support the motivating example under the reported settings: small Euclidean discrepancies coexist with appreciable CHSW$_1$ separation, although their numerical scales are not directly comparable. We also agree that exact equality of group distributions is independent of the choice of geometry. We have clarified in Section 1 (page 2) that our claim concerns approximate discrepancy control. The quantitative comparisons have been added to Figure 1 (page 2) and Appendix A (page 14).

Q2: Connection to MLP heads through sliced approximation, Lipschitz bounds, and explicit sliced classifiers.

A: Thank you for the suggestion. We added two SPD experiments, reporting mean ±\pm standard deviation over five training seeds.

1. Empirical approximation error and slice-wise Lipschitz bounds. We froze the trained encoders and MLPs used in the supplementary comparison below, and fitted their predicted probabilities using 120 full-SPD directions and slope-constrained piecewise-linear slice functions with 32 knots each.

Lipschitz budgetMAEηtest\eta_{\mathrm{test}}Slice-wise Lipschitz bound
10.100±0.0290.100\pm0.0290.205±0.0640.205\pm0.0641.000±0.0001.000\pm0.000
40.003±0.0030.003\pm0.0030.060±0.0950.060\pm0.0954.000±0.0004.000\pm0.000
160.001±0.0010.001\pm0.0010.026±0.0250.026\pm0.0258.905±4.6168.905\pm4.616

Here, $\eta_{\mathrm{test}} =\max_{z_i\in\mathcal D_{\mathrm{test}}} |h_{\mathrm{MLP}}(z_i)-\widetilde h(z_i)|$. This measures the maximum observed test error, not a certified support-wide bound on η\eta. These results show that the trained MLP probabilities can be closely approximated on held-out samples by sliced functions with explicitly controlled Lipschitz constants, providing empirical support for the sliced approximation discussed in Appendix C.5.

2. Performance of explicit finite-slice classifiers. We kept encoder architectures, data splits, and training budgets identical.

HeadACC ↑\uparrowΔDP↓\Delta_{\mathrm{DP}}\downarrowΔDPsoft↓\Delta_{\mathrm{DP}}^{\mathrm{soft}}\downarrow
MLP0.795±0.0340.795\pm0.0340.210±0.1450.210\pm0.1450.079±0.0680.079\pm0.068
Sliced, budget 10.785±0.0400.785\pm0.0400.324±0.1680.324\pm0.1680.181±0.1110.181\pm0.111
Sliced, budget 40.790±0.0280.790\pm0.0280.271±0.1510.271\pm0.1510.162±0.1320.162\pm0.132
Sliced, budget 160.793±0.0210.793\pm0.0210.225±0.0760.225\pm0.0760.117±0.0610.117\pm0.061

All three sliced heads achieve mean test accuracy within 1.0 percentage point of the MLP head. At budget 16, accuracy differs by only 0.2 percentage points (0.793 versus 0.795), with mean hard-DP and soft-DP gaps of 0.225 and 0.117, compared with 0.210 and 0.079 for the MLP. These results support the empirical feasibility of explicit sliced classification with closely preserved predictive accuracy, while quantifying the remaining fairness differences.

We have included these results in Appendix D.5.4 (pages 25–26), with all changes highlighted in red, while clearly distinguishing the empirical evidence from the conditional population guarantee in Theorem 4.5.

Q3: Fixed-map comparisons, including pullback SW on Adult, and clarification of manifold selection.

A: Thank you for raising this issue. The original baselines share the same MLP encoder architecture and classification head, but not identical manifold maps. Euclidean SW acts directly on z=fθ(x)z=f_\theta(x), whereas CHSW applies manifold mappings and projections within the regularizer. All classifiers take the pre-map zz as input. We added two controls with identical map definitions and network architectures; encoder parameters are trained separately for each method.

1. Adult: Euclidean SW in identical pullback coordinates. With φ(z)=asinh⁡(z)\varphi(z)=\operatorname{asinh}(z), Appendix F.3.4 (formerly Appendix E.3.4) gives $CHSW_p(\mu_s,\mu_t)=SW(\varphi_{\sharp}\mu_s,\varphi_{\sharp}\mu_t) $.

We therefore directly applied ordinary SW to asinh⁡(z)\operatorname{asinh}(z). With matched directions, quantiles, group weights, and normalization, implementation checks yield zero differences in both loss and gradients with respect to zz.

MethodACC ↑\uparrowΔDP\Delta_{\mathrm{DP}} ↓\downarrowΔDPsoft\Delta_{\mathrm{DP}}^{\mathrm{soft}} ↓\downarrow
Original CHSW0.840±0.0070.840\pm0.0070.098±0.0340.098\pm0.0340.082±0.0330.082\pm0.033
SW⁡(asinh⁡(z))\operatorname{SW}(\operatorname{asinh}(z))0.842±0.0040.842\pm0.0040.100±0.0310.100\pm0.0310.087±0.0200.087\pm0.020
SW⁡(z)\operatorname{SW}(z)0.825±0.0400.825\pm0.0400.102±0.0590.102\pm0.0590.085±0.0480.085\pm0.048

The similar performance in identical pullback coordinates agrees with the theoretical equivalence.

2. SPD: identical manifold mapping, different discrepancies. Both methods use X=exp⁡(Sym⁡(z))X=\exp(\operatorname{Sym}(z)), with identical architectures and classifier inputs. Euclidean SW linearly slices the Frobenius-isometric coordinates svec⁡(X)\operatorname{svec}(X), whereas SPD-AIRM CHSW uses manifold projections.

MethodACC ↑\uparrowΔDP\Delta_{\mathrm{DP}} ↓\downarrowΔDPsoft\Delta_{\mathrm{DP}}^{\mathrm{soft}} ↓\downarrow
SW⁡(svec⁡(X))\operatorname{SW}(\operatorname{svec}(X))0.794±0.0280.794\pm0.0280.220±0.1260.220\pm0.1260.112±0.0850.112\pm0.085
SPD-AIRM CHSW0.795±0.0340.795\pm0.0340.210±0.1450.210\pm0.1450.079±0.0680.079\pm0.068

With the SPD mapping fixed and comparable mean accuracy, CHSW achieves lower mean hard and notably lower soft DP, providing additional evidence for the role of the manifold discrepancy in the non-flat setting. We have added these controlled experiments and a discussion of the respective roles of coordinate transformations and manifold-aware discrepancies to Appendix F.5 (pages 34–36).

Manifold selection considers the validation fairness–accuracy trade-off, rather than minimizing fairness gaps alone. We have revised Appendix F.3.1 to clarify that we select among predefined candidate manifolds based on their joint predictive and fairness performance.

Q4: Fairness--accuracy comparisons, variability, and validation-based operating-point selection.

A: Thank you for the comment. We agree that “improves fairness substantially” overstates the hard-DP results in Table 1. We have revised this wording in the abstract and updated Section 5.2.1 (pages 7-8) to discuss the modest mean improvements and across-seed variability, and to emphasize the fairness–accuracy balance.

The key challenge in fair representation learning is to improve fairness while maintaining predictive utility. In Table 1, CHSW achieves modest mean hard-DP improvements together with higher mean accuracy on four datasets. For example, on SPD, MAVGM_{\mathrm{AVG}}, the second-best method in mean hard DP, achieves DP of 0.2150.215 and ACC of 0.7840.784, whereas CHSW achieves DP of 0.2100.210 and ACC of 0.795\mathbf{0.795}. On Hyperbolic, WB achieves a lower mean hard-DP gap (0.1690.169 versus 0.2450.245), whereas CHSW achieves a smaller mean soft-DP gap (0.0050.005 versus 0.0260.026), with comparable mean accuracy (0.5210.521 versus 0.5180.518). We now explicitly discuss this case in Section 5.2.1.

Furthermore, Figure 3 (page 9) compares the trade-offs across regularization strengths and now reports mean curves with error bars indicating ±1\pm1 standard deviation over five random-seed runs. These curves provide complementary evidence of higher accuracy at comparable fairness levels, particularly for soft DP and on SPD. Together, these results support a favorable fairness–accuracy balance, rather than uniformly large improvements in hard DP.

Q5: Clarification of validation criteria, run counts, and matched-accuracy comparisons on real data.

A: Thank you for requesting this clarification. All reported results are means and standard deviations over five random-seed runs. Since the intrinsic geometry of real-world representation spaces is unknown, we evaluate several candidate manifolds and select the best per dataset. This yields pullback Euclidean for Adult and hyperbolic geometry for LFWA, as described in Section 5.1.2 (page 7).

At the reported real-data operating points, CHSW achieves higher mean accuracy and lower or tied mean DP gaps. The key challenge in fair representation learning is balancing accuracy and fairness. For example, on Adult in Table 1, SW, the second-best method in DP, achieves an accuracy of 0.8250.825, whereas CHSW achieves 0.840\mathbf{0.840} together with a lower DP gap. These results support a favorable fairness–accuracy balance. Figure 3 provides complementary trade-off evidence on the synthetic datasets. We have clarified the validation criteria for selecting the regularization coefficient and manifold Section 5 (page 7) and Appendix F.3.1(page 32), and specify the five-seed evaluation protocol in Section 5 (page 7).

Reviewer_ihL5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a fair representation learning technique that measures the discrepancy between group-conditional representation distributions on a curved manifold instead of in Euclidean coordinates. The authors argue that learned representations often lie on a curved space. And two groups can look similar under a Euclidean measure but still be far apart, so a model could tell them apart. Instead, they use a Hadamard manifold, such as hyperbolic space or the space of SPD matrices, and add a fairness penalty based on the Cartan-Hadamard sliced Wasserstein (CHSW) distance. The author show that this penalty bounds the demographic parity gap on flat manifolds. Experiments on three synthetic and two real datasets shows the effectiveness of the approach.

Based on the rebuttal from the authors I have increased my score.

Critical strengths
  1. The paper measures the fairness penalty in the geometry of the latent space when that space is curved instead of using the Euclidian approach.
  2. CHSW compares the representation of sensitive groups along random one-dimensional directions, so it costs about the same as Euclidean sliced Wasserstein.
  3. This paper proposed a fairness regularizer as penalty that pulls all groups toward one shared barycenter and it is defined for any number of groups.
  4. It also connects the penalty to fairness with proofs, e.g. Theorem 4.4 bounds the DP gap by the penalty on pullback Euclidean manifolds.
  5. Experiments show CHSW is about 2 to 20 times faster than the Sinkhorn baseline on every dataset.
Critical weaknesses
  1. CHSW, its properties are already existing knowledge [1], and the other proofs use standard triangle-inequality, smoothing techniques. So the contribution is incremental. Also, in the abstract, it’s mentioned that CHSW is differentiable, however the paper never discusses this.
  2. In table 1, CHSW improves DP over the best baseline by at most 0.005 on four datasets, which is smaller than its own standard deviations (0.12 to 0.14). And it’s worse than WB on Hyperbolic dataset. So the claim that the method "improves fairness substantially" is not supported.
  3. The presented method and experiments only cover binary classification with one binary sensitive attribute. Multiple sensitive attributes and their intersections are not discussed.
  4. It is not clear whether the gains come from the geometry. On the pullback manifolds, Appendix E.3.4 states that CHSW reduces to Euclidean sliced Wasserstein after a fixed map ϕ\phi. Also the paper does not clearly say whether the baselines use the same manifold encoder.
  5. In the experiment, only two real datasets has been used and the Adult dataset has some known issues [2]. Only demographic parity (DP) has been used as an evaluation metric, with no equalised odds.

[1] Bonet, Clément, Lucas Drumetz, and Nicolas Courty. "Sliced-Wasserstein distances and flows on Cartan-Hadamard manifolds." Journal of Machine Learning Research 26.32 (2025): 1-76. [2] Ding, Frances, et al. "Retiring adult: New datasets for fair machine learning." Advances in neural information processing systems 34 (2021): 6478-6490.

Questions for authors
  1. In figure 1, what are the Euclidean SW, MMD and Sinkhorn values between samples of each pair, and can you show where these are small but CHSW is large?
  2. Do the baselines use the same manifold valued encoder as CHSW? What happens if that encoder is trained with Euclidean SW instead?
  3. With several sensitive attributes, how would the group distributions look on your manifold?
3 comments

We thank the reviewer for the careful review and constructive feedback. We appreciate your recognition of our geometry-aware fairness formulation, CHSW’s efficiency, the barycenter regularizer’s applicability to multiple sensitive groups, the theoretical connection to demographic parity on pullback Euclidean manifolds, and the substantial speedups over Sinkhorn in our experiments. Below, we address your concerns point by point. All manuscript changes are highlighted in red.

Q1: Additional Euclidean discrepancy measurements for the motivating example.

A: Thank you for the suggestion. We computed the Euclidean discrepancies between the same samples used in Figure 1:

PairEuclidean SW$_1$MMD (σ=1\sigma=1)Sinkhorn divergenceCHSW$_1$
A2.85×10−42.85\times10^{-4}2.05×10−42.05\times10^{-4}2.16×10−52.16\times10^{-5}3.283.28
B4.64×10−54.64\times10^{-5}2.98×10−52.98\times10^{-5}1.09×10−91.09\times10^{-9}1.891.89

Pair B provides a particularly clear example: all three Euclidean discrepancies are small, whereas CHSW$_1$ remains 1.891.89. These results illustrate that near alignment in ambient Euclidean coordinates can coexist with substantial intrinsic separation on the manifold. We have added these quantitative comparisons to Figure 1 (page 2) and Appendix A (page 14).

Q2: Role of geometry under comparable encoder and mapping settings.

A: Thank you for raising this issue. The original baselines share the same MLP encoder architecture and classification head, but not identical manifold maps. Euclidean SW acts directly on z=fθ(x)z=f_\theta(x), whereas CHSW applies manifold mappings and projections within the regularizer. All classifiers take the pre-map zz as input. We added two controls with identical map definitions and network architectures; encoder parameters are trained separately for each method.

1. Adult: Euclidean SW in identical pullback coordinates. With φ(z)=asinh⁡(z)\varphi(z)=\operatorname{asinh}(z), Appendix F.3.4 gives $CHSW_p(\mu_s,\mu_t)=SW(\varphi_{\sharp}\mu_s,\varphi_{\sharp}\mu_t) $.

We therefore directly applied ordinary SW to asinh⁡(z)\operatorname{asinh}(z). With matched directions, quantiles, group weights, and normalization, implementation checks yield zero differences in both loss and gradients with respect to zz.

MethodACC ↑\uparrowΔDP\Delta_{\mathrm{DP}} ↓\downarrowΔDPsoft\Delta_{\mathrm{DP}}^{\mathrm{soft}} ↓\downarrow
Original CHSW0.840±0.0070.840\pm0.0070.098±0.0340.098\pm0.0340.082±0.0330.082\pm0.033
SW⁡(asinh⁡(z))\operatorname{SW}(\operatorname{asinh}(z))0.842±0.0040.842\pm0.0040.100±0.0310.100\pm0.0310.087±0.0200.087\pm0.020
SW⁡(z)\operatorname{SW}(z)0.825±0.0400.825\pm0.0400.102±0.0590.102\pm0.0590.085±0.0480.085\pm0.048

The similar performance in identical pullback coordinates agrees with the theoretical equivalence. This is expected. On a pullback Euclidean manifold, the geometry is fully encoded by the map ϕ\phi, and CHSW specifies the coordinates in which alignment should be performed. Without this geometric choice, ordinary SW on zz yields lower and more variable accuracy (0.825 ± 0.040 vs. 0.840 ± 0.007).

2. SPD: identical manifold mapping, different discrepancies. Both methods use X=exp⁡(Sym⁡(z))X=\exp(\operatorname{Sym}(z)), with identical architectures and classifier inputs. Euclidean SW linearly slices the Frobenius-isometric coordinates svec⁡(X)\operatorname{svec}(X), whereas SPD-AIRM CHSW uses manifold projections.

MethodACC ↑\uparrowΔDP\Delta_{\mathrm{DP}} ↓\downarrowΔDPsoft\Delta_{\mathrm{DP}}^{\mathrm{soft}} ↓\downarrow
SW⁡(svec⁡(X))\operatorname{SW}(\operatorname{svec}(X))0.794±0.0280.794\pm0.0280.220±0.1260.220\pm0.1260.112±0.0850.112\pm0.085
SPD-AIRM CHSW0.795±0.0340.795\pm0.0340.210±0.1450.210\pm0.1450.079±0.0680.079\pm0.068

With the SPD mapping fixed and comparable mean accuracy, CHSW achieves lower mean hard and notably lower soft DP, providing additional evidence for the role of the manifold discrepancy in the non-flat setting. We have added these controlled experiments and a discussion of the respective roles of coordinate transformations and manifold-aware discrepancies to Appendix F.5 (pages 34–36).

Q3: Extension to multiple sensitive attributes and intersectional group distributions.

A: Thank you for these related questions. Our theoretical results apply to AA sensitive groups under the stated assumptions. Sections 3.1 (page 3) and 4.1 (page 4) already support AA sensitive groups. Multiple attributes can be encoded jointly as intersectional groups, whose conditional distributions are aligned on a shared representation manifold. This is independent of whether prediction labels are binary.

We added an Adult experiment with sex and White/Non-White race, yielding four groups [2]. Across multiple seeds, CHSW has the highest mean ACC and lowest mean intersectional hard/soft DP:

MethodACC ↑\uparrowΔDP\Delta_{\mathrm{DP}} ↓\downarrowΔDPsoft\Delta_{\mathrm{DP}}^{\mathrm{soft}} ↓\downarrow
CHSW0.846 ±\pm 0.0020.142 ±\pm 0.0060.121 ±\pm 0.008
SW0.845±0.0000.845 \pm 0.0000.150±0.0110.150 \pm 0.0110.125±0.0070.125 \pm 0.007
MSNKM_{\mathrm{SNK}}0.845±0.0010.845 \pm 0.0010.171±0.0110.171 \pm 0.0110.146±0.0110.146 \pm 0.011
MAVGM_{\mathrm{AVG}}0.843±0.0010.843 \pm 0.0010.157±0.0040.157 \pm 0.0040.142±0.0030.142 \pm 0.003
MMMDM_{\mathrm{MMD}}0.845±0.0010.845 \pm 0.0010.156±0.0130.156 \pm 0.0130.147±0.0120.147 \pm 0.012
WB0.843±0.0010.843 \pm 0.0010.145±0.0060.145 \pm 0.0060.128±0.0030.128 \pm 0.003

We visualize test distributions in ϕ(Z)=asinh⁡(Z)\phi(Z)=\operatorname{asinh}(Z) using a common PCA basis fitted to pooled training representations (Figure 6 in Appendix F.4). Colors identify the four groups; normalized contours enclose approximately 50%50\%, 80%80\%, and 95%95\% probability mass. The panels compare training without the fairness penalty and with CHSW, using separate axis ranges for detail and shared within-panel bandwidths. Under CHSW, the principal density regions and contours overlap more closely, suggesting improved alignment of the four intersectional group distributions on the shared manifold.

Beyond this visualization, we evaluate all six pairwise empirical CHSW22\mathrm{CHSW}_2^2 discrepancies in the full 16-dimensional space, normalized by mean coordinate variance to remove uniform global rescaling (Figure 7 in Appendix F.4). Compared with the model trained without the fairness penalty, CHSW reduces the mean scale-normalized pairwise discrepancy from 0.2704 to 0.0487, an 82.0% reduction, indicating closer alignment of the intersectional group distributions even after accounting for uniform global rescaling. Together with the multi-seed results above, these observations illustrate the applicability of CHSW to multiple sensitive attributes and their intersections.

We have added the paragraph “Multiple sensitive attributes performance.” to Section 5.2.1 (pages 7-8), together with the corresponding experimental details and Figures 6 and 7 in Appendix F.4 (pages 34–35).

[2] Meinson C et al. FairST: A novel approach for machine learning bias repair through latent sensitive attribute translation. Information and Software Technology, 2025.

Q4: Clarification of the contribution and differentiability.

A: Thank you for the feedback. We agree that CHSW and its basic properties originate from Bonet et al. [1], and will clarify this attribution throughout the paper. Our contribution is to use CHSW for group-distribution alignment on non-Euclidean manifolds, addressing the mismatch between Euclidean fairness discrepancies and representation geometry. We develop a barycenter regularizer and connect it to hard DP on pullback Euclidean manifolds (Theorem 4.4) and soft DP on general Hadamard manifolds (Theorem 4.5), under their respective explicit assumptions. Appendix D.2 further shows that the tractable slice-wise relaxation preserves the pairwise CHSW control required by these guarantees. Although the proofs use standard tools, these fairness-specific connections constitute our intended theoretical contribution.

Regarding differentiability, we agree that the original explanation was insufficient. For p=2p=2, the empirical regularizer combines manifold-aware projections, sorting, linear quantile interpolation, and squared deviations. Under smooth manifold maps and projections, it is almost everywhere differentiable and supports backpropagation. We have added an explanation of this computation to Section 4.4 (page 6) and Appendix E.2.1 (page 28), and revised the claim in the abstract (page 1) accordingly.

[1] Bonet C et al. Sliced-Wasserstein distances and flows on Cartan-Hadamard manifolds. JMLR, 2025.

Q5: Interpretation of fairness improvements and variability across datasets.

A: Thank you for the comment. We agree that “improves fairness substantially” overstates the hard-DP results in Table 1. We have revised this wording in the abstract and updated Section 5.2.1 (pages 7-8) to discuss the modest mean improvements and across-seed variability, and to emphasize the fairness–accuracy balance.

The key challenge in fair representation learning is to improve fairness while maintaining predictive utility. In Table 1, CHSW achieves modest mean hard-DP improvements together with higher mean accuracy on four datasets. For example, on SPD, MAVGM_{\mathrm{AVG}}, the second-best method in mean hard DP, achieves DP of 0.2150.215 and ACC of 0.7840.784, whereas CHSW achieves DP of 0.2100.210 and ACC of 0.795\mathbf{0.795}. On Hyperbolic, WB achieves a lower mean hard-DP gap (0.1690.169 versus 0.2450.245), whereas CHSW achieves a smaller mean soft-DP gap (0.0050.005 versus 0.0260.026), with comparable mean accuracy (0.5210.521 versus 0.5180.518). We now explicitly discuss this case in Section 5.2.1.

Furthermore, Figure 3 (page 9) compares the trade-offs across regularization strengths and now reports mean curves with error bars indicating ±1\pm1 standard deviation over five random-seed runs. These curves provide complementary evidence of higher accuracy at comparable fairness levels, particularly for soft DP and on SPD. Together, these results support a favorable fairness–accuracy balance, rather than uniformly large improvements in hard DP.

Q6: Dataset coverage, the use of Adult, and evaluation beyond DP.

A: Thank you for the comment. We acknowledge Adult’s known limitations, but it remains an established benchmark used in recent fairness research, including ICLR 2026 work [3]. We selected Adult and LFWA to facilitate comparison with existing methods on tabular income prediction and visual attribute classification, respectively.

Our focus on DP follows our theoretical analysis of distributional demographic parity. The contribution of this work is geometry-aware distribution alignment, with DP serving as an established evaluation criterion. The same construction can target equalized odds by aligning P(Z∣S=s,Y=y)P(Z\mid S=s,Y=y) across groups separately for each label yy, under analogous assumptions.

To broaden the evaluation, we additionally measured EO on LFWA. CHSW achieves the lowest mean EO gap among all compared methods, while retaining the highest mean accuracy:

MetricMAVGM_{\mathrm{AVG}}MMMDM_{\mathrm{MMD}}MSNKM_{\mathrm{SNK}}WBSWCHSW
ACC ↑\uparrow0.900±0.0050.900\pm0.0050.902±0.0040.902\pm0.0040.894±0.0090.894\pm0.0090.902±0.0050.902\pm0.0050.899±0.0100.899\pm0.0100.905$\pm$0.004
ΔDP\Delta_{\mathrm{DP}} ↓\downarrow0.056±0.0130.056\pm0.0130.056±0.0080.056\pm0.0080.056±0.0130.056\pm0.0130.058±0.0110.058\pm0.0110.051±0.0150.051\pm0.0150.048$\pm$0.012
ΔDPsoft\Delta_{\mathrm{DP}}^{\mathrm{soft}} ↓\downarrow0.051±0.0130.051\pm0.0130.049±0.0070.049\pm0.0070.050±0.0150.050\pm0.0150.046±0.0070.046\pm0.0070.048±0.0120.048\pm0.0120.046$\pm$0.010
EO gap ↓\downarrow0.078±0.0180.078\pm0.0180.060±0.0140.060\pm0.0140.053±0.0140.053\pm0.0140.081±0.0180.081\pm0.0180.054±0.0140.054\pm0.0140.044$\pm$0.027

These additional results demonstrate favorable performance beyond the DP metrics used in our main evaluation. We have added these results to Appendix F.8 on pages 38-39 of the revised manuscript.

[3] Liu Y et al. Fairness via Independence: A General Regularization Framework for Machine Learning. ICLR, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.