acceptodds
Under review as a conference paper at ICLR 2027

Robust Mixture Models for Algorithmic Fairness Under Latent Heterogeneity

Abstract

Machine learning models optimized for average performance can perform poorly on vulnerable subpopulations. Existing group-robust approaches often rely on groups specified in advance, yet fairness-relevant subgroup structure may be latent, intersectional, and driven by complex interactions among continuous and discrete attributes. We introduce ROME (RObust Mixture Ensemble), a framework that learns latent group structure while optimizing worst-group predictive performance. ROME connects latent-variable modeling with distributionally robust optimization (DRO) through two complementary approaches: an Expectation-Maximization formulation with robust aggregation for linear models and a neural Mixture-of-Experts formulation for nonlinear settings. Across simulations and three real-world regression datasets, ROME improves worst-group performance while maintaining competitive overall accuracy, including in comparisons with established group-aware and group-label-free robust learning methods. ROME provides a flexible approach to robust prediction when fairness-relevant attributes are available for subgroup discovery but their direct use in group-specific outcome models is restricted.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Algorithmic fairness, group robustness, distributionally robust optimization, Expectation-Maximization, mixture of experts, fair regression

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_cC4nOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The contribution is to interpret the multiple sources in \citet{wang2023} as latent variables.

Critical strengths

None

Critical weaknesses

First, the closed-form solution in Equation (4) is not verified. The reward function in \citet{wang2023} is defined as RT(f):=E(X,Y)∼T[Y2−(Y−f(X))2].\mathbf{R}_{\mathbf{T}}(f):=\mathbb{E}_{(X, Y) \sim \mathbf{T}}\left[Y^2-(Y-f(X))^2\right]. It is helpful to prove Equation (4) because the it is different from Equation (4).

Second, the paper seems to interpret the sources in \citet{wang2023} as latent variables.

Questions for authors

Could you propose a more substantial new framework for statistical fairness, or verify that your framework is distinctly different from \citet{wang2023}?

Reviewer_QbNKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROME (RObust Mixture Ensemble), a framework for improving worst-group predictive performance when fairness-relevant subgroups are unknown and potentially determined by complex interactions among sensitive attributes. ROME connects latent-variable modeling with distributionally robust optimization (DRO) through an Expectation-Maximization formulation with robust aggregation for linear models and a neural Mixture-of-Experts formulation for nonlinear settings.

Critical strengths
  1. The paper addresses a genuine limitation of existing fairness methods: predefined demographic groups may not capture complex or intersectional heterogeneity. Learning latent group structure while improving worst-group predictive accuracy is a worthwhile research direction.
Critical weaknesses
  1. The mathematical connection between ROME-EM's DRO formulation and the stated fairness objective is insufficiently justified. I think more explanation is needed for equation (3) - (5).
  2. Authors mention in lines 261-263: 'The key architectural restriction in ROME-MoE is that S can influence group assignment through the gating network but is not provided directly to the expert prediction functions, which receive only A.' It is unclear what fairness property the proposed architectural restriction guarantees.
Questions for authors

See weaknesses.

Reviewer_GZkqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROME, a framework for regression under latent subgroup heterogeneity. ROME-EM fits a mixture of linear regressions with membership probabilities informed by fairness-relevant attributes, then aggregates the fitted component predictors using an existing distributionally robust aggregation formulation. ROME-MoE uses a neural gating network and experts, combining average prediction loss with a maximum over learned soft-group losses. Sensitive attributes may enter the gate, while the neural experts receive only non-sensitive features. Validation includes a mixture-regression simulation and experiments on three real-world regression datasets, with performance audited using externally defined subgroups.

Critical strengths
  • The problem setting is practically relevant. Predictive disparities can arise from unknown interactions among continuous and categorical attributes, making a predefined demographic partition insufficient. Studying this problem for regression broadens the focus beyond the classification settings common in subgroup robustness research.

  • The evaluation distinguishes learned groups from external auditing groups. Assessing performance on external subgroup definitions is more informative than evaluating only the partitions produced by the model itself. Multiple auditing schemes and repeated runs are useful elements of the experimental design.

Critical weaknesses
  1. The ROME-MoE objective does not adequately represent worst-group conditional risk and admits degenerate grouping behavior. Algorithm 1 defines each group loss as the weighted sum of prediction errors divided by the number of selected observations, rather than their total membership weight. Consequently, the loss changes with membership confidence even when predictions remain unchanged. For the reported choices G∈2,3,4G \in {2, 3, 4}, uniform assignments give IjI_j equal to the entire batch and Lj=Lavg/GL_j = L_{avg}/G for every group. The purported worst-group term then becomes a scaled average loss. Moreover, the gate is trained to minimize this same objective, without a separate identification objective or constraint preventing groups from becoming uninformative. Thus, the model can reduce its robustness surrogate by changing the partition rather than improving performance on a vulnerable subgroup. Correcting the normalization alone would not resolve the broader issue of jointly minimizing over the subgroup definition.

  2. The empirical evidence does not sufficiently isolate a benefit from robust subgroup learning. The principal “fair” baselines use A-only prediction functions, whereas ROME uses sensitive attributes at inference through its gate. Excluding S from individual experts does not make the composite predictor independent of S, so these comparisons mix the proposed robustness mechanism with access to additional predictive information. The paper includes unrestricted MLP and MoE baselines and an α-ablation, which are helpful, but its strong significance claims primarily concern the A-only baselines. Furthermore, the ablation interpretation is overstated: Figure 2 shows generally increasing worst-group MSE at larger α on Law School, and Figure 8 does not support the caption’s claimed improvement from α\alpha = 0 to α\alpha = 1. These results do not currently establish that the proposed worst-group term, rather than architecture or sensitive-feature access, explains the reported gains.

Questions for authors
  1. Does the implementation use the normalization shown in Algorithm 1? What prevents the gate from reducing the robustness objective through uniform or otherwise uninformative assignments? Please distinguish control of group membership weights from improvements in actual conditional prediction risk.

  2. Under the same permitted inference-time use of S as ROME, how do input-matched robust baselines perform? How much improvement remains relative to the identical ROME architecture trained with α\alpha = 0, with matched model selection and uncertainty estimates?

  3. Can you reconcile the claim that increasing α generally improves worst-group performance with Figures 2 and 8? Which externally defined subgroup scheme is used in each ablation plot, and how does it relate to the corresponding main-table evaluation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.