acceptodds
Withdrawn from ICLR 2027

Tracing Group Fairness through Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

When a trained model performs much worse on some demographic groups than others, a natural question is: which training examples contribute to the disparity? Fairness-aware learning typically aims to reduce such gaps without attributing them to individual training examples, while standard data attribution methods focus on predictive performance rather than fairness. We propose FAIR-DS, which assigns each example a score for its accumulated contribution to a chosen group-aware objective in a single training run. Building on In-Run Data Shapley, the method uses group validation statistics to identify harmful examples under popular fairness criteria. Our analysis bounds the attribution error and connects the computed scores to changes in the selected fairness criteria. These guarantees clarify how individual examples contribute to fairness changes along the training trajectory. Empirical studies on label-bias detection and attribution-guided data repair demonstrate competitive performance on real-world tasks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Fairness, Trustworthy Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8e2kOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

FAIR-DS extends in-run data attribution to group-aware validation objectives. It accumulates each training example’s gradient contribution to a selected fairness objective, provides bounds for aggregate attribution, and tests detection and repair of synthetically biased labels on three datasets.

Critical strengths

The method targets a specified fairness objective while reusing the realized training trajectory, avoiding repeated subset retraining for attribution.

Comparisons across datasets and inspection budgets show strong top-ranked detection; the analysis distinguishes smooth fairness surrogates from hard-prediction parity.

Critical weaknesses

The theoretical bound concerns aggregate attribution, not the accuracy or causal effect of each individual score; good aggregate fit could coexist with misleading top-ranked examples.

The repair protocol restores labels only for selected samples known to be synthetically flipped. This oracle-assisted correction does not measure the cost or errors of real human inspection.

The experiments rely on injected label bias and a clean group-labeled validation set. Sensitivity to naturally occurring errors, noisy protected attributes, and small subgroups remains unclear.

Questions for authors

At a fixed annotation budget, how does the method perform when an annotator must review every selected example, including uncorrupted ones?

Do top-ranked examples cause the predicted fairness change under actual leave-one-out or targeted retraining on a tractable subset?

How do noisy validation labels and rare protected groups change the rankings and repair outcome?

Reviewer_7XxPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies which training examples cause group disparities in a trained model. The authors propose FAIR-DS, extending In-Run Data Shapley from the validation loss to a group-aware fairness objective. The score of each example is the step size times the inner product of its gradient with the objective's gradient, summed over training. Since hard fairness criteria have zero gradients, the authors replace them with smooth versions: a log-sum-exp of group losses for worst-group risk, and variances of mean predicted probabilities for disparity criteria. They bound the gap between the summed scores and the change in the original criteria. Experiments were conducted on three datasets, and show that the choice of objective changes the outcome of label repair.

Critical strengths
  1. The paper is well presented and easy to follow. The decomposition in Eq. (6) makes it clear why the same example gets different scores under different criteria, and the theory states its limits clearly.

  2. The repair study in Table 3 is helpful to show that the choice of objective matters, not just the number of corrected labels.

Critical weaknesses
  1. τ=0.01\tau = 0.01 is used for results in Table 2. With this τ\tau value, the group weights qgq_g are almost uniform, so the score is close to In-Run Shapley with a group-balanced validation loss. The ablation study in Figure 2 shows that as τ\tau increases, the detection AUROC decreases. So the more the method focuses on the worst group, the worse it does. This suggests the gains come from group balancing, not worst-group focus. Also, in Theorem 5.2, even with G=2, the smoothing term log⁡G/τ\log G / \tau is over 30 (log 2 / 0.01) with this τ\tau value, so the theory gives no guarantee for the reported runs.

  2. The gains over In-Run Shapley are small on two out of three datasets: Adult and UTKFace.

  3. The connection between the scores and fairness after retraining is validated in a narrow way. Removal is tested only on synthetic data, and label repair only on FairFace (Tables 3 and 6). Could you show on real data that the summed scores predict the actual change in a fairness metric after retraining? This is the main use case described in the introduction.

  4. Some experimental setup details are missing: the real-data experiments use one noise rate (ρ=0.4\rho = 0.4), and runtime or memory costs are not reported. Also, it's not clear which race groups correspond to s=1s = 1 and s=2s = 2.

Questions for authors
  1. For the Table 2 setting, what are the group weights qgq_g during training? Could you show detection results at a τ\tau value where the weights are clearly non-uniform?

  2. With τ=0.01\tau = 0.01, what does Theorem 5.2 guarantee for the reported runs? Is there a τ\tau value where both the bound and detection are useful?

  3. On one real dataset, could you remove or correct the top-K examples for several K, retrain, and plot the summed scores against the actual change in the hard fairness metric?

  4. Could you clarify the experimental details in W4 above?

Reviewer_yXs8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Fair-DS, that investigates which training examples make a model unfair to some demographic groups. The author states that fairness methods reduce group gaps without explaining why, and that current fairness data valuation needs retraining and can be unreliable for deep networks. Based on In-Run Data Shapely, Fair-DS scores each training example in one training run by how much it raises a fairness objective on a clean validation set. Experiments on three datasets (one tabular, two images) and one synthetic dataset compare the performance of Fair-DS against several baselines over multiple seeds.

Critical strengths
  1. The paper investigates which training examples cause a group disparity, Fairness methods usually reduce the gap without explaining, and a answer to this question can help with data inspection, removal, and relabeling.
  2. Fair-DS needs only one training run, while retraining-based Shapley needs many retraining.
  3. The paper separates the smooth objective from the original fairness criteria and bounds the gap between them.
  4. The experiments includes three real datasets (both tabular and image) and ten baselines, including fairness-oriented, performance-oriented, and report the mean and standard deviation over 10 seeds.
Critical weaknesses
  1. The approach is incremental. Its a small step beyond In-Run Data Shapely. Section 4.1 states that after linearization, "each example's Shapely value is its corresponding term in eq (4)". So, definition 4.1 is TracIn [1] computed at every step, with the validation loss replaced by a group objective.
  2. All real data results use one symmetric label-flip pattern (ρ\rho = 0.4), and detection counts every flipped label as harmful, even though in the introduction it is stated "a biased label does not necessarily harm every fairness criterion". So detection AUROC measures how well the method finds label noise, not how well it finds harm to fairness.
  3. On Adult, Fair-DS ties with In-Run Shapely in AUROC, and the score of AP and Precision@100 gains are within one standard deviation. On UTKFace, its AUROC is below KNN-Shapely. So, Fair-DS is not getting the best performance on every evaluation.
  4. Each experiment uses one sensitive attribute (gender on Adult, race on the face datasets), and the paper never tests multiple attributes together or their intersections.
  5. The DP objective does not use any validation labels. So when the true base rate differ across groups, Fair-DS can score a correctly labelled example as harmful just because it pushes the group's positive rate apart.

References

[1] Pruthi, Garima, et al. "Estimating training data influence by tracing gradient descent." Advances in neural information processing systems 33 (2020): 19920-19930.

Questions for authors
  1. How does Fair-DS compare with In-Run Shapley using a group-balanced validation loss, given that the LSE weights are almost uniform at τ\tau = 0.01? Also why did you choose τ\tau = 0.01?
  2. How are the race groups in UTKFace and FairFace defined, how many are there, and how is the two-group flip rule applied to them?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.