acceptodds
Under review as a conference paper at ICLR 2027

SEWISE: Signed Error-Weighted Informed Subgroup Estimation

Abstract

Many machine learning classifiers can achieve good average accuracy but sometimes fail systematically on subgroups of their inputs. Subgroup discovery methods (SDMs) surface such underperforming groups by using knowledge about which samples were misclassified. As common SDMs use a binary indicator to represent an error, when a model leaves one subgroup under-detected and its complement over-detected, the two error types cancel. For example, on CheXpert, where a pleural effusion classifier’s errors track patient sex, the two groups are far apart in signed error rate but nearly indistinguishable once direction is dropped. Additionally, the discovered errors are only part of a subgroup and therefore SDMs might miss the correctly predicted inputs. We present SEWISE, which represents each mistake as a signed error score, which is positive for false negatives and negative for false positives and is scaled by the model’s confidence, and diffuses it over a nearest-neighbour graph constructed from the classifier’s features to recover additional subgroup examples from their failing neighbours. Across ISIC-2017, CheXpert, CelebA and Waterbirds, SEWISE outperforms SPOTLIGHT, DOMINO and GEORGE on the majority of tasks. We further show that error-guided discovery may not be suitable when the classifier already has a good performance, as there is little error signal to utilise.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Subgroup Discovery, Failure Subgroups, Model Bias, Error-Guided Learning, Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XN16OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The work proposes a new subgroup discovery method based on signed error score. The authors tested the method across a bunch of benchmarks. The proposed method outperforms some baseline subgroup discovery methods across a number of baselines.

Critical strengths

The subgroup discovery problem, where the subgroup corresponds to the erroneous group, is an interesting one. The proposed idea of subgroup discovery based on signed error and neighborhood graph construction is quite interesting.

Critical weaknesses

The main weakness of the work is the experiment set ups. The benchmarks are primarily synthetically induced shortcuts, where the proposed method fails to outperform at some benchmarks and the ablation studies are inadequate. The variance of their performance is quite high.

Questions for authors
  1. The error direction advantage is not clear. The advantage is mostly discussed in experiments in section 5.1 but how it helps the ranking process (intuitively/ mathematically) in Eq (3) is not clear. Moreover, no controlled ablation study is shown with unsigned error.

  2. The authors should appropriately define the subgroup. Does the subgroup correspond to any semantically meaningful attribute, such as race or sex? Do the evaluations assume only 1 target subgroup? How does the method recognize the target group(s) if there are multiple of such failing subgroups or subgroups at the insertion of two or more attributes.

  3. The authors should explain the high standard deviations across the folds, when they claim no run-to-run variance. The authors should also explain the performance drop in celebA.

  4. The advantages of the proposed methods are shown primarily with the synthetically induced shortcuts in ISIC datasets. “High classifier accuracy may limit the information available for error-guided subgroup discovery.” Further realistic benchmarks, choice of model architectures, or controlling errors through controlled model training are required to prove the claim and the generalizability of the method.

  5. No systematic ablation studies are shown varying T, the with/ without PCA, varying dimensions of PCA. Local Linear Embedding (LLE) seems to be a suitable alternative dimensionality reduction method for the framework.

  6. Why only pleural effusion on the CheXpert dataset? There are other labels within the dataset.

  7. How does the method extend to a multi-class classification problem?

  8. The rationale behind the confidence formula is not quite clear. Why not the direct probability from model output pip_i is used?

  9. The authors should report the number of test samples.

3 comments

We thank the reviewer for their detailed feedback. We have uploaded the revised manuscript (with additions in blue and new tables in grey). Below, we provide point by point responses to the reviewer concerns:

  • W1 [Benchmark diversity and performance]: The experiments in our submission include synthetic shortcuts in ISIC and Waterbirds as these are commonly studied benchmarks for subgroup discovery and bias related works, alongside the real demographic attribute-related biases in CheXpert and CelebA. In the revised version, we additionally provide results for an edema classifier on CheXpert and a multiclass classifier on Spawrious.
  • Q1 [Error ablation]: We thank the reviewer for this question. In the original submission, we provided a controlled signed-versus-unsigned comparison in Section 5.1 (Figure 2), keeping the graph, confidence weights and diffusion depth fixed. In the revised Section 3.2.2 (Equation 5), we further explain how positive and negative error contributions propagate through the graph and may reinforce or cancel each other.
  • Q2 [Definitions]: In the original submission, we evaluated the retrieval of one predefined target subgroup at a time, including subgroups defined by semantic attributes such as patient sex or background. In the revised Section 3.1, we clarify that a target subgroup may span multiple task classes. In our approach, we assign an error-based score to each sample and compute a single ranking instead of explicitly separating multiple or intersecting failing subgroups.
  • Q3 [Deviation and CelebA]: The diffusion process is deterministic for fixed inputs and parameters. The standard deviations reflect variations across data splits and classifier training, which can affect predictions, embeddings and retrieval performance. For CelebA, the classifier achieves approximately 98% accuracy, resulting in relatively few errors for diffusion. We discuss in Section 6 how the distribution of these errors and the embedding space may also influence retrieval performance. The additional ablation in Section 5.4 shows that using the full prediction residual improves CelebA retrieval with CLIP features.
  • Q4 [Generalisation]: In the original submission, we evaluated our approach on four datasets, including CheXpert, CelebA and Waterbirds, with improvements on CheXpert and Waterbirds using both embedding spaces. In the revised manuscript, we additionally evaluate an Edema classifier on CheXpert and a multiclass classifier on Spawrious to examine generalisation across different tasks. We also provide a full-residual ablation in Section 5.4, which shows that the choice of error representation influences retrieval performance on CelebA. In Section 6, we further discuss how the number and distribution of classifier errors and the embedding space may contribute to the lower performance on CelebA.
  • Q5 [PCA and diffusion-depth ablations]: We thank the reviewer for this question and in the revised version of the paper, we have included the PCA ablation in Table 4, covering 64, 128 and 256 components and the full-dimensional representation without PCA reduction, where we show that the performance is stable across different dimensions. We also provide a diffusion-depth ablation in Table 7, where we observe that the preferred depth varies across datasets and representations. The validation-selected depths result in a test AUC within 1.8 percentage points of the highest observed values.
  • Q6 [CheXpert pleural effusion]: We chose the pleural effusion task as it is clinically meaningful and standard for a CheXpert analysis. In the revised version, we additionally provide results using an Edema classifier on CheXpert in Appendix A.2 where we observe higher AUC of the signed score vs the unsigned score.
  • Q7 [Multiclass extension]: In the revised version, we have added a multiclass extension in Section 3.2.1, Equation (2) where we define a confidence-weighted error vector, which is diffused through the graph to obtain a single ranking based on its magnitude. We also evaluate a multiclass classifier on the Spawrious dataset and provide the corresponding subgroup-retrieval results in Table 8.
  • Q8 [Confidence weighting]: We use max⁡(pi,1−pi)\max(p_i,1-p_i) because pip_i represents the positive-class probability, whereas our formulation requires the confidence assigned to the predicted class. We have clarified the same in Section 3.2.1. In the revised manuscript, we also include an ablation comparing the confidence-weighted score with the unweighted signed error and probability-based scores (Section 5.4, Table 3). The results show that confidence weighting generally provides a small positive improvement in retrieval AUC, although the differences are modest.
  • Q9 [Test sample sizes]: We have provided the test set details in Appendix A.2 and have also updated the details in the revised version to mention further details about the shortcuts in ISIC and data for CheXpert.
AuthorsOpenReview ↗

Follow-Up to Our Rebuttal

We thank the reviewer for their time and helpful feedback. We have addressed your concerns and revised the manuscript accordingly. We welcome further discussion and would be grateful if you would take our revisions into account when reconsidering your score and overall assessment.

I feel that the authors addressed most of my concerns. I have increased my score.

Reviewer_VVNMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a method for discovering underperforming subgroups without bias annotations by propagating signed, confidence-weighted error scores through a nearest-neighbor graph. Experiments show improved subgroup retrieval over three baselines in most settings. However, the work remains incomplete, as its effectiveness in mitigating bias has not been evaluated.

Critical strengths
  1. Annotation-free subgroup estimation is an important and appealing research topic in the context of bias mitigation.

  2. The proposed diffusion mechanism provides an interesting way to recover more complete subgroups beyond misclassified samples, offering useful insights for related research.

Critical weaknesses
  1. The presentation and methodological motivation need improvement. The paper contains numerous formatting issues, and its theoretical analysis is limited. The method section focuses primarily on implementation details, without sufficiently explaining the underlying rationale or how the proposed design addresses key challenges in bias mitigation. In particular, the paper lacks a convincing explanation of why preserving error direction can improve subgroup retrieval AUC. Furthermore, the definition of a subgroup is not sufficiently clear.

  2. The experimental evaluation is incomplete. Subgroup estimation is ultimately intended to facilitate bias mitigation, yet the paper does not evaluate whether the proposed method improves downstream bias mitigation. Consequently, its effectiveness for the ultimate task remains unclear. In addition, several hyperparameters require careful selection, but the paper lacks sufficient sensitivity analyses to assess their influence on performance.

Questions for authors
  1. Line 16: How should the phrase “its complement over-detected” be interpreted? The meaning of “its complement” is unclear.

  2. Line 130: The paper appears to define a subgroup that spans multiple classes, which seems inconsistent with the conventional definition of a group. However, this aspect is not discussed in the Method section. Could the authors clarify the definition and explain the motivation and benefits of allowing subgroups to span multiple classes?

  3. Section 3.1, around Line 168: Is (\sigma) defined in relation to (e) or to (p_{a_i})? Please clarify the notation and its relationship to the preceding definitions.

  4. Equations before Section 3.2: These equations appear to be unnumbered. Please check the equation numbering and formatting.

  5. Figure 1: The presentation appears to be closer to an algorithm or procedural pipeline than a conventional figure. The authors should consider whether it would be more appropriate to present it as an algorithm or revise its presentation accordingly.

  6. Writing style: Some parts of the paper are overly informal. For example, the caption of Figure 1 could be revised to adopt a more formal academic style.

  7. Section 3.2: This section appears incomplete and requires further development.

  8. Line 202: The explanation is confusing. First, for binary classification, if a sample is misclassified, the predicted probability assigned to the incorrect class must exceed 0.5 (assuming a standard probability-threshold decision rule). Therefore, it is unclear why the maximum operation is necessary. Second, the formulation appears to be specific to binary classification. How would it be extended to multi-class classification?

  9. Hyperparameter sensitivity: How sensitive is the proposed method to the number of PCA components, the choice of (k) in the nearest-neighbor graph, and the self-retention parameter (\lambda)? A systematic sensitivity analysis would help establish the robustness of the method.

  10. Assumptions about the embedding space: The method relies on nearest-neighbor relationships in the embedding space to propagate error scores. This implicitly assumes that local feature-space proximity is meaningful for identifying subgroup membership. However, the paper does not sufficiently discuss or validate this assumption, nor does it introduce mechanisms to ensure that the embedding space preserves the local structure required for reliable diffusion.

  11. Line 245: The statement that “there is no random initialization and no iterative fitting” seems overly broad. Although the diffusion procedure itself may not involve random initialization or iterative model fitting, the encoder used to extract representations is trained beforehand and may be affected by random initialization. The authors should clarify the scope of this claim and distinguish the proposed diffusion procedure from the training of the upstream encoder.

  12. Selection of diffusion depth (T): The authors select (T) using ground-truth subgroup labels on the validation split. This raises concerns about the consistency of the evaluation protocol with the paper's annotation-free motivation, as subgroup annotations are required for hyperparameter selection. Could the authors clarify this design choice and discuss whether (T) can be selected without access to ground-truth subgroup labels?

5 comments
AuthorsOpenReview ↗

Response Part 1

We thank the reviewer for their detailed feedback. We have uploaded the revised manuscript (with additions in blue and new tables in grey). Below, we provide point by point responses to the reviewer concerns:

  • W1 [Methodological motivation and presentation]: We have revised the Introduction and Sections 3.1–3.2 to clarify the motivation for signed errors, the subgroup definition and the diffusion mechanism. In Section 3.2.2, we explain how positive and negative error contributions propagate through neighbouring samples and may reinforce or cancel each other. We also discuss in Section 5.3 when preserving error direction increases the difference between subgroup mean scores and evaluate its effect on AUC in Section 5.1, where retaining error direction increases AUC in most of the experiments.
  • W2 [Downstream mitigation and sensitivity]: We agree that evaluating downstream bias mitigation using the discovered subgroups would be beneficial. In our work, like many other works on SDMs, we focus on subgroup retrieval as the discovery stage, following the evaluation setup of DOMINO, which can subsequently support approaches such as Group DRO or last-layer retraining. Downstream bias mitigation is outside the scope of this work. In the original submission, we evaluated the contributions of error assignment and diffusion through the ablations in Section 5.2 (Table 2). In the revised manuscript, we additionally provide sensitivity analyses for PCA dimensionality, diffusion depth, neighbourhood size, and self-retention in Section 5.5 and the Appendix.
  • Q1 [Ambiguous complement terminology]: We have removed the ambiguous wording from the abstract. We also discuss in Section 5.3 how the target subgroup and its complement can exhibit different false-positive and false-negative rates despite having similar overall error rates.
  • Q2 [Subgroups spanning classes]: We have clarified in Section 3.1 that a subgroup may be defined by a semantic attribute and can span multiple task classes. This allows us to consider subgroups such as the wrong-background images in Waterbirds, which include both waterbirds on land and landbirds on water backgrounds. Our approach assigns an error-based score to each sample and produces a single ranking for subgroup retrieval.
  • Q3 [Definition of σi\sigma_i]: We have clarified the notation in Sections 3.1–3.2. The signed error ςi\varsigma_i represents the direction of the classification error, which we weight by the classifier's confidence to obtain the initial score ϕi\phi_i. We then diffuse this score through the neighbourhood graph and rescale it to obtain the final score σi\sigma_i.
  • Q4 [Equation formatting]: We thank the reviewer for pointing this out. We have revised the equation formatting in the manuscript. We define the preliminary variables in Section 3.1 and present the principal mathematical formulations as numbered equations in the subsequent sections.
  • Q5: [Algorithm presentation]: In the original submission, we presented the computational procedure as numbered pseudocode in Figure 1. In the revised manuscript, we have updated its caption to describe the main steps of our approach, including preprocessing, signed-error score construction and neighbourhood-based diffusion.
  • Q6 [Writing style]: We have revised the Introduction to improve the motivation for preserving error direction and propagating errors through neighbouring samples. We have also updated the caption of Figure 1 and improved the presentation of the method in Section 3.2, where we describe the main components of our approach through separate subsections and formulations.
  • Q7 [Section 3.2]: We thank the reviewer for this suggestion. We wanted to present the subsequent Methods' sections as subsections of Section 3.2. In the revised manuscript, we have reorganised Section 3.2 into three subsections covering signed error scoring (3.2.1), diffusion over feature neighbours (3.2.2), and diffusion-depth selection (3.2.3).
  • Q8 [Confidence formula and multiclass extension]: We have clarified in Section 3.2.1 that max⁡(pi,1−pi)\max(p_i,1-p_i) represents the confidence assigned to the predicted class under binary classification. For false negatives, this confidence is 1−pi1-p_i, while for false positives it is pip_i. In the revised manuscript, we have also added a multiclass extension in Equation (2), where we construct class-wise signed error scores and apply diffusion to obtain a single ranking based on the magnitude of the resulting vector. We further evaluate this extension using a four-class classifier on the Spawrious data and obtain AUCs of 95.03% with CLIP features and 85.53% with classifier features. We provide the corresponding results in Appendix A.9 (Table 8).
AuthorsOpenReview ↗

Response Part 2

  • Q9 [Hyperparameter sensitivity] We thank the reviewer for this suggestion. In the revised manuscript, we have included sensitivity analyses for PCA dimensionality, diffusion depth, neighbourhood size (k), and self-retention λ\lambda. We present the PCA results in Section 5.5 (Table 4) and the sensitivity analyses in Appendices A.6 and A.10 (Tables 7, 9 and 10). We observe relatively stable retrieval performance across PCA dimensions, while the effects of neighbourhood size and diffusion depth depend on the dataset and representation. We also observe that removing neighbourhood propagation by setting λ=1\lambda=1 generally reduces retrieval AUC. For diffusion depth, we find that validation-selected values achieve test AUC within 1.8 percentage points of the highest observed values across the evaluated comparisons.
  • Q10 [Embedding-space assumptions]: We agree that the neighbourhood structure of the embedding space is important for the diffusion process. We discuss in Section 3.2.2 how our approach uses local similarity to propagate error information to neighbouring samples, including correctly classified members of a subgroup. Our ablation experiments in Section 5.2 (Table 2) demonstrate the contribution of error assignment and diffusion for the evaluated embeddings. In the original submission, we evaluated our approach using both CLIP and ResNet-18 representations (Tables 1 and 5). In the revised manuscript, we further study the sensitivity to PCA dimensionality and neighbourhood size in Section 5.5 and Appendix A.10.
  • Q11 [Determinism and variability]: The diffusion process is deterministic for fixed inputs and parameters. The standard deviations reflect variations across evaluation splits and, where applicable, classifier training, which can affect predictions, embeddings, neighbourhood graphs and performance.
  • Q12 [Selection of diffusion depth]: We have clarified in Section 3.2.3 that subgroup annotations are used to select diffusion depth and ranking direction on the validation data. We do not use these annotations when computing sample scores. We then apply the selected depth and ranking direction to the test data. We fix k=15k=15 and λ=0.5\lambda=0.5 in the main comparisons and study their sensitivity separately in Appendix A.10. We also select DOMINO’s likelihood weight (γ\gamma) using validation subgroup annotations (Appendix A.5, Table 6).
AuthorsOpenReview ↗

Follow-Up to Our Rebuttal

We thank the reviewer for their time and helpful feedback. We have addressed your concerns and revised the manuscript accordingly. We welcome further discussion and would be grateful if you would take our revisions into account when reconsidering your score and overall assessment.

Reviewer_vgAkOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a method for discovering underperforming subgroups for classification. The idea is motivated by the fact that current methods for discovering subgroups can miss failures by treating false positives and false negatives similarly (i.e. by only focusing on mispredictions). The paper proposes 1) assigning each sample a signed error score (positive for FNs and negative for FPs) and 2) propagating these using a nearest neighbour graph based on the model’s representations, to identify additional examples. The paper shows that the method outperforms prior work on different datasets.

Critical strengths
  1. subgroup discovery is important for scenarios where group annotations are not available, and the main idea is motivated well, and is especially important for medical applications.
  2. The results are strong on CheXpert, where the target groups are different in FPR and FNR.
  3. The paper includes ablations for both the signed error representation that shows that preserving the direction of the error can be useful in certain settings.
Critical weaknesses
  1. The method does use subgroup labels on the validation data, which seems inconsistent with the subgroup discovery motivation. In a realistic setting, we might not even have access to validation subgroup labels. (See Auto-D3M from https://arxiv.org/pdf/2406.16846 for a setup where no subgroup labels are available)
  2. The proposed signed error score is essentially the prediction residual (y_i-p_i) for misclassified examples, while correctly classified examples receive a score of zero. A natural baseline would be to use (y_i-p_i) for all examples. This would preserve the direction and confidence of predictions and also gives information about correctly classified examples as well. Additionally, it would be useful to ablate the confidence weighting.
  3. Some of the baselines might not be particularly suitable for the evaluated tasks. For example, GEORGE clusters examples within each target class, while the subgroups considered in the experiments span both classes. Comparisons with more recent error slice discovery methods are missing.
  4. Although the paper reports results on 9 different settings, 6 of these are variations of the same synthetic shortcut on ISIC-2017. The results on the remaining datasets are more mixed, especially on CelebA, where SEWISE performs worse than the baselines. I think more diverse experiments would help show how well the method generalizes to other types of subgroups. Also, for ISIC it would be useful to report whether the injected subgroup actually has worse classifier performance than other data points.
  5. The benefit of signed errors seems to depend on the types of errors within the subgroup. For example, if a subgroup includes both FPs and FNs, their signed scores might cancel during diffusion. The waterbirds results, where unsigned errors perform slightly better seem consistent with this weakness. It would be useful to better understand when signed errors are preferable to unsigned errors.
Questions for authors

See weaknesses mentioned above.

2 comments

We thank the reviewer for their detailed feedback. We have uploaded the revised manuscript (with additions in blue and new tables in grey). Below, we provide point by point responses to the reviewer concerns:

  • W1 [Subgroup labels during validation]: We have clarified in Section 3.2.3 that validation subgroup labels are used to select diffusion depth and ranking direction. We do not use subgroup labels when constructing the initial error scores or the neighbourhood graph, and the selected parameters are subsequently applied to the test data. We acknowledge that this evaluation setup requires subgroup annotations on the validation set. We also select DOMINO’s likelihood weight (γ\gamma) using validation subgroup annotations (Appendix A.5, Table 6).
  • W2 [Full residual and confidence weighting]: We agree that the prediction residual yi−piy_i-p_i is a natural baseline. In the revised manuscript, we provide a comparison of our proposed score with the full prediction residual and model prediction probability pip_i in Section 5.4 (Table 3). We observe that the full residual improves retrieval on some datasets, including CheXpert and CelebA with CLIP features, while our proposed score performs better on others. We therefore do not observe a consistent advantage for either error representation across datasets and embeddings. We also compare the confidence-weighted score with the unweighted signed error in the same table, where we observe relatively small differences in retrieval AUC below two percentage points across all experiments.
  • W3 [Baseline suitability]: We have expanded Related Work (Section 2) to discuss GEORGE's class-conditional clustering and its implications for subgroups spanning multiple task classes, such as Waterbirds. In the original submission, we also provided a validation-selected likelihood-weight comparison for DOMINO in Appendix A.5 (Table 6), where we observe that our approach generally achieves higher retrieval AUC even after tuning this parameter. We now additionally compare our approach with Failure Directions [1] in Appendix A.11 (Table 11) where SEWISE achieves higher retrieval AUC on CheXpert with both representations and on CelebA with ResNet-18 features and, Failure Directions performs better on Waterbirds with both representations and on CelebA with CLIP features.
  • W4 [Experimental diversity]: We thank the reviewer for raising this concern. In the original submission, we evaluated our approach on ISIC-2017, CheXpert, CelebA and Waterbirds, including six variations of the injected shortcut in ISIC. In the revised manuscript, we additionally provide results for an Edema classifier on CheXpert in Appendix A.2 and a multiclass classifier on Spawrious in Appendix A.9. We also analyse classifier performance on the predefined target subgroups in Appendix A.2 and observe that accuracy on square-injected ISIC images is lower by 9.6–24.6 percentage points than on images without the square across the six experiments and both classifiers.
  • W5 [Signed versus unsigned errors]: We agree that preserving error direction is not necessarily beneficial for every target subgroup. In the original submission, we compared signed and unsigned error scores in Section 5.1 (Figure 2), keeping the graph, confidence weights and diffusion depth fixed. We observed that the signed score achieves higher retrieval AUC in seven of the nine experiments, while the unsigned score performs better on CelebA and slightly better on Waterbirds. In the revised manuscript, we discuss in Section 3.2.2 (Equation 5) how error contributions reinforce or cancel each other. We also analyse in Section 5.3 (Equation 6) when retaining direction increases the difference between subgroup mean scores. On CheXpert, signed errors distinguish groups with similar overall error rates but different error compositions. On Waterbirds, pooling false positives and false negatives gives greater mean-score separation, which is also reflected in the unsigned score’s higher retrieval AUC.

[1] Jain, S., Lawrence, H., Moitra, A., & Madry, A. (2022). Distilling model failures as directions in latent space. arXiv preprint arXiv:2206.14754.

AuthorsOpenReview ↗

Follow-Up to Our Rebuttal

We thank the reviewer for their time and helpful feedback. We have addressed your concerns and revised the manuscript accordingly. We welcome further discussion and would be grateful if you would take our revisions into account when reconsidering your score and overall assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.