- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
I read and evaluated the manuscript all by myself, the review is my own analysis and academic judgment. I only use chatgpt for language polishing to improve the readability.
My original text as as follows:
This paper proposes an unsupervised debiasing method. It combines Gaussian Noise Intervention (GNI) for pseudo-group discovery, reliability-aware multi-level debiasing, and D-CORE for inference-time residual-shortcut correction. Experiments on four benchmark datasets shows improved worst-group accuracy, it doesn’t need spurious-attribute annotations during training and model selection. However, the framework is designed very complex, the D-CORE is limited to binary classification, which limits its practical application.
-
Debiasing without access to group annotations is a practical and interesting topic.
-
Using trained bias experts to correct residual shortcut effects at inference time may be a good and novel direction for adaptive debiasing.
1.The motivation of the paper, the necessity of the designed techniques are insufficiently justified. The paper does not discuss the limitations of existing bias-mitigation methods, which the paper aims to address.
2.论文的关键模块在设计、理论支撑和实验验证方面不足。it is not sure whether the theoretical of GNI translate into more accurate pseudo-group prediction, whether D-CORE can isolate and correct reliance on residual shortcut.
3.The framework involves multiple components, loss terms, and hyperparameters, raising concerns about implementation complexity and tuning costs. Moreover, D-CORE is currently restricted to binary classification, limiting the method's broader applicability.
-
摘要关于本文的motivation过于笼统了. Many bias-mitigation methods already exist, the paper should clarify what kind of limitation existing methods have, what kind of limitation it try to address, what advantages it brings compare to existing methods.
-
Figure 1, caption, "correctly classified" is used without specifying what a correctness refers to, does it means correct target-label prediction or does it means correct pseudo-group assignment. The evaluation criterion used should be clearly descripted.
-
Related work is not up to date, more recent works should be discussed.
-
Since the work builds on existing method NSF, it should be introduced in preliminary for easy understanding.
-
符号不统一,different parts of the paper use different notations (rho and l) for the layer index.
-
The authors should clarify, the hyperparameter selection, evaluate the practical tuning costs.
-
Concern about GNI technique: Reduce the variance of geometric statistic does not necessarily improve pseudo-group prediction accuracy because the statistic may remain systematically biased. There is a gap between the theoretical analysis and the practical implementation, please discuss it. The motivation that multi-view stability improves pseudo-group discovery need further verification.
-
Can the method maintain normal performance on data without spurious correlations? a method that degrades performance in this setting is not practical.
-
D-CORE 将bias expert对factual representation的响应与对zero-feature reference的响应之间的差异,理解为residual-shortcut signal。However,由于偏置专家是使用target labels而非伪相关属性annotation进行监督训练,他会同时捕获shortcut-related and target-related information。when spurious correlations is weak,target-related information甚至可能占据主导地位。而用zero vector replace input representation 不一定能够分离出伪相关特征所产生的贡献。 相关说法需要更充分的理论依据,建议设计对应的 controlled study区分residual-bias correction与传统auxiliary-model correction的关系与区别。
-
它在 multi-shortcut 情况下如何?, because it’s the case in real-world application.
Critical strengths
-
Debiasing without access to group annotations is practically appealing.
-
Using trained bias experts to correct residual shortcut effects at inference time is a potentially promising direction for adaptive debiasing.
Critical weaknesses
1.The motivation and necessity of the proposed design are insufficiently justified. The paper does not clearly establish the specific limitations of existing bias-mitigation methods that the proposed framework aims to address.
2.The theoretical justification and empirical validity of key components remain insufficiently established. In particular, it is unclear whether the theoretical properties of GNI translate into more accurate pseudo-group discovery, and whether D-CORE genuinely isolates and corrects residual shortcut effects.
3.The framework involves multiple components, loss terms, and hyperparameters, raising concerns about implementation complexity and tuning costs. Moreover, D-CORE is currently restricted to binary classification, limiting the method's broader applicability.
Questions for authors
-
The motivation stated in the abstract is too generic. Many bias-mitigation methods already exist; the paper should clarify which specific limitation of existing methods it addresses and what advantage the proposed method offers over them.
-
The caption of Figure 1 uses "correctly classified" without specifying what correctness refers to—whether correct target-label prediction or correct pseudo-group assignment. The authors should explicitly clarify the evaluation criterion.
-
Related work is insufficient; more recent works should be discussed.
-
Since the work builds on existing NSF, NSF should be introduced as a preliminary so that the paper is easier to follow.
-
The layer index is denoted by both rho and l in different parts of the paper; this notation should be unified.
-
The authors should clarify the hyperparameter selection process and evaluate the associated tuning costs in practice.
-
Concern about GNI: Reducing the variance of the geometric statistic does not necessarily improve pseudo-group assignment accuracy, since the statistic may remain systematically biased. The authors should clarify the gap between the theoretical analysis and the practical implementation, and verify whether multi-view stability improves pseudo-group discovery.
-
Can the method maintain normal performance on data without spurious correlations? This matters because a method that degrades performance in this setting would have limited practical value.
-
D-CORE interprets the difference between a bias expert's responses to the factual representation and to a zero-feature reference as a residual-shortcut signal. However, since the bias experts are trained with target labels but without explicit spurious-attribute supervision, they may capture both shortcut-related and target-relevant information, with the latter potentially dominating when spurious correlations are weak. Moreover, replacing the input representation with a zero vector does not necessarily isolate the contribution of spurious features. The authors should provide stronger justification for this interpretation and conduct controlled experiments to distinguish genuine residual-bias correction from generic auxiliary-model correction.
-
How robust is the method under multiple-shortcut settings, which are more common in real-world applications?
Response to Reviewer 3 on Method Motivation and Validation
W1&Q1&Q2&Q4&Q5: We thank the reviewer for the feedback on the manuscript’s structure and presentation. We revised the abstract and replaced “correctly classified” with “correct target-label predictions” in the Figure 1 caption. We expanded the related work to compare D-CORE with recent counterfactual debiasing methods, introduced NSF in the preliminaries, and standardized the notation for layer indices throughout the manuscript. These revisions strengthen the motivation for our framework and clarify which limitations of existing methods it addresses. W2&Q7&Q9: We thank the reviewer for the questions. NSF’s sign-based grouping can fail under weak bias. To address this limitation, GNI uses intra-class ranking, which can correctly separate two groups under weaker bias conditions in our theoretical model. Using multiple perturbed views can further improve pseudo-group quality. Under our assumptions, the upper bound on the probability of reversing score order decreases exponentially with the number of independent views. Detailed proofs are provided in Appendix A. We added clean-score baseline that applies the same intra-class ranking to noise-free scores with all other settings held fixed (Appendix D, Table 4). We also varied the number of perturbed views to assess effects on pseudo-group quality and downstream performance (Appendix D.1, Table 5).In Appendix B.1, we added analysis of residual correction under controlled interventions on spurious features. These results show that zero-feature responses provide an effective reference for isolating residual shortcut signals and that aggregating these signals to correct predictions improves prediction consistency under the interventions. To compare D-CORE with a generic auxiliary-model correction in our seven image-only classification settings, we used the official CF-VQA code to adapt its correction and substitute it for D-CORE, keeping training hyperparameters fixed. D-CORE achieves higher WGA and mean accuracy in all seven settings (Appendix D.4, Table 8). W3&Q6: We thank the reviewer for the questions. We also added hyperparameter sensitivity analysis (Appendix D1&D7, Table5&Table12). Performance is sensitive to the voting threshold, which determines which samples are assigned to pseudo-groups and thus affects their quality. In Appendix I, we compare total runtime and peak GPU memory usage with those of baselines. On MIMIC-CXR+NIH at bias ratio 0.90, our method takes longer to run but has the lowest peak GPU memory usage among the competing methods. It also achieves the best WGA among all competing methods. We will focus on reducing training time in future work. In Appendix B.2, we prove that, under the stated assumptions, D-CORE can also reduce the model’s reliance on spurious features in multiclass classification. Q8&Q10:We thank the reviewer for raising this question about practical applicability. We evaluated our method on an unbiased variant of Waterbirds (Appendix H, Table 16). The results show that our method maintains strong predictive performance in the absence of spurious correlations. We also evaluated our method on HAM10000 with age and sex as spurious attributes (Appendix G, Table 15). Our method improves both worst-group and overall accuracy, demonstrating its ability to handle multiple coexisting shortcuts in this real-world image classification setting.