- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates the role of spurious correlations during model training, demonstrating that in small Transformer models, the presence of spurious correlations can actually improve generalization on bias-conflicting samples. The authors argue that the utility of spurious correlations depends on the deviation of the bias ratio from the random-chance baseline. Empirical evaluations across four synthetic tasks (e.g., Max-Equal-Sum-Parity) and various bias-ratio sweeps highlight this phenomenon and prompt questions regarding when, how, and what training signals contribute during optimization.
Critical strengths
-
The conceptualization of the uninformative shortcut (where the bias-aligned sample ratio matches the random-chance baseline) offers a fresh angle on understanding shortcuts.
-
The experiments conducted on small Transformers across different bias-ratio settings provide evidence supporting the proposed claims.
Critical weaknesses
-
Limited practical significance and scope. My primary concern lies in the practical implications of this study. The finding that spurious correlations benefit model training is currently validated only on small Transformers and synthetic tasks, while reducing the bias ratio to random chance is shown to impair generalization. However, the paper lacks discussion on whether these conclusions hold for larger, real-world models (such as pre-trained Language Models) or image classification models where spurious correlations are extensively studied. The broader practical value of these insights remains unclear.
-
Missing connection to existing debiasing literature. In addressing spurious correlations, the paper focuses primarily on adjusting the bias ratio of bias-aligned samples. However, existing debiasing research relies on countermeasures such as sample re-weighting, data augmentation, and advanced training algorithms, which have proven effective at mitigating shortcut overfitting and improving generalization without explicitly introducing uninformative shortcuts. The submission lacks a discussion or comparison with these advanced debiasing techniques and should broaden its scope to contextualize its findings within the existing literature.
-
Clarity of mechanistic analysis. Section 5 offers a mechanistic interpretability analysis, including gradient dynamics, circuit evolution, and QK/OV fingerprints. However, this analysis is restricted to a single synthetic task and is currently hard to follow. A broader, more structured presentation is needed to substantiate these mechanistic claims.
-
Lack of vision generalization. The evaluation of the submission is limited to auto-regressive generation text tasks. Given that spurious correlations are heavily studied in image classification (e.g., CNNs or CLIP models) with explicit shortcut signals like backgrounds, it remains uncertain whether vision models behave similarly. Expanding the study to image datasets or vision-language models would significantly enhance the real-world impact and applicability of the paper.
-
Minor issues.
- I recommend that full names are provided upon first introduction (e.g., line 047 MESP), otherwise it is hard to follow the submission.
- Broken sentence: line 048, There and on two…
- As noted in the limitations (Section 6), the paper would benefit from broader studies or ablation experiments across different transformer architectures, learning rates, optimizers, training epochs, batch sizes, and tasks to confirm the robustness of the proposed insights.
Questions for authors
Please refer to the Weaknesses.