- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies spurious correlations in pretrained models. The key idea is: instead of retraining the model or changing the classifier, identify and suppress spurious concepts in the latent representation before classification.
SPOCK differs from last-layer retraining methods because those methods leave the spurious information encoded in the latent representation and only adapt the final classifier. In contrast, SPOCK directly modifies the latent representation while keeping the original classifier fixed.
SPOCK assumes that the latent representation (z) is dense and entangled. The authors therefore train a Sparse Autoencoder (SAE) on the model's latent representations, which maps them into a much larger set of sparse concepts.
After training the SAE, SPOCK learns a mask over the SAE concepts. The mask suppresses some concepts and preserves others. The masked concept representation is then decoded back into the original latent space and passed to the original classifier.
The mask is learned using a small manually constructed group-balanced validation set, where examples from different groups are balanced so that the model does not simply preserve concepts associated with the majority groups.
Overall, evaluation shows that SPOCK improved worst-group accuracy (WGA) while maintaining relatively high average accuracy.
Critical strengths
- No last-layer retraining required. A key strength of SPOCK is that it can mitigate spurious correlations without retraining or modifying the original classifier head. Instead, it learns a mask over the SAE concepts and applies it before the fixed classifier, making the approach lightweight and potentially more efficient than methods that require last-layer retraining. This is consequential because it provides a lightweight post-hoc approach that improves worst-group accuracy without requiring expensive end-to-end retraining.
- Broad experimental evaluation. The paper evaluates SPOCK across multiple datasets and model architectures, including Waterbirds, CelebA, BAR, and MultiNLI, and compares against several strong baselines. The experiments also examine label efficiency, worst-group versus average accuracy, and whether the learned mask actually suppresses spurious information.
- Label efficiency. The experiments show that SPOCK can achieve meaningful improvements in worst-group accuracy with only a very small group-annotated validation set, while maintaining relatively high average accuracy. This is practically important because obtaining group annotations for the full training set can be expensive, and the results across four datasets provide evidence that the approach is not limited to a single benchmark.
Critical weaknesses
- The evaluation does not sufficiently establish the generality of the approach. The experiments use relatively small base models (ResNet-18 for vision and BERT for MultiNLI), while the proposed mechanism is motivated as a general post-hoc representation-level method. It would be important to evaluate whether the approach remains effective for larger and more modern pretrained models and whether its claimed advantage over last-layer retraining persists in those settings.
- Dependence on predefined group annotations. SPOCK requires group-annotated validation examples to learn its concept mask. Consequently, its ability to mitigate spurious correlations depends on the groups represented in the validation set. Biases arising from unknown or unannotated attributes may remain unmitigated, since SPOCK is not explicitly designed to discover arbitrary sources of bias.
Questions for authors
- How much does performance depend on SAE architecture, latent layer, expansion factor, sparsity coefficient, and mask regularization?
- Would the approach work with much larger/modern foundation models rather than ResNet-18 and BERT?
- How much additional inference cost does SAE encode -> mask -> decode introduce?
4 comments
References
- Bricken, Trenton, et al. "Towards monosemanticity: Decomposing language models with dictionary learning." Transformer Circuits Thread, 2023.
- Huben, Robert, et al. "Sparse autoencoders find highly interpretable features in language models." ICLR, 2024.
- Kirichenko, Polina, Pavel Izmailov, and Andrew Gordon Wilson. "Last layer re-training is sufficient for robustness to spurious correlations." ICLR, 2023.
Thanks for your response and additional experiments. These address most of my concerns, particularly regarding newer backbones and inference overhead.
My main remaining concern is the reliance on group information to learn the mask. The BAR results are promising, but it remains unclear how well SPOCK handles unknown or unannotated spurious attributes. But I find the approach interesting, and the additional experiments have strengthened the paper.
Thank you very much for your follow-up and for taking the time to engage in the rebuttal process. We are glad that our responses and additional experiments have addressed several of your concerns.
We appreciate your acknowledgement of SPOCK's performance on BAR, even with approximate group balancing. As previously indicated in our rebuttal, one limitation of our method is its reliance on group annotations to construct a balanced held-out set for mask optimisation, which we also acknowledge in the paper.
Nevertheless, our annotation-budget experiment (Figure 5) demonstrates that SPOCK can achieve competitive performance with only 1 example per group on CelebA, reducing its reliance on extensive group-annotated data. We agree that extending SPOCK to settings with unknown spurious attributes is a promising direction for future research, particularly by investigating whether SPOCK's learned masks can help identify additional sources of spurious correlation and subsequently guide more informed data collection and augmentation.
If you have any further questions or concerns, please do not hesitate to let us know. Thank you once again for engaging in this discussion and for your valuable feedback.
Dear Reviewer SaaW,
Thank you for taking the time to review our work. We greatly appreciate you describing our experimental evaluation as "broad" and acknowledging its coverage across multiple benchmarks. Below, we address your questions and concerns.
(W1 / Q2) Does SPOCK remain effective on larger and more modern pretrained models, and does its advantage over last-layer retraining persist in these settings?
We thank you and reviewer QWw6 for raising questions about SPOCK's effectiveness on modern backbone architectures. Following your suggestions, we have added Appendix E, which details the results of evaluating vision and text benchmarks using CLIP ViT-B/32 and ModernBERT-base, respectively.
Our experiments show that SPOCK outperforms DFR in both WGA and average accuracy across the vision benchmarks and remains competitive with DFR on MultiNLI (Figure 10). Together with our original ResNet-18 and BERT experiments, these results demonstrate that SPOCK generalises across different vision and language backbones while maintaining a favourable WGA–average accuracy trade-off. Please see our response to reviewer QWw6 (Q2) for a detailed comparison with DFR across these benchmarks.
(W2) SPOCK depends on predefined group annotations and may not address unknown or unannotated biases.
We acknowledge SPOCK's reliance on group annotations to construct a balanced held-out set for mask optimisation and explicitly identify this as one of the main limitations in our paper. We further agree that SPOCK is not designed to discover unknown biases. However, we would like to clarify that SPOCK does not assume access to privileged group information beyond what existing methods already use, such as group-annotated validation data for hyperparameter selection.
Nevertheless, our results on BAR show that SPOCK performs well even when the group-balanced set is constructed using only limited knowledge of the distribution shift, without predefined group annotations. This suggests that SPOCK can remain effective even when group annotations are approximate, although identifying unknown biases remains an open challenge.
(Q1) How much does performance depend on SAE architecture, latent layer, expansion factor, sparsity coefficient, and mask regularization?
We previously investigated the effects of the SAE expansion factor and mask regularisation coefficient λm. Following your and reviewer jJ6k's suggestions, we additionally conducted an ablation on SAE sparsity coefficient λs. All results are reported in Appendix G, and we summarise the key findings below:
We do not ablate the choice of layer in the backbone architecture, as our method builds on the premise that last-layer representations encode both task-relevant and spurious information [3], as exploited by DFR, AFR, and other LLR-based methods. We also focus on the last layer to ensure a fair comparison with these methods and demonstrate the effectiveness of bias mitigation without retraining the classification head.
For the SAE architecture, we adopt a standard single-hidden-layer design commonly used in the mechanistic interpretability literature for concept discovery [1,2]. The primary purpose of using an SAE is to obtain an overcomplete representation that facilitates the separation of task-relevant and spurious concepts, which is crucial for mask learning (see Figure 8 for experiments showing this). As acknowledged in our limitations, SAEs are not guaranteed to discover semantically disentangled concepts, and our findings are supported by empirical evidence rather than a theoretical guarantee.
(Q3) How much additional inference cost does SAE encode -> mask -> decode introduce?
We thank you and reviewer jJ6k for raising questions about SPOCK's inference overhead. Following your suggestions, we have added Appendix F, which reports inference latency (ms) and peak GPU memory (MiB).
Our results show that the relative inference latency and peak gpu memory overhead decreases with increasing batch size (Figure 11). For instance, for ResNet-18, the latency overhead decreases from 4.9% to 1.4%, while the memory overhead decreases from 16.2% to 1.7% as batch size increases from 1 to 128.
Overall, these results show that SPOCK introduces minimal inference latency overhead, particularly at larger batch sizes. Please see our response to reviewer jJ6k (Q4 / W4) and Appendix F, where we report a detailed comparison of inference latency and peak GPU memory across different batch sizes.