- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Steering, or intervening on a model's representation at test time, is becoming a strategy in self-supervised learning to mitigate shortcut learning by vision models. The authors argue that the well-studied "steering" in the language space mirrors "concept removal" in the vision space. While there are existing benchmarks to assess steering, this paper introduces SWORDBench, a benchmark for steering image representations at test time. The authors evaluate SWORDBench across a range of backbones and concept removal tasks. In addition to providing a benchmark, the authors provide novel metrics to assess steering performance and use those metrics to discover insights.
Critical strengths
- Thorough evaluation on 5 popular datasets (and investigation into two more)
- Thorough evaluation metrics and steering metrics that are well discussed
- Well-identified gap in the literature of steering behaviors on image spaces.
- Interesting novel metrics. I found the added metrics of cross concept robustness and collateral damage to be really interesting contributions of this work. I think nice extensions of these metrics could contribute to characterizing bias-accuracy tradeoffs that are rampant in the field.
Critical weaknesses
-
I am not entirely convinced that "steering" in language models is the same as "concept removal" in the vision space. The authors seem to take this analogy for granted. While steering can be used to mitigate or remove an effect in language models, it can also be used to add an effect [1, 2]. Thus, I have an issue with the overall framing of the paper that this is steering for vision models. I believe that either the claims should be adjusted to follow concept removal or that additional experiments need to be included for additive effects of steering.
-
Limited novelty in experimentation. Orthogonalization is used for concept erasure which is a well-established approach in the debiasing space, even for CLIP vision models (as used in this paper) [3]. To me, it is not clear that this experiments sufficiently expand on Chuang et al.'s work [3]. Chuang et al. also used orthogonalization to improve performance of classifiers by removing concepts on a similar set of datasets and backbones. The primary difference seems to be this submission's framing of steering instead of debiasing. However, as mentioned in my first weakness, I'm not sure that this steering at all (seems like debiasing called something else).
-
There is a lack of tie to real-world use (although a little in the Appendix). While broad claims are made about the findings on the particular datasets studied here, it is not clear to me what a user would do after running their dataset on SWORDBench. I think this is an important note for a benchmark paper.
-
(Minor). Lack of explanatory figure. Given the complicated pipeline and evaluation, I would have found this helpful for comprehension but not necessary.
[1] https://arxiv.org/abs/2402.01618 [2] https://arxiv.org/html/2602.00333v1#bib.bib3 [3] https://arxiv.org/pdf/2302.00070
Questions for authors
Why do you believe steering in language is equivalent to concept removal in images?
How would you expect results from SWORDBench to be used in the real-world? I.e. how are these insights actionable for downstream users?