- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Authors extract SAE directions as basis for CLIP prompt tokens. They initialise learnable queries that then attend over this basis and the resulting tokens are injected into vision and text layers. They test it on base-to-novel, domain generalization, cross-dataset transfer, and few-shot learning tasks using unified VL-SAE concepts.
Critical strengths
Authors perform quite a lot of experiments which is good.
The paper is well written.
Critical weaknesses
I am wondering if SAEs used to build E seen evaluation images and/or class names? This would be useful to check to find potential leakage.
The method is totally dependable by SAE quality. The datasets used by the authors are quite standard and most concepts will be recognised by SAEs, however it is not clear whether the SAE concepts would be good initialisation when SAEs are weak or a dataset is truly outside of SAEs concepts. I think EuroSAE is quite representative of this where the method doesn’t not perform well (Table 6). Additional experiments on OOD data such as medical or other remote-sensing datasets or with deliberately weaker SAEs (smaller dictionaries, less training data) would show whether the method is robust or only works where the SAE is already strong.
Can authors clarify how parameters k, N and and layers have been chosen? What split was used etc?
Authors claim that prompt tokens are interpretable because as they come from SAEs. But the attention and projection layers can mix and change those directions. Could authors provide a quantitative test showing the learned tokens are still interpretable? As for example readable prompts in CPL or IPO it is clear that they are interpretable.
Questions for authors
See weaknesses
5 comments
Thank you for the response. The weaker-SAE experiment is still missing. For hyperparameters, k seems to have been chosen on the evaluation datasets so please clarify whether a separate validation split was used. The convex selector helps, but tokens still pass through the projections and no quantitative interpretability test was given.
Clarification regarding weaker SAE, K and quantitative interpretability test
We sincerely thank the reviewer for responding to our rebuttal. Below we clarify the issues:
Weaker SAEs. Our results already include IPL being driven by weaker interpretability signals. Vision-only and text-only VL-SAE dictionaries capture a single modality of the unified concept space. With these weaker signals, IPL performance actually degrades and we can see this evidently in Table 1. In our response to #WgfD, Q1, table 3, we also performed an experiment where used bottom 256 and random 256 directions from the same SAE and observed that using top 256 activated directions yields better results. Retraining a deliberately weakened SAE was not feasible within the discussion period and is slightly out of scope of our central research question (which is to to verify the conceptual feasibility of whether already existing SAE based interpretability tools can be used as guide for adaptation).
Validation split for . No separate validation split was used. was selected from Fig. 3c , i.e., on base-to-novel test accuracy averaged over the 11 datasets (we have discussed this in our ablation section too), and the injection layers and were selected on ImageNet (Figs. 3b and 6), following common practice in prompt-learning ablations. Each is a single value shared by all datasets and all four tasks, and the domain-generalization target domains were never used in any selection.
Quantitative interpretability test. We report this in our response to Reviewer #WgfD (Q2), measured on the actual prompt tokens, including after the projections. With the convex selector, every token's residual outside the span of the concept directions is 0.000, and each token equals its selected concept mixture (cosine 1.00) and we report other quantitative metrics like geometric distortion and concept alignment too. We will also include all these supporting experiments in our main draft.
We hope these clarifications address remaining concerns. If so, we would be really grateful if the reviewer would consider updating score.
Rebuttal by Authors
We sincerely thank reviewers for carefully reading our paper and raising meaningful question and concerns
W1:
The SAE’s are trained without class labels and VL-SAE specifically is trained on CLIP embeddings of CC3M image-caption pairs. The dictionary E is selected using base-class training data only, and prompts are trained on base classes only, novel images and names are used only during evaluation. Hence, IPL’s protocol ensures that there is no explicit data leakage. Also, it's important to note that any semantic overlap between web data and downstream categories might be present through CLIP’s own pretraining on web image-text pairs, which all the baseline use and that’s why the zero shot performance of CLIP is already 74.22 on novel classes. Many prompt learning methods including IPL does not aim to learn from data that the pretrained model has never seen, rather it adds context in the form of concept-composed prompt tokens that adapts an already pretrained model to downstream tasks which includes novelty and distribution shifts. The standard benchmarks therefore measure generalization beyond the data used for prompt learning, which is the setting in which IPL and all baselines are evaluated.
W2:
Following the reviewer’s suggestion, we evaluate on RESISC45, an aerial remote sensing dataset outside the standard benchmarks. We report the results in the table below and all methods use the same splits and seeds and other configurations.
Table 5
We observe that IPL improves over MMRL by +2.83 on novel classes and +1.82 on HM with less variance in accuracy across seeds on novel classes. The SAE concepts therefore remain a useful basis on a domain outside the standard benchmarks. IPL's effectiveness does depend on the interpretability tool, and we explicitly acknowledge this as a limitation in Sec. 5. Since the main goal of our work is to investigate a conceptual feasibility question of whether interpretability tools can act as training guides during adaptation, this dependence is therefore inherent to the question. Hence, IPL is plug-and-play with respect to the interpretability tool and any new tool that yields concept directions and concept strengths can be used without changes to the method. We therefore expect IPL to benefit directly as these tools improve.
W3:
The concept dictionary size K=256, the number of prompt tokens N=5, the injection layers 6-12 and alpha=0.7 form a single setting shared by all datasets and all four tasks; none of them is tuned per dataset. They were chosen from the ablations in Fig. 3b (layers, on ImageNet), Fig. 3c (K, averaged over the 11 datasets) and Fig. 6 (N, on ImageNet), using the described split in our experiments section. Training settings (epochs, learning rate, batch size, optimizer) follow MMRL.
These choices are not the optimum ones. We choose those because performance saturates at N=5 (Fig. 6), insertion at layers 3-12 performs similarly to 6-12 (Fig. 3b), and our choice depended on similar rationales for K and lambda ablations.
W4:
Firstly, we would like to clarify that IPL’s prompt tokens are not natural language strings as in CPL or IPO. We define the prompt tokens’ interpretability at the level of concept directions derived from the interpretability tools like SAE’s. Taking reviewers suggestions into consideration, we have proposed a variant of attention selector such that the prompt tokens constructed lie in the span of original concept directions (for more details please refer to our response to #Lage, W1/Q1). We also note that the results of the convex selector variant matches the accuracy of the earlier variant (82.15 - convex selector vs. 82.01 - naive MHA average HM over four datasets) indicating that the results are mostly driven by SAE’s original concept directions E irrespective of the transformation it undergoes via attention selector.