- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper
- Investigates the failure pattern in safety classification from VLMs that the model may correctly classify each individual modality but miss the joint context.
- It extracts activation-based features from the individual and joint forward pass and then fits a multinomial classifier on the normalized features
- The features include the final-layer logits, probabilities and confidence (7d) extracted for each view
- as well as features from dot products with linear directions in intermediate layer activations that correspond to average activations on missed Unsafe and undetected Unsafe errors in the feature space.
- It shows classification results substantially better than the base model and slightly better than with just the 21-d logits features.
- It compares to extensive baselines and controls that make a reasonable case for the value of the learned features from the hidden layers.
Critical strengths
- The paper demonstrates substantial performance improvements in joint classification performance compared to the base-model’s joint classification. It also shows that using information from the hidden layers provided a small but robust additional gain.
Critical weaknesses
- There are two important baselines missing:
- a) how much of the gain is explained by just re-calibrating the logit features from the joint view? Could the performance improvement be just due to calibrating the zero-shot LLM classification?
- b) for the hidden-layer features one important baseline seems missing: Just a strongly regularized linear model from all the hidden features for the 3 views directly towards predicting the label, trained on train_a and train_b. This is how probe-based safety classification is typically done in practice.
- No deeper analysis in terms of where the gains come from in the features. The majority of the gains come from the logits across the 3 views and are read-out by a linear model. It would be straightforward to look into the mechanism that explains which features exactly lead to the gain in performance
- No deeper analysis of what kind of datapoints the model fixes and on what datapoints the model regresses. For safety classification not all errors have the same cost, so looking at what the new model trades-off would be important
Questions for authors
- The cross-benchmark binary results seems lacking critical details. Where is the training data coming from for the non-VLSU results? And what do you do on benchmarks that don’t have image-only and text-only annotations? How do you fit the hidden-layer features on those?