- Rating
- 4
- Confidence
- 1(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies which privileged context is useful for safety-oriented on-policy self-distillation in vision-language models. SA^2-OPSD maps image, request, and pair safety labels to a short instruction telling a training-only self-teacher whether to answer or refuse and identifying the risk source. The student learns from the teacher's distributions on its own responses, without requiring the privileged context at inference. The central finding is that stating the action works better than supplying a complete reference response. The paper also decomposes attack success by observable harm mentions and examines dependence on visual-token states.
Critical strengths
- The idea is novel to me.
- The controlled evidence for the main finding is convincing.
- The robustness checks are unusually comprehensive for the claimed comparison.
Critical weaknesses
- Only evaluate in 4B, 9B, and 12B models, It would be nice we can check the gap in commercial models, e.g., gpt 6/6,1, fable 5.1, or opus 5.5 and see if the observation still exists.
Questions for authors
- See weakness above
Response to Reviewer W2Xj
We sincerely thank you for the encouraging evaluation. We deeply appreciate your recognition of the novelty of the idea, the "convincing" controlled evidence for the main finding, and the "unusually comprehensive" robustness checks. Below we address your suggestion on commercial models.
W1: Does the observation still exist on commercial models?
Yes, the observation still exists on all four commercial models you name, and the gain of our method also holds on a 32B backbone.
Table a. Commercial models on USU. Cp is the share of responses that mention the harmful image content, and pASR is the ASR among those responses. The SA²-OPSD row is a three-run mean.
All four commercial models mention the harmful image content in 29.5 to 42.2% of their responses and still comply in 67.7 to 74.8% of those, close to the 79.2% of the open base model. SA²-OPSD at 4B mentions the harm at least as often as any of them and complies in only 43.5% of those responses.
Table b. Results on Qwen3-VL-32B-Instruct (one run per method, same judge as the paper).
SA²-OPSD also achieves the lowest ASR at 32B, in every unsafe category and on both external benchmarks. Its mASR is 22.5%, against 31.8% for OPSD and 53.3% for the base model. General capability is preserved: the average over the five benchmarks of Table 2 is 86.4%, against 86.5% for the base model.
These results show that the observation holds on commercial models and that the gain of SA²-OPSD holds at 32B. Thank you again for your suggestion.