acceptodds
Under review as a conference paper at ICLR 2027

Actions Speak Louder than Answers: Privileged-Context Self-Distillation for VLM Safety

Abstract

Vision-language models (VLMs) that refuse harmful text requests often comply when a benign-looking request is paired with an image that makes it harmful. On-policy self-distillation (OPSD) supervises every token of a model's own responses with a teacher that sees privileged context, usually a complete reference response, but it is unclear what this context should contain for safety. A response-level decomposition shows that naming the harm does not ensure refusal: four Qwen3-VL-4B baselines that we train for safety still fail to refuse in 67.0–82.6% of the responses that mention the harmful image content, close to the base model's 79.2%. We therefore give the teacher the action instead of an answer. Safety-Axis-Aware OPSD (SA²-OPSD) maps the image, request, and pair safety labels of each training pair to a short instruction that states whether to answer or refuse and where the risk lies, and the student, which sees only the image and request, learns from the teacher's next-token distributions on its own responses. Across two model scales and three safety benchmarks, SA²-OPSD achieves the lowest attack success rate (ASR), the share of unsafe inputs not clearly refused, in five of six settings. Against OPSD in the same pipeline, it lowers HoliSafe-Bench mean ASR over three runs from 37.7 to 25.8% at 4B and from 32.2 to 25.8% at 8B, and, in single 4B runs, ASR among responses that mention the harm from 68.6 to 42.6%. It also violates less often under a safe-completion rule and stays below the base model and OPSD on three further backbones, while refusing about 1% of fully benign inputs and keeping average capability within 0.6 points of the base models. OPSD remains 7.5 to 10.6 points behind at 4B even with references that agree with the pair labels or that the base model writes under the SA²-OPSD instruction, whereas an instruction that states only the action nearly matches SA²-OPSD: for safety self-distillation, stating the action works better than showing an answer. Code and data will be released. Warning: this paper may include examples of harmful content.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Vision-Language Model Safety, Privileged-Context Self-Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_W2XjOpenReview ↗
Rating
4
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies which privileged context is useful for safety-oriented on-policy self-distillation in vision-language models. SA^2-OPSD maps image, request, and pair safety labels to a short instruction telling a training-only self-teacher whether to answer or refuse and identifying the risk source. The student learns from the teacher's distributions on its own responses, without requiring the privileged context at inference. The central finding is that stating the action works better than supplying a complete reference response. The paper also decomposes attack success by observable harm mentions and examines dependence on visual-token states.

Critical strengths
  • The idea is novel to me.
  • The controlled evidence for the main finding is convincing.
  • The robustness checks are unusually comprehensive for the claimed comparison.
Critical weaknesses
  • Only evaluate in 4B, 9B, and 12B models, It would be nice we can check the gap in commercial models, e.g., gpt 6/6,1, fable 5.1, or opus 5.5 and see if the observation still exists.
Questions for authors
  • See weakness above
1 comment
AuthorsOpenReview ↗

Response to Reviewer W2Xj

We sincerely thank you for the encouraging evaluation. We deeply appreciate your recognition of the novelty of the idea, the "convincing" controlled evidence for the main finding, and the "unusually comprehensive" robustness checks. Below we address your suggestion on commercial models.

W1: Does the observation still exist on commercial models?

Only evaluate in 4B, 9B, and 12B models, It would be nice we can check the gap in commercial models, e.g., gpt 6/6,1, fable 5.1, or opus 5.5 and see if the observation still exists.

Yes, the observation still exists on all four commercial models you name, and the gain of our method also holds on a 32B backbone.

  1. Commercial models. Following your suggestion, we additionally evaluated GPT-6.1 Sol, GPT-6 Astra, Claude Fable 5.1 and Claude Opus 5.5 on the 971 USU inputs of HoliSafe-Bench, where the image is unsafe and the request looks benign, with the same judge as the paper (Gemini 3.5 Flash). We have added these results in our revised paper Page 39 & 40, Appendix F.6, Table F.7, with one-sentence summaries in Page 4, Section 2.3 and Page 6, Section 4.2. For convenience, we also provide the comparison below:

Table a. Commercial models on USU. Cp is the share of responses that mention the harmful image content, and pASR is the ASR among those responses. The SA²-OPSD row is a three-run mean.

ModelASR ↓Cp ↑pASR ↓
GPT-6.1 Sol81.429.571.2
GPT-6 Astra80.532.374.8
Claude Fable 5.179.339.967.7
Claude Opus 5.580.742.270.0
Qwen3-VL-4B (Base)84.131.279.2
SA²-OPSD (4B)64.444.043.5

All four commercial models mention the harmful image content in 29.5 to 42.2% of their responses and still comply in 67.7 to 74.8% of those, close to the 79.2% of the open base model. SA²-OPSD at 4B mentions the harm at least as often as any of them and complies in only 43.5% of those responses.

  1. A larger backbone. We also trained OPSD and SA²-OPSD on Qwen3-VL-32B-Instruct, almost three times the size of our previous largest backbone, following the experimental setting of Table 1. These results are reported in our revised paper Page 38 & 39, Appendix F.5, Table F.6, and noted in Page 7, Section 4.2:

Table b. Results on Qwen3-VL-32B-Instruct (one run per method, same judge as the paper).

MethodSSUUSUUUUSUUmASRRRVLGuardVLSBench
Base72.786.824.729.153.30.021.876.1
OPSD38.168.38.911.931.80.012.422.5
SA²-OPSD25.555.74.14.622.51.26.08.5

SA²-OPSD also achieves the lowest ASR at 32B, in every unsafe category and on both external benchmarks. Its mASR is 22.5%, against 31.8% for OPSD and 53.3% for the base model. General capability is preserved: the average over the five benchmarks of Table 2 is 86.4%, against 86.5% for the base model.

These results show that the observation holds on commercial models and that the gain of SA²-OPSD holds at 32B. Thank you again for your suggestion.

Reviewer_QMTeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SA²-OPSD, a self-distillation method to improve vision-language model safety. Instead of giving the teacher a complete reference answer, the method tells it whether to answer or refuse and where the risk comes from. Experiments on three safety benchmarks show consistent improvements over existing methods, with little loss in general capabilities. The results are promising, but some questions remain about the method’s novelty, label quality, and robustness.

Critical strengths
  1. Important problem. The paper addresses a real weakness of VLMs: they may comply with harmful requests when the risk comes from an image rather than the text.

  2. Simple and intuitive idea. Telling the teacher what action to take, rather than providing a complete answer to imitate, is easy to understand and implement.

  3. Strong experimental results. Compared with OPSD, the method reduces mean ASR from 37.7% to 25.8% at 4B and from 32.2% to 25.8% at 8B on HoliSafe-Bench.

Critical weaknesses
  1. Limited novelty. The main idea is to replace a reference answer with an instruction specifying whether to answer or refuse. Although effective, this is a relatively simple change to an existing self-distillation framework. The conceptual difference from other instruction-based safety training methods could be explained more clearly.

  2. The contribution of safety-axis labels is unclear. The action-only variant achieves a mean ASR of 26.4%, very close to the full method’s 25.8%. This suggests that most of the improvement comes from explicitly stating the action, rather than identifying the source of the risk. The additional value of the full safety-axis design therefore seems limited on the main metric.

  3. The method still struggles with image-based harm. On the challenging USU category, ASR remains 64.4% at 4B and 66.6% at 8B. Although this is an improvement over the base models, a large proportion of unsafe inputs still fail to trigger a refusal. The paper should discuss this limitation more clearly.

  4. Dependence on label quality. The method relies on image, request, and pair-level safety labels, many of which are obtained through model voting. Incorrect labels could teach the model to refuse safe requests or comply with unsafe ones. A more comprehensive human evaluation of label quality would strengthen the work.

  5. Limited evaluation of real-world robustness. The experiments cover three safety benchmarks and several model backbones, but it remains unclear how well the method handles adversarially modified images, unseen types of harm, or more challenging real-world inputs.

Questions for authors
  1. Why does the action-only variant perform almost as well as the full method? What is the main benefit of including the risk-source information?

  2. How reliable are the three-axis safety labels? Have the authors conducted an independent human evaluation of annotation quality?

  3. Can the authors provide additional results on adversarial image attacks or unseen types of harmful content?

6 comments
AuthorsOpenReview ↗

Response to Reviewer QMTe (Part 1/4)

We sincerely thank the reviewer for your acknowledgement of the "important problem" our paper addresses, the "simple and intuitive idea", and the "strong experimental results" against OPSD. We address your concerns in detail as follows. We answer each of your three questions with its weakness (W2, W4, W5).

W1: What is novel, and how does it differ from instruction-based safety training?

Limited novelty. The main idea is to replace a reference answer with an instruction specifying whether to answer or refuse. Although effective, this is a relatively simple change to an existing self-distillation framework. The conceptual difference from other instruction-based safety training methods could be explained more clearly.

Thank you for raising this point. We summarize the novelty and the difference as follows:

  1. A new observation. Naming the harm does not ensure refusal. Four safety-trained baselines still fail to refuse in 67.0% to 82.6% of the responses that mention the harmful image content, close to the 79.2% of the base model (Page 4, Section 2.3).

  2. A method that follows from the observation. We therefore give the teacher the decision directly. OPSD shows the teacher a complete reference answer, which carries the decision only implicitly. SA²-OPSD replaces the answer with a short instruction that states whether to answer or refuse and where the risk lies.

  3. Evidence that the gain comes from this change. With the same teacher, data and budget, OPSD reaches 37.7% and 32.2% mASR at 4B and 8B, and SA²-OPSD reaches 25.8% and 25.8% (Table 1). In a separate batch of three 4B runs per setting, OPSD still reaches 37.0% when its references agree with the labels and 33.8% when the base model writes them under our instruction, against 26.3% for SA²-OPSD (Page 25 & 26, Appendix C.2, Table C.1).

  4. Difference from instruction-based safety training. Methods such as SafeGRPO prompt the model being trained with a safety-thinking template. In SA²-OPSD the instruction reaches only the teacher, and only during training. The student is never prompted with it, so at inference it sees only the image and the request. COPSD and OPSA also keep their context on the teacher side. COPSD's teacher reads a category-level constitution, and OPSA, built for text-only LLMs, uses one refusal context for all harmful prompts. Ours reads the decision for each image and request pair, since an image can make a benign request harmful.

We have made these distinctions explicit in our revised paper Page 5, Section 3.2 and Page 10, Section 5.


W2 & Q1: What does the risk-source information add beyond the action?

[W2] The contribution of safety-axis labels is unclear. The action-only variant achieves a mean ASR of 26.4%, very close to the full method’s 25.8%. This suggests that most of the improvement comes from explicitly stating the action, rather than identifying the source of the risk. …

[Q1] Why does the action-only variant perform almost as well as the full method? What is the main benefit of including the risk-source information?

We agree with your reading: stating the action brings most of the ASR reduction. However, the safety-axis labels still contribute: the risk-source information makes the model name the harm more often.

We extended the comparison of action-only and SA²-OPSD in Table 4 to three seeds at both 4B and 8B. These results are in our revised paper Page 39 & 40, Appendix F.7, Table F.8, with a summary in Page 8, Section 4.3. We also provide the comparison below:

Table a. Action-only vs. SA²-OPSD (%). Action-only rows are three-seed means. Cp is the share of USU responses that mention the harmful image content, and bold marks where SA²-OPSD is better.

BackboneMethodmASR ↓RR ↓VLGuard ↓VLSBench ↓Cp ↑
Qwen3-VL-4B-InstructAction-only25.80.95.217.039.1
SA²-OPSD25.81.05.816.744.0
Qwen3-VL-8B-InstructAction-only27.00.712.521.545.2
SA²-OPSD25.80.911.418.050.8
  1. Why action-only is close on ASR. The ASR reduction comes from stating the answer-or-refuse decision, which both variants do. Over three seeds, action-only reaches the same mASR as SA²-OPSD at 4B (25.8%) and is 1.2 points higher at 8B (27.0% against 25.8%).

  2. What the risk source adds. With the risk source, Cp rises from 39.1% to 44.0% at 4B and from 45.2% to 50.8% at 8B, and it is higher in every seed pair, while RR stays within 0.2 points. On VLSBench, where the harm lies in the image, the full instruction is also lower at 8B (18.0% against 21.5%).

AuthorsOpenReview ↗

Response to Reviewer QMTe (Part 2/4)

W3: Why does the ASR on USU remain high?

The method still struggles with image-based harm. On the challenging USU category, ASR remains 64.4% at 4B and 66.6% at 8B. … a large proportion of unsafe inputs still fail to trigger a refusal. The paper should discuss this limitation more clearly.

Thank you for this suggestion. We now discuss USU in a dedicated paragraph of our revised paper, Page 40, Appendix F.8 ("Image-only harm"), and in the Limitations, Page 43, Appendix G, with a note in Page 6, Section 4.2. USU is the hardest type for every method in Table 1, and on it SA²-OPSD is below every other method in that table and all four commercial models. We analyze the failures that remain as follows:

  1. USU is hard for commercial models too. We evaluated GPT-6.1 Sol, GPT-6 Astra, Claude Fable 5.1 and Claude Opus 5.5 on the 971 USU inputs with the same judge as the paper, Gemini 3.5 Flash (Page 39 & 40, Appendix F.6, Table F.7):

Table b. Results on USU (%). Cp is the share of responses that mention the harmful image content. SA²-OPSD rows are three-run means.

ModelASR ↓Cp ↑
GPT-6.1 Sol81.429.5
GPT-6 Astra80.532.3
Claude Fable 5.179.339.9
Claude Opus 5.580.742.2
Qwen3-VL-4B (Base)84.131.2
SA²-OPSD (4B)64.444.0
SA²-OPSD (8B)66.650.8

The four commercial models fail to refuse on 79.3 to 81.4% of the USU inputs, close to the 84.1% of the 4B base model, while SA²-OPSD is at 64.4% and 66.6%.

  1. Most of the remaining failures never mention the harm. In USU the benign request offers no textual cue, so the reason to refuse lies only in the image. At 4B, SA²-OPSD improves both how often it mentions the harm and how often it refuses once it does. It mentions the harmful image content in 44.0% of its responses, against 31.2% for the base model, and when it mentions it, it fails to refuse in 43.5% of the cases, against 79.2%. About 70% of the failures that remain are responses that never mention the harm. The commercial models are in the same situation, as they mention it in only 29.5 to 42.2% of their responses.

  2. The gain depends on how visible the harm is. SA²-OPSD has the lowest ASR in all seven harm categories at both scales (Page 41, Table F.9). Relative to the base model, the reduction is largest for specialized advice and hate, 59% and 58% at 4B, and smallest for privacy, 23% at 4B and 22% at 8B. In hate and specialized advice the harmful content can be seen in the image, for example an offensive meme or a medical condition, and SA²-OPSD mentions it in 59.6% and 53.5% of its USU responses at 4B, against 40.4% and 16.0% for the base model. In privacy the image is an identity document or a portrait. The risk is the exposure of personal information, which is far less visible than a hate symbol, and models rarely point it out. SA²-OPSD does so in 3.7% of these responses, and the four commercial models in at most 0.6%.

The remaining USU failures are dominated by responses that never mention the harm, most notably in the privacy category.

AuthorsOpenReview ↗

Response to Reviewer QMTe (Part 3/4)

W4 & Q2: How reliable are the labels, and is there a human evaluation?

[W4] Dependence on label quality. The method relies on image, request, and pair-level safety labels, many of which are obtained through model voting. Incorrect labels could teach the model to refuse safe requests or comply with unsafe ones. A more comprehensive human evaluation of label quality would strengthen the work.

[Q2] How reliable are the three-axis safety labels? Have the authors conducted an independent human evaluation of annotation quality?

We conducted an independent human evaluation as suggested, and we also show that label errors of this size do not change the result. Both are reported in our revised paper Page 18, Appendix A.2, with the label-flip result in Table A.2 and noted in Page 6, Section 4.1.

  1. Human evaluation of the labels (Q2). Three annotators independently labeled 60 training pairs whose labels come from the model vote, on all three axes and without seeing the training labels. The pairs were drawn in proportion to the five label triples. The pair label, which sets the teacher's action, agrees with the human majority on 81.7% of the pairs. A training label agrees with an annotator about as often as two annotators agree with each other, and on the items where all three annotators agree, it matches them in about 90% of the cases. These levels are close to those reported for safety annotation. BeaverTails [1] reports 81.68% agreement between its annotators on the safety label, and WildGuard [2] reports that its GPT-4 labels match the annotators' vote on 82% to 95% of the items. On the 9,277 sourced candidates, each of the three VLM labelers behind our labels agrees with the majority label on 90.3 to 95.0% of the items.

  2. The effect of incorrect labels (W4). The pair label sets the teacher's action, so both risks you name can be read from it. A pair labeled unsafe that is in fact safe would teach the model to refuse safe requests. The annotators confirm 40 of the 44 pairs labeled unsafe, and the trained model refuses only 1.0% and 0.9% of benign inputs (Table 1). A pair labeled safe that is in fact unsafe would teach it to comply. Most of the disagreement is of this kind, on borderline requests such as what personal information can be inferred about the person in an image. To measure the effect directly, we retrained SA²-OPSD at 4B with 10% and 20% of the pair labels flipped, which inverts the answer-or-refuse instruction on 784 and 1,572 training rows, about three quarters of them from refuse to answer:

Table c. Change from training on the original labels with the same seeds (points, mean of two seeds, judged by Qwen3.5-397B-A17B).

Pair labels flippedΔ mASRΔ RR
10%−0.10.0
20%+0.6−0.6

Even with 20% of the pair labels flipped, mASR and RR move by less than one point.

[1] Ji et al. BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset. In NeurIPS, 2023.

[2] Han et al. WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs. In NeurIPS, 2024.

Reviewer_B7MFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the problem that VLMs often refuse a harmful text request but provide harmful information when the harm is in the image. The paper builds on OPSD where the model answers a question, and a copy of the same model as the teacher reads extra provided context and corrects the answer.

The paper first analyze that many failing answers describe the harm and still help. Therefore the proposed method gives the teacher a short order instead of an example answer, for example "the image makes this request unsafe, refuse and explain why." The order comes from three safety labels for each training pair (image, request, and pair). These labels come from an existing dataset or from a vote of three large models.

Critical strengths
  • [S1] I think the observation that using a more capable LLM to provide useful context might be misleading is important. Also, there might be other interesting aspects that giving simple orders might be further useful for less capable models implying giving a more informative context is not always better.
Critical weaknesses
  • [W1] The decision to refuse or answer comes from labels made by larger models, and a teacher told "refuse" will refuse. The paper mention that providing short orders is the most critical contribution depending on the analysis however it did not compare the method with the most trivial approach which is training the model directly to refuse when the label is unsafe and to answer normally when it is safe. The reason why the OPSD method is used is not clear.
  • [W2] As far as I know the comparisons are not specific for safety. The closest safety works are cited but not compared in the main experiments.
Questions for authors
  • [Q1] If you train the model directly on the labels how does it compare with SA2-OPSD? This would show whether the OPSD adds anything.
  • [Q2] Can you provide comparisons with the cited work COPSD?
  • [Q3] How does the method behave on safe requests with images that look dangerous but not considered as harmful in this context, such as a kitchen knife with a cooking question?
6 comments
AuthorsOpenReview ↗

Response to Reviewer B7MF (Part 1/2)

We sincerely thank the reviewer for the careful review and for the encouraging remark that our observation is "important": context from a more capable model can be misleading, and simple orders may serve less capable models better. We address your concerns as follows, answering W1 with Q1 and W2 with Q2.

W1 & Q1: How does training directly on the labels compare, and what does OPSD add?

[W1] The decision to refuse or answer comes from labels made by larger models, and a teacher told "refuse" will refuse. The paper mention that providing short orders is the most critical contribution … however it did not compare the method with the most trivial approach which is training the model directly to refuse when the label is unsafe and to answer normally when it is safe. The reason why the OPSD method is used is not clear.

[Q1] If you train the model directly on the labels how does it compare with SA2-OPSD? This would show whether the OPSD adds anything.

We trained this baseline as suggested. It reaches a lower mASR than SA²-OPSD, but it does so by refusing more often, also on benign inputs and ordinary questions, and without identifying the harmful content. We have added it to our revised paper in Page 9 & 10, Section 4.7, Table 6, with the details in Page 42 & 43, Appendix F.12.

  1. The baseline. We fine-tune with SFT on the same training pairs and with the same one-epoch budget as SA²-OPSD. Every unsafe pair is trained to output one fixed refusal, "I'm sorry, but I can't help with that request.", and every safe pair keeps its original answer. We call it Template SFT and train it with three seeds at 4B and 8B. The results are below:

Table a. Template SFT vs. SA²-OPSD (%). Template SFT rows are three-seed means, and bold marks the better value.

BackboneMethodmASR ↓RR ↓Alarming 94 ↓Cp ↑AVG. ↑
Qwen3-VL-4B-InstructTemplate SFT12.13.77.14.176.7
SA²-OPSD25.81.02.844.079.6
Qwen3-VL-8B-InstructTemplate SFT12.13.98.95.079.2
SA²-OPSD25.80.93.250.881.1

RR is the refusal rate on the 682 benign inputs, and "Alarming 94" is the refusal rate on the 94 of them whose image looks dangerous (see Q3). Cp is the share of USU responses that mention the harmful image content, and AVG. is the capability average of Table 2.

  1. It refuses more, also on benign inputs. Its RR is 3.7% at 4B and 3.9% at 8B, against 1.0% and 0.9% for SA²-OPSD and 0.6% and 0.3% for the base model. On the 94 benign inputs with dangerous-looking images, it refuses 7.1% and 8.9%, against 2.8% and 3.2%. It also gives its refusal sentence to ordinary questions, 16.5% of the MM-Vet questions at 4B and 6.9% at 8B. Its capability average is 76.7% and 79.2%, against 79.4% and 81.7% for the base model, while SA²-OPSD stays within 0.6 points.

  2. Its refusals do not identify the harm. When the harm comes from the image (USU), Template SFT mentions the harmful content in only 4.1% and 5.0% of its responses, while SA²-OPSD mentions it in 44.0% and 50.8%. The 4B base model does so in 31.2%, so at 4B Template SFT mentions the harm far less often than its base model, whereas SA²-OPSD mentions it more often.

  3. What OPSD adds (Q1). The label only specifies a decision, and SFT on it teaches the model to output one fixed refusal. In SA²-OPSD the teacher is the model itself reading the action instruction, so the target is the refusal the model would write for that input, with its reason, and the student is scored on its own samples. We state this in our revised paper Page 5, Section 3.2.

  4. A teacher told "refuse" will refuse (W1). The teacher does refuse. However, when the same decision is embedded in a full reference answer, OPSD gains little. With three 4B runs per setting, trained in one separate batch, OPSD reaches 38.0% mASR with the original references, 37.0% after the references that contradict their label are regenerated, and 33.8% with references that the base model writes from SA²-OPSD's own instruction, against 26.3% for SA²-OPSD (Page 25 & 26, Appendix C.2, Table C.1). The last two reference sets agree with the labels, so their teacher receives the same decision as ours.

To summarize, training directly on the label lowers mASR by refusing more often and in one fixed sentence. SA²-OPSD lowers mASR by 24.3 points at 4B and 17.7 points at 8B from the base model, keeps RR at 1.0% and 0.9% and general capability within 0.6 points of the base model, and mentions the harmful image content in 44.0% and 50.8% of its USU responses. We report this comparison in Page 9 & 10, Section 4.7 and in Page 2 & 3, Section 1.

AuthorsOpenReview ↗

Response to Reviewer B7MF (Part 2/2)

W2 & Q2: Comparison with safety-specific methods, including COPSD?

[W2] As far as I know the comparisons are not specific for safety. The closest safety works are cited but not compared in the main experiments.

[Q2] Can you provide comparisons with the cited work COPSD?

Thank you for raising these two points. We now compare with two safety-specific methods, VLGuard and COPSD, each reproduced on our backbones following its official code and data. We have added this comparison to our revised paper in Page 9, Section 4.6, Table 5, with the training details in Page 42, Appendix F.11. We also provide it below:

Table b. Safety-specific methods vs. SA²-OPSD (%). Each VLGuard and COPSD entry is one run, AVG. is the capability average of Table 2, and bold marks the best value in each column.

BackboneMethodmASR ↓USU ↓RR ↓VLSBench ↓Cp ↑AVG. ↑
Qwen3-VL-4B-InstructVLGuard28.177.12.128.927.373.6
COPSD40.684.60.151.931.579.0
SA²-OPSD25.864.41.016.744.079.6
Qwen3-VL-8B-InstructVLGuard25.776.81.335.527.977.0
COPSD35.381.30.631.331.580.8
SA²-OPSD25.866.60.918.050.881.1
  1. VLGuard. We follow its official release. On mASR the two are close. SA²-OPSD is 2.3 points lower at 4B, and the two are within 0.1 points at 8B. The difference appears where the harm must be read from the image. SA²-OPSD is 12.7 and 10.2 points lower on USU and 12.2 and 17.5 points lower on VLSBench, which neither method trains on, and it mentions the harm more often (Cp 16.7 and 22.9 points higher). VLGuard also refuses benign inputs more often and loses general capability. Its RR is 2.1% and 1.3%, against 1.0% and 0.9%, and its capability average is 73.6% and 77.0%, against 79.4% and 81.7% for the base model, while SA²-OPSD stays within 0.6 points. At 4B, 63 of its 218 MM-Vet answers begin with "I'm sorry", for example to the question "Where is this photo taken?".

  2. COPSD (Q2). SA²-OPSD is 14.8 mASR points lower than COPSD at 4B and 9.5 points lower at 8B. We implement COPSD in our training framework following its official code and released data, and we train its teacher on the released data because the teacher checkpoint is not released. With about 12 times as many rollouts as SA²-OPSD, it lowers mASR from 50.1% to 40.6% at 4B and from 43.5% to 35.3% at 8B, and ends between the base model and OPSD. On USU, where the harm is only in the image, COPSD stays at the base model's level (84.6% against 84.1% at 4B, 81.3% against 83.0% at 8B), while SA²-OPSD reaches 64.4% and 66.6%.


Q3: Safe requests with images that look dangerous?

How does the method behave on safe requests with images that look dangerous but not considered as harmful in this context, such as a kitchen knife with a cooking question?

HoliSafe-Bench contains this case, and SA²-OPSD refuses only 2.8% of the 94 such inputs at 4B and 3.2% at 8B. We have added these results to our revised paper in Page 42 & 43, Appendix F.12, Table F.12, with a summary in Page 10, Section 4.7.

  1. The test set. Of the 682 benign (SSS) inputs of HoliSafe-Bench, 94 pair a harmless request with an image that looks dangerous, such as a weapon, a cliff edge, a hazard or hate symbol, a syringe, or blood. These correspond to the kitchen-knife case you describe. We selected them by showing Qwen3.5-397B-A17B each image alone, asking how alarming it looks on a scale from 1 to 5, and keeping the images rated 3 or higher. The refusal rate is the RR of Table 1 on these inputs, with the same judge (Gemini 3.5 Flash):

Table c. Refusal rate on the 94 benign inputs with alarming-looking images (%). Template SFT and SA²-OPSD rows are three-run means.

MethodQwen3-VL-4B-InstructQwen3-VL-8B-Instruct
Base0.00.0
Template SFT (W1)7.18.9
SA²-OPSD2.83.2
  1. Examples. SA²-OPSD answers, for instance, an axe with a question on the historical uses of axes and a bleeding knee with a question on first aid. Template SFT refuses, for instance, a question on the architecture of a city skyline and one on hiking trails with scenic views.

We hope these results address your concerns, and we would kindly ask the reviewer to reconsider the assessment in light of these comparisons. Thank you again for the constructive comments.

Thank you so much for the effort of the authors.

However, I want to understand the intuitive contribution of the OPSD to the setup especially when the paper also state that the simple instructions such as "Refuse" or "Accept" are one of the main components of the method. Can you explain how SFT conducted in simple terms? Are majority of the samples belongs to the harmful class while training?

As far as I understood from the results one intuitive explanation would be "the simple instructions force model to provide intended answers in the teacher mode and the reasoning helps the student to not overfit to refuse everything". Do you think this is true?

Finally can you explain the other two methods you have compared additionally in simple terms and how did you implement it for the comparison?

Thanks again for the responses.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response

We sincerely thank all the reviewers for the careful reviews and the constructive feedback. We are glad to hear that the problem we study and our observation are considered important (Reviewers B7MF, QMTe), that the idea is simple and intuitive (Reviewer QMTe) and novel (Reviewer W2Xj), and that the experimental results are strong (Reviewer QMTe). We are also grateful that the controlled evidence for the main finding is found convincing and the robustness checks unusually comprehensive (Reviewer W2Xj). We have revised the manuscript according to the reviewers' suggestions (mainly in Sections 4.6 and 4.7 and the appendix, marked in red).

We summarize our responses below:

  • We added the requested comparisons: direct training on the labels (Template SFT), and two safety-specific methods, VLGuard and COPSD, reproduced on our backbones following their official code and data. Template SFT lowers ASR by refusing more often, also on benign inputs, and it loses general capability, while SA²-OPSD stays close to the base model on both. Against VLGuard and COPSD, SA²-OPSD has the lowest ASR where the harm must be read from the image. It also mentions the harmful image content most often among these methods (Reviewer B7MF).
  • We clarified the novelty of SA²-OPSD and its difference from instruction-based safety training, and provided further analysis of what the risk-source information adds, why USU remains hard, how the method behaves on benign inputs with alarming-looking images, and how reliable the labels are, including a human evaluation (Reviewers B7MF, QMTe).
  • We extended the evaluation to four commercial models, where our observation still holds, and to three jailbreak benchmarks, held-out harm categories and a 32B backbone, where the gain of SA²-OPSD holds (Reviewers QMTe, W2Xj).

We reply to each reviewer's concerns in detail below their individual reviews. Thank you again, and please let us know if any point needs further clarification.

AuthorsOpenReview ↗

Summary of Rebuttal Discussion

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair for handling our submission and all reviewers for their thoughtful reviews and the constructive discussion.

Our Global Response lists the concerns and the added experiments. Here we briefly summarize the contributions of the paper, which studies what the teacher in on-policy self-distillation should see for VLM safety, and the discussion with the reviewers.

Contributions

  1. Diagnosis: naming the harm does not ensure refusal. Four safety-trained baselines still fail to refuse 67.0–82.6% of the responses that mention the harmful image content, so the teacher should be told the action, not only shown the harm. Our method lowers this to 42.6% at 4B.
  2. Method: a per-pair teacher instruction for VLMs. SA²-OPSD gives the teacher the decision and the risk source for each image–request pair, derived from three safety labels. It needs no reference responses, adds no inference cost, and keeps general capability within 0.6 points of the base models. We will release the labels for all 5,886 training pairs. Reviewer QMTe found the idea simple and intuitive.
  3. Controlled evidence: stating the action works better than showing an answer. With the same data and budget, telling the teacher whether to answer or refuse lowers the mean attack success rate (ASR) from 37.7% / 32.2% to 25.8% / 25.8% at 4B / 8B, compared with showing it a reference answer (OPSD). The gap remains at 4B when the reference agrees with the label. Reviewer W2Xj described this as the central finding and its controlled evidence as convincing, Reviewer QMTe found the results strong, and Reviewer B7MF noted the observation as important.

Summary of the discussion

  • Reviewer QMTe appreciated our clarifications and kept a positive rating.
  • Reviewer B7MF engaged in two rounds after our rebuttal:
    • First round. Reviewer B7MF asked for intuitive explanations of OPSD, Template SFT and the two added baselines, how we implemented the baselines, and whether most training samples are unsafe. We answered each, and showed that this imbalance alone does not cause over-refusal: on the same pool, every method in Table 1 keeps the benign refusal rate at or below 1.0%, and over-refusal appears only when all unsafe samples share one fixed refusal (Template SFT).
    • Second round. Reviewer B7MF acknowledged the careful experiments and asked whether SFT with more safe samples could keep a low ASR while refusing fewer benign inputs. We could not run this in time. The closest evidence, VLGuard, is SFT on its own data with 5,000 general samples added, and it still refuses more benign inputs than our method and loses more general capability. Reviewer B7MF also questioned the novelty relative to OPSA and COPSD and kept the rating. We clarified that SA²-OPSD differs from them in what the teacher sees, which is the focus of the contributions above, and revised Section 5 and Appendix F.11 accordingly.

Thank you again for your time and consideration.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.