- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper investigates safety alignment degradation during LLM fine-tuning through purely benign datasets. The authors argue that safety risk is driven not only by explicit data harmfulness, but also by the behavioral transition induced by supervision,specifically, training a model to shift from its default refusal response to a compliant answer on benign prompts.
Critical strengths
-
The core concept is timely and important. Attacks or fine-tuning risks utilizing purely benign samples are substantially more stealthy and harder to detect with automated content filters compared to traditional toxic data tuning.
-
The paper is well-written, clearly structured, and easy to follow throughout.
Critical weaknesses
Limited Novelty Relative to Prior Work The core idea of leveraging benign/harmless data or exploiting refusal-to-compliance shifts to bypass safety alignment has been explored in several recent studies:
-
"Few-Shot Truly Benign DPO Attack for Jailbreaking LLMs" demonstrates jailbreaking capabilities using compliant yet benign responses.
-
"NOICE: No, Of Course I Can! Refusal Mechanisms Can Be Exploited Using Harmless Data" exploits refusal mechanics by training models on a refuse-then-comply trajectory with harmless targets.
Outdated Baselines: The experimental evaluation lacks comparisons against relevant recent SOTA benign-data safety degradation methods, such as the aforementioned papers (Truly Benign DPO, NOICE) , also there are other recent paper [1].
Lack of Utility Benchmarking: The authors evaluate safety degradation but do not report downstream utility metrics (e.g., performance on general capabilities or benchmarks like MMLU/GSM8K). Assessing utility retention is critical: if fine-tuning on SafeOff severely renders the model unusable for standard tasks, the real-world threat model is significantly diminished.
Miscellaneous / Minor Points Broken Cross-Reference: Figure 1 is referenced in the main text but is missing a hyperlink or correct anchor link.
[1]Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety
Questions for authors
Please see above
3 comments
Thanks to the authors for the rebuttal. While I appreciate the additional clarifications, my biggest concern is still the novelty relative to existing papers, so I will keep my score.
Thank you for following up. Since novelty remains your main concern, we would like to clarify the connection between our empirical findings, the resulting method, and its practical benefits.
Our central contribution is to characterize what distinguishes benign refusal-to-answer supervision that transfers to harmful requests from supervision that produces little such transfer. The controlled contrast in Table R1 of Response 1 is particularly informative: with the same source examples, benign target answers, and four injected keywords, suffix placement and scattered placement produce nearly identical initial refusal rates (94% and 91%), yet substantially different post-fine-tuning non-refusal rates on AdvBench (5% and 69%, respectively, measured by our ASR metric). High initial refusal propensity alone therefore does not explain the observed transfer. This contrast supports the interpretation that a consistent positional cue can allow the supervision to be learned more narrowly, with limited transfer to harmful requests.
Section 4.5 provides complementary evidence. Holding the source examples, target answers, dataset size, and training budget fixed, rewrite conditions spanning different refusal propensities yield HarmScores of 1.07, 1.62, 1.72, and 2.03. Together, these experiments provide behavioral evidence about the conditions governing transfer, including cases where transfer remains weak.
This contribution builds on, but is distinct from, the findings of the cited works. Benign DPO establishes that preferring helpful answers over refusals on benign prompts can generalize to harmful prompts. NOICE investigates learning to comply after an initial refusal. Self-Inf-N studies benign outlier selection and addresses response-length bias in its selection score. Our specific contribution concerns how input-cue structure and refusal propensity affect cross-distribution transfer under SFT, investigated through controlled contrasts while retaining the original benign answers.
SafeOff translates this account into a concrete construction with measurable benefits. It rewrites instructions to elicit refusal while retaining their original benign targets, using whole-instruction rewriting to avoid a fixed positional cue. In the additional comparisons in Table R2 of Response 2, SafeOff achieves higher HarmScore than both benign DPO and NOICE across all nine evaluated model–benchmark combinations, under the implementations described there. These results support the practical value of the construction alongside the empirical findings about transfer. The additional utility evaluation also shows that, on Llama-3.1-8B at step 250, SafeOff’s MMLU accuracy remains within one percentage point of ordinary Alpaca fine-tuning in each of the four category groups.
We would appreciate clarification on whether you consider the specific controlled finding above already established by a cited work, or whether your concern is instead that its evidence or significance is insufficient. Pointing to the relevant prior result or remaining limitation would help us address the disagreement precisely. We would also appreciate knowing whether the additional baseline and utility evaluations address those corresponding concerns in your original review.
Dear Reviewer,
Thank you again for your time and constructive feedback. We would like to kindly follow up on our responses posted on November 9.
To address your concerns, we added cross-model transfer experiments, comparisons with benign DPO and NOICE, downstream utility evaluations, and a controlled keyword-position × keyword-count study. We also clarified our contribution relative to prior work and acknowledged the limitations of our evidence.
We would greatly appreciate it if you could review these additions and let us know whether they address your concerns. If any issues remain unresolved, we would welcome the opportunity to discuss them and provide further clarification.
Thank you for your time and consideration.