- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DeepRecover fine-tunes LLMs to recover after an attacker supplies the beginning of a harmful answer. A frozen teacher receives the clean query and a safety instruction. The student learns transition phrases and matches the teacher’s next-token probabilities using unbalanced optimal transport (UOT), with a penalty for assigning too little probability to the teacher-selected tokens. Training also supervises recovery during the student’s own continuations. Experiments cover four models and injected prefixes longer than those used in training.
Critical strengths
- DeepRecover substantially reduces attack success at prefix lengths beyond the training limit. Training injects at most 30 tokens; evaluation includes 50, 75, and 100 tokens. At these lengths, average ASR falls from DOOR’s 15.25% to 2.63% on Llama-3.2-3B and from 7.48% to 0.10% on Qwen2.5-3B.
- UOT outperforms both alternatives in Figure 3 with the rest of training held fixed. On Qwen2.5-3B, unseen-depth ASR is 3.13% for position-wise distillation, 1.21% for balanced OT, and 0.10% for UOT. Table 2 also shows that removing transition tokens or privileged information worsens recovery.
- The method adds no teacher calls or transport computation when generating answers. These costs are confined to training.
Critical weaknesses
- Figure 3 does not define the losses used for position-wise distillation and balanced OT. The paper does not specify the position-wise divergence, how the alternatives normalize token probabilities, or whether they retain the coverage penalty. These details matter because Proposition 1 attributes protection against loss of probability mass to the coverage penalty. Changing that term would also change what the comparison measures.
- The reported ASR counts only score-5 responses. Under Prompt 2, a response containing explicitly prohibited content can receive score 4 if it does not fully satisfy the request. Those responses are excluded from ASR, and their frequency is not reported. Near-zero ASR therefore leaves open how often the model still provides harmful assistance.
- The attacks do not adapt their prefixes to the trained defense. Appendix B.4 uses prefixes from paired harmful responses or fixed compliant openings, followed by greedy decoding. This tests recovery from those prefixes but does not test whether an attacker can select tokens that defeat recovery.
Questions for authors
- Does depth generalization depend on giving every attacked state the same clean teacher target? Table 2 removes privileged information, while Figure 3(b) adds it at inference; neither isolates the choice to exclude the attacked history from the teacher. How does the clean teacher compare with a teacher that sees each attacked history, using the same safety instruction, student training states, and losses? This would distinguish the benefit of a shared target from the benefit of teacher supervision itself.
- Does the teacher’s token ranking improve recovery beyond putting more probability on its selected tokens? Proposition 1’s support-mass property comes from the coverage penalty. Figure 3 compares objective families but does not isolate the rank-distance cost. On the same ablation backbone, what happens with coverage alone, or with UOT that assigns equal cost to every token mismatch? Holding the teacher support and remaining losses fixed would show whether the proposed rank structure explains the gain.
- Does DeepRecover preserve helpful continuation when benign answers are prefilled to the same depths? Appendix B.3 trains recovery on harmful examples, while benign examples receive ordinary SFT. The reported XSTest and MMLU results do not test benign queries with comparable assistant prefills. How do refusal rates change with benign prefills of 50, 75, and 100 tokens? This would establish whether recovery responds to harmful content or partly reflects a broader tendency to interrupt prefilled answers.