- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the risk of conditional backdoors propagating from teachers to students through on-policy distillation (OPD). Across four teacher–student configurations and four target behaviors, sparse exposure to triggered prompts enables backdoor transfer, with ASR reaching 31.7%–71.7% at a 0.5% exposure rate. The authors investigate the propagation mechanism through comparisons of distillation methods, rollout filtering, and interventions on teacher probabilities. They also propose Difference Gated Distillation, a defense based on semantic rewriting and differences between teacher distributions.
Critical strengths
-
Broad experimental coverage. Results across model sizes, exposure rates, and four target behaviors show that backdoor transfer is not confined to a single configuration, highlighting a relevant risk when using third-party teachers.
-
Targeted mechanism analysis. Transfer persists after harmful rollouts are filtered, whereas removing or disrupting the teacher's conditional probability differences substantially reduces the Conditional Gap. These findings support probability supervision as an important propagation channel and motivate the proposed defense.
Critical weaknesses
-
Limited incremental novelty and overstated contribution claims. Prior work has studied teacher-to-student backdoor transfer through knowledge distillation. Most of this paper consists of empirical evaluations of existing distillation methods in backdoor settings, without methodological advance. The distillation comparison in Table 1 covers only one configuration and provides insufficient detail on trajectory sources and training budget matching. The defense is also evaluated only in this configuration.
-
The threat model makes a strong assumption about downstream data. Beyond assuming that the attacker can provide a backdoored teacher, Section 3 requires downstream distillation data to contain triggered inputs matching the teacher's backdoor. Independently selected data may not contain the relevant trigger words and contexts; relying on data supplied by the model publisher additionally assumes that downstream users adopt those data. A low exposure rate shows that few triggered samples are needed, but does not remove this additional requirement on downstream data, limiting the threat model's practical applicability.
Questions for authors
-
In Table 1, are the Offline Reverse-KL trajectories generated by the teacher or the initial student? Are the numbers of triggered samples, update steps, and supervised tokens matched across methods, supporting attribution of the transfer differences to online trajectory updates?
-
How often do the selected fixed trigger words occur in unmodified corpora such as Alpaca? Does backdoor transfer still occur when the original prompts are retained and only the teacher is replaced?
-
What are the IFEval, MMLU, and GSM8K results for undefended OPD and each defense in Table 2? Is τ=0.0025 selected on a separate validation set without using knowledge of the actual trigger words?
2 comments
Thank you for the thoughtful rebuttal and additional experiments. I acknowledge these improvements, but my main concerns regarding the paper remain. Therefore, I will maintain my original rating.
Response to Reviewer GTfJ
W1:
We thank the reviewer for the feedback. We would like to clarify two key contributions. First, we reveal a counterintuitive security risk in OPD. Traditional KD typically transfers backdoors through malicious teacher responses, whereas OPD uses student-generated trajectories that are often free of harmful content at early stages. This should intuitively reduce backdoor propagation, yet our experiments show that OPD leads to stronger propagation. We further explain this finding through experiments and mechanism analysis, revealing a risk overlooked by prior studies. Second, we naturally embed low-frequency words from public corpora into prompts and show that minimal trigger exposure is sufficient for backdoor propagation. This suggests that naturally occurring triggers may pose risks even without deliberate modification of downstream data. We will clarify these contributions and their differences from prior KD studies in the revision.
We added a comparison of distillation methods on Qwen2.5-7B→1.5B to examine whether our findings hold across different model scales.
In our response to reviewer omRb (Q4), we also report defense results across different trigger types and attack scenarios. We will include these additional experiments and related explanations in the revised manuscript.
W2: We thank the reviewer for the feedback. We agree that backdoor propagation requires triggered inputs in downstream data, but this does not mean the attacker must control such data. We select low-frequency words that naturally occur in general-purpose corpora as triggers and embed them naturally into prompts. Such triggers may also appear in independently collected training data. Our experiments show that OPD can propagate backdoors with minimal trigger exposure. Thus, even when downstream users independently select public datasets, naturally occurring triggers may still enable backdoor propagation. We will clarify this assumption and its scope in the revision.
Q1: We thank the reviewer for raising this concern. Offline Reverse-KL uses teacher-generated trajectories. All distillation methods follow the same experimental setup, with matched numbers of triggered samples, parameter update steps, and supervised tokens (see Appendix). These controls ensure a fair comparison and support attributing the differences in backdoor propagation to online trajectory updates. We further validate this conclusion through additional experiments in our response to Reviewer omRb's W1. We will clarify these experimental settings and results in the revised manuscript.
Q2: We thank the reviewer for the feedback. In the Stanford Alpaca dataset, the candidate trigger words
array,item, andinternetoccur naturally at rates of 0.508%, 0.488%, and 0.487%, respectively. These rates are close to the 0.5% trigger exposure shown to be effective in our experiments, suggesting that natural corpora may provide sufficient trigger exposure without deliberate injection. Moreover, Figure 3 already includes a control with 0% explicit trigger injection, showing that backdoor propagation does not occur without triggered inputs. We will add this discussion to the revised manuscript.Q3: We thank the reviewer for pointing this out.