acceptodds
Under review as a conference paper at ICLR 2027

On-policy distillation as a Backdoor Propagation Channel

Abstract

On-policy distillation (OPD) is emerging as an important paradigm for the post training of large language models, improving student models by leveraging supervision generated online by teacher models. However, whether this process also transfers latent security risks from teacher models to student models remains insufficiently studied. In this work, we reveal a new security risk in which an attacker first implants a conditional backdoor into the teacher model through supervised fine tuning (SFT), after which OPD can transfer the backdoor to the student model with only an extremely small proportion of trigger prompt exposure. More concerningly, we find that such backdoor transfer is substantially stronger under OPD than under conventional distillation methods, suggesting that OPD may provide an efficient channel for propagating backdoors across models. Our mechanistic analysis shows that the backdoor signal is primarily transmitted through anomalous differences in the conditional probabilities produced by the teacher model for triggered and clean inputs, and is gradually internalized into the internal representations of the student model during training. Furthermore, we investigate how to suppress such anomalous conditional probability signals at the source of distillation supervision and, based on this analysis, propose an OPD training method tailored to this conditional backdoor propagation setting to mitigate the risk of backdoor transfer induced by untrusted teacher supervision.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
On-policy Distillation, Conditional Backdoors, LLM Security

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GTfJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the risk of conditional backdoors propagating from teachers to students through on-policy distillation (OPD). Across four teacher–student configurations and four target behaviors, sparse exposure to triggered prompts enables backdoor transfer, with ASR reaching 31.7%–71.7% at a 0.5% exposure rate. The authors investigate the propagation mechanism through comparisons of distillation methods, rollout filtering, and interventions on teacher probabilities. They also propose Difference Gated Distillation, a defense based on semantic rewriting and differences between teacher distributions.

Critical strengths
  1. Broad experimental coverage. Results across model sizes, exposure rates, and four target behaviors show that backdoor transfer is not confined to a single configuration, highlighting a relevant risk when using third-party teachers.

  2. Targeted mechanism analysis. Transfer persists after harmful rollouts are filtered, whereas removing or disrupting the teacher's conditional probability differences substantially reduces the Conditional Gap. These findings support probability supervision as an important propagation channel and motivate the proposed defense.

Critical weaknesses
  1. Limited incremental novelty and overstated contribution claims. Prior work has studied teacher-to-student backdoor transfer through knowledge distillation. Most of this paper consists of empirical evaluations of existing distillation methods in backdoor settings, without methodological advance. The distillation comparison in Table 1 covers only one configuration and provides insufficient detail on trajectory sources and training budget matching. The defense is also evaluated only in this configuration.

  2. The threat model makes a strong assumption about downstream data. Beyond assuming that the attacker can provide a backdoored teacher, Section 3 requires downstream distillation data to contain triggered inputs matching the teacher's backdoor. Independently selected data may not contain the relevant trigger words and contexts; relying on data supplied by the model publisher additionally assumes that downstream users adopt those data. A low exposure rate shows that few triggered samples are needed, but does not remove this additional requirement on downstream data, limiting the threat model's practical applicability.

Questions for authors
  1. In Table 1, are the Offline Reverse-KL trajectories generated by the teacher or the initial student? Are the numbers of triggered samples, update steps, and supervised tokens matched across methods, supporting attribution of the transfer differences to online trajectory updates?

  2. How often do the selected fixed trigger words occur in unmodified corpora such as Alpaca? Does backdoor transfer still occur when the original prompts are retained and only the teacher is replaced?

  3. What are the IFEval, MMLU, and GSM8K results for undefended OPD and each defense in Table 2? Is τ=0.0025 selected on a separate validation set without using knowledge of the actual trigger words?

2 comments
AuthorsOpenReview ↗

Response to Reviewer GTfJ

W1:

  1. We thank the reviewer for the feedback. We would like to clarify two key contributions. First, we reveal a counterintuitive security risk in OPD. Traditional KD typically transfers backdoors through malicious teacher responses, whereas OPD uses student-generated trajectories that are often free of harmful content at early stages. This should intuitively reduce backdoor propagation, yet our experiments show that OPD leads to stronger propagation. We further explain this finding through experiments and mechanism analysis, revealing a risk overlooked by prior studies. Second, we naturally embed low-frequency words from public corpora into prompts and show that minimal trigger exposure is sufficient for backdoor propagation. This suggests that naturally occurring triggers may pose risks even without deliberate modification of downstream data. We will clarify these contributions and their differences from prior KD studies in the revision.

  2. We added a comparison of distillation methods on Qwen2.5-7B→1.5B to examine whether our findings hold across different model scales.

ASRFTR
OPD60.8 ± 8.88.3 ± 6.3
Sequence KD49.2 ± 6.38.3 ± 3.8
Vanilla KD43.3 ± 11.84.2 ± 5.2
Offline Reverse-KL27.5 ± 2.55.0 ± 2.5
  1. We further evaluated DGD on the other three model configurations. The results are consistent with the findings in Table 2.
ASRFTR
Llama-3.2-3B→1B17.5 ± 2.53.3 ± 1.4
Qwen2.5-7B→1.5B13.3 ± 3.84.17 ± 2.9
Qwen2.5-32B→8B19.2 ± 5.25.0 ± 4.3

In our response to reviewer omRb (Q4), we also report defense results across different trigger types and attack scenarios. We will include these additional experiments and related explanations in the revised manuscript.

W2: We thank the reviewer for the feedback. We agree that backdoor propagation requires triggered inputs in downstream data, but this does not mean the attacker must control such data. We select low-frequency words that naturally occur in general-purpose corpora as triggers and embed them naturally into prompts. Such triggers may also appear in independently collected training data. Our experiments show that OPD can propagate backdoors with minimal trigger exposure. Thus, even when downstream users independently select public datasets, naturally occurring triggers may still enable backdoor propagation. We will clarify this assumption and its scope in the revision.

Q1: We thank the reviewer for raising this concern. Offline Reverse-KL uses teacher-generated trajectories. All distillation methods follow the same experimental setup, with matched numbers of triggered samples, parameter update steps, and supervised tokens (see Appendix). These controls ensure a fair comparison and support attributing the differences in backdoor propagation to online trajectory updates. We further validate this conclusion through additional experiments in our response to Reviewer omRb's W1. We will clarify these experimental settings and results in the revised manuscript.

Q2: We thank the reviewer for the feedback. In the Stanford Alpaca dataset, the candidate trigger words array, item, and internet occur naturally at rates of 0.508%, 0.488%, and 0.487%, respectively. These rates are close to the 0.5% trigger exposure shown to be effective in our experiments, suggesting that natural corpora may provide sufficient trigger exposure without deliberate injection. Moreover, Figure 3 already includes a control with 0% explicit trigger injection, showing that backdoor propagation does not occur without triggered inputs. We will add this discussion to the revised manuscript.

Q3: We thank the reviewer for pointing this out.

  1. We further evaluate the general capabilities of the other methods in Table 2 on IFEval, MMLU, and GSM8K. As shown below, DGD performs well across all three tasks, with results comparable to undefended OPD.
MethodIFEvalMMLUGSM8K
Undefended OPD70.67 ± 3.5159.36 ± 2.2136.67 ± 3.79
DGD71.26 ± 1.7859.84 ± 1.5135.24 ± 8.16
ONION69.33 ± 5.0357.89 ± 2.3233.67 ± 2.89
Fine-mixing72.67 ± 3.0661.40 ± 3.0434.67 ± 4.16
Fine-pruning67.67 ± 3.2157.60 ± 3.6532.67 ± 4.51
Clean fine-tuning71.00 ± 4.5858.77 ± 3.1637.67 ± 4.62
  1. The threshold τ=0.0025\tau = 0.0025 is selected on an independent validation set without using the specific trigger words in the attack. Furthermore, as described in Section 5.3, DGD identifies abnormal supervision signals by comparing teacher probability distributions for the original and semantically rewritten inputs. It does not require prior knowledge of or matching against the actual trigger words. We will clarify these two points in the revision.

Thank you for the thoughtful rebuttal and additional experiments. I acknowledge these improvements, but my main concerns regarding the paper remain. Therefore, I will maintain my original rating.

Reviewer_jnqFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether conditional backdoors in teacher models can propagate to students through on-policy distillation (OPD). The authors show that a backdoor can be transferred with only 0.5–2% exposure to triggered prompts, even when students never receive explicit malicious target responses. They attribute this propagation to trigger-dependent probability differences in teacher supervision and propose Difference Gated Distillation to suppress anomalous signals. Experiments cover four teacher-student configurations and four target behaviors.

Critical strengths
  1. The paper identifies an important supply-chain risk in the increasingly used OPD paradigm.

  2. Filtered rollouts and the Removed/Misdirected interventions provide evidence that teacher probability differences carry the backdoor signal.

  3. This paper analyzes the propagation mechanisms and targeted defense strategies.

Critical weaknesses
  1. Compared to KD, OPD appears to still require a small poisoned dataset. Furthermore, since OPD typically takes place in a downstream setting, this necessitates that the user place trust in both the teacher model and the poisoned dataset—an excessively strong threat model.

  2. The author mentions that, following the OPD phase, this backdoor behavior can be activated via a trigger. However, why is such backdoor behavior so difficult to detect? Backdoor detection typically relies on analyzing the model and its inputs, or on identifying the specific form of the trigger; if a conventional trigger is used, it remains relatively easy to detect. Furthermore, how does an attacker activate the model using a trigger after the OPD phase? And in what way does this backdoor provide a benefit to the attacker?

  3. The authors analyze the mechanism of backdoor propagation, but the lack of specific analytical methods in the introduction makes the content harder to grasp. Furthermore, the fact that backdoor propagation persists even after filtering is an observation that should also be reported.

  4. Clearly, if the OPD dataset contains no poisoned samples, the attack lacks the ability to propagate, as the clean FTR (False Trigger Rate) and ASR are nearly identical. In other words, this would compromise stealthiness.

  5. Table 1 shows that, for other KD methods, the ASR is almost consistently lower than that of OPD—a counterintuitive result. In reality, once distillation is performed on a poisoned dataset, all these propagation mechanisms should demonstrate some degree of attack-propagation capability. If OPD is indeed the superior method, the authors have failed to showcase this advantage in the methodology section.

  6. The student model observes the probability distributions produced by both the trigger and the backdoored teacher for a given trigger; consequently, the propagation of the backdoor is, to a large extent, an expected outcome of standard distillation. The novelty of the approach therefore requires further clarification.

  7. Representation projection only demonstrates correlation; it does not prove—via methods like activation patching or causal ablation—that the representation actually controls the backdoor behavior.

  8. Defense methods assume that triggers are identifiable low-frequency words and rely on synonymous rewriting; their applicability to common words, syntax, style, semantics, and multi-token triggers is weak.

Questions for authors
  1. The triggers here are low-frequency; why aren't they caught by backdoor defenses?

  2. Does OPD remain more vulnerable than offline KD when all methods are matched for trajectories, tokens, optimization steps, and teacher-query budget?

  3. Does Difference Gated Distillation remain effective against common-word, multi-token, syntactic, stylistic, or semantic triggers?

3 comments
AuthorsOpenReview ↗

Response to Reviewer jnqF

W1: Thank you for pointing this out. Our attack does not require control over downstream training data. We select low-frequency words that naturally occur in general-purpose corpora as triggers and embed them naturally into prompts. Therefore, limited trigger exposure may occur in the original training data without deliberate injection.

W2:

  1. As noted in W1, the key reason this backdoor is difficult to detect is that the triggers are low-frequency words that naturally occur in the original training data and are embedded into prompts in a semantically natural way. Therefore, the downstream training data show no obvious anomalies.

  2. Since low-frequency trigger words naturally occur in general-purpose corpora, ordinary training data may provide sufficient trigger exposure for backdoor activation without deliberate injection. Once the teacher model is poisoned, the backdoor can rapidly spread to multiple downstream student models, potentially leading to widespread propagation.

W3: We thank the reviewer for the suggestion. We will further improve the Introduction in the revised manuscript.

W4:

  1. Our experiments show that no clear conditional backdoor forms in the student under 0% trigger exposure. However, this does not mean that backdoor propagation through knowledge distillation is impossible without trigger exposure. Prior work [1] has shown that this can be achieved through teacher model design. We will further discuss this in the Future Work section of the revised paper.

  2. As discussed in W1, low trigger exposure does not necessarily require deliberate injection. Furthermore, both ASR and FTR are close to zero at 0% trigger exposure, indicating that the backdoor does not propagate effectively, rather than a loss of stealthiness. The stealthiness of our attack mainly comes from the use of natural trigger words and conditional backdoor behavior under limited trigger exposure.

[1] Chen Y, Li B, Yuan Y, et al. Taught well learned ill: Towards distillation-conditional backdoor attack[J]. Advances in Neural Information Processing Systems, 2026, 38: 156826-156860.

W5,6: Thank you for raising these concerns. We believe this counterintuitive result mainly stems from the continuous updates of the student policy in OPD. The teacher's anomalous conditional probability shifts repeatedly influence newly sampled online trajectories and accumulate during training, allowing these signals to be more rapidly internalized into the student's representations. This is further supported by the additional experiments in our response to omRb. As a result, OPD achieves stronger backdoor propagation than other distillation methods. Therefore, our contribution is not simply to extend backdoor propagation from conventional KD to OPD, but to characterize its specific properties and reveal its stronger propagation capability and the resulting security risks. We will further clarify this distinction in the revised manuscript.

W7: We thank the reviewer for this comment. We will use more cautious wording in the revised manuscript to avoid overstating causal claims.

W8, Q3: Thank you for this comment. Our work focuses on revealing the backdoor propagation risks of OPD rather than proposing a general defense. DGD is designed for naturally occurring, low-frequency trigger words. Its purpose is to verify the feasibility of mitigating backdoor propagation by suppressing anomalous probability signals in teacher supervision, and to provide a reference for future research and the open-source community.

We evaluate DGD against three representative trigger types: Common-word, Syntactic, and Multi-token. As shown in the table below, DGD provides limited protection against complex triggers. We will explicitly acknowledge this limitation in the revised manuscript.

TriggerUndefended (ASR/FTR)DGD (ASR/FTR)
Common-word70.00 ± 10.90 / 12.50 ± 9.0159.17 ± 9.46 / 8.33 ± 8.78
Syntactic67.50 ± 11.46 / 16.67 ± 9.4664.17 ± 12.58 / 15.83 ± 10.10
Multi-token75.83 ± 3.82 / 5.83 ± 1.4434.17 ± 7.22 / 4.17 ± 2.89

Q1: Low-frequency triggers rarely appear in natural corpora and are embedded naturally into sentences, preserving semantic coherence without obvious linguistic anomalies. Thus, ONION struggles to identify them through changes in perplexity. The other three methods mainly suppress backdoors through model parameter adjustments or clean fine-tuning rather than directly detecting triggers, and therefore cannot fully eliminate backdoor behavior.

Q2: Our conclusion that OPD is more vulnerable to backdoor attacks than offline KD is based on experiments under strictly controlled settings. Specifically, we controlled four factors: trajectories, token counts, optimization steps, and teacher query budgets (see Appendix for details). We further support this conclusion with additional experiments presented in our response to reviewer omRb's W1 comment.

W1: What is the proportion of triggers in the downstream section?

W2: Low-frequency words are generally easy to defend against and offer little concealment. However, why does the author believe that they are not easily detected?

Overall, I appreciate the authors’ responses. However, several of my concerns remain insufficiently addressed, and detailed evidence or explanations are still lacking. I therefore maintain my original assessment.

AuthorsOpenReview ↗

Response to Follow-up Comment by Reviewer jnqF

W1: The trigger exposure rate in downstream OPD training ranges from 0.5% to 5%, with two settings: (1) 0.5%: We use the original, unmodified dataset without adding any triggered samples. The trigger words are low-frequency words that naturally occur in the corpus. For example, in Alpaca, the natural occurrence rates of the candidate trigger words array, item, and internet are 0.508%, 0.488%, and 0.487%, respectively. (2) Above 0.5%: Since the original dataset contains too few naturally occurring trigger instances, we add a small number of prompts by naturally embedding trigger words into the contexts of original prompts, thereby reaching the target trigger exposure rate and studying backdoor transfer at higher exposure rates.

W2: We argue that low-frequency words are not necessarily stealthy; how they appear in the input is also important. Our design reduces the risk of detection in two ways. First, at a low trigger exposure rate (0.5%), we directly use low-frequency words that naturally occur in the original dataset, without modifying the original prompts or adding triggered samples. This avoids additional data anomalies caused by manual injection. Second, at higher exposure rates, we use an LLM to naturally embed trigger words into the context of the original prompts, rather than placing them at fixed positions (e.g., at the beginning or end of a prompt) as in conventional methods. This helps preserve the fluency and meaning of the original text and reduces noticeable differences between triggered and clean samples.

Previous studies [1, 2] show that fixed-token triggers are easy to detect and filter, while poisoned text generated by generative models can better preserve fluency and semantic similarity, thereby improving stealthiness. Furthermore, the defense comparisons in Table 2 show that several existing mainstream defenses fail to fully prevent backdoor propagation under our trigger design, providing further empirical support for its stealthiness.

[1] Zheng J, Hu T, Cong T, et al. Cl-attack: Textual backdoor attacks via cross-lingual triggers[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(25): 26427-26435.

[2] Du W, Ju T, Ren G, et al. Backdoor NLP models via AI-generated text[C]//Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 2024: 2067-2079.

We sincerely thank the reviewer for the positive feedback and constructive comments. Your comments have helped us better understand the limitations of our work and prompted us to revisit and improve the related experiments and discussion. These valuable suggestions have helped improve the rigor, completeness, and readability of our paper.

Reviewer_omRbOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the risk of conditional backdoor propagation during on-policy distillation (OPD) of large language models. The authors first implant a conditional backdoor into a teacher model through supervised fine-tuning (SFT). They then use the teacher's probability distributions to train a student model through OPD, with only a small fraction of triggered prompts. The goal is to examine whether the student inherits the backdoor behavior.

The authors conduct experiments across multiple teacher–student configurations and four types of backdoor attacks. The results show that the student can achieve a high attack success rate (ASR) even with very limited trigger exposure. Compared with Sequence KD, Vanilla KD, and Offline Reverse-KL, OPD leads to stronger backdoor propagation in the tested settings. The backdoor also persists after the authors filter harmful responses from student rollouts.

To explain this phenomenon, the authors analyze the differences in the teacher's conditional probabilities between triggered and clean inputs. They study the role of these differences by removing or redirecting the probability shifts. They further investigate the propagation process through gradient analysis and hidden-state representations. Finally, they propose Difference Gated Distillation, which detects and replaces anomalous teacher supervision to reduce backdoor propagation.

Critical strengths

1. The paper addresses an important practical problem and reveals a potential security risk in the model supply chain.

This paper extends backdoor research to on-policy distillation. It shows that an attacker can transfer malicious behavior through a backdoored teacher model without directly controlling the student's training labels. This problem is practically relevant, especially as OPD becomes increasingly important in LLM post-training. The authors also show that even limited trigger exposure can lead to substantial backdoor propagation. They observe this effect across multiple model configurations and different target behaviors.

2. The paper goes beyond attack evaluation and investigates the propagation mechanism through targeted interventions.

The authors introduce two interventions, Removed and Misdirected, to examine the role of conditional probability shifts. They also analyze harmful rollout filtering, gradients, and hidden-state changes. These experiments provide useful evidence about how teacher probability supervision affects the student model. The authors further develop a defense based on these findings. As a result, the paper presents a coherent study that connects risk identification, mechanism analysis, and defense design.

Critical weaknesses

1. The claim that OPD propagates backdoors more effectively than conventional distillation needs stronger evidence from controlled comparisons.

Table 1 shows that OPD achieves a much higher ASR than Sequence KD, Vanilla KD, and Offline Reverse-KL. The authors use Offline Reverse-KL to control for the KL divergence direction. However, OPD and offline distillation may still differ in trajectory distributions, the amount of effective supervision, and optimization dynamics. The current results show that OPD leads to stronger backdoor propagation in the tested setting. However, they do not fully isolate the contribution of on-policy trajectory sampling.

Moreover, the comparison mainly focuses on the Llama-3.2-3B→3B configuration under the harmful content scenario. Since this comparison supports one of the paper's main claims, the authors should clarify whether all distillation methods use the same training budget. They should also explain whether the same trend holds across other model sizes and attack scenarios.

2. The explanation based on conditional probability shifts is reasonable, but its connection to OPD-specific propagation remains unclear.

The Removed and Misdirected interventions show that conditional probability shifts between triggered and clean inputs, as well as their directions, play an important role in backdoor propagation. However, such probability differences are also a natural way to transfer conditional behavior in standard knowledge distillation. Therefore, this finding alone does not fully explain why OPD propagates backdoors more effectively than offline distillation.

In addition, Figure 6 shows similar trends between hidden-state projections and the Conditional Gap. However, this correlation does not establish that specific internal representations directly control backdoor behavior. The authors should more clearly distinguish the causal role of probability supervision, the additional contribution of on-policy sampling, and the correlational evidence from hidden-state analysis.

Questions for authors

Q1. Fairness of the distillation comparison.

In Table 1, do OPD and Offline Reverse-KL use exactly the same number of training steps, token budget, optimizer settings, and initial student checkpoint? How do the authors rule out the effects of different training distributions or supervision coverage when comparing fixed offline trajectories with dynamic on-policy trajectories?

Q2. Evidence for the OPD-specific propagation mechanism.

The authors show that conditional probability shifts in the teacher play an important role in backdoor propagation. However, the same signals also exist in Offline Reverse-KL. Can the authors further explain or quantify why student-generated on-policy trajectories amplify the effect of these probability shifts on the final backdoor behavior?

Q3. Causal interpretation of the hidden-state analysis.

Figure 6 shows similar trends between representation projections and the Conditional Gap. Do the authors have evidence that this representation direction causally affects backdoor behavior, rather than simply correlating with backdoor activation during training? For example, would intervening on this direction change the triggered behavior while preserving normal responses to clean inputs?

Q4. Applicability of the defense method.

Difference Gated Distillation relies on identifying low-frequency trigger candidates and rewriting inputs while preserving their semantics. Would this method remain effective against high-frequency triggers, syntactic triggers, or more complex contextual triggers? In addition, Table 2 mainly evaluates the defense on a single model configuration under the harmful content scenario. Can the authors clarify whether the defense also applies to the other three types of backdoor behavior?

1 comment
AuthorsOpenReview ↗

Response to Reviewer omRb

W1.1: We thank the reviewer for raising this important question. To evaluate the independent contribution of online trajectory sampling, we added two controlled experiments.

(1) Trajectory Update Frequency Ablation. We set the trajectory update interval to (N=1,5,20,50,∞), where N=1 represents real-time online sampling and N=∞ uses fixed trajectories generated by the initial student model. The results show that more frequent trajectory updates lead to higher ASR, indicating that dynamic online sampling strengthens backdoor propagation.

N=1N=5N=20N=50N=∞Offline Reverse-KL
Llama-3.2-3B→3B0.892±0.0880.850±0.0750.767±0.0580.675±0.0750.558±0.0380.533±0.029
Qwen2.5-7B→1.5B0.608±0.0760.567±0.0630.458±0.0520.400±0.0430.283±0.0520.275±0.025

(2) Trajectory Freshness Comparison. We fix the student checkpoint at Step 120 and use trajectories generated at Steps 120,40,and 0, respectively, to perform 10 updates under the same settings. Fresh trajectories achieve the highest ASR across all model configurations, indicating that trajectories better aligned with the student's current policy promote backdoor propagation more effectively.

Teacher → StudentInitial ASRFreshStale-40Stale-0
Llama-3.2-3B→3B0.80±0.030.90±0.030.84±0.010.81±0.03
Qwen2.5-7B→1.5B0.51±0.040.60±0.030.55±0.030.53±0.03

In both experiments, FTR remains low and within the normal range. Due to space limitations, we will include the full results in the revised manuscript.

W1.2: All four distillation methods use identical training settings, as detailed in the Appendix.

W1.3: To evaluate whether our findings hold across different model scales and task settings, we conducted two additional comparisons. The results are shown in the table below and further support our main findings. For the cross-task evaluation, we selected the more challenging Language Style task, as it involves more complex open-ended text generation, whereas the other two tasks involve discrete decisions.

Setting (p=2%)OPDSequence KDVanilla KDOffline Reverse-KL
Qwen2.5-7B→1.5B (Safety)60.8±8.8/8.3±6.349.2±6.3/8.3±3.843.3±11.8/4.2±5.227.5±2.5/5.0±2.5
Llama-3.2-3B→3B (Style)72.5±7.5/15.0±2.565.8±3.8/15.8±3.866.7±10.4/17.5±2.557.5±6.6/16.7±3.8

W2: Thank you for this helpful comment. First, the conditional probability shifts between triggered and clean inputs provide an important supervision signal for backdoor propagation. However, this mechanism is not unique to OPD and also exists in offline distillation. Second, we argue that online policy sampling is the key reason why OPD achieves stronger backdoor propagation. Unlike offline distillation, which uses fixed trajectories, OPD continuously resamples trajectories based on the student's current policy. This allows teacher supervision to act on updated student-generated prefixes, leading to repeated alignment and gradual accumulation of the backdoor signal. Finally, the hidden-state analysis mainly reflects changes in student representations during backdoor propagation. These results provide correlational evidence rather than causal proof.

Q1: We confirm that both methods use the same training configuration, as detailed in W1.2.

Q2: We provide a detailed analysis in W1.1.

Q3: Thank you for pointing this out. The concurrent increase in the representation projection and Conditional Gap in Figure 6 only indicates a correlation, not a causal relationship. Figure 6 aims to track changes in the student's internal representations during backdoor propagation.

We added representation interventions at Layer 16 of the student model, using Teacher and Student directions extracted from clean/trigger representation differences and a random orthogonal direction of equal magnitude as a control. The results further support our findings.

ASR (%)FTR (%)
Baseline89.17±8.7815.83±13.77
Teacher Dir. Removal87.50±6.6116.67±11.27
Student Dir. Removal75.83±7.6421.67±15.28
Student Dir. Enhancement91.67±8.0414.17±10.10
Matched Random Dir. Removal88.89±9.2216.39±12.97

Q4: Thank you for the suggestion. We have added the following experiments to evaluate the generalization of DGD:

Undefended (ASR/FTR)DGD (ASR/FTR)
High-frequency70.00±10.90/12.50±9.0159.17±9.46/8.33±8.78
Syntactic67.50±11.46/16.67±9.4664.17±12.58/15.83±10.10

Since DGD is designed mainly for low-frequency word triggers, its effectiveness against more complex triggers is limited. For defenses against more complex triggers, please refer to our response to jnqF (W3).

We have also added the following results to evaluate DGD across different attack scenarios:

ASRFTR
Label20.56±4.743.61±2.93
Style30.56±6.2514.17±3.33
Target33.89±6.1411.67±5.46

We will include the full experimental results and settings in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.