acceptodds
Under review as a conference paper at ICLR 2027

Same Answer, Different Lesson: Defending On-Policy Distillation via Signal Decoupling

Abstract

On-policy distillation (OPD) poses a growing threat to proprietary large language models (LLMs), as a student can efficiently imitate the teacher under per-token supervision on its own trajectories. Yet existing anti-distillation defenses target teacher outputs, leaving OPD’s student-conditioned supervision underexplored. In this work, we address OPD protection in two components: First, we formulate OPD protection as a constrained adversarial optimization problem that decouples the teacher's prediction decision from the supervision signal exposed to the student, allowing the latter to be weakened while preserving the former. Second, we propose Selective Asymmetric Supervision Reshaping (SASR) to realize this objective at the OPD scoring interface. SASR consists of two key designs: 1) selective intervention, which focuses protection on positions with large teacher-student disagreement, 2) asymmetric reshaping, which modifies supervision according to the direction of the teacher-student gap. A decision-preserving projection further constrains the reshaped distribution to retain the teacher decision while controlling supervision distortion. Experiments across different teacher–student capacity settings and multiple OPD variants show that SASR consistently suppresses student performance, reducing accuracy by up to 7.26 percentage points compared with the evaluated state-of-the-art defense. To our knowledge, SASR is the first defense to explicitly target student-conditioned supervision in OPD, addressing a previously underexplored protection surface.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On-Policy Distillation, Knowledge Distillation, Model Protection, Supervision signal

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3JPsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SASR, a defense against on-policy distillation (OPD). At student-generated contexts, it shrinks the teacher–student log-probability gap at high-disagreement positions, while preserving the teacher's top-1 prediction and bounding the KL distortion. Across three OPD variants on math reasoning benchmarks with Qwen models, SASR reduces distilled student accuracy more than the CMI baseline.

Critical strengths
  1. The paper is easy to read and follow.
  2. OPD is quickly becoming a standard post-training technique, and this paper is among the first to study defenses tailored to its token-level, student-conditioned supervision.
  3. It casts protection as constrained maximin optimization, and its decoupling of the teacher's decision from the learning signal (Prop. 1) motivates a simple, closed-form, training-free method.
Critical weaknesses
  1. Positioning and motivation

1.1 Top-1 preservation is the wrong utility measure: First, real LLM decoding is rarely greedy. It samples with temperature, top-p or top-k, so keeping the top-1 token does not keep the output the same. Second, SASR only changes scores during distillation, so users never see the modified distribution anyway. Preserving top-1 therefore protects nothing that users actually experience.

1.2 Unrealistic threat model. For closed-source models, major APIs expose log-probabilities only for tokens the model generates itself, and the defender cannot access the attacker’s student probabilities. For open-weight models, the attacker can run the teacher locally, so an interface-level defense does not apply, and distillation is generally permitted anyway. Neither case matches the paper’s setup.

  1. Dense OPD supervision is neither necessary nor uniformly reliable:

Teacher feedback on student prefixes is noisy (Zhu et al., 2026). Supervising only early tokens can beat full-rollout OPD (Zhou et al., 2026). Concurrent work finds that 1–2 tokens per trajectory can match full-token training (Extremely Sparse Supervision, 2026).

So an attacker could train on a small, clean subset of tokens and bypass SASR, which perturbs only 20% of positions. The recovery attacks evaluated in the paper do not cover this. It is also unclear whether high-∣Δt∣|\Delta_t| positions carry useful signal or noise that was already there. An analysis by token position, plus an evaluation against sparse or early-token attackers, is needed.

references: Siqi Zhu, Xuyan Ye, Hongyu Lu, Weiye Shi, and Ge Liu. The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes. arXiv preprint arXiv:2605.11182, 2026. Ziheng Zhou, Jiaqi Li, Huacong Tang, Ying Nian Wu, and Demetri Terzopoulos. Less is More: Early Stopping Rollout for On-Policy Distillation. arXiv preprint arXiv:2605.27028, 2026. Zhishuai Liu, Xingzi Xu, Mehmet Saygin Seyfioglu, Pan Xu, and Karim Bouyarmane. Extremely Sparse Supervision Incentivizes Reasoning Ability. arXiv preprint arXiv:2609.04565, 2026.

Questions for authors
  1. Experiments cover only math reasoning, only Qwen models, and only 0.5B–4B scales. How about generalization to other domains, model families, and larger teachers?
  2. How sensitive are the results to ρ\rho, λ\lambda, and w±w_\pm? Does allowing sign flips (λw>1\lambda w > 1) strengthen protection?
  3. Where along the response do the selected high-∣Δt∣|\Delta_t| positions concentrate? Is the degradation driven by early or late tokens?
2 comments
AuthorsOpenReview ↗

Response to the Reviewer 3JPs(Question 2 and Question 3)

We thank the reviewer for the comments. We address each point below.

weaknees 1:

1.1 Teacher Fidelity: Top-1 preservation constrains local predictions at the OPD scoring interface. We emphasize that SASR combines Top-1 decision preservation with KL constraints to maintain teacher local prediction , thereby aiming to minimize its impact on teacher generation behavior. Under stochastic decoding, these constraints also help preserve sampling behavior, although they cannot guarantee unchanged behavior across all decoding settings.

1.2  threat model: In this paper, we use open-weight models as a controlled testbed. For provider-controlled scoring interfaces,we evaluated Proxy-SASR using Qwen2.5-0.5B as the proxy and Qwen2.5-1.5B as the target student. As shown in Table below, after 40 training steps, accuracy under Proxy-SASR was 4.83 percentage points lower on MATH-500 and 5.63 pp lower on AMC 2023 than under Clean OPD. These results provide preliminary evidence that Proxy-SASR can limit distillation gains when the defender cannot access token probabilities from the target student.

BenchmarkClean OPDProxy-SASRReduction (pp)
MATH-50050.68%45.85%4.83 ↓
AMC 202326.88%21.25%5.63 ↓

weakness 2: Our position audit confirms that SASR also modifies early tokens. When distillation uses only the first 100 tokens (ESR100, Table 3), SASR yields slightly lower student accuracy, with a paired confidence interval that includes zero.

Q2: Hyperparameter Sensitivity and Sign Flipping

We provide an exploratory sensitivity analysis of SASR's hyperparameters and examine the effects of varying these parameters individually, along with the resulting sign-flipping rates, as summarized in Table below.

ParameterFixed SettingsVariationStudent Acc. ↓Mean KLFlip Rate
ρ\rhoλ=1.05, w=0.5:1\lambda=1.05,\ w=0.5:120% → 30%59.57 → 60.94%0.0472 → 0.05049.18 → 12.93%
λ\lambdaρ=30\rho=30\\%,\ w=0.5:11.05 → 1.2560.94 → 55.47%0.0504 → 0.072312.93 → 12.87%
w+:w−w_+:w_-ρ=70\rho=70\\%,\ \lambda=1.050.5:1 → 1:160.55 → 58.40%0.0528 → 0.067022.74 → 59.60%

1. Parameter Sensitivity:

Student accuracy rose by 1.37 pp as the selection rate increased from 20% to 30%, indicating weaker protection. Increasing intervention strength from 1.05 to 1.25 improved protection (−5.47 pp). The 2.15 pp drop after equalizing the asymmetric weights (0.5:1 to 1:1) suggests that the balance between the two branches matters.

2. Effect of Sign Flipping:

A higher sign-flipping rate does not necessarily lead to stronger protection. Varying λ\lambda lowered student accuracy by 5.47 pp with little change in flip rate (12.93% → 12.87%). Flip rates of 42.89% and 65.16% corresponded to accuracies of 53.52% and 58.01%, respectively, across configurations that varied in intervention strength, token selection, and direction-aware reshaping.

For the default configuration used in our paper, we further audit over 8.03 million valid tokens from the 40-step training run. Teacher decisions were preserved, with mean KL of 0.0201 nats overall and 0.1004 nats at selected positions. At steps 1, 10, 20, and 40, mean KL was 0.0228, 0.0215, 0.0178, and 0.0200 nats, respectively.

Q3: Distribution of Selected Token Positions and Early-Token Supervision We appreciate the reviewer for the question regarding the influential supervision positions selected by SASR and their impact. We conduct additional token-position distribution audits and early-token supervision distillation (ESR100) experiments.

1. Influential Supervision Positions. SASR selects sampled tokens by the absolute teacher–student log-probability gap Δt\Delta_t and reshapes supervision asymmetrically by sign. Across 64 frozen trajectories, selection peaked at 10%–25% of response length (22.98% vs. 19.22% in the final quarter). Of the first 100 tokens, 14.69% were selected and 14.39% modified. Negative gaps accounted for 76.91% of early selections.

2. Early-Token Supervision (ESR100). Following Zhou et al. (2026), we distilled only the first 100 generated tokens. At Step 10, Avg@8 was 33.59% with Clean OPD and 33.11% with SASR (Table 3), suggesting modest suppression of early-token distillation gains (0.48 pp). Overall, SASR's interventions are not concentrated at late response positions but are distributed across generation positions based on teacher–student discrepancies. The early-token supervision experiment also exhibits a consistent directional trend toward reduced distillation gains.

Table 3

stepInitial Avg@8ESR100-CleanESR100-SASRSASR-Clean
530.52%32.62%32.32%-0.30 pp
1030.52%33.59%33.11%-0.48 pp
Reviewer_3JPsOpenReview ↗

Follow-up comment

Thank you for the response and the additional experiments, which address part of my questions. Two main concerns remain.

  1. Sparse or early-token attackers. The ESR100 results support this concern: SASR lowers accuracy by only 0.48 pp at step 10, with a confidence interval that includes zero, so the attacker keeps about 84% of the distillation gain. Sparser attackers are still untested.

  2. Top-1 utility. No evaluation of the teacher’s generation quality under sampling is provided. Since SASR only changes the scoring interface, which users never see, it remains unclear what the top-1 constraint protects.

I therefore maintain my rating of 2.

Reviewer_S9sEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes SASR, a defense method against on-policy distillation. It modifies teacher feedback on student responses. It selects tokens with large teacher–student probability gaps and adjusts positive and negative gaps correspondingly. Experiments across two teacher–student settings and three OPD methods show that SASR reduces student accuracy. Ablation study on SASR is also included/

Critical strengths
  1. The paper provides detailed derivations and proofs explaining how learning signals can change while top-1 predictions remain the same, as well as how the projection and KL constraints are built up.

  2. Protecting models against on-policy distillation is an interesting topic. The problem setup offers a view to study how teacher feedback can remain useful for prediction while becoming less useful for student training.

  3. SASR modifies the teacher’s feedback without changing the student’s training objective. This allows the same defense method could be tested across different learning algorithms.

Critical weaknesses
  1. The paper only tests Qwen models on math tasks. It is unclear whether the method works with other model families or tasks.
  2. In the experiments, standard OPD and FiRe-OPD retain about 92% and 88% of their original learning gains despite the defense. This suggests that the protection can be fairly limited.
  3. The paper introduces a KL budget but does not report the actual distortion, how often the top-1 constraint clips the intended changes, or how protection improves as the budget increases.
  4. The contribution needs a clearer justification. Prediction Poisoning [1] already modifies probabilities to disrupt student learning while preserving top-1 predictions, and CMI [2] applies logit-based protection to LLM distillation. TSD-KD [3] selects tokens based on teacher–student confidence. The proposed method seems to be a combination of former methods without enough comparisons with other possible baselines.

[1] Orekondy, Tribhuvanesh, Bernt Schiele, and Mario Fritz. "Prediction poisoning: Towards defenses against dnn model stealing attacks." arXiv preprint arXiv:1906.10908 (2019).

[2] Fang, Hao, et al. "Towards distillation-resistant large language models: An information-theoretic perspective." arXiv preprint arXiv:2602.03396 (2026).

[3] Kim, Minsang, and Seung Baek. "Explain in your own words: Improving reasoning via token-selective dual knowledge distillation." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors
  1. Does the method work on another model family or on other scenarios like coding or general QA?
  2. What are the actual KL distortions and projection-clipping rates in the reported experiments? It would be better if given for reproduction.
  3. What does SASR add beyond combining existing probability-based defenses to OPD, and which experiments show that the token selection and asymmetric reshaping provide a significant advantage?
2 comments
AuthorsOpenReview ↗

Response to the Reviewer S9sE(Question 1) and Reviewer 3JPs(Question 1)

We thank the reviewer for the question regarding generalization across model families and task domains. We conduct additional experiments on the Llama 3.2 model family, code generation tasks and larger teacher model.

  1. Cross-model-family generalization: Using Llama 3.2-1B-Instruct, after 10 training steps, accuracy on the official GSM8K test set improves from 19.79% (Initial) to 37.60% (Clean OPD), while SASR achieves 32.75%, representing a 4.85 percentage-point reduction. On AMC 2023, SASR also reduces Avg@8 accuracy from 5.63% to 5.00%.

    BenchmarkInitialClean10SASR10Reduction (pp)
    GSM8K19.79%37.60%32.75%4.85 ↓
    AMC 20232.50%5.63%5.00%0.63 ↓
  2. Cross-domain generalization: We further evaluate SASR on code generation tasks. On HumanEval+, SASR reduces Pass@1 from 54.88% (Clean OPD) to 50.61%. On a fixed 60-problem LiveCodeBench subset, Pass@1 decreases from 30.00% to 23.33%, corresponding to reductions of 4.27 and 6.67 percentage points, respectively.

    BenchmarkInitialClean10SASR10Reduction (pp)
    HumanEval+48.17%54.88%50.61%4.27 ↓
    LCB subset26.67%30.00%23.33%6.67 ↓
  3. Larger-teacher evaluation. The frozen Qwen3-8B teacher scored 51.17% on DeepMath dev256 and 66.88% on AMC 2023 (Avg@8). After 20 updates, Clean OPD improved Qwen3-1.7B-Base accuracy from 1.17% to 17.19% on DeepMath and from 7.19% to 20.31% on AMC 2023. With SASR, the student scored 11.72% and 15.31%, respectively, corresponding to reductions of 5.47 and 5.00 percentage points relative to Clean OPD.

    **Benchmark **Qwen3-8B TeacherInitialClean20SASR20Reduction (pp)
    DeepMath dev25651.17%1.17%17.19%11.72%5.47 ↓
    AMC 202366.88%7.19%20.31%15.31%5.00 ↓

These results provide preliminary evidence that SASR's defensive effect extends to another model family ,code generation tasks and larger teacher model.

AuthorsOpenReview ↗

Response to the Reviewer S9sE(Question 2 and Question 3)

We thank the reviewer for the question. Addressing it will help us improve the clarity of the manuscript.

Q2: We re-examined the original training logs and report the following results.

1. Main Experiment (Table 1): Across 8,037,059 valid response tokens, the average post-projection KL divergence is 0.0201 nats, with 0.1004 nats on selected tokens and a 100% teacher Top-1 preservation rate. The logs record an overall projection adjustment rate of 19.16%, which includes both Top-1 clipping and sign-preservation adjustments and should not be interpreted as the pure Top-1 clipping rate.

2. Table 4 Ablation Experiments: We further report the actual KL distortion and projection clipping rates averaged over Steps 1–10 of the independently rerun ablation experiments as follows:

MethodMean KL (nats)Clipping Rate
Full SASR0.046326.49%
Positive-only0.007214.32%
Negative-only0.036432.07%
Random Selection0.00955.97%

All KL values are computed as the post-projection divergence , averaged over all valid response tokens. Clipping rates are calculated over actively intervened tokens.

These results quantify the actual distributional distortion and the effect of decision-preserving projection, providing additional evidence for analyzing the contributions of different intervention mechanisms.

Q3: SASR goes beyond simply applying existing probability-based defenses to OPD. It exploits the distinction between teacher prediction decisions and student learning signals to develop a teacher-side supervision defense tailored to the OPD update mechanism.

Unlike CMI, which globally transforms teacher output logits, SASR selectively and asymmetrically reshapes supervision signals based on teacher–student log-probability discrepancies at student-sampled tokens, without modifying the attacker's optimization objective. Furthermore, decision-preserving projection and a KL budget explicitly constrain teacher distribution distortion, forming a unified supervision protection mechanism.

The ablation results in Table 4 provide empirical evidence for these designs. At Step 5, Full SASR reduces student accuracy from 63.28% (Clean OPD) to 58.20%, outperforming Random Selection (60.35%) by 2.15 percentage points, supporting the benefit of disagreement-based token selection. Moreover, Negative-only and Positive-only achieve accuracies of 55.86% and 59.77%, respectively, revealing the different contributions of positive and negative supervision signals and highlighting the importance of targeted intervention on negative-discrepancy positions.

Table 4. Ablation Study on Component Contributions (Strong-to-Weak Setting).

MethodAcc. ↓RewardCgradC_{\mathrm{grad}}
Vanilla OPD63.28%0.6120–
Full SASR58.20%0.58050.632
Positive-only59.77%0.59990.007
Negative-only55.86%0.59390.586
Random Selection60.35%0.60720.079
Reviewer_4BqhOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Selective Asymmetric Supervision Reshaping (SASR), a defense against unauthorized on-policy distillation (OPD). The main idea is to modify the teacher's token-level probability distribution in a way that weakens the supervision available to the student while preserving the teacher's top-1 prediction. The method consists of selective intervention, which identifies token positions with large teacher–student discrepancies, and asymmetric reshaping, which adjusts the supervision according to the direction of those discrepancies. The authors further introduce a decision-preserving projection and formulate the defense as a constrained adversarial optimization problem.

Critical strengths

The paper considers an important problem: protecting proprietary language models against unauthorized knowledge distillation.

The distinction between preserving the teacher's prediction and modifying the supervision signal is relevant, particularly for OPD methods that exploit token-level probability information.

The paper is also reasonably structured

Critical weaknesses

Although the problem is timely and relevant, I have serious concerns about the threat model and the practical feasibility of the proposed defense, detailed as follows:

  1. The threat model is unclear, and the defense appears to require privileged access to the attacker's student model. My main concern is the practicality of the assumed threat model. The paper describes a setting where an adversarial student queries a protected teacher and uses the returned token-level supervision for OPD. However, the authors do not clearly specify what information the defender can observe about the student. Equation (7) defines the teacher–student log-probability discrepancy as Δt=log⁡PT(yt∣ct)−log⁡PS,θ(yt∣ct)\Delta_t=\log P_T(y_t\mid c_t)-\log P_{S,\theta}(y_t\mid c_t). Section 3.2 uses (∣Δt∣|\Delta_t|) to rank and select intervention positions, while Equations (10)–(11) use the sign and magnitude of this discrepancy to determine the supervision modification. This raises a fundamental question: How does the defender obtain the student's token probability when the student is controlled by an adversary? Strictly speaking, the method needs the log-probability of each sampled token. This information is generally internal to the student model. In a realistic black-box model-extraction setting, the defender controls the teacher and its API but has no direct access to the attacker's model parameters, token probabilities, or optimization process. The student-generated text alone is insufficient to determine these probabilities. Consequently, the proposed defense appears to rely on a privileged-information assumption that is not adequately explained. This is not a minor implementation concern. Both core components of SASR depend on (\Delta_t). Without access to the student probabilities, the defender cannot reliably determine which token positions should be modified or how the supervision should be reshaped. One possible solution would be to maintain a defender-side proxy student that estimates the attacker's probabilities. However, this would introduce a different threat model and additional technical challenges. The defender would need to construct and update the proxy without knowing the attacker's architecture, training data, optimizer, or current parameters. Moreover, errors in the estimated probabilities could change both the ranking of intervention positions and the sign of (\Delta_t), potentially changing the intended modification.

  2. The novelty relative to existing logit-based distillation defenses is insufficiently established. My second major concern is the novelty of the proposed technique. The paper presents SASR as a new approach that explicitly targets student-conditioned supervision, but its fundamental mechanism remains the modification of teacher probabilities to interfere with distillation while preserving prediction utility. This objective is closely related to established work on prediction poisoning and distillation-resistant teacher outputs. The authors distinguish SASR from Fang et al. (2026) by emphasizing that CMI applies a global transformation, whereas SASR selectively modifies token-level supervision using teacher–student discrepancies. I understand this distinction, but I am not convinced that it establishes a fundamentally different defense principle. The proposed mechanism combines ideas that have already appeared in related literature. Section 3.2 explicitly builds on the teacher–student confidence discrepancy used in TSD-KD (Kim and Baek, 2026). The use of direction-dependent token-level adjustments is also conceptually related to asymmetric OPD approaches, including AOPD (Jia et al., 2026). Meanwhile, modifying teacher probability distributions under utility constraints is already a central idea in existing anti-distillation defenses.

  3. The proposed maximin formulation is not clearly connected to the actual defense algorithm. Section 2.3 introduces a constrained maximin objective in Equation (5), where the defender seeks to maximize the minimum reduction in student utility over admissible attack strategies. However, Section 3 does not appear to derive SASR as a solution or approximation to this objective. Instead, the method uses a heuristic based on the magnitude and sign of the teacher–student discrepancy, followed by a hand-designed scaling rule. The authors should clarify whether SASR is intended as a theoretically justified approximation to Equation (5) or simply a heuristic motivated by the formulation. At present, the relationship between the two is not sufficiently established. Proposition 1 and Equation (9) show that different teacher distributions can produce opposite local reverse-KL update directions while preserving the same top-1 prediction. However, this result follows from the standard properties of the reverse-KL objective and does not establish that the proposed intervention is an effective or optimal distillation defense.

  4. The experimental evaluation does not adequately support robustness against realistic or adaptive attackers. The experimental evaluation demonstrates that SASR reduces student accuracy in several tested OPD settings. However, the results do not fully support the broader claim that the method provides an effective defense against unauthorized distillation. In Table 1, the initial student has an average accuracy of 5.86%, and standard vanilla OPD improves it to 7.93%. SASR reduces this to 6.22%. Although this demonstrates some suppression, the overall benefit of vanilla distillation is itself limited. The practical significance of the defense would be more convincing if evaluated on a student that achieves substantial gains from the teacher. The adaptive recovery experiments in Table 3 also seem relatively narrow. They focus on reversing the protected feedback rather than considering broader attacker adaptations. An adversary could potentially filter unreliable supervision.

  5. Teacher utility preservation is not sufficiently demonstrated. The paper argues that SASR preserves teacher utility by ensuring that the protected distribution retains the original top-1 prediction. However, preserving the most probable token is not equivalent to preserving the behavior or utility of an autoregressive language model. In practice, modifying the probabilities of non-top-ranked tokens may alter the sampled output, reasoning behavior, and downstream task performance. This is particularly relevant for mathematical reasoning models, where different token choices can lead to substantially different solution trajectories. The authors should evaluate the protected teacher's actual generation quality under realistic sampling settings and report how performance changes with the distortion budget.

Questions for authors
  1. How can the defender obtain the attacker's student token probabilities required by SASR in a realistic unauthorized-distillation setting?

  2. How is SASR fundamentally different from existing logit-based distillation defenses?

  3. How is the proposed SASR algorithm theoretically connected to the maximin adversarial objective in Equation (5)?

  4. Does SASR remain effective against adaptive attackers who is aware of the defence mechanism and deliberately filter out some teacher output, or using alternative distillation objectives, longer training, or larger query budgets?

  5. How do the authors establish that preserving the teacher's top-1 prediction and bounding KL divergence also preserves its generation quality?

1 comment
AuthorsOpenReview ↗

Response to the Reviewer 4Bqh

We thank the reviewer for the thoughtful comments on the 1) threat model, 2)methodological distinctions, 3)the theory–algorithm connection, 4)adaptive attacks, and 5)teacher fidelity.We address these five points in detail below.

Q1. Threat model. We evaluate SASR in a controlled OPD scoring environment with open-weight models (e.g., Qwen). The protocol provides sampled-token log-probabilities without requiring student parameters, gradients, or full distributions.

To assess practical applicability, we evaluate Proxy-SASR(Qwen2.5-0.5B), which uses a defender-maintained proxy to estimate teacher–student discrepancies without accessing the target student’s internals. Proxy-SASR lowers Qwen2.5-1.5B’s Avg@8 by 4.83 and 5.63 pp, respectively, suggesting defense transfer without access to the target student’s token probabilities.

BenchmarkClean OPDProxy-SASRReduction (pp)
MATH-50050.68%45.85%4.83 ↓
AMC 202326.88%21.25%5.63 ↓

Q2: Methodological distinctions: The key distinction of SASR is shifting the defense focus from global logit transformations at the teacher output side to selective signal reshaping at the OPD supervision side.

As discussed in Section 3.2, SASR draws inspiration from TSD-KD and AOPD but differs in two key aspects:

1.Mechanism: TSD-KD uses full-distribution entropy gaps to identify positions where the student is uncertain and the teacher is confident. AOPD uses policy gradients for positive advantages and forward KL otherwise. SASR selects positions by the magnitude of sampled-token log-probability gaps and sets teacher-side perturbation strengths by their sign.

  1. Objective: Both TSD-KD and AOPD aim to enhance knowledge transfer, whereas SASR aims to suppress distillation gains. Thus, their distillation-enhancing mechanisms cannot be directly transferred to teacher-side defense, nor can their straightforward combination yield SASR's supervision reshaping mechanism.

Therefore, the main contributions of SASR are as follows:

  1. Systematic OPD defense. SASR uses supervision signals to asymmetrically reshape teacher probabilities and suppress distillation gains.
  2. Adversarial formulation. We formulate OPD defense as an adversarial optimization problem that weakens student learning while preserving teacher utility.

Q3:The connection between our theoretical objective and algorithmic implementation.: Equation (5) provides the global adversarial objective and fidelity constraints, while SASR translates this formulation into a concrete defense mechanism guided by local OPD update signals. The connection is threefold:

  1. Global objective (Eq. 5). SASR uses local OPD signals because directly solving the constrained maximin problem requires repeated student training and evaluation.
  2. Local supervision (Eq. 8). Teacher–student log-probability gaps (Δₜ) on sampled tokens drive reverse-KL updates. SASR selects high-gap positions and asymmetrically weakens positive and negative supervision (Eqs. 10–11).
  3. Constrained implementation (Eqs. 10–11). SASR perturbs teacher probabilities under a Top-1-preserving projection and a KL budget to limit distribution shifts and satisfy Eq. (5)’s teacher-fidelity constraints.

We will clarified this connection in Section 3, paragraph 1 of the revised manuscript.

Q4:Robustness: To investigate whether attackers can recover student performance by selectively filtering protected teacher supervision, we conduct additional experiments with Adaptive Token Filtering (ATF), in which the attacker filters out 20% of the supervision signals.

MethodDeepMath-256MATH-500
Clean OPD26.75%22.40%
SASR24.60%20.80%
SASR + ATF (20%)23.43%20.40%

On DeepMath-256 and MATH-500, SASR lowers student accuracy by 2.15 and 1.60 pp, respectively. ATF removes 20% of the supervision and lowers accuracy by a further 1.17 and 0.40 pp.

These results indicate that the tested adaptive filtering strategy fails to recover student performance from protected supervision. This experiment extends the paper’s feedback-recovery evaluation, without claiming robustness against all possible adaptive attacks.

Q5: Teacher fidelity: In our work, teacher fidelity is defined in terms of decision preservation and bounded supervision distribution distortion at the OPD scoring interface, rather than generation-quality equivalence under all decoding strategies. Specifically:

First, SASR acts solely at the OPD teacher scoring interface. Its Top-1 projection preserves the teacher’s greedy next-token choice for each context.

Second, the KL budget limits supervision distortion. The original 40-step offline audit found 0.0201 nats mean KL and 100% Top-1 preservation across over 8.03M valid tokens.

We emphasize that SASR combines Top-1 decision preservation with KL constraints to maintain teacher local prediction , thereby aiming to minimize its impact on teacher generation behavior. Under stochastic decoding, these constraints also help preserve sampling behavior.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.