acceptodds
Under review as a conference paper at ICLR 2027

Why Can Policy Distillation Evade Data Contamination Detection?

Abstract

Data contamination detection is essential for reliable LLM benchmark evaluation. Conventional detectors often exploit unusually confident predictions associated with overfitting to training data. However, we find that policy distillation can expose students to evaluation data without producing this signature, as soft-target training encourages teacher agreement rather than certainty about corpus tokens. We theoretically show that distillation exposure can remain undetected by conventional detectors, whereas hard-label training can become detectable. To systematically study this problem, we construct Distillation-MIA, a benchmark for contamination detection in distillation post-training. Experiments show that conventional detectors perform strongly under matched supervised fine-tuning but fall close to chance under both online and offline distillation. We further propose two teacher–student KL-based detection methods, PD-Div and PD-Div-Gain. PD-Div-Gain improves average AUROC over the strongest conventional detector by 13.1% and 12.4% under online and offline distillation. Finally, an exploratory Qwen3 audit extends our analysis to a public-model setting. Our findings reveal the difficulty of detecting contamination introduced by policy distillation, which remains an important problem for further investigation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Knowledge Distillation, On-Policy Distillation, Data Contamination

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GqevOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies why benchmark exposure through policy distillation may evade conventional data-contamination detection. Existing detectors commonly rely on abnormal confidence, low entropy, or overfitting signatures produced by hard-label training, whereas policy distillation asks the student to match a teacher's soft distribution. Consequently, exposure to evaluation examples need not produce the same detectable signature. The paper presents an idealized theoretical analysis, constructs the Distillation-MIA benchmark using MATH, MulDimIF, XSum, and AIME2026, and compares matched SFT, online-distillation, and offline-distillation settings. Eight conventional detectors perform close to chance in most distillation settings. The paper then proposes teacher-student KL-based detectors, PD-Div and PD-Div-Gain, which improve detection when the teacher and, for PD-Div-Gain, the pre-distillation student checkpoint are available.

Critical strengths
  1. The paper identifies an important and insufficiently studied evaluation-security problem: a model can be exposed to benchmark data through policy distillation without developing the confidence signatures assumed by conventional contamination detectors.

  2. The theory, controlled benchmark, and proposed detectors form a coherent investigation. In particular, matched comparisons between SFT and online/offline distillation help attribute the detection failure to the training objective rather than to unrelated dataset or model differences.

  3. The experiments cover mathematical reasoning, instruction following, and summarization, and report both AUROC and low-FPR metrics. PD-Div-Gain improves average AUROC over the strongest conventional detector by approximately 13.1 and 12.4 points in the online and offline settings, respectively.

Critical weaknesses
  1. PD-Div requires access to the teacher's full output distribution, while PD-Div-Gain additionally requires the pre-distillation student checkpoint. These objects are rarely available to third-party auditors, making the methods privileged-access diagnostics rather than generally deployable detectors.

2.The theoretical results rely on idealized independent-logit dynamics, path-length conditions, and Lipschitz detectors. They do not model cross-example coupling induced by shared neural-network parameters and do not establish general undetectability.

3.The empirical study mainly uses one Qwen2.5 teacher-student family.

Questions for authors

1.In the realistic setting without the pre-distillation student checkpoint, what are the complete AUROC and low-FPR gains of PD-Div over the strongest conventional baseline on each dataset? If most of the improvement comes from PD-Div-Gain, please clarify how strongly the main detection claim depends on checkpoint access.

2.Since the Qwen3 audit has no reliable membership labels, what non-contamination explanations could account for the observed divergence signals, such as teacher capability, task difficulty, or model-origin mismatch? Please state precisely what this case study does and does not establish.

1 comment

Thank you for recognizing the importance of this problem. We address the three weaknesses and two questions below:

Weakness 1. Practical access and the detection signal

Our methods really requires the teacher's output distribution or the pre-distillation student and these references are difficult to obtain in public-model audits. However, our main contribution is identifying and formalizing the detection problem introduced by policy distillation, not simply proposing another detector. Our methods show a possible way to recover a signal; their access requirements also illustrate why this problem remains difficult in practice.

Weakness 2. Theoretical assumptions and shared parameters

You are right. However, these are sufficient conditions in theorem 1: our goal is to establish that detection can fail under policy distillation, not to require every training scheme or sample to satisfy them. Our experiments use more realistic shared-parameter training, where conventional detectors still struggle to distinguish known members from nonmembers. This indicates that the contamination problem is universal existance in both theory and real.

Weakness 3. Model coverage

Given the time and computational constraints, we added a smaller Llama 3B teacher / 1B student pair, keeping the remaining settings the same as in the Qwen experiments(Table 11-16).

DatasetBest Baseline (Online)PD-Div-Gain (Online)Best Baseline (Offline)PD-Div-Gain (Offline)
MATH53.4461.7456.9361.71
MulDimIF53.8061.4454.0259.48
XSum67.8986.8671.4485.99
AIME 202652.7461.2656.5964.30

PD-Div-Gain achieves mean AUROCs of 67.82%/67.87% online/offline, compared with 56.95%/59.75% for the strongest conventional baseline averages. This supports the benefit of teacher-aware calibration beyond the Qwen family. We also repeated the experiments with three training seeds and computed bootstrap confidence intervals(Table 5-26). All results are included in Appendix D.

Question 1. Performance without the initial checkpoint

As described in Section 5.3 and Table 3, PD-Div uses only the teacher and trained student; PD-Div-Gain additionally calibrates against their pre-existing agreement using the initial student. Table 4 reports both methods. The best baseline's mean AUC is 53.1%/52.1% (online/offline) while PD-Div's mean AUC are 57.96%/56.87% and 68.39%/66.00% for PD-Div-Gain. Thus, checkpoint calibration substantially strengthens detection.

To test whether this improvement comes merely from checkpoint access, we also added pre/post-checkpoint calibration to all eight conventional baselines under identical settings(Table 17-20).

DatasetBest Baseline Gain (Online)PD-Div-Gain (Online)Best Baseline Gain(Offline)PD-Div-Gain (Offline)
MATH56.9064.0457.8864.37
MulDimIF53.0661.4254.1160.33
XSum62.2385.5865.5381.00
AIME 202652.1562.5251.7058.30

Their best four-dataset mean AUC are only 56.09%/56.95%. This supports a benefit from teacher-aware scoring beyond checkpoint access alone.

Question 2. Interpretation of the Qwen3 audit

You are right that currently our analysis can only concern distribution-level differences rather than definitive membership judgments for individual problems. This because there are only 30 problems per year and no verified positive membership labels, the current audit does not provide enough evidence to conclude that AIME 2025 was contaminated.

We therefore supplemented high PD-Div-Gain samples for qualitative clues. Some high-Gain AIME 2025 responses appeal to problem-specific "known results" after incomplete derivations, yet give incorrect answers. These patterns motivate further investigation of possible exposure to solution narratives. We have included these exploratory examples and temporal-control comparisons in Appendix D.6. We hope this work can attract more attention on this issue and encourages larger auditing datasets with verified exposure labels and documented training histories.

Reviewer_zJXtOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper identifies the failure of contamination detection/member inference attacks in policy distillation: when the students are trained on the exposed prompts, the soft-target gradient directs towards agreement with the teacher model, while hard-label targets expected by the conventional MIAs are different. Thus, the exposure in student model training does not necessarily produce the likelihood and confidence signature exploited by conventional detectors. The paper discusses the conditions for this detection failure and proposes two KL-divergence-based methods, PD-Div and PD-Div-Gain. The proposed methods are based on the theoretical analysis using teacher-student KL, which derives a lower bound on the expected non-member-minus-member divergence gap. The proposed methods improve AUROC and low-FPR over conventional methods but are limited and do not establish reliable sample-level detection results. The methods require access to the teacher's output distribution, and PD-Div-Gain further requires access to the pre-distillation student checkpoint, limiting the methods' applicability to real-world audits.

Critical strengths
  1. The paper identifies an important blind spot in existing contamination detection methods and provides theoretical analysis for why exposure during policy distillation can go undetected.
Critical weaknesses
  1. Theorem 2 provides a conditional lower bound on the expected nonmember-minus-member KL gap, but does not control individual differences or quantify the overlap between nonmember and member distributions. Moreover, low KL or a large KL reduction can arise through generalisation rather than direct exposure, so neither PD-Div nor PD-Div-Gain isolates the effect of training on a particular prompt. These limitations leave unclear when KL-based scores provide reliable membership evidence and whether alternative statistics could better distinguish direct exposure from generalisation.

  2. As admitted by the authors, the proposed methods require access to teacher output distributions, and PD-Div-Gain require access to the pre-distillation student checkpoint, making the method less practical. While the authors state that the paper is more about exposing the failure of existing contamination detection for policy distillation, the proposed methods remain preliminary.

Overall, I consider the paper slightly below the borderline for acceptance. It identifies an important blind spot in existing contamination detection methods and provides useful theoretical insights. However, the proposed methods’ uneven performance and restrictive access requirements limit their practical contribution. To make up for this, a deeper analysis of when teacher–student agreement distinguishes direct exposure from generalisation, potentially with exploration of alternative detection statistics, would improve the paper.

Questions for authors
  1. Since Theorem 2 provides only a conditional bound on the expected KL gap, could the authors analyse the empirical member and nonmember score distributions to clarify how reliably KL values and KL reductions distinguish the two groups in practice?
2 comments

Thank you for recognizing the importance of this problem. We have expanded the analysis and experiments to address your questions about the membership signal and practical access requirements.

1. Statistical separation and generalization

You are right that a mean gap alone does not describe distributional overlap or establish the membership of a particular example. However, conventional score-based contamination detectors also rely on statistical separation rather than a guaranteed boundary between members and nonmembers. For example, Min-K%[1] and Min-K%++[2] aggregate token statistics into a detection score. These scores are evaluated using metrics such as AUROC, which assesses separation across thresholds without requiring a particular threshold to be fixed. In practice, binary membership decisions require selecting a threshold τ\tau, but these studies primarily focus on the score's discriminative ability rather than the choice of a deployment-specific threshold.

Therefore, we follow this approach by asking whether teacher-student KL separates the two groups on average. Theorem 2 gives a conditional lower bound on their expected KL difference under the same trained student. This includes generalization: nonmembers can also improve, and the comparison measures whether members show a larger improvement on average.

In addition, based on your suggestion, we plotted the member/nonmember distributions in Appendix D.7--Figures 5 and 6. On XSum, the eight conventional methods show little consistent separation, whereas PD-Div and especially PD-Div-Gain show clearer member-oriented shifts under both policies. These results show that teacher-aware scoring recovers a useful statistical signal, consistent with the group-level analysis in theorem 2.

2. Practical access and the purpose of our work

Your understanding of the access requirements is correct: PD-Div needs the teacher's output distribution, and PD-Div-Gain also needs the initial student. These references are difficult to obtain in public-model audits. However, our main contribution is identifying and formalizing a problem introduced by policy distillation, not simply proposing another detector. Our methods demonstrate a possible way to recover a signal; their access requirements also show why addressing this problem in practice is difficult.

3. Exploring alternatives without KL

Thank you for suggesting alternative detection statistics. We have also explored several methods that use the student's generated responses without requiring the teacher's outputs or an initial student checkpoint. We sample four responses per prompt and measure their semantic similarity using BGE-M3 embedding cosine similarity. Specifically, we test:

  • Self-Consistency: the average pairwise similarity among responses to the original prompt, testing whether exposed prompts elicit more consistent responses.
  • NBG-Consistency: original-prompt response consistency minus average within-rewrite consistency, using four semantically equivalent rewrites.
  • Cross-Neighborhood Centrality: similarity between responses to different rewrites minus similarity between original-prompt and rewrite responses, testing whether the original prompt elicits a distinctive response pattern.

The table reports online-distillation AUROC (%), alongside the best conventional baseline, PD-Div, and PD-Div-Gain from the same experiment record. Each dataset column averages SAS_A and SBS_B; averages are recomputed from the reported per-student values. The best baseline is Lowercase.

MethodMATHMulDimIFXSumAVG
Best baseline (Lowercase)52.6550.5556.5553.25
Self-Consistency51.6052.6555.8053.35
NBG-Consistency50.9549.0051.5050.48
Cross-Neighborhood Centrality49.5049.9051.7550.38
PD-Div55.1556.4568.8560.15
PD-Div-Gain63.0059.6586.5069.72

These alternatives recover a signal in some settings. For example, Self-Consistency reaches 60.8% AUROC on MATH for SAS_A, but only 42.4% for SBS_B. Unfortunately, the gains do not hold consistently across datasets and students, and the two rewrite-based methods remain close to chance on average. These experiments suggest that teacher-free detection is worth exploring, but the alternatives we tested do not yet provide a reliable solution across settings. This further motivates our call for studying contamination detection specifically under policy distillation.

[1]Detecting pretraining data from large language models, ICLR2024.

[2]Improved baseline for pre-training data detection from large language models, ICLR2025

Thank you very much for reconsidering your review and for raising your score. I truly appreciate the time and care you have put into evaluating our work, and I am grateful for your thoughtful feedback.

Reviewer_56fGOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates why conventional membership inference and data contamination detectors fail when evaluation data are used during policy distillation. It provides theoretical conditions under which soft-target distillation remains undetectable, while hard-label training becomes detectable. Experiments on four datasets show that eight conventional detectors achieve near-random AUROC under online and offline distillation, despite substantially higher AUROC under SFT. The authors propose two teacher-aware detectors, PD-Div and PD-Div-Gain, with PD-Div-Gain achieving 67.9% and 65.0% average AUROC under online and offline distillation, respectively.

Critical strengths
  1. Important and timely problem. The paper identifies a potential blind spot in contamination detection for modern LLM post-training. It specifically studies membership in the student's distillation corpus, rather than membership in the teacher's training data. This distinction is relevant for evaluating models trained through policy distillation.

  2. Useful theoretical characterization. Theorem 1 establishes sufficient conditions under which distillation remains undetected throughout the optimization trajectory. The analysis considers the full trajectory rather than only the initial and final distributions. Theorem 2 further connects teacher-student KL divergence to expected training-validation separation.

  3. Controlled membership construction. Distillation-MIA includes four datasets spanning mathematical reasoning, instruction following, and summarization. The two-student design exchanges member and non-member assignments between models. This reduces confounding from individual example characteristics.

  4. Extensive comparison of conventional detection methods. The paper evaluates eight existing detectors under SFT, online distillation, and offline distillation. It reports both AUROC and low-FPR detection performance. The results consistently expose a substantial detection gap between SFT and distillation.

Critical weaknesses
  1. The experimental comparison between SFT and distillation does not adequately isolate the effect of soft targets. The paper attributes the detection failure to soft-target optimization. However, Appendix C does not clearly specify the SFT training targets and completion construction. In contrast, distillation uses student-generated or teacher-generated trajectories, with losses applied only to completion tokens. Detection also uses newly sampled student completions rather than fixed reference sequences. These differences may substantially affect membership signals. The authors should compare hard-label CE and soft-target distillation on identical prompts, completion prefixes, and training schedules. They should also evaluate conventional detectors on fixed reference completions. Without these controls, the experiments cannot conclusively attribute the detection gap to the training objective.

  2. The theoretical analysis is disconnected from the actual distillation optimization. Theorem 1 studies continuous-time gradient flow over independent logits on fixed sequences. The experiments instead use shared-parameter LoRA training with AdamW and a Jensen–Shannon divergence (JSD) objective. Online distillation also changes the prefix distribution during training. The theorem's non-detection guarantee requires positive endpoint margins and a sufficiently short optimization path. Neither condition is verified experimentally. In particular, the paper does not measure the relevant detector margins or provide a bound on the actual trajectory length. Therefore, the theoretical result does not establish that the observed detection failures follow from the proposed mechanism.

  3. Theorem 2 provides a potentially vacuous guarantee and does not directly characterize detection performance. Equation (17) lower-bounds the expected validation-minus-training KL gap. However, the bound subtracts the initial population suboptimality term, Δ₀. This term can dominate the positive contribution, especially when the initial student is not population-optimal. The authors do not estimate the bound or demonstrate that it is positive in their experiments. Furthermore, a positive expected KL gap only implies the existence of a threshold with positive membership advantage. It does not guarantee a useful AUROC or low-FPR detection rate. The analysis is also based on a general empirical-risk gradient flow argument. Its implications are not specific to policy distillation.

  4. Important checkpoint-calibrated detection baselines are missing. PD-Div-Gain uses both the teacher and the initial student checkpoint. However, all eight conventional baselines operate on the final student alone. This comparison cannot establish whether the improvement comes from teacher information or simply from access to the initial checkpoint. For example, the authors could compare changes in sequence likelihood, predictive entropy, or student prediction distributions between the initial and final checkpoints. These baselines would control for checkpoint access without requiring the teacher. Such comparisons are necessary to establish the additional value of teacher–student KL divergence.

  5. The experimental scale and statistical validation are insufficient. All controlled experiments use one model pair: Qwen2.5-1.5B-Instruct and Qwen2.5-7B-Instruct. Training uses rank-16 LoRA for three epochs, with only 100 designated members for each of MATH, MulDimIF, and XSum. AIME 2026 contains just 15 members and 15 non-members per student. The experiments also use a single random seed. No confidence intervals or statistical significance tests are reported. It remains unclear whether the findings generalize across model scales, teacher quality, distillation temperatures, training durations, or contamination ratios. The AIME low-FPR metric also uses 13.3% FPR, while the other datasets use 5%. Averaging these TPR values obscures their different operating points.

  6. The proposed detector has limited practical applicability and uneven effectiveness. PD-Div-Gain reaches 67.9% and 65.0% average AUROC, but its strongest results come from XSum, with 86.5% and 80.7%. Excluding XSum, average AUROC drops to approximately 61.7% and 59.7%. Thus, the detection signal remains relatively weak on mathematical reasoning and instruction-following tasks. More importantly, PD-Div-Gain requires the actual distillation teacher and the precise pre-distillation checkpoint, including full-vocabulary probability distributions. These are rarely available when auditing public models. The Qwen3 case study uses proxy checkpoints and has no verified positive membership labels. It therefore provides no direct evidence that the proposed method works under realistic model-access constraints.

Questions for authors

see weakness

1 comment

Thank you for the detailed review. We address the six concerns below:

1. Training objectives and completion prefixes

Following your suggestion, we added a control with the same prompts, fixed reference completions, and training settings as SFT. Detection also uses these fixed completions; supervision changes from hard-label CE to the teacher's soft distribution. The best conventional AUROCs are:

DatasetBest AUC(SFT)Best AUC(PD)
MATH72.152.56
MulDimIF92.167.50
XSum99.549.06
AIME 202698.153.70

The detailed result can be seen in Table 1 and Table 25. These results show that soft-target training can weaken conventional detection even when the prefixes are held fixed. The detection gap therefore cannot be explained solely by changing OPD prefixes or generated evaluation completions. We will clarify the matched setup and reference sources in Appendix C.

2. Theorem 1 and practical optimization

You are right that Theorem 1 uses idealized conditions. These are sufficient conditions and our goal is to establish that this failure can occur, rather than require every training scheme or sample to satisfy these conditions. Although our experiments are differ from the theoretical model, conventional detectors still struggle to distinguish known members from nonmembers. Therefore, the contamination problem is universal existance in both theory and real. In the revision, we will clearly separate the theoretical guarantee from this practical evidence.

3. What Theorem 2 establishes

You are right that Theorem 2 does not guarantee a positive gap for every training run. Establishing this would require stronger assumptions on the initial student. To address your concern about the specific value of KL, we have added the following result, which compares detectors under the same distillation process(Appendix A.4).

Additional theorem. Under Theorem 2's sampling and KL-gradient-flow setup,

\qquad \frac{|\Gamma_r'(0)|}{\sigma_r} \le\frac{\Gamma_k'(0)}{\sigma_k} =\frac{\sigma_k}{N} \quad(\sigma_r,\sigma_k>0).$$ This gives a concrete reason for choosing KL: **at the same gradient-fluctuation scale, KL produces the fastest initial growth of the expected membership gap.** In addition, we examined whether this signal appears in practice by plotting the member/nonmember distributions in **Appendix D.7--Figures 5 and 6**. The eight conventional methods show little consistent separation, whereas PD-Div and especially PD-Div-Gain show clearer shifts under both policies. These results show that teacher-aware scoring recovers a useful membership signal and support the group-level analysis in Theorem 2. ## 4. Checkpoint-calibrated baselines We added pre/post-checkpoint calibration to baselines on identical settings(**Table 17-20**). Their best mean AUC is 56.09(online)/56.95(offline), versus 68.39/66.00 for PD-Div-Gain. This supports a benefit from teacher-aware scoring beyond checkpoint access alone. ## 5. Model coverage and statistical validation To address your concern, we first repeat each experiment with three seeds and show confidence interval(**Table 5-26**). Then we repeated the experiments with a Llama 3B teacher and 1B student (consider limited time)(**Table 11-16**). PD-Div-Gain achieves mean AUC of 67.82%/67.87% online/offline (best baseline is 56.95%/59.75%), which maintain the conclusion on Qwen. ## 6. Practical access and uneven performance We would like to emphasize that, even excluding XSum, its mean AUC are 62.66%/61.00% online/offline, compared with 52.32%/51.37% for the best baseline. Moving from near-chance performance to around 0.6 provides a useful detection signal and a viable starting point, even though the absolute scores remain modest. We agree that obtaining the actual teacher and initial student is difficult in real audits. However, our main contribution is identifying and formalizing a problem introduced by policy distillation, not simply proposing another detector. The access needed to recover a signal further illustrates why this problem is difficult to address with current auditing tools. As discussed in our response to point 3, our theoretical evidence concerns group-level differences. With only 30 problems per year and proxy checkpoints, the Qwen3 audit provides clues of possible exposure rather than a firm conclusion. We also examined suspicious responses, including appeals to problem-specific known results without a valid derivation, and will include these examples and temporal-control comparisons in **Appendix D.6**. We hope this study draws attention to the problem and encourages larger auditing datasets with verified exposure labels and better-documented training histories.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Rebuttal Summary

 We sincerely thank the reviewers for their constructive feedback, which has greatly helped us improve the quality of our work. We have carefully addressed their concerns and updated the manuscript accordingly. At the camera-ready stage, we will integrate these revisions into the main text and upload the updated manuscript.  Key strengths recognized by reviewers: 

  • Importance and novelty: Reviewer 56fG highlighted an "Important and timely problem," while Reviewer zJXt recognized "an important blind spot in existing contamination detection methods."
  • Theory and experimental design: Reviewer 56fG highlighted "Useful theoretical characterization," "Controlled membership construction," and "Extensive comparison of conventional detection methods." Reviewer Gqev noted that "The theory, controlled benchmark, and proposed detectors form a coherent investigation."
  • Presentation: All three reviewers rated the presentation as "Clear and easy to follow."  Key rebuttal updates: 
  • Controlled comparisons and theoretical analysis (Reviewer 56fG): We added matched SFT–distillation experiments using fixed reference completions, a new theoretical result on KL-based detection (Theorem 3), and checkpoint-calibrated baselines. We also conducted three-seed experiments with confidence intervals and additional Llama experiments, addressing concerns about statistical reliability and generalizability.
  • Membership signals (Reviewer zJXt): We added member–non-member score distribution analyses (Appendix D.7) and explored teacher-free detection alternatives. The reviewer subsequently raised their score, and we sincerely appreciate their reconsideration.
  • Practical applicability (Reviewer Gqev): We clarified the access requirements and limitations of our detectors, provided additional checkpoint-calibrated comparisons, and expanded the analysis of the exploratory Qwen3 audit.  We hope these substantive revisions and positive discussion progress will be considered in the final assessment. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.