- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes the Zero-CoT Probe (ZCP), a contamination detection method that truncates the Chain-of-Thought process to expose latent shortcut mappings. The proposed method can detect data contamination without requiring access to the model's token probabilities.
Critical strengths
The authors aim to develop a dataset-level contamination detection method, which is important for ensuring reliable evaluation of LLMs.
Critical weaknesses
-
Lack of baseline comparisons. The manuscript does not compare the proposed method against existing detection methods, making it unclear whether it achieves superior performance.
-
Limited novelty. Existing work has explored the use of performance differences between the original and reference datasets to detect contamination [1]. Additionally, introducing a reference dataset does not necessarily isolate memorization from the model's intrinsic ability. The authors do not provide experimental evidence to support this claim.
-
Unconvincing empirical evaluation. The experimental results do not convincingly demonstrate the effectiveness of ZCP. In particular, the authors should compare Zero-CoT and Full-CoT under the same controlled setting. Additionally, it is unclear whether the performance gap stems from ZCP or differences between the target and reference datasets. The experimental results in Table 18 show that the measured metrics are highly sensitive to the dataset.
Reference: [1] ConStat: Performance-Based Contamination Detection in Large Language Models, Neurips 2024
Questions for authors
-
In Figure 2, what is the experimental setup? How are the contaminated questions and clear questions subsets established?
-
In line 219, what does 'where response prefixes cannot be explicitly pre-filled' mean? From my perspective, the instruction can be placed directly into the prompt.
-
What motivates the use of a Bayesian evidence framework and an upper bound on the Bayes factor?
-
In Figure 2, why are solid lines used in the legend, while dashed lines are used in the plot?
6 comments
Response to Reviewer iRCZ 2/3
W3: Compare Zero-CoT and Full-CoT under the same controlled setting; is the performance gap caused by ZCP or by differences between the target and reference datasets; Table 20 (Table 18 in the original submission) shows that the metrics are highly sensitive to the dataset
In the original submission, Appendix I.1 (Table 17) compares the two settings only for Qwen-Math on GSM8K. We now evaluate all existing-model settings of Table 4 and the OLMo models of Table 6 with both Full-CoT and zero-CoT, on the same original examples and the same reference data (Table R12 in the rebuttal and Table 19 in the revised paper).
Table R12. on original / reference ( / , %) and with Full-CoT and zero-CoT. All rows are means over 3 seeds.
| Model | Data | Full-CoT | Zero-CoT |
|---|---|---|---|
| Qwen-Math | GSM8K original | 95.9 / 96.2, 0.500 | 45.7 / 33.2, 1.000 |
| Qwen-Math | GSM8K paraphrased | 94.3 / 96.3, 0.500 | 42.3 / 33.2, 0.996 |
| Qwen-Math | MATH original | 78.6 / 76.4, 0.675 | 49.3 / 31.3, 1.000 |
| Qwen-Math | MATH paraphrased | 74.5 / 76.3, 0.500 | 43.1 / 31.3, 1.000 |
| DeepSeek-Math | GSM8K original | 96.6 / 93.7, 0.942 | 29.7 / 21.7, 0.993 |
| DeepSeek-Math | GSM8K paraphrased | 90.2 / 93.9, 0.500 | 27.3 / 21.7, 0.917 |
| DeepSeek-Math | MATH original | 59.4 / 49.5, 1.000 | 37.3 / 20.4, 1.000 |
| DeepSeek-Math | MATH paraphrased | 52.4 / 50.0, 0.616 | 30.0 / 20.4, 1.000 |
| OLMo-2-Instruct | MATH | 33.8 / 32.9, 0.517 | 13.0 / 10.0, 0.929 |
| OLMo-3-Instruct | MATH | 76.0 / 75.0, 0.629 | 22.1 / 17.0, 0.990 |
Full-CoT finds strong evidence () only on the original GSM8K and MATH data of DeepSeek-Math, and little or no evidence on any paraphrased (evasive) setting (). On GSM8K, Full-CoT accuracies on the original and reference data are close. With zero-CoT, in all 10 settings, including all paraphrased ones. This is the masking effect of Sec. 3.3: the model's reasoning also solves the reference problems and hides the memorization.
Table R12 also shows that the gap comes from ZCP rather than from differences between the original and reference datasets. Full-CoT and zero-CoT are evaluated on exactly the same original and reference data, so any difference between the two datasets affects both settings in the same way. Yet with Full-CoT, the accuracies on the original and reference data are close in most settings (e.g., 94.3% vs. 96.3% for Qwen-Math on paraphrased GSM8K), and strong evidence appears only on the original data of DeepSeek-Math; with zero-CoT, the same data give in all 10 settings (e.g., 42.3% vs. 33.2%). Zero-CoT does not create a gap by itself either: on the uncontaminated GSM1K and Dataset U, of zero-CoT is 0.500–0.567 (W2 (a)).
Table 20 varies the number of evaluated examples N. As an evidence score derived from a p-value, is expected to depend on N: with few examples, even a real gap provides limited evidence. For the same contaminated model and a similar gap, of rises from 0.578 at N = 50 to 1.000 at N = 1000. The logit-based metrics are much less sensitive to N: of is at least 0.946 from N = 50, and that of is at least 0.997 from N = 100. As recommended in Appendix I.2 (l.1314–1328), and should be preferred whenever logits are available; and are a compromise for cases where logits or ground-truth answers are unavailable, and they require a sufficiently large N.
Q1: What is the experimental setup of Figure 2, and how are the contaminated and clean question subsets established?
We thank the reviewer for raising this question. Figure 2 uses Qwen-Math and DeepSeek-Math. The contaminated questions are the 700 MATH training examples of Table 4, on which both models were trained (Sec. 4.1), and the clean questions are their counterparts in the reference dataset, which the models have not seen. For each question, we keep the first X% (by characters) of the model's Full-CoT solution and append The final answer is: \[ \boxed{ for the model to complete the answer; X = 0% is zero-CoT, and X = 100% is the full solution. Δ is the accuracy on the contaminated questions minus that on the clean ones. We have added this description to Appendix I.1 (l.1261–1266).
Response to Reviewer iRCZ 3/3
Q2: What does "where response prefixes cannot be explicitly pre-filled" (l.224–225; l.219 in the original submission) mean? The instruction can be placed directly into the prompt.
For open-weight models, we write the beginning of the model's response ourselves (The final answer is: \[ \boxed{, l.222–224), so the model must answer immediately without reasoning. The APIs of closed-source models such as GPT do not allow us to edit the model's response directly. We can only edit the user prompt, so for them we place the instruction like "Please ONLY put your final answer within \boxed{} directly without any other content before or after it (e.g., reasoning or explanation)" in the prompt instead (l.224–228). Since an instruction does not guarantee that reasoning is skipped, we measure how often the models comply: GPT-4o and GPT-5.1 output the boxed answer directly in 92.8–100% of cases on MATH500 and GSM8K (Table 24, Appendix J). We have clarified this in Sec. 3.3 (l.224–230).
Q3: What motivates the use of a Bayesian evidence framework and an upper bound on the Bayes factor?
We thank the reviewer for raising this question. First, a binary clean/contaminated label cannot capture how the evidence of contamination varies with the exposure frequency and the proportion of leaked data (l.310–312). We therefore want a continuous measure of evidence rather than an accept/reject decision at a certain threshold. A p-value does not consider how likely the gap is under contamination, and by itself it does not provide a good measure of evidence for a hypothesis (Wasserstein & Lazar, 2016). The Bayes factor compares how likely the observed data are under contamination and under no contamination, and a p-value can be converted into a bound on it (Sellke et al., 2001; Bayarri et al., 2016).
Second, an exact Bayes factor requires a model of the data under contamination, which depends on the unknown amount of exposure and varies across models and benchmarks. The bound only requires a valid p-value. With a neutral prior of 0.5, it gives , where 0.5 means no evidence. Because it is based on an upper bound, is an evidence score rather than a calibrated probability, which we now state in Sec. 3.6 (l.341–343).
Q4: In Figure 2, why are solid lines used in the legend, while dashed lines are used in the plot?
Thank you for pointing this out. The DeepSeek-Math curve is dashed, and its legend entry uses the same dashed style, but the legend line is too short for the dashes to be visible. We have lengthened the legend lines in Figure 2 so that the line styles are clearly visible.
References
- Bayarri et al. (2016). Rejection Odds and Rejection Ratios: A Proposal for Statistical Practice in Testing Hypotheses. Journal of Mathematical Psychology, 72, 90–103. https://doi.org/10.1016/j.jmp.2015.12.007
- Sellke et al. (2001). Calibration of p Values for Testing Precise Null Hypotheses. The American Statistician, 55(1), 62–71. https://doi.org/10.1198/000313001300339950
- Wasserstein and Lazar (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108
Response to Reviewer iRCZ 1/3
Thank you for the detailed review and valuable advice. We have revised the paper (changes in dark green); section, table, figure, and line numbers below refer to this version.
W1: Lack of baseline comparisons with existing detection methods
Sec. 3.2 already evaluates existing methods on the same contaminated models: the scores of DPCC (Table 1) and data reconstruction (Table 2) drop sharply from the original to the paraphrased data (e.g., for Qwen-Math on MATH, DPCC 0.730 → 0.191, ROUGE-L 0.621 → 0.267).
Existing methods produce different outputs (a thresholded fraction of low-RMIA examples for DPCC, ROUGE-L and accuracy for reconstruction, per-example scores for membership inference), and ZCP's paired test needs reference data paired one-to-one with the original data. We therefore do not rank the methods by one score, but test whether each method's own signal, which is stronger in the direct (original) setting, still raises an alarm in the evasive (paraphrased) setting.
We add Min-K% (Shi et al., 2023), a widely used membership inference method (Appendix I.4). Following Shi et al., we score the question of each example with k = 20%, treat the original (or paraphrased) examples as positives and their unseen reference examples as negatives, and report AUROC and the true positive rate (TPR) at 5% false positive rate.
Table R11. Min-K%: AUROC / TPR at 5% FPR, with the reference examples as negatives (random guessing: 0.5 / 5%).
On the original data, Min-K% ranks the contaminated examples above their references (AUROC 0.545–0.862, TPR up to 38.2%). On the paraphrased data, the signal disappears: TPR is at most 0.2%, and AUROC falls well below 0.5. Like DPCC and data reconstruction, Min-K% relies on the exact tokens of the training data, which paraphrasing removes.
W2: Limited novelty, as existing work (ConStat) uses performance differences on reference data; a reference dataset may not isolate memorization from intrinsic ability
We agree that comparing performance on original and reference data is an established idea, and now cite and discuss related work such as ConStat (Dekoninck et al., 2024) in Sec. 2. However, ZCP differs from it in two ways:
Our results show that the reference data isolate memorization from intrinsic ability:
References