- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
Texts that appear in the training corpora of unsupervised or supervised learning algorithms tend to be modeled with higher likelihood, and can therefore be distinguished from novel texts at rates better than chance. This manuscript studies the problem of detecting prompts that have been used in RL training procedures based on the likelihoods and distribution of the rollouts they cause a model to generate. Eleven diagnostic features in four categories are proposed: Sharpened likelihood distribution within a rollout, Trajectory collapse, Answer saturation, and sharpened likelihood distribution across Rollouts.
Critical strengths
- The four feature categories are each well-motivated by existing literature and theory. The eleven features that compose them are certainly not comprehensive, but are well-motivated representative examples of each category.
- Experimental results demonstrate the claim that no single feature category or single baseline provides detection AUC better than a combination of the four feature categories.
- The label-free multi-axis weighted aggregation algorithm consists of a well-motivated initialization and a well-motivated refinement, though both are marred by heuristic hyperparameters.
- Robustness analysis shows that the four-axis feature combination outperforms any single feature even if features are uniformly combined, suggesting that the algorithm is robust to the hyperparameters of the label-free weight estimation.
- In each of the four controlled experiments, half of a benchmark's prompts are included in training data, and half are not, so there is no systematic difference between in-training and non-training prompts other than whether or not they were used in RL training. Equivalence of distribution between in-training and non-training prompts is further verified by the near-chance performance of SCAR using the pre-RL version of each model.
Critical weaknesses
- The refinement stage of the label-free refinement algorithm assumes that the top-scoring and bottom-scoring 30% from the first stage are mostly true positives and false positives, respectively. This hyperparameter might be reasonable if exactly 50% of all tested prompts were in the training data, but is unlikely to be reasonable in any real-world deployment.
- The distribution of ground truth labels would also affect the thresholds used for detection. Even with uniform weights across SCAR, how robustly can thresholds be estimated if ground truth positive rate is much different from 50%?
- Although SCAR performs 0.09 better than the best baseline, its AUC is still only 0.73. Isn't that still too low to be useful for the in-training vs. out-of-training diagnosis of a real-world benchmark?
Questions for authors
- Some of the measures are defined too succinctly to be intelligible, e.g., what is mink20(Li)?
- Can you run experiments with a ground truth distribution substantially different than 50%, to judge model robustness?
2 comments
Dear Reviewer,
As the discussion period ends in two days, we kindly wanted to follow up on our rebuttal. We would greatly appreciate it if you could review our responses and share any remaining concerns before the deadline.
Thank you for your time and consideration.
We thank the reviewer for their review. Below, we address the concerns and questions raised.
W1/W2/Q2: Refinement and contamination rates
The top- and bottom-scoring 30% of the audit pool are used as pseudo-seen and pseudo-unseen groups for refinement. This procedure does not assume a 50% membership rate, although highly imbalanced pools may affect its performance.
To directly address these concerns, Appendix F.3 (Figure 9) evaluates SCAR-Cal and SCAR-Uniform across contamination rates ranging from 15% to 85% on OLMo/GSM8K and DeepSeek/AIME25. Both methods maintain informative membership rankings across the tested rates without requiring knowledge of the true membership proportion.
For extreme cases where the audit pool contains only members or only non-members, a practical approach is to augment the pool with reference questions known to be members and non-members, when available, and rerun the audit. This introduces membership variation into the pool and avoids contamination rates near 0% or 100%, although it requires access to suitable reference questions.
W3: Practical usefulness
We would like to clarify that SCAR-Cal achieves a mean ROC-AUC of 0.764, not 0.73, compared with 0.673 for the strongest baseline, Min-kNN. SCAR-Uniform achieves 0.737.
Although this does not guarantee correct membership decisions for every question, it provides more informative rankings for identifying potentially contaminated benchmark questions. Furthermore, all eleven label-free aggregation rules outperform the strongest baseline in macro-average ROC-AUC (Figure 3).
Q1: Signature definitions
We thank the reviewer for highlighting the need for clearer notation. The signature mink20(Li) denotes the average log-probability of the least probable 20% of tokens in rollout i. We then average this quantity across the N sampled rollouts to obtain the signature for each question. Complete notation and definitions are provided in Appendix A.1.
We hope our responses have addressed your concerns and clarified the points raised in your review.