- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces DUG, a layerwise framework combining token-prediction probes, logit-lens scores, and difference-in-differences to study how training changes membership signals. Its experiments suggest that SFT changes both decodability and readout, while reconstruction-based RL attacks primarily change readout; a controlled RLVR case further separates behavioral membership effects from likelihood-based detection.
Critical strengths
- DUG separates target-token decodability from logit-lens accessibility, providing more detail than membership AUROC alone.
- Question-level cross-fitting, shuffled-label controls, and comparisons against the SFT initialization strengthen the membership-specific analysis.
- Exposure-matched RLVR and SFT experiments reveal a useful dissociation between behavioral changes and likelihood-based membership signals.
Critical weaknesses
- Near-zero DiD[D] from probes trained on small samples cannot exclude representational changes beyond their statistical sensitivity.
- Intermediate-layer logit-lens scores conflate vocabulary alignment with information actually used downstream, weakening the mechanistic interpretation of U.
- RLVR evidence covers only 64 selected members and two pool epochs, limiting conclusions about prolonged contamination.
Questions for authors
- Can you vary probe-training size, capacity, and vocabulary coverage, reporting confidence intervals for the stability of DiD[D]?
- Can you align trained and base representations using disjoint non-member activations, then test whether DiD[U] survives this alignment control?
- Does the likelihood/behavior dissociation persist across additional membership splits, problem difficulties, and longer RLVR runs?
4 comments
Follow-up on Author Response
Dear Reviewer,
As the author–reviewer discussion period is drawing to a close, we wanted to check whether our response has addressed your concerns, and whether there is anything further we can clarify or provide. We welcome any comments and suggestions you may have at any time, and would be glad to discuss with you how to further improve our paper.
Thank you again for your time and thoughtful review.
Thanks for the authors' response. I will raise my score to 4. Please be sure to include the discussion into your final version (if accepted).
Follow-up on Author Response
We sincerely thank the reviewer for their support and encouragement. We will make sure to incorporate the additional discussion into the final version if accepted.
Response to Reviewer 7bHh
We sincerely thank the reviewer for their valuable suggestions, which have encouraged us to examine several aspects of our work more thoroughly. We have conducted additional experiments to address the reviewer's questions and provide our responses below.
W1&Q1: Different Probe size, Capacity, and Vocabulary Coverage.
We thank the reviewer for this valuable suggestion. Following the reviewer's recommendation, we varied probe-training size, capacity, and vocabulary coverage to assess the stability of DiD[D]. We summarize the results below.
1.1 Varied Probe-training Size
We varied the amount of probe-training data to examine the stability of DiD[D]. Specifically, we train probes using 50%, 75%, and 100% of the available training examples in each fold, while keeping the evaluation examples fixed. Across both datasets and RL-based MIA methods, DiD[D] ranges from −0.018 to +0.031 nats, with 95% CIs including zero for 70 of 84 layer-wise estimates. On AIME, the few significant effects appear at early layers (L4/L8) with reduced training data and disappear when using the full set of available probe-training examples. Layer-averaged DiD[D] also shows no increasing trend with training size. These results provide little evidence that increasing probe-training data would lead to larger DiD[D] effects.
1.2 Varied Capacity
Our default probe is full-rank, allowing the maximum possible rank for a linear probe. We further examine the effect of reducing probe capacity to rank 64. Across both full-rank and rank-64 probes, DiD[D] for RL-based MIA methods ranges from −0.011 to +0.044 nats, with 95% CIs including zero for 38 of 40 layer-wise estimates. Moreover, the absolute value of D appears relatively insensitive to probe rank (2.948/2.955/2.938 nats for ranks 64/128/256 at Olympia L32, respectively). These results suggest that our findings are relatively robust to probe capacity.
1.3 Varied Vocabulary Coverage
By default, we retain only tokens appearing in the suffixes (App. A.3). To examine the effect of vocabulary coverage, we further restrict the vocabulary to the most frequent tokens accounting for 80%, 90%, and 95% of token occurrences, mapping the remaining tokens to UNK. Across ADRA and ADRA+, DiD[D] ranges from −0.014 to +0.032 nats, with 95% CIs including zero in 94 of 112 comparisons and no systematic trend across coverage levels. These results suggest that our main conclusions are relatively robust to the choice of vocabulary coverage.
W2&Q2: Representation Alignment Control.
Following the reviewer’s suggestion, we fitted layer-wise transformations from the trained to the base representations using a disjoint set of non-member activations. We applied several alignment maps of increasing flexibility, from mean shift to ridge-regularized affine alignment. Alignment effectively reduced non-members’ ΔU from −0.197 to near zero (−0.017 to +0.021). DiD[U] remained statistically significant across all alignment methods, ranging from +0.045 to +0.053 (95% CIs exclude zero; permutation p ≤ 0.028), compared with +0.069 (95% CI [0.019, 0.117]) before alignment. These results confirm that the observed member-specific readout effect persists after representation alignment, supporting our original conclusion. We thank the reviewer for this valuable suggestion and will incorporate these additional results into the final version of the paper.
W3&Q3: Different Data Splits and Longer RLVR Runs
We appreciate the reviewer's concern regarding the robustness of our RLVR findings. We would like to clarify our experimental design and the rationale behind the training duration.
3.1 Data Selection
Both member and non-member examples were sampled from MATH-500, restricted to problems with 0<pass@32<1, and matched by MATH difficulty level. As reported in Sec. 5.3 and Table 1, the two groups exhibit comparable pre-training accuracy (0.141 vs. 0.132), while Loss and Min-K% yield AUROCs of 0.491 and 0.492, respectively. These results suggest that the two groups are not readily distinguishable by their initial difficulty or likelihood-based signals, helping mitigate potential confounding effects arising from dataset selection in our RLVR experiments.
3.2 Explanation on Longer RLVR Runs
As in the main text, we stopped training when policy entropy fell below half its step-1 value, which occurred at step 20 (App. C.1). As further shown in App. C.2, reward variance and the fraction of groups with non-zero advantages were already declining, suggesting diminishing learning signals under GRPO. Therefore, extending training substantially beyond step 20 may offer limited additional insight in our setting.