acceptodds
Under review as a conference paper at ICLR 2027

From Representation to Readout: Mechanistic Analysis of Membership Signals in Language Models

Abstract

Membership inference aims to determine whether a particular example was included in a model’s training data. Such membership signals are often broadly described as memorization, while the underlying mechanisms can involve both how information is represented internally and how it is accessed by the model to produce behaviors. We therefore present a mechanistic study of membership signals in large language models (LLMs). We introduce Decodable Information Score, Used Information Score, and Readout Accessibility Gap (DUG), a set of complementary measures that separately quantify membership information decodable from internal representations and the extent to which it is exposed through the model’s readout. Across supervised fine-tuning (SFT) and reinforcement learning (RL)-based membership inference methods, membership-specific effects appear in both the amount of decodable information and readout under SFT, while under RL-based membership inference methods they appear primarily in readout. Activation patching further localizes this membership-specific effect to intermediate layers. To examine whether these findings extend to a more general RL setting, we further study reinforcement learning with verifiable rewards (RLVR) contamination, where the training reward is independent of the reference solution. In this setting, we find that behavioral changes can emerge without a corresponding increase in likelihood-based membership inference signals.

Primary area:
interpretability and explainable AI
Keywords:
Interpretability, Explainability, MIA

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_7bHhOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces DUG, a layerwise framework combining token-prediction probes, logit-lens scores, and difference-in-differences to study how training changes membership signals. Its experiments suggest that SFT changes both decodability and readout, while reconstruction-based RL attacks primarily change readout; a controlled RLVR case further separates behavioral membership effects from likelihood-based detection.

Critical strengths
  1. DUG separates target-token decodability from logit-lens accessibility, providing more detail than membership AUROC alone.
  2. Question-level cross-fitting, shuffled-label controls, and comparisons against the SFT initialization strengthen the membership-specific analysis.
  3. Exposure-matched RLVR and SFT experiments reveal a useful dissociation between behavioral changes and likelihood-based membership signals.
Critical weaknesses
  1. Near-zero DiD[D] from probes trained on small samples cannot exclude representational changes beyond their statistical sensitivity.
  2. Intermediate-layer logit-lens scores conflate vocabulary alignment with information actually used downstream, weakening the mechanistic interpretation of U.
  3. RLVR evidence covers only 64 selected members and two pool epochs, limiting conclusions about prolonged contamination.
Questions for authors
  1. Can you vary probe-training size, capacity, and vocabulary coverage, reporting confidence intervals for the stability of DiD[D]?
  2. Can you align trained and base representations using disjoint non-member activations, then test whether DiD[U] survives this alignment control?
  3. Does the likelihood/behavior dissociation persist across additional membership splits, problem difficulties, and longer RLVR runs?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 7bHh

We sincerely thank the reviewer for their valuable suggestions, which have encouraged us to examine several aspects of our work more thoroughly. We have conducted additional experiments to address the reviewer's questions and provide our responses below.

W1&Q1: Different Probe size, Capacity, and Vocabulary Coverage.

We thank the reviewer for this valuable suggestion. Following the reviewer's recommendation, we varied probe-training size, capacity, and vocabulary coverage to assess the stability of DiD[D]. We summarize the results below.

1.1 Varied Probe-training Size

We varied the amount of probe-training data to examine the stability of DiD[D]. Specifically, we train probes using 50%, 75%, and 100% of the available training examples in each fold, while keeping the evaluation examples fixed. Across both datasets and RL-based MIA methods, DiD[D] ranges from −0.018 to +0.031 nats, with 95% CIs including zero for 70 of 84 layer-wise estimates. On AIME, the few significant effects appear at early layers (L4/L8) with reduced training data and disappear when using the full set of available probe-training examples. Layer-averaged DiD[D] also shows no increasing trend with training size. These results provide little evidence that increasing probe-training data would lead to larger DiD[D] effects.

1.2 Varied Capacity

Our default probe is full-rank, allowing the maximum possible rank for a linear probe. We further examine the effect of reducing probe capacity to rank 64. Across both full-rank and rank-64 probes, DiD[D] for RL-based MIA methods ranges from −0.011 to +0.044 nats, with 95% CIs including zero for 38 of 40 layer-wise estimates. Moreover, the absolute value of D appears relatively insensitive to probe rank (2.948/2.955/2.938 nats for ranks 64/128/256 at Olympia L32, respectively). These results suggest that our findings are relatively robust to probe capacity.

1.3 Varied Vocabulary Coverage

By default, we retain only tokens appearing in the suffixes (App. A.3). To examine the effect of vocabulary coverage, we further restrict the vocabulary to the most frequent tokens accounting for 80%, 90%, and 95% of token occurrences, mapping the remaining tokens to UNK. Across ADRA and ADRA+, DiD[D] ranges from −0.014 to +0.032 nats, with 95% CIs including zero in 94 of 112 comparisons and no systematic trend across coverage levels. These results suggest that our main conclusions are relatively robust to the choice of vocabulary coverage.

W2&Q2: Representation Alignment Control.

Following the reviewer’s suggestion, we fitted layer-wise transformations from the trained to the base representations using a disjoint set of non-member activations. We applied several alignment maps of increasing flexibility, from mean shift to ridge-regularized affine alignment. Alignment effectively reduced non-members’ ΔU from −0.197 to near zero (−0.017 to +0.021). DiD[U] remained statistically significant across all alignment methods, ranging from +0.045 to +0.053 (95% CIs exclude zero; permutation p ≤ 0.028), compared with +0.069 (95% CI [0.019, 0.117]) before alignment. These results confirm that the observed member-specific readout effect persists after representation alignment, supporting our original conclusion. We thank the reviewer for this valuable suggestion and will incorporate these additional results into the final version of the paper.

W3&Q3: Different Data Splits and Longer RLVR Runs

We appreciate the reviewer's concern regarding the robustness of our RLVR findings. We would like to clarify our experimental design and the rationale behind the training duration.

3.1 Data Selection

Both member and non-member examples were sampled from MATH-500, restricted to problems with 0<pass@32<10 < \text{pass@32} < 1, and matched by MATH difficulty level. As reported in Sec. 5.3 and Table 1, the two groups exhibit comparable pre-training accuracy (0.141 vs. 0.132), while Loss and Min-K% yield AUROCs of 0.491 and 0.492, respectively. These results suggest that the two groups are not readily distinguishable by their initial difficulty or likelihood-based signals, helping mitigate potential confounding effects arising from dataset selection in our RLVR experiments.

3.2 Explanation on Longer RLVR Runs

As in the main text, we stopped training when policy entropy fell below half its step-1 value, which occurred at step 20 (App. C.1). As further shown in App. C.2, reward variance and the fraction of groups with non-zero advantages were already declining, suggesting diminishing learning signals under GRPO. Therefore, extending training substantially beyond step 20 may offer limited additional insight in our setting.

AuthorsOpenReview ↗

Follow-up on Author Response

Dear Reviewer,

As the author–reviewer discussion period is drawing to a close, we wanted to check whether our response has addressed your concerns, and whether there is anything further we can clarify or provide. We welcome any comments and suggestions you may have at any time, and would be glad to discuss with you how to further improve our paper.

Thank you again for your time and thoughtful review.

Thanks for the authors' response. I will raise my score to 4. Please be sure to include the discussion into your final version (if accepted).

AuthorsOpenReview ↗

Follow-up on Author Response

We sincerely thank the reviewer for their support and encouragement. We will make sure to incorporate the additional discussion into the final version if accepted.

Reviewer_qvNpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how membership signals are represented and exposed in large language models under different training procedures. The authors propose the DUG framework, consisting of Decodable Information Score (D), Used Information Score (U), and Readout Accessibility Gap (G), to distinguish information decodable from internal representations from information accessible through the model's readout. Combined with a difference-in-differences analysis, the framework measures membership-specific changes across layers.

Experiments on supervised fine-tuning (SFT) and RL-based membership inference methods suggest that SFT affects both decodable information and readout, whereas RL-based methods primarily influence readout. The authors further employ activation patching to localize these effects and investigate an RLVR contamination setting, demonstrating that behavioral membership differences do not necessarily translate into detectable likelihood-based membership signals.

Critical strengths
  • Intuitive analytical framework. The proposed DUG framework provides a relatively straightforward way to distinguish between decodable information and readout accessibility. Combining these metrics with difference-in-differences analysis helps isolate membership-specific effects from general training-induced changes.

  • Multiple complementary experiments. The paper investigates SFT, RL-based membership inference, and RLVR contamination. The inclusion of activation patching and steering experiments provides additional evidence for understanding the observed membership signals.

Critical weaknesses
  • Limited justification for separating representation and readout. The proposed framework uses linear probing to estimate decodable information and the logit lens to measure used information. However, these measurements do not necessarily provide a clean separation between information encoded in representations and information actually used by the model. In particular, a linear probe only measures linearly accessible information, while the logit lens does not capture the computation performed by subsequent layers. Therefore, the interpretation of DUG as distinguishing representation changes from readout changes requires stronger justification.

  • Limited experimental scale and generalizability. The experiments primarily focus on mathematical reasoning datasets and a small number of 7B-scale models. Moreover, the controlled contamination experiments involve relatively small evaluation sets. It remains unclear whether the observed distinctions between SFT and RL-based methods generalize to larger models, other domains, or more diverse training objectives.

  • Insufficient causal evidence for the proposed mechanism. Although the paper includes activation patching experiments, these experiments primarily identify the layers where membership-specific effects emerge. They do not directly establish that RL-based methods selectively modify readout mechanisms rather than other aspects of the representations.

Questions for authors
  • How reliable is the separation between decodable and used information? Since D relies on linear probes and U relies on the logit lens, could the observed differences arise from the limitations of these measurement tools rather than genuine differences between representation and readout? Have the authors considered alternative probe architectures or readout methods to validate the conclusions?(e.g. using some nonlinear probing like MLP probing instead of linear probing)

  • How well do the findings generalize beyond the current experimental settings? Most experiments use mathematical reasoning datasets and relatively small models. Would the same distinction between SFT and RL-based membership effects hold for larger language models or non-mathematical datasets?

  • Can the authors provide stronger causal evidence for readout-specific changes? The current activation patching experiments mainly identify the layers where membership-specific effects emerge, but do not clearly distinguish changes in representations from changes in downstream computation. Could the authors conduct a controlled representation–readout swapping experiment, where hidden states from the base and RL-trained models are independently combined with their respective downstream networks? Such an experiment could help clarify whether the observed membership-specific effects are primarily driven by changes in hidden representations or by how subsequent layers process them.

5 comments
AuthorsOpenReview ↗

Response to Reviewer qvNp (Part 1/2)

We sincerely thank the reviewer for their constructive and insightful feedback, which has encouraged us to examine several aspects of our work more thoroughly. Below, we provide our responses to the weaknesses and questions, along with additional experimental results.

Q1&W1: Reliability of D and U Measurements: Nonlinear Probing and Representation Alignment.

To test whether our conclusions depend on the choice of measurement tools, we conducted two additional experiments to improve our reliability, as follows.

1.1 Nonlinear probe

To test whether member-specific changes in D are encoded nonlinearly and thus missed by a linear probe, we trained an MLP probe. As a control, we also trained linear-sel, a linear probe with the same hyperparameter selection as the MLP. All other settings follow Sec. 5.2, i.e., same datasets for each RL-based MIA method. We find:

(1) Weak evidence for a consistent member-specific increase in D. The MLP is significant in 1/10 settings for each RL-based MIA method (ADRA: Olympia Math L12, +0.0097, p = 0.050; ADRA+: AIME L12, +0.0207). This matches the paper’s linear probe findings of D as in Sec.5.1.

(2) MLP probes show comparable decoding performance to linear probes. On θ₀, linear-sel achieves higher scores in three of the four settings, suggesting that the benefit of nonlinear probing may be limited in our experiments.

1.2 Alignment control

We fitted layer-wise transformations from the trained to the base representations using a disjoint set of non-member activations, which can remove global representational shift that could change the logit-lens readout. Alignment indeed reduced non-members’ ΔU from −0.197 to near zero (−0.017 to +0.021). However, DiD[U] remained statistically significant across all alignment methods, ranging from +0.045 to +0.053 (95% CIs exclude zero; permutation p ≤ 0.028), compared with +0.069 (95% CI [0.019, 0.117]) before alignment. These results confirm that the observed member-specific readout effect persists after representation alignment, supporting our original conclusion. We will add the full results of these experiments to the paper.

Q2&W2: Generalizability Across Domains, Dataset Sizes, Model Scales, and Training Objectives.

2.1 Domain coverage

While our main experiments primarily focus on mathematical datasets, our analysis also includes WikiMIA-2024-Hard, which consists of non-mathematical Wikipedia text (Sec. 5.1-5.2, App. A.5). In addition, all three datasets in our preliminary study (Sec. 3) are non-mathematical. We will make this coverage clearer in the revised version.

2.2 Dataset size

We understand the reviewer's concern regarding the limited evaluation size. We would like to provide additional context for our design.

(1) Our setting focuses on controlled contamination of evaluation benchmarks during post-training, where the number of contaminated examples can be inherently small. Our experimental setup follows ADRA (Yin et al., 2026), which also studies small-scale controlled post-training contamination on AIME and Olympia Math. Similarly, Self-Critique (ICLR 2026) injects 50% of AIME 2024/2025 into RL training, corresponding to approximately 15 member and 15 non-member problems per benchmark.

(2) Larger benchmarks are not necessarily more reliable for membership inference due to potential distribution shifts (Duan et al., 2024; Meeus et al., 2025; Das et al., 2025). For instance, model-independent classifiers achieve 99.0% AUROC on WikiMIA and 99.9% on WikiMIA-24, compared with only 57.7% on WikiMIA-Hard. Such distribution shifts could confound our analysis of membership signals in internal representations.

2.3 Generalizability and scale

We included the RLVR case study on MATH-500 in Sec. 5.3 to extend our analysis beyond membership-targeted training objectives and provide an additional reference point under a standard verifiable-reward objective. Regarding model scale, we acknowledge that our experiments are limited to 7B-scale models. This is primarily because a controlled SFT-versus-RL comparison requires models trained with different objectives from the same base model, and the publicly available RL-trained models we study are limited to the 7B scale. Our computational budget also constrained independent RL training at larger scales. We will explicitly acknowledge this limitation in the revised manuscript and consider extending the comparison to larger models an important direction for future work.

AuthorsOpenReview ↗

Response to Reviewer qvNp (Part 2/2)

Q3&W3: Causal Evidence: Representation–Readout Swapping and Downstream Computation.

We thank the reviewer for highlighting the need for stronger causal evidence. The proposed representation–readout swapping experiment was in fact implemented in Section 5.5 (Activation Patching). We provide further clarification below and will highlight this experiment more clearly in the final version.

Specifically, in L461–462, we stated that “we run θ0\theta_0 and θa\theta_a in parallel and swap the residual stream at the suffix positions of layer ℓ\ell in both directions.” As defined in L465–467, these two directions correspond to the reviewer’s proposed combinations:

(1) Forward patching (θfwd,ℓ\theta_{\mathrm{fwd},\ell}): RL-trained representations combined with the base model’s downstream network.

(2) Reverse patching (θrev,ℓ\theta_{\mathrm{rev},\ell}): Base model representations combined with the RL-trained model’s downstream network.

In L468–471, we define the corresponding ratios RfwdR_{\mathrm{fwd}} and RrevR_{\mathrm{rev}}. We report their difference, Rfwd−RrevR_{\mathrm{fwd}}-R_{\mathrm{rev}}, to identify the boundaries of the layer window where the membership-specific effect emerges. We further report the full forward and reverse patching ratios at each layer as follows.

Table: Bidirectional activation patching results.

DatasetLayerRfwdR_{\mathrm{fwd}}RrevR_{\mathrm{rev}}Rfwd−RrevR_{\mathrm{fwd}}-R_{\mathrm{rev}} [95% CI]
Olympia Math40.020.97−0.95 [−1.04, −0.78]
80.060.97−0.91 [−1.20, −0.63]
120.170.73−0.56 [−1.28, −0.13]
160.470.26+0.21 [−0.36, +0.68]
200.710.09+0.62 [+0.31, +0.93]
240.820.06+0.76 [+0.51, +0.98]
280.940.04+0.90 [+0.73, +1.07]
AIME4−0.040.98−1.02 [−1.64, −0.84]
80.000.74−0.75 [−1.41, −0.48]
120.130.49−0.37 [−1.52, +0.29]
160.230.23−0.00 [−1.47, +0.60]
200.040.18−0.13 [−3.04, +0.59]
240.030.14−0.11 [−3.26, +0.70]
280.040.10−0.06 [−3.42, +0.78]

On Olympia Math, Rfwd−RrevR_{\mathrm{fwd}}-R_{\mathrm{rev}} is significantly negative at ℓ≤12\ell \leq 12 and significantly positive at ℓ≥20\ell \geq 20. AIME shows consistent results at ℓ=4\ell=4 and ℓ=8\ell=8, while deeper-layer estimates are inconclusive due to near-zero denominators. We highlight layers 8 and 20 in our main text as representative boundaries of the transition: layer 8 is the deepest layer with a significantly negative difference on both datasets, while layer 20 is the first layer with a positive difference on Olympia Math.

AuthorsOpenReview ↗

Follow-up on Author Response

Dear Reviewer,

As the author–reviewer discussion period is drawing to a close, we wanted to check whether our response has addressed your concerns, and whether there is anything further we can clarify or provide. We welcome any comments and suggestions you may have at any time, and would be glad to discuss with you how to further improve our paper.

Thank you again for your time and thoughtful review.

Reviewer_S7ZYOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether training changes membership signals by changing what information present in model internal representations or how that information is read out. In particular, it defines three layer-wise scores on the target tokens of each example:

  • D\textbf{D} (decodable information): how much a cross-fitted linear probe reduces cross-entropy relative to an intercept-only baseline.
  • U\textbf{U} (used information): the same reduction, measured through the model’s own unembedding (logit lens).
  • G\textbf{G} := D\textbf{D} − U\textbf{U}: the gap between the two.

A difference-in-differences (DiD) compares how each score changes from the base model for members versus non-members. The experiments use Tulu2-7B on AIME and Olympia Math with controlled SFT and two RL-based membership inference attacks (ADRA and ADRA+ [Yin et al., 2026]), plus Qwen2-7B on WikiMIA-2024-Hard. In particular, the paper claims:

  • SFT changes both D and U.
  • RL-based MIAs act mainly on readout.
  • Activation patching places the effect in intermediate layers.
  • In an RLVR contamination case study, membership shows up in behavior but not in likelihood-based MIA scores.
Critical strengths
  • The problem is clearly specified and well positioned within related work, and the experiment/measurement design is carefully performed:

    • DiD separates member-specific changes from drift shared with non-members (Eq. 6).
    • Probes are cross-fitted with folds grouped by question.
    • Regularization is chosen with a shuffled-label control.
    • Confidence intervals come from a problem-level bootstrap (App. A.3).
    • Choice of the logit lens over a tuned lens is justified empirically: a trainable readout injects spurious member/non-member differences (as referenced in App. B: 0.196 nats at layer 0).
  • RLVR case study is practically relevant and well controlled:

    • Reward ignores the reference solution, and checkpoints are matched in exposure to the SFT runs (App. C.1).
    • (Longer) Training is shown to be effective (L397–399, App. C.2).
    • Result is a clean dissociation: likelihood-based MIAs stay at chance (Table 1, every Δ\DeltaAUROC confidence interval includes 0), while the member/non-member accuracy gap widens to 0.082 (L408–410).
    • The proposed mechanism is checked directly: the loss on the reference solution shifts by the same amount for members and non-members (L413–416).
Critical weaknesses
  • The central contrast, that SFT changes D\textbf{D} and U\textbf{U} while RL-based MIAs change “primarily readout,” is confounded by how distinctively sensitive D\textbf{D} and U\textbf{U} are:

    • At the final layer, DiD[U\textbf{U}] is the member/non-member gap in the change of log-likelihood of the reference solution (Eqs. 3, 4, 6). That is the loss-based MIA signal, which ADRA’s reconstruction reward targets directly.
    • D\textbf{D} depends on a probe with 1,328–2,630 output classes, fit on 16–32 problems per group per seed (App. A.1, A.3). Even at the last layer, it falls well short of the model’s own head (Fig. 2).
    • In Fig. 4, ADRA+’s DiD[D\textbf{D}] point estimates (about +0.02 on AIME at layers 12–28, about +0.03 on Olympia at layer 28) match SFT-2ep’s significant ones; with wider confidence intervals.
    • Table 3 reports a significant DiD[D\textbf{D}] at AIME layer 8 under the RL-trained model (θa\theta_a; the caption doesn’t say whether that is ADRA or ADRA+).
    • SFT-2ep also moves U\textbf{U} several times more than D\textbf{D} (Fig. 5: about +0.48 vs about +0.10 at layer 28). So the SFT-versus-RL distinction looks numerical, and partly a power asymmetry, rather than qualitative insight.
  • RL evidence leans on a confounded method and on small samples tested many times.

    • ADRA+’s reward gates the suffix with a Min-K%++ membership prior (L263–264, L781–783), so part of its member-specific effect is put there by design. The authors note that effect sizes track the prior’s AUROC (L784–785).
    • ADRA without the prior shows weaker, scattered effects (Fig. 4). On WikiMIA, ADRA+ is close to zero (Fig. 5).
    • Samples are 16 vs 16 problems per seed on AIME and 32 vs 32 on Olympia, and the number of seeds is not mentioned for result consistency evaluation.
    • Significance stars are reported across layers, scores, methods, and datasets with no stated multiple-comparison correction (Fig. 4).
    • Binary probe on the same data swings between 0.29 and 0.55 AUROC (A.4, L763), which shows how noisy the considered scale is.
    • Localization by patching rests on two layers, and the AIME result is inconclusive (L473–477).
Questions for authors
  • At the final layer, does DiD[U\textbf{U}] reduce to the loss-MIA DiD over the restricted vocabulary? If so, what does DUG add beyond tracking the logit-lens loss per layer? Also, the abstract calls D and U “membership information” (L019–021). They measure target-token information, and membership enters only through the DiD.
  • Can you show that D\textbf{D} has the power to detect a member-specific change as large as the observed DiD[U\textbf{U}] at these sample sizes, for example with a positive control or a power analysis? And why is G(L)<0G(L) < 0 attributed to a “rank-limited” probe (L227–228) when the AIME and Olympia probes are full-rank (L746–747)?
  • How much of ADRA+’s readout effect remains if the Min-K%++ gate is removed or randomized?
  • How many seeds are used, and are the significance markers in Fig. 4 corrected for multiple comparisons?
  • How are AIME members and non-members assigned? The 120/30 split matches four years versus one (L707–708). If the split is by year, it is temporal, and the earlier years may overlap with the pre-training data of Tulu2’s Llama-2 base.
  • How does the RLVR result relate to Tao et al. (2025) and Zhang et al. (2026), which already detect RLVR contamination through non-likelihood signals (L083–087)? Would those methods detect the behavioral gap in your setup, and does the gap (CI [+0.008, +0.159]) hold across seeds?
8 comments
AuthorsOpenReview ↗

Response to Reviewer S7ZY (Part 1/5)

We sincerely thank the reviewer for the careful reading and constructive comments, which have helped us improve the paper. Below, we respond to each point, providing further explanations and additional experiments.

Q1: Interpretation of DUG.

Q1.1 At the final layer, does DiD[U] reduce to the loss-MIA DiD over the restricted vocabulary?

Yes. At the final layer, DiD[U] is equivalent to the negative of the restricted-vocabulary loss DiD, since the null baseline cancels out. This equivalence serves as a consistency check, showing that U recovers the conventional loss-based signal at the final layer.

Q1.2 If so, what does DUG add beyond tracking the logit-lens loss per layer?

We appreciate the reviewer's thoughtful question and agree that U itself is closely related to the layer-wise logit-lens loss. However, tracking the logit-lens loss alone does not distinguish between: (1) how much target-token information is linearly decodable from the representations and (2) how much of that information is exposed through the model's readout. Our goal is to provide a unified analytical framework that combines D, U, and DiD to better understand how training-induced membership-specific changes manifest in decodability and readout.

Q1.3 Also, the abstract calls D and U “membership information” (L019–021).

We thank the reviewer for this helpful suggestion. We agree that D and U directly measure target-token information, while membership specificity is introduced through DiD. This is consistent with our motivation in Sec. 4 (L158–159), where we state that we use “their target-predictive information to characterize membership rather than a membership label directly.” We will revise the corresponding statement in the abstract accordingly.

Q2 + W1.2: Probe Sensitivity

Q2.1 Can D detect a member-specific change as large as the observed DiD[U] at these sample sizes, as assessed by a positive control or power analysis?

We consider SFT-2ep as a positive control, since it is trained directly on the member solutions, so a member-specific increase in D is expected. Using the same datasets, sample sizes, and probes as for the RL-based methods, D indeed detects this increase (details in our response to Q4).

We further conducted a power analysis for the RL-based methods using the same bootstrap. For ADRA and ADRA+, the upper 95% confidence bound of DiD[D] is below the observed DiD[U] of +0.069 in all 32 layer-wise estimates, and the power to detect a DiD[D] of this size (two-sided, α = 0.05) is above 99% in 31 of the 32 estimates and 92% in the remaining one. At the two layers where DiD[U] is significant, DiD[D] is close to zero (+0.005 and +0.002) (details in Q4). A member-specific change in D as large as the observed change in U would thus have been detected.

Q2.2 Clarification on “rank-limited.”

We appreciate the reviewer for pointing out that our wording could be more precise. We will clarify this in the main text. The intended explanation is that “the logit lens uses the model’s own output head, whereas the probe is estimated from a small number of problems, so G(L) < 0 is expected” (L227–228).

W1.2 Probe Capacity and Reliability.

We agree that the probe is weaker than the model’s own head, which is why we interpret D as a lower bound, as stated in L229 of the main text. Our main conclusions, however, are based on DiD[D], which compares differences across groups and checkpoints under the same probe settings. Thus, shared limitations may largely cancel out in DiD[D], with limited impact on our main conclusions. We have also further examined whether increasing probe capacity or training data substantially affects DiD[D] for the RL-based methods. We observed no clear systematic increase with more probe-training data (50%–100%; 95% CIs include zero in 70 of 84 estimates), a smaller output vocabulary (80%–95% token coverage; 94 of 112), or a different probe rank (38 of 40), with DiD[D] remaining between −0.018 and +0.044 nats across these settings. Similarly, replacing the linear probe with an MLP yielded statistically significant DiD[D] in only 1 of 10 settings for each method, suggesting that the findings are largely consistent across different probe settings. Full results will be added to the final version.

AuthorsOpenReview ↗

Response to Reviewer S7ZY (Part 2/5)

W1.3, W1.5: SFT/RL Contrast

W1.3 Clarification of DiD[D] Effect Sizes and Statistical Uncertainty.

We agree that the DiD[D] point estimates of ADRA+ are comparable to those of SFT-2ep at some layers, and our intervals cannot exclude effects of this size. Our central claim, however, is that RL-based methods show no detectable increase in D (upper bounds: +0.036 for ADRA, +0.054 for ADRA+, details in Q2), not that D is unchanged. We test whether the two differ beyond this under W1.5.

W1.5 Clarification of the SFT–RL Distinction and Differences in Effect Size.

Fig. 5 reports total changes averaged over members and non-members, whereas our contrast concerns member-specific changes (DiD, Fig. 4). We do not claim that SFT changes only D, as stated in our main text, L319-320, “both DiD[D] and DiD[U] show larger differences when the number of epochs increases under SFT.” To test whether the contrast reflects only a power asymmetry, we computed the ratio DiD[D]/DiD[U] with paired bootstrap resamples at the layers where DiD[U] is significant after BH correction. For SFT-2ep, the ratio is 0.22–0.46 at L28 and L32 on both datasets, with 95% CIs excluding zero. For ADRA+ on AIME L32, DiD[U] is larger than that of SFT-2ep at the same layer (+0.185 vs. +0.070), yet the ratio is 0.01 (95% CI [−0.16, 0.22]), below the interval for SFT-2ep (0.46, 95% CI [0.23, 0.74]). For ADRA on Olympia Math L16, the ratio is 0.07 (95% CI [−0.13, 0.40]). Thus, a larger readout change is accompanied by a smaller change in D, which cannot be fully explained by differences in effect size or statistical power alone.

Q3 + W1.1, W2.1, W2.2: Reward Design and Membership Prior

We appreciate the reviewer’s careful consideration. We understand the reviewer's concern that the positive DiD[U] under RL-based MIA methods may arise from the membership prior and would like to provide some further clarification as below.

Q3 How much of ADRA+’s readout effect remains if the Min-K%++ gate is removed or randomized?

Removing the Min-K%++ gate from ADRA+ essentially reduces it to ADRA, as described in the original paper (Yin et al., 2026, Eqs. 5–6). The ADRA results reported in our paper (Figs. 3–6) may therefore serve as a reference for the gate-removed case.

W1.1 At the final layer, DiD[U] reflects the loss-based MIA signal directly targeted by ADRA’s reconstruction reward.

To our understanding, ADRA’s reward is defined on sampled rollouts, scoring their lexical similarity to a candidate pool, and is applied in the same way to members and non-members (Yin et al., 2026, Eq. 5). Although y⋆ is in the pool, the reward acts on the lexical overlap of sampled rollouts rather than on the teacher-forced likelihood of y⋆, and is therefore related to that likelihood only indirectly. Consistent with this, the reference log-likelihood decreases after RL for both members and non-members (Fig. 3, with final-layer ΔU of about −0.25 to −0.83). If the reward were directly increasing this likelihood, we would expect it to rise.

W2.1. ADRA+ uses a Min-K%++ membership prior to gate its reward, introducing member-specific effects by design.

Beyond what we discussed above, a more fundamental point is that if the concern is that the prior makes the supervision membership-dependent, SFT is the extreme case of membership-dependent supervision, since only members are trained on and y⋆ is the direct target. At the final layer on AIME, SFT (2 ep) yields a significant DiD[D] of +0.035 together with a DiD[U] of +0.08. ADRA+ yields a DiD[U] of +0.18, about twice that of SFT, while its DiD[D] is close to zero. The results suggest that membership-dependent supervision alone does not account for the readout-specific pattern we observe.

W2.2. ADRA without the prior shows weaker, scattered effects (Fig. 4). On WikiMIA, ADRA+ is close to zero (Fig. 5).

We acknowledge that ADRA exhibits weaker and less consistent effects than ADRA+ in some settings. Nevertheless, the effect is still observed without the prior: ADRA’s DiD[U] remains significant on Olympia Math (L16,L32, Fig. 4) and WikiMIA-2024-Hard (L16–24, Fig. 5), while its DiD[D] is not significant on any dataset. On WikiMIA, where the prior is close to chance, the gate may effectively remove y⋆ from the matching pool at random, potentially diluting the reconstruction signal. Consistent with this explanation, ADRA shows a larger effect than ADRA+ in Fig. 5. These results further support RL-based MIA’s member-specific readout amplification without a membership prior or a detectable increase in D.

AuthorsOpenReview ↗

Response to Reviewer S7ZY (Part 3/5)

Q4 + W2.3, W2.4, W2.5: Statistical Reliability and Multiple Comparisons.

We greatly value the reviewer's suggestions on statistical reliability. Below, we clarify our experimental setup and provide additional results with multiple-comparison correction.

Q4.1 How many seeds are used?

For non-member sampling, the results reported in the paper are averaged over three random seeds: 1, 2, and 12 for Olympia Math, and 7, 21, and 42 for AIME.

Q4.2, W2.4 Multiple-Comparison Correction.

Following the reviewer's suggestion, we recomputed the confidence intervals in Fig. 4 using 10,000 bootstrap resamples, independently sampling problems from the member and non-member groups. We then applied the Benjamini–Hochberg (BH) correction to the resulting two-sided bootstrap p-values across all 192 tests (2 datasets, 4 training settings, 8 layers, and 3 metrics), with an FDR threshold of q = 0.05. After correction, 22 results remain significant, consistent with our main findings. Specifically, all significant DiD[D] results arise from SFT, particularly SFT-2ep on AIME (L12, L20–L32) and Olympia Math (L28–L32), supporting increased member-specific decodable information under SFT. In contrast, neither RL-based MIA method shows significant DiD[D], while positive DiD[U] remains significant for ADRA (Olympia Math, L16) and ADRA+ (AIME, L32). These results further support our main finding that RL-based MIA methods primarily amplify membership-specific readout, without a corresponding detectable increase in decodable information.

W2.5 Binary Probe Stability.

The 0.29–0.55 AUROC range in App. A.4 (L763) corresponds to different configurations across 9 layers and two pooling strategies (mean/last), rather than repeated evaluations under the same setting. The below-chance AUROCs indeed indicate instability in the binary probing baseline, which is also the limitation we intended to highlight in App. A.4. Binary probing assigns only one label per problem, making it difficult to reliably estimate membership signals. For this reason, we do not adopt binary probing in our main experiments and instead use token-level DUG analysis.

W2.3 Dataset Size.

We understand the reviewer's comment regarding the “16 vs. 16 examples on AIME and 32 vs. 32 examples on Olympia Math” as a concern about the sample size. We would like to provide some additional context regarding our experimental setting, as follows.

(1) Our study focuses on controlled contamination of evaluation benchmarks, where the number of contaminated examples can be inherently small. The experimental setup follows ADRA (Yin et al., 2026), which also studies small-scale controlled post-training contamination on AIME and Olympia Math. Similarly, Self-Critique (ICLR 2026) injects 50% of AIME 2024/2025 into RL training, corresponding to approximately 15 member and 15 non-member problems per benchmark.

(2) Larger benchmarks are not necessarily more reliable for membership inference due to potential distribution shifts (Duan et al., 2024; Meeus et al., 2025; Das et al., 2025). For instance, model-independent classifiers achieve 99.0% AUROC on WikiMIA and 99.9% on WikiMIA-24, compared with only 57.7% on WikiMIA-Hard. Such distribution shifts could confound our analysis of membership signals in internal representations.

Q5: AIME Dataset.

As noted in App. A.1 (Datasets and Models, L712–713), “All examples used as members are randomly selected from their respective member pools.” We would also like to clarify that membership in our setting is introduced through controlled SFT (post-training), and all effects are measured relative to the base model θ₀ (Eq. 6). Therefore, potential exposure during Llama-2 pre-training is already reflected in θ₀ and should be largely accounted for when taking the difference.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their thoughtful feedback and valuable suggestions, which have helped us further strengthen our work. We are grateful that the reviewers recognized the value of our DUG framework in distinguishing decodable information from readout accessibility (qvNp, 7bHh), the experimental design with question-level cross-fitting and shuffled-label controls (S7ZY, 7bHh), and the complementary experiments across SFT, RL-based MIA, and RLVR, including activation patching and steering analyses (qvNp). We summarize our main clarifications and additional experimental findings below:

  1. Additional Probing Experiments (S7ZY, qvNp, 7bHh). We conducted additional experiments with nonlinear MLP probes, varying probe-training size, capacity, and vocabulary coverage. With RL-based methods, we found no consistent member-specific increase in decodable information, consistent with our finding that these methods primarily affect readout.

  2. Statistical Robustness (S7ZY). We performed additional power analysis, paired bootstrap comparisons, and multiple-comparison correction with 10,000 bootstrap resamples. After correction, member-specific readout gains persisted for RL-based MIA methods , while only SFT showed a detectable member-specific gain in decodable information.

  3. Representation Alignment (qvNp, 7bHh). Following the reviewers' suggestions, we introduced representation alignment controls using disjoint non-member activations. The membership-specific readout effect remained significant after alignment, consistent with our finding that RL-based MIA suppresses readout less for members than for non-members.

  4. Role of the Membership Prior (S7ZY). We further examine the influence of reward design and membership priors, showing that member-specific readout effects can emerge without an explicit membership prior.

  5. Causal Analysis (S7ZY, qvNp). Section 5.5 highlights L8 and L20 to narrow down where the membership-specific effect emerges. We clarify that we performed bidirectional activation patching across layers on both datasets and report the full layer-wise results in our response.

  6. Membership Signals under RLVR (S7ZY). At the reviewer’s suggestion, we evaluated Min-kNN Distance as a non-likelihood membership detector. We also repeated RLVR training with two additional seeds to assess the member–nonmember accuracy gap. Our RLVR case study shows that this gap can widen without a corresponding increase in likelihood-based membership signals.

We are deeply grateful for the reviewers' insightful suggestions, and we will include these additional experiments and analyses into our final version of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.