- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies the membership inference problem for aligned language models. The goal is to determine whether a candidate sample participated in training when the specific alignment method is unknown and the model may have undergone multi-stage training. The authors integrate prompt statistics, preference response likelihoods, token probability changes, and reward distributions into a 19-dimensional detector vector, hoping to capture the complementary signals left by different training objectives. The method includes two types of scoring: ENV, which requires no aligned shadow model, sorts the detector vector and calibrates it against rank statistics in a reference population, taking the largest degree of abnormality; and SLR, which relies on shadow models, trains one shadow model for each of SFT, DPO, NPO, GRPO, and PPO, uses fingerprint distances on a reference prompt set to blend member and non-member distributions, and then computes likelihood ratios using QDA, naive Bayes, or an AUC-gated weighted version. The experiments cover five model families, use IMDB, HH-RLHF, and GSM8K, and examine continuous rewards, binary verifiable rewards, and four training chains. The authors report that on Llama-3.2-1B, SLR-NB-W achieves good average AUC results under both reward types, and that detectable signals are also preserved in multi-stage experiments.
Critical strengths
-
The research problem has practical value. The paper not only examines the situation under a single training objective, but also incorporates unknown alignment methods and multi-stage training into the membership inference problem. By integrating behavioral signals and likelihood signals from different sources into a unified vector, it enables a clearer comparison of how different alignment methods each leak information.
-
The method provides choices under different resource conditions. ENV requires no training of aligned shadow models, while SLR uses five types of shadow distributions for calibration; QDA, NB, and NB-W clearly reflect the trade-offs among covariance modeling, independence assumptions, and signal selection. The sorted rank calibration is also a design worth verifying.
Critical weaknesses
-
The contribution of this paper is not clearly stated. The Gaussian likelihood ratio, QDA, and NB of SLR all have mature statistical foundations. The existing results are insufficient to show whether the improvement mainly comes from the new signals, the five shadows, the fingerprint, or the scorer. If the features and shadow budget could be fixed, and LR/RF, NB, and QDA compared, with the fingerprint, rank calibration, and dimension weighting removed separately, the experiments would be more convincing.
-
The multi-stage experiments do not sufficiently demonstrate the preservation or overwriting of membership signals across stages. Table 3 does not clearly specify whether the same training data are used across stages. It also does not distinguish samples used only in SFT, only in later stages, in multiple stages, or in none of the stages. Without such distinctions, it is difficult to determine which stage contributes to the final membership inference results.
Questions for authors
-
Could the authors clarify whether the improvement comes from the new signals, shadows, fingerprint, or scorer?
-
Would it be possible to compare LR/RF, NB, and QDA under fixed features and a fixed shadow budget, and to provide corresponding ablation results?
-
The multi-stage experiments do not state whether the data in each stage are the same, nor do they distinguish different stage member types, making it difficult for readers to determine which stage the detection results come from. Please clarify the data setup and the definition of member types at each stage.
4 comments
The authors have supplemented Table 4 and Table 5; however, they still have not fully addressed my concerns. For example, regarding specific quantitative attribution, the ablation experiments lack comparative results for LR/RF, NB, and QDA under fixed features and a fixed shadow budget, and the definition of member type at each stage in Question 3 remains insufficiently clear. Some issues have not been effectively addressed, so I maintain the original score.
Response to Comment 1
We thank the reviewer for this comment. We believe there may be a misunderstanding about what Table 4 already reports. The comparison the reviewer requests, namely LR/RF, NB and QDA under fixed features and a fixed shadow budget, is exactly the design of the component ablation table. This holds for both the one shadow block and the five shadow block. In both blocks the target data are the same batch of target data, so the features are identical across scorers. The shadows are trained on the same batch of shadow data, so the shadow budget is also fixed. The blocks report all five scorers side by side, including LR/RF, NB and QDA. The requested fixed feature and fixed shadow comparison is therefore already present in the revised paper.
We are not fully certain what additional comparison the reviewer has in mind. If the reviewer means a table restricted to a different fixed feature set, or a different shadow budget, we would be grateful for a more precise pointer. Please specify which features, which budget and which table or column, so that we can address it directly.
Response to Comment 2
We thank the reviewer for pressing us to define member type operationally. In plain language, a prompt is a member of a stage if and only if that stage’s training data contained that prompt. The three prompt groups are F (the forget set, 300 prompts), R (the RL set, 1200 prompts) and S (the SFT only set, 1000 prompts). These three groups are mutually exclusive and together cover all 2500 prompts. SFT is the canonical sft@1500 over all 2500 prompts, so every prompt is an SFT member. GRPO trains on F ∪ R, which is 1500 prompts, so F and R are GRPO members and S is not. NPO trains on F only, which is 300 prompts, so only F is an NPO member.
Therefore the member type of each prompt is as follows. F is a member of SFT, GRPO and NPO. R is a member of SFT and GRPO. S is a member of SFT only. The three groups are disjoint, so no prompt belongs to more than one of F, R and S. This is what we mean by stage specific membership.
To explain the purpose of each stage, SFT is the warm up that all 2500 prompts go through, because RL run directly on the base model is unstable and produces unreadable output, so the standard pipeline finetunes first and then runs RL from the SFT checkpoint, and in our chain SFT also plays the role of making a prompt a member at all since unlearning presupposes that something was learned.
GRPO is the RL alignment stage, where the model is updated against a reward, which is a reward model in the HH RLHF regime and a rule verifier in the RLVR regime, and its purpose is to align the SFT checkpoint with that reward.
NPO is the unlearning stage, where naive gradient ascent on the forget data collapses the model, so NPO instead unlearns the forget set while staying anchored to the reference model, and its purpose is to remove the forget set without destroying the model.
The chain matters because real models are trained in stages and a later deletion request hits a model that has already been aligned, so membership is stage specific, and the exposure nesting lets us attribute the trace stage by stage.
The member type definitions, the purpose of each stage and the per stage audits are now stated explicitly in the revised paper and quantified by the experiment above. We would be grateful if the reviewer could let us know whether any further concern remains. If so, please state it explicitly so that we can address it directly.
Thank you for the additional clarifications. I think most of my concerns have been addressed, so I’m willing to raise my score to 3.
Answer to Weakness 1.
We thank the reviewer and clarify the contributions as follows. Our pipeline starts with 19-dim signal extraction and branches into ENV (shadow-free rank calibration) for the no-shadow budget setting and SLR (fingerprint-based multi-shadow calibration). After SLR, the scoring variants are DIBA (classifier trained on shadow data) and the shadow-statistics scorers QDA, NB, NB-W, top-K. These designs are not parallel, they are layered, as shown in the hierarchy added in Figure 3 in Section 5.3 of the revised version.
Specifically, rank calibration is used only for ENV, i.e., the shadow-free multi-signal budget. Multi-shadow calibration is the combined name for the five shadows, fingerprint, and dimension weighting. Only with multiple shadows do we obtain different shadow fingerprints and then use softmax weighting for multi-shadow calibration. After multi-shadow calibration, the scoring variants can be either our proposed LLR-based statistics family or DIBA's LR/RF scorer.
Our contributions can be grouped into three parts: new signals, multiple shadow calibration, and LLR-based scoring variants. To further demonstrate the effect of each of our technique components, we explicitly list the performance of different combinations of our proposed techniques as an ablation study in Table 4, which we have added in the revised version Section 5.3, so that the individual contribution of each technique is clear.
Another advantage of our scorers is that LR/RF trains a classifier on shadow data, whereas our LLR-based scorers require no training: the blended Gaussians are fit in closed form from the selected shadow's statistics, so the scorer adds no optimizer, no iteration, and no failure mode of its own.
Answer to Weakness 2.
Our multi-stage setup is stated in the Multi-Stage Alignment section: every stage of every curriculum trains on the same 1000 prompts. To additionally separate which stage contributes the detectable trace, we add a per-stage trace analysis on a more realistic multi-stage chain.
The new data split as follows. Configs: SFT = 1500 steps on all prompts; GRPO = 1500 steps on F∪R, the reference model is sft; NPO = 1500 steps on F∪R, the reference model is sft; NPO = 1500 steps on F, the reference model is grpo.
Table 5. AUCs of SLR-NB on different data splits (S, R, F).
Analysis:
Answer to Question 1.
Please refer to Answer to Weakness 1.
Answer to Question 2.
Please refer to Answer to Weakness 1.
Answer to Question 3.
Please refer to Answer to Weakness 2.