acceptodds
Under review as a conference paper at ICLR 2027

When Membership Signals Lie: Per-Sample Membership Inference via Intermediate Checkpoints

Abstract

Membership Inference Attacks (MIAs) are the primary tool for privacy auditing and machine unlearning evaluation. Population-based MIAs, which typically derive membership scores solely from the target model's output, are computationally cheap and widely adopted under the assumption that they provide a weak but informative signal. Per-sample MIAs such as LiRA, in contrast, account for individual sample hardness but require training tens to hundreds of shadow (reference) models. Challenging the weak-but-informative assumption, we introduce a vulnerability-stratified evaluation protocol and show that population-based MIAs can produce an inverted signal on vulnerable samples—those at greatest risk of privacy leakage—assigning them lower membership scores than typical non-members. Avoiding both the inverted signal of population-based MIAs and the prohibitive cost of per-sample MIAs, we propose Ghost-LiRA, a per-sample membership inference that requires no additional model training. Motivated by the observation that vulnerable samples tend to be memorized in the later stages of training, we find that on these samples, mid-stage checkpoints, naturally saved during training, behave almost as if the sample had been excluded, providing the proxies LiRA needs. Ghost-LiRA thus replaces LiRA's spatial ensemble of shadow models with a temporal ensemble drawn from a single training trajectory. Across four datasets and architectures, Ghost-LiRA matches the privacy-critical [email protected]%FPR of online LiRA with 16–64 shadow models at a roughly reduction in computational cost. Applied to unlearning evaluation, Ghost-LiRA exposes substantial residual leakage on vulnerable samples—leakage that population-based MIAs fail to detect.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Membership inference attacks, Machine unlearning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xF92OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Ghost-LiRA, a per-sample membership inference attack that exploits the observation that privacy-vulnerable samples tend to be memorized later in training. Based on this observation, and assuming that an intermediate training checkpoint of the target model is available, the method leverages the gap between the intermediate and final checkpoints to identify training samples that were learned toward the end of the training process (hence more vulnerable to privacy attacks). The authors further suggest that Ghost-LiRA can be used by model owners to conduct privacy audits of their models and to evaluate the effectiveness of machine unlearning.

Critical strengths

Ghost-LiRA suggests a compelling per-sample MIA under limited computational budgets and with no required shadow model training, but under a strong assumption (availability of a training checkpoint of the target model being attacked).

The authors conducted extensive experiments across multiple datasets and model architectures that support their observation.

Critical weaknesses

The biggest drawback of the proposed approach is the assumption that an intermediate training checkpoint of the attacked target model is available to the attacker. If such a checkpoint is unavailable, the method becomes obsolete.

The methodological novelty of the paper is limited by prior works such as TrajectoryMIA and SeqMIA, which already exploit intermediate checkpoints for membership inference. In contrast to GhostLiRA, the previous methods did not assume availability of target checkpoints, but rather tried to estimate those. Hence, the comparison with SeqMIA in appendix H does not fully isolate the contribution of the respective attack mechanism. Ghost-LiRA uses the true intermediate checkpoints while SeqMIA is evaluated using approximated ones.

Questions for authors

Section 3 analyzes the limitations of population-based MIA. Mainly, it highlights the inversion of membership signals of valuable samples. However, previous works already well established the limitations of population-based MIA, and sample-based attacks are considered SoTA for a while. What new insight does Section 3 provide, and how is it relevant to the proposed method?

7 comments

Thank you for recognizing per-sample MIA under a limited compute budget and our experiments across datasets and architectures. We have revised the PDF in response to your comments and list the revised locations below.

Weaknesses

1. Checkpoint assumption. The method remains applicable without target checkpoints. The original submission already described this setting in Section 5, where an adversary without the target's checkpoints trains a single model under the standard assumption of per-sample MIAs and uses its checkpoints as the OUT reference, and evaluated it in Appendix J.4. Since our main results used the target's intermediate checkpoints, the revision makes this setting, which does not use the target's checkpoints, the main threat model. The revisions are as follows, with locations referring to the revised PDF.

  • Section 5, ‘Threat model’ paragraph: we moved the setting of Appendix J.4 of the original submission into this paragraph as the main threat model. Under the standard threat model of LiRA and RMIA, the adversary trains one reference model and uses its intermediate checkpoints as the OUT reference, querying the target model only for its final outputs. Using the target model's own checkpoints is described as the setting for privacy auditing and unlearning verification by model owners, and as a limited case for external adversaries, such as a checkpoint leak.
  • Section 6.1, ‘Ghost-LiRA settings’ paragraph: we evaluate both settings, denoted Ghost-LiRA (N=1) and Ghost-LiRA (N=0), where N is the number of models trained for the attack. Figures 3 and 4 and the result figures in the appendix report both.
  • Abstract, Section 1, Section 6.2 ‘Effect of the number of trained models’ paragraph, and Table 1 (new): the main performance and cost claims are restated for N=1.

2. Novelty and comparison with TrajectoryMIA and SeqMIA. The asymmetry you describe was present in the submitted SeqMIA comparison. Both works assume the adversary sees only the target's outputs and approximate the trajectory by distillation, whereas the submitted Ghost-LiRA used the target's true intermediate checkpoints. To isolate this factor, we keep SeqMIA's setting unchanged and add to the same comparison table the Ghost-LiRA (N=1) result from Appendix J.4 of the submission. This variant does not use target checkpoints.

  • Appendix H, Table 7 (formerly Table 6): we added the Ghost-LiRA (N=1) row, and the caption states that this variant does not access the target's intermediate checkpoints. Under this same access condition, Ghost-LiRA (N=1) achieves higher [email protected]%FPR than SeqMIA both on the full population (7.48% vs. 0.79%) and on the top-30% vulnerable subgroup (18.12% vs. 1.25%), with a similar overall AUC (0.837 vs. 0.835).
  • Appendix H, ‘Findings’ paragraph, and the footnote in Appendix B.1: we revised the related wording accordingly.

Our novelty does not lie in using intermediate states. We share with both works the intuition that the training process carries membership information. Both works start from the observation that member and non-member trajectories differ at the population level, feed the entire trajectory to a classifier, and leave it to classifier training to determine which state is informative for which sample. We start by answering that question directly, and the difference is twofold. (i) Rationale: we show experimentally that vulnerable samples tend to be memorized late in training (Section 4, Figure 2a), and this is our reason for using intermediate checkpoints. On the vulnerable subgroup, intermediate checkpoints lie closer to the true OUT mean than the final checkpoint does (Appendix J.1, formerly Table 10, now Table 11). (ii) Use: an intermediate checkpoint serves as each sample's OUT reference for per-sample calibration, and the resulting score is a monotone transform of the global-variance LiRA offline statistic (Appendix E). Both works apply a shared classifier to all samples and therefore have no per-sample calibration in this sense (Section 2).

Questions

Section 3's new insight and its relation to the proposed method. The limitations of population-based MIAs are known. What Section 3 adds is that the limitation is not merely a weak signal: on the most vulnerable samples, it can make leaking samples look safe.

Carlini et al. [A] report that the Loss attack falls below chance at low FPR on the whole population. Watson et al. [B] observe that uncalibrated attacks yield high FPR on easy non-members, but they also report results on the whole population. Neither work separately evaluates how population-based attacks behave on vulnerable samples. Meanwhile, unlearning evaluation has continued to rely on population-based attacks because of their low cost, a practice that treats their signal as weak but valid (Section 1). A weak signal would underestimate leakage while still pointing in the right direction. We address this gap by stratifying samples by privacy score and evaluating population-based attacks separately on the most vulnerable samples.

Aggregate metrics make the population-based signal look weak but valid. On the full CIFAR-10 forget set of the original model, Loss gives 1.14% TPR@1%FPR and 0.536 AUC, slightly above random. On the top-5% vulnerable subgroup of the same model, however, it gives 0.65% TPR@1%FPR and 0.441 AUC, ranking these vulnerable members below typical non-members (formerly Table 1, now Table 2). Our results thus show that on the most vulnerable samples, the population-based signal can be inverted. In unlearning, this difference changes the conclusion. After SCRUB, the aggregate Loss AUC is 0.417, which would conventionally read as successful forgetting. On the retrained model, which was never trained on the vulnerable samples of this forget set, Ghost-LiRA gives 0.06% TPR@1%FPR, the level expected when these samples are truly absent. After SCRUB, it gives 6.67% on the same samples, showing that Ghost-LiRA detects the residual leakage that the aggregate evaluation misses. An unlearning run that looks successful under aggregate population-based evaluation can therefore leave membership traces on vulnerable samples. On the other benchmarks and strata, population-based attacks likewise fall to or below random on the vulnerable subgroup (Figure 3b, Figures 7 and 8).

This diagnosis leads to the proposed method. If the final output alone cannot separate sample difficulty from membership, a per-sample reference is needed. Section 4 shows that, because vulnerable samples tend to be memorized late, intermediate checkpoints can provide this reference, and Ghost-LiRA subtracts it from the final output. On the same vulnerable subgroup of the original model, Ghost-LiRA reaches 30.32% TPR@1%FPR (formerly Table 1, now Table 2).

We believe this addresses the concerns raised, and would be glad to clarify any remaining point.

[A] N. Carlini et al. Membership inference attacks from first principles. (IEEE S&P 2022)

[B] L. Watson et al. On the importance of difficulty calibration in membership inference attacks. (ICLR 2022)

I thank the authors for their detailed response and revisions. I find the proposed idea interesting and potentially valuable, particularly the N=1N=1 setting, which enables membership inference using a single independent reference model without requiring access to the target model's intermediate checkpoints. However, while N=1N=1 is now presented as the paper's primary threat model, I believe its underlying mechanism and robustness warrant further investigation to support this more central role. A new approach of this kind requires a deeper empirical understanding of why it works, when it is effective, and where its limitations lie.

Some aspects that I believe remain insufficiently explored include:

1. Role of intermediate checkpoints. It remains unclear why intermediate checkpoints from an independent reference model should provide a better OUT reference than its final checkpoint, particularly for samples excluded from the reference model's training set.

2. Validity of the OUT approximation. The proxy-quality analysis in Appendix J considers only N=0N=0, leaving open whether the same approximation properties hold for an independently trained reference model (N=1N=1).

3. Performance relative to LiRA and RMIA. Ghost-LiRA (N=1N=1) is promising and efficient, but its advantage varies across datasets, FPR thresholds, and shadow-model budgets. The conditions under which it outperforms existing attacks remain insufficiently characterized.

4. Robustness and limitations. The checkpoint-selection and sensitivity analyses also focus exclusively on N=0N=0, leaving the robustness of N=1N=1 across checkpoint choices and training conditions insufficiently explored.

5. Scaling with additional reference models. It remains unclear how Ghost-LiRA would behave with multiple independent reference trajectories, whether additional reference models would improve its calibration, and how this would affect its performance–computation trade-off relative to LiRA and RMIA.

Overall, I appreciate the authors' efforts, the substantial experimental evaluation, and the improvements made during the rebuttal. My main reservation is not about the proposed idea, but about the gap between demonstrating that a new method works and developing a clear understanding of why and under what conditions it works. The points above illustrate aspects that remain insufficiently explored rather than a specific list of required experiments. Given the central role of N=1N=1 in the revised paper, I believe this setting deserves a more thorough investigation. I therefore maintain my original score.

Reviewer_3jdbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper propose Ghost-LiRA, a per-sample membership inference attack where intermediate training checkpoints are used as reference models. The methods achieves comparable performance comparing to LiRA.

Critical strengths
  1. Per-sample membership is a very critical and important area to study. I appreciate the authors timely attention on this topic.
  2. The hard samples memorization patterns is an interesting stepping stone for many attacks, so I like how the authors used this piece of information as well.
Critical weaknesses
  1. The major claim in this paper is that Ghost-LiRA can achieve similar attack performance as LiRA with much lower computational cost. However, I find this claim difficult to support. First of all, this paper has a very different threat model (and less practical) comparing to LiRA. The paper assumes that intermediate target model checkpoints are available to the attacker. This is a very very strong assumption. Not all models or applications naturally save their checkpoints and make them available to the public. Thus, it need lots of justification with evidence to make this assumption sound.

  2. A few details in Ghost-LiRA algorithm made this attack even less realistic. 1) The algorithm has a component to select checkpoints (Equation 9), this is rarely the case in real world and can cause significant privacy concerns (by letting an attacker picking whatever checkpoints they think might be useful, it's exposing the training trajectories). 2) Instead of saying Ghost-LiRA doesn't require extra training, I'm more interested how the number and quality of mid-stage checkpoints can affect the attack performance . 3) for the vulnerability-stratified evaluation protocol, the privacy score still requires training 256 shadow models (Appendix F), though precomputed, it still sounds like a lot of computation cost (this is minor comparing to the first two subpoenas, since it doesn't look like Ghost-LiRA used privacy score for training).

  3. The unlearning section is not very informative, it was hard to tell if Ghost-LiRA performs well in unlearning or not simply by reading the finding paragraph.

Suggestions: Some of the concepts about the hard-to-fit samples are memorized later in the training feel connected to the concept of curriculum learning (Y. Bengio, et al. Curriculum learning. In ICML, 41–48, 2009), there are privacy study in this area too. Might be worth it for the authors to take a look on it. I think per-sample MIA is still a valuable direction, but I'd suggest the authors to rethink the threat model.

Questions for authors
  1. Have you tested how the number and quality of mid-stage checkpoints can affect the attack performance? Figure 2 suggests that there's a sweet spot for a good mid-stage checkpoint to be useful. What if that one checkpoint is not available? Having an evaluation on the effect on this factor would be valuable.

  2. How can an external attacker obtain the intermediate checkpoints, and what evidence supports this assumption?

2 comments

Thank you for recognizing the importance of per-sample membership inference and our use of the late memorization of hard samples. We have revised the PDF in response to your comments and list the revised locations below.

Weaknesses

1. Threat model and comparison with LiRA. The main scenario of the original submission was that of model owners. Section 5 stated that the natural use case is privacy auditing and unlearning verification by model owners, and the main results and cost comparison were made from this perspective, using the target's own intermediate checkpoints. The original submission also covered the setting in which the adversary trains one model itself. Section 5 stated that an adversary without the target's checkpoints can train a single model under the standard assumption of per-sample MIAs and use its checkpoints as the OUT reference, and the experiment was given in Appendix J.4. In the revision, we extend the main text to make this external adversary setting the main threat model and align it accordingly. The revisions are as follows, with locations referring to the revised PDF.

  • Section 5, ‘Threat model’ paragraph: we moved the setting of Appendix J.4 of the original submission into this paragraph as the main threat model, following the standard threat model of LiRA and RMIA. The adversary queries the target model as a black box, knows the training recipe, and has auxiliary data from the same distribution. Under these assumptions, it trains one reference model and uses its intermediate checkpoints as the OUT reference. Using the target model's own checkpoints is described as the setting for privacy auditing and unlearning verification by model owners, and as a limited case for external adversaries, such as a checkpoint leak.
  • Section 6.1, ‘Ghost-LiRA settings’ paragraph: we evaluate both settings, denoted Ghost-LiRA (N=1) and Ghost-LiRA (N=0), where N is the number of models trained for the attack. Figures 3 and 4 and the result figures in the appendix report both.
  • Abstract, Section 1, Section 6.2 ‘Effect of the number of trained models’ paragraph, and Table 1 (new): the cost comparison is restated for one trained model.
  • Abstract, Section 1, and Section 5: we removed phrases that presumed checkpoints are already available, such as "naturally saved during training" and "byproduct of standard training".

2. Details of Ghost-LiRA. (1) The original submission did not assume that an external adversary selects among the target's checkpoints. Selecting among the target's checkpoints is possible for the model owner, whereas an external adversary selects among the checkpoints of the model it trains itself, applying Eq. 9 with that model's validation accuracy. We made this clearer in the ‘Selecting the OUT window via validation accuracy’ paragraph of Section 5.1. The case in which an external adversary obtains a leaked checkpoint is treated separately. A single mid-training checkpoint then suffices without validation accuracy, and various mid-training checkpoints each give 1.16 to 2.18% [email protected]%FPR (Appendix J.3, formerly Table 11, now Table 12).

(2) We answer the number and quality of checkpoints in Question 1.

(3) The 256 shadow models are used only to define the vulnerability strata for evaluation, not in the attack (Appendix F.4). As you note, Ghost-LiRA itself does not need them, and they are not included in the cost comparison above. The stratification is an evaluation device that shows how population-based and per-sample MIAs judge vulnerable samples differently. Privacy scores are computed once per benchmark and will be released with the code (Reproducibility statement), thus researchers using this protocol do not need to pay the same cost again.

3. Unlearning. As an evaluator of unlearning, Ghost-LiRA detects residual membership signals on vulnerable samples that population-based attacks miss. The unlearning section is one case in which the inverted signal on vulnerable samples (Section 3) changes a practical conclusion. Unlearning evaluation commonly relies on population-based attacks (Section 1), so these attacks can judge vulnerable samples as forgotten even when membership traces remain. On the vulnerable samples of the CIFAR-10 forget set, Ghost-LiRA detects members on the original model (30.32% TPR@1%FPR), stays near zero on the retrained reference, which never saw the forget set (0.06%), and flags the residual signal above this reference after SCRUB (6.67%). On the same samples, Loss attack gives 0.00% both on the retrained model and after SCRUB, and does not detect this difference (formerly Table 1, now Table 2).

To make this finding clearer, we rewrote the ‘Finding’ paragraph in Section 7. Because Ghost-LiRA is an evaluation tool rather than an unlearning algorithm, the paragraph now concludes with what it shows as an evaluator, and it states that the comparison is against the retrained reference.

Suggestions. We added a ‘Training order and privacy’ paragraph to Appendix B.2. Curriculum learning [A] presents examples in an easy-to-hard order by design, and Chen et al. [B] show that this ordering makes membership inference slightly more effective, especially on difficult samples. The paragraph also notes that our analysis instead concerns standard training without a designed order, in which hard samples are fit late. Your suggestion on the threat model is addressed by the revision described in Weakness 1.

Questions

1. Yes, we evaluated this, and performance does not depend on any single checkpoint.

With fewer checkpoints, performance stays at a similar level. On CIFAR-100, [email protected]%FPR is 2.98% with four checkpoints and 2.18% with the best single checkpoint (Appendix J.3, formerly Table 11, now Table 12).

The sweet spot is wide. Sweeping the OUT window on all four benchmarks shows a broad plateau at an upper α of 0.95 to 0.97, within which performance barely changes as the lower α varies (Appendix J.2, Figure 10). Our setting (0.90, 0.97) lies at the center of this plateau on every benchmark.

We measured checkpoint quality by rank agreement with, and distance to, the OUT mean estimated from shadow models (Appendix J.1, formerly Table 10, now Table 11). Because Ghost-LiRA's performance depends on the ranking across samples, rank agreement relates most directly to performance. On all four benchmarks, intermediate checkpoints behaved closer to OUT models than the final checkpoint on vulnerable samples, with higher rank agreement and smaller distance.

If a particular checkpoint is unavailable, other available checkpoints can be used (Appendix J.3, now Table 12). The case with no target checkpoints at all is now evaluated in the main text as the main setting (N=1) (Section 6, Figures 3 and 4).

2. We do not claim that external adversaries can generally obtain the target's intermediate checkpoints. The original submission mentioned such access as a possibility in limited cases, such as a leak, and stated that an external adversary without checkpoints can obtain them by training one model itself under the standard assumptions of per-sample MIAs (Section 5). The revision emphasizes this distinction. The main threat model follows LiRA and RMIA: the adversary trains one model and queries the target model for its final outputs (Section 5, ‘Threat model’ paragraph). Access to the target's checkpoints is described as the owner setting and, for external adversaries, as a limited case such as a leak.

We believe this addresses the concerns raised, and would be glad to clarify any remaining point.

[A] Y. Bengio et al. Curriculum learning. (ICML 2009)

[B] J. Q. Chen et al. A comprehensive study of privacy risks in curriculum learning. (PoPETs 2025)

Reviewer_snZTOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the limitations of population-based membership inference attacks (MIAs), showing that they can produce inverted membership signals on privacy-vulnerable samples. To expose this issue, the authors introduce a vulnerability-stratified evaluation protocol that separately assesses attack performance across different privacy-risk groups.

The paper further proposes Ghost-LiRA, a computationally efficient per-sample MIA that replaces LiRA's independently trained shadow models with intermediate checkpoints from a single training trajectory. Motivated by the observation that vulnerable samples tend to be memorized late in training, Ghost-LiRA uses intermediate checkpoints to approximate the OUT-model reference confidence and estimates membership through confidence changes between intermediate and final checkpoints.

Experiments across four datasets and architectures demonstrate that Ghost-LiRA achieves competitive performance with substantially lower computational cost. The authors also apply Ghost-LiRA to machine unlearning evaluation, revealing residual membership signals that conventional population-based attacks fail to detect.

Critical strengths
  1. Addressing findings on the limitations of population-based MIAs. The paper identifies a meaningful failure mode in which conventional MIAs can assign lower membership scores to highly vulnerable members than to non-members. The proposed vulnerability-stratified evaluation provides a more informative assessment of privacy risks than aggregate metrics alone, particularly for machine unlearning evaluation.

  1. Simple and computationally efficient attack design. Ghost-LiRA offers an interesting way to approximate per-sample membership inference using intermediate checkpoints instead of training additional shadow models. The method is straightforward to implement and achieves competitive performance across multiple datasets and architectures, with substantial computational savings. Its application to detecting residual privacy leakage after unlearning further demonstrates its practical relevance.
Critical weaknesses
  1. Insufficient justification of intermediate checkpoints as OUT-model proxies. The central assumption of Ghost-LiRA is that intermediate checkpoints approximate models trained without the queried sample. However, these checkpoints have already been trained on that sample and are highly correlated. Moreover, confidence changes between intermediate and final checkpoints may reflect general learning dynamics rather than membership-specific memorization. Although Figure 2 provides empirical support for the approximation, it remains unclear how reliably this assumption holds across different training regimes, particularly when samples are memorized early or repeatedly encountered. The method also ignores per-sample variance in the OUT distribution, which is an important component of LiRA's calibration.

  1. Concerns about the theoretical claims and unlearning evaluation. Proposition 1 establishes the existence of incorrectly ordered member–non-member pairs under an assumption that essentially already implies this ordering. It does not establish population-level signal inversion or a general impossibility result for output-based MIAs. Additionally, Table 1 reports substantially below-random AUC values ​​even for the retrained reference model (e.g., Ghost-LiRA AUC = 0.198 on highly vulnerable samples). This suggests possible distributional or sample-selection effects in the evaluation. Further analysis is needed to distinguish residual membership leakage from systematic differences between the selected sample groups and from training-induced confidence shifts.
Questions for authors
  1. How accurately do intermediate checkpoints approximate the true per-sample OUT distributions, particularly for early-memorized or repeatedly sampled examples?

  2. Proposition 1 follows almost directly from Assumption 1. Can the authors establish a stronger distribution-level result supporting systematic signal inversion?

  3. Why does the retrained reference achieve substantially below-random AUC (0.198) on highly vulnerable samples? Could sample-selection bias or distributional differences affect the residual-leakage conclusions?

2 comments

Thank you for recognizing the failure mode of population-based MIAs, the stratified evaluation, and the design of Ghost-LiRA and its application to unlearning evaluation. We have revised the PDF in response to your comments and list the revised locations below.

The original submission also covered an external adversary that trains one reference model and uses its intermediate checkpoints as the OUT reference (Section 5, Appendix J.4). In response to other reviewers, the revision makes this setting the main threat model. We denote it Ghost-LiRA (N=1), where N is the number of models trained for the attack, and the setting that uses the target model's own checkpoints, the main setting of the original submission, Ghost-LiRA (N=0). This change is summarized in our general comment. Table and figure numbers refer to the revision, with former numbers given where they changed.

Weaknesses

1. Justification of the OUT proxy. Of the four points you raise, the first three apply to Ghost-LiRA (N=0), which takes the intermediate checkpoints from the target's own trajectory: (a) the checkpoints have already been trained on the sample and are correlated, (b) confidence changes may reflect general learning dynamics, and (c) how reliably the approximation holds across training regimes. (d) Per-sample variance applies to both settings.

For N=0, the manuscript states (a) as a limitation (Appendix A(ii), (iii)), and we examined all three experimentally.

  • (a) We measured the gap to the true OUT mean directly. On vulnerable samples, the Ghost reference lies closer to it than the final checkpoint does (Appendix J.1, Question 1).
  • (b) Members and non-members are split at random from the same data (Section 6.1), thus confidence changes unrelated to membership appear with the same distribution in both groups, and a score reflecting only such changes would give TPR equal to FPR. On the top-5% vulnerable samples of the CIFAR-10 forget set, Ghost-LiRA reaches 30.32% TPR@1%FPR, compared with 0.06% on the retrained model, which never saw these samples (formerly Table 1, now Table 2).
  • (c) The four benchmarks differ in optimizer and schedule, regularization and stopping point, and initialization (training from scratch and fine-tuning from a pretrained model) (Appendix F.1, formerly Table 4, now Table 5). The result in (a) holds in all four.

In the main setting of the revision, Ghost-LiRA (N=1), this issue does not arise. The reference model is trained separately, on a different random subset and with a different seed (Section 6.1), so its checkpoints are not correlated with the trajectory that produced the target's final output.

(d) Not using per-sample variance is a design choice, and the manuscript states the reason as a limitation (Appendix A(ii)). Checkpoints within the window are correlated, thus their spread underestimates the true OUT variance; Ghost-LiRA therefore approximates global-variance LiRA offline (Appendix E). Carlini et al. [A] also report that global variance outperforms per-sample variance with fewer than 64 shadow models. Reflecting your comment on per-sample variance, we revised the footnote in Appendix E so that it presents this result together with the reason it applies to Ghost-LiRA. With this design, Ghost-LiRA (N=0), which uses the target's checkpoints, achieves higher TPR at low FPR than LiRA online with tens of shadow models on all four benchmarks, without any shadow model (Figures 4 and 9).

2. Proposition 1 and Table 1. We answer these in Questions 2 and 3, respectively.

[A] N. Carlini et al. Membership inference attacks from first principles. (IEEE S&P 2022)

Questions

1. Accuracy of the OUT approximation by intermediate checkpoints. This question concerns Ghost-LiRA (N=0) (Weakness 1). On all four benchmarks, compared per sample with the true OUT mean estimated from 256 shadow models, the Ghost reference of vulnerable samples has a smaller gap and better rank agreement than the final checkpoint (Appendix J.1, formerly Table 10, now Table 11; Appendix G.1, Figure 6). We did not compare variances, for the reason given in Weakness 1.

For early-memorized samples, which are typically easy, the IN and OUT distributions overlap heavily, so per-sample attacks in general find them hard to detect (Figure 2a, bottom-30% curve). Ghost-LiRA likewise gives them low scores, because the gap between their final output and the reference is small (Section 5). In contrast, on the top-30% vulnerable subgroup, where privacy risk concentrates, LiRA online applied to intermediate checkpoints stays near zero [email protected]%FPR until late in training (Figure 2a, Figure 5). Because the memorization time of each sample is not observed, we did not measure accuracy broken down by memorization time.

Our experiments follow standard training, in which every sample is seen once per epoch, and the results above hold under that condition. Repeatedly encountered samples, such as duplicated or oversampled ones, were not evaluated, and we added this to Appendix A(v) as a limitation.

2. Distribution-level result. A result over the full data distribution would depend on how large the hard-to-fit and easy non-member groups are, which varies with data and model, so a stronger general result is difficult to establish without further assumptions on these sizes. Our formal results therefore establish the inversion on the hard-to-fit subset, and at the population level we provide empirical evidence. As you note, Proposition 1 follows closely from Assumption 1; it formalizes the form the inversion takes under that assumption. Appendix D.1 extends it to the rate level: on the hard-to-fit subset, the loss-threshold attack has zero TPR at a positive FPR, which is strictly worse than random guessing at the same FPR. Empirically, population-based attacks fall to or below random on the vulnerable subgroup of all four benchmarks (Figure 3b, Figures 7 and 8). On the top-5% vulnerable samples of CIFAR-10, Loss gives 0.65% TPR@1%FPR and 0.441 AUC, ranking these members below typical non-members (formerly Table 1, now Table 2).

This result concerns the analysis of population-based MIAs in Section 3. Ghost-LiRA needs a per-sample reference because output-only scores are confounded by sample difficulty, as prior work has shown [B]; Section 3 shows how severe this confound can be on vulnerable samples. Whether the inversion holds at the population level therefore does not affect Ghost-LiRA, whose reference and performance are evaluated directly.

3. AUC of the retrained row. The below-0.5 AUC arises from differences between sample groups. Vulnerable samples are atypical, thus even on the retrained model, which never saw them, they receive lower confidence than typical non-members. For a subgroup AUC computed against a common non-member pool, 0.5 is only the pool-wide average. Thus, the reference for fully forgotten vulnerable samples is the Retrained row, not 0.5.

This reference separates residual leakage from group differences and from training-induced confidence shifts. Every row of the table compares the same top-5% vulnerable forget samples against the same 25k non-member pool, and positives and negatives are scored by the same model in each state. Group differences are therefore equally present in the Retrained row, and a model state that shifts the confidence of all samples together is absorbed by the threshold. Only the excess over the Retrained row is read as residual leakage. Against this reference, Ghost-LiRA gives 0.06% TPR@1%FPR on the retrained model and 6.67% after SCRUB, whereas Loss gives 0.00% in both states (formerly Table 1, now Table 2).

In the original submission, this labeling was explained only in the footnote of an appendix table (Table 15). The revision now moves the footnote to Table 2 in the main text and adds what the negatives are and that positives and negatives are scored by the same model in each state. We also revised the ‘Finding’ paragraph of Section 7 to compare against the retrained reference.

As you note, sample selection affects this AUC. What we aim to show is that this phenomenon can occur on vulnerable samples, not that it occurs on all samples. Had less vulnerable samples been chosen as the forget set, the AUC of the Retrained row could have been higher.

We hope these revisions and clarifications are helpful, and remain glad to clarify anything further.

[B] L. Watson et al. On the importance of difficulty calibration in membership inference attacks. (ICLR 2022)

Comments on the paper

2 comments

We sincerely thank the reviewers for their thoughtful and constructive comments. We believe the paper has become stronger thanks to them. We have revised the PDF accordingly and list the revised locations in each reviewer's response. All results of the original submission are retained, and the revision treats the external adversary scenario as the main threat model. This comment summarizes the main changes.

We are grateful that the reviewers found the failure mode of population-based MIAs on vulnerable samples meaningful and the vulnerability-stratified evaluation more informative than aggregate metrics (snZT). They recognized per-sample membership inference as a critical and timely direction (3jdb), and Ghost-LiRA as a compelling per-sample MIA under limited computational budgets (xF92) that builds on the late memorization of hard samples (3jdb). They also noted its simple and efficient design with competitive performance (snZT), the extensive experiments across datasets and architectures (xF92), and its practical relevance for detecting residual leakage after unlearning (snZT).

We changed the main threat model (Reviewers 3jdb and xF92).

Reviewers 3jdb and xF92 raised the concern that assuming access to the target model's intermediate checkpoints is a strong assumption. The original submission also covered an external adversary without this access. Section 5 stated that an adversary without the target's checkpoints can train a single model under the standard assumption of per-sample MIAs and use its checkpoints as the OUT reference, and the experiment was in Appendix J.4. However, because this setting had a secondary place in the original paper, the revision moves it from Appendix J.4 into Section 5 and the main experiments as the main threat model.

As a result, the main text now adopts the standard threat model of LiRA and RMIA as its default setting. The adversary queries the target model for its final outputs, trains one reference model, and uses its intermediate checkpoints as the OUT reference (Ghost-LiRA (N=1), where N is the number of models trained for the attack). The setting that uses the target model's own checkpoints (N=0) serves privacy auditing and unlearning verification by model owners. The evaluations in the revision use both settings, and the unlearning evaluation, which corresponds to the owner setting, uses N=0. With N=1, Ghost-LiRA exceeds in [email protected]%FPR the LiRA online configurations that cost about 87× more on average (Table 1).

We revised the related text accordingly. For example, "requires no additional training" in the Abstract and Section 1 is now modified to "requires at most one trained model", and phrases that presumed checkpoints are already available, such as "naturally saved during training", were removed. Because the cost comparison was added to the main text as Table 1, later table numbers changed, and each response gives the former numbers as well.

Revisions by reviewer

  • Reviewer snZT asked about the justification of intermediate checkpoints as OUT proxies and the interpretation of the retrained AUC in the unlearning table. The labeling explanation, which in the original submission appeared in the footnote of an appendix table, now accompanies Table 2 in the main text, together with a description of the negatives. The related text in Section 7 and Appendices A and E was revised accordingly.
  • Reviewer 3jdb, in addition to the threat model, asked about the selection of checkpoints, noted that the unlearning results were hard to judge from the ‘Finding’ paragraph, and suggested the connection to curriculum learning. We clarified in Section 5.1 that selecting among the target's checkpoints is possible for the model owner, whereas an external adversary selects among the checkpoints of its own model, rewrote the ‘Finding’ paragraph of Section 7 to compare against the retrained reference, and added a paragraph on training order and privacy to Appendix B.2.
  • Reviewer xF92, in addition to the threat model, raised the concern that the SeqMIA comparison does not isolate the difference in access to target checkpoints. A result that isolates this difference was in Appendix J.4 of the original submission. We added this result for Ghost-LiRA (N=1), which does not use target checkpoints, to the SeqMIA comparison table in Appendix H, so the two can be compared directly under the same access condition. The related text in Appendices B.1 and H was revised accordingly. In the discussion, Reviewer xF92 also asked why the N=1 setting works and whether the analyses of Appendix J hold for it. We added an explanation to Section 5.1, the N=1 results to the proxy-quality analysis (Table 11, Appendix J.1) and the OUT-window sweeps (Appendix J.2, Figures 12 and 13), and the scope of a single reference trajectory to Appendix A(vi).
AuthorsOpenReview ↗

Summary of the discussion

Dear AC and reviewers,

We thank the reviewers for taking the time to read our paper and share their feedback. We believe their comments have made the paper stronger. As this is our last comment, we summarize how the concerns of each review were addressed.

Reviewers snZT and 3jdb. The two reviewers did not post further comments during the discussion period.

  • Reviewer snZT rated the paper as weak acceptance. We answered the concerns of the review (the justification of the OUT proxy, the scope of Proposition 1, and the reference for the unlearning table) in our response and the revised PDF.
  • The main concern of Reviewer 3jdb, the threat model, was addressed by the revision described below. We also revised the manuscript for the comments on checkpoint selection and number (Section 5.1, Appendix J), the unlearning findings (Section 7), and curriculum learning (Appendix B.2).

Reviewer xF92: the threat model and the scope of the revision.

  • In the review, Reviewer xF92 called the assumption of access to the target's checkpoints "the biggest drawback of the proposed approach" and wrote that "if such a checkpoint is unavailable, the method becomes obsolete." Reviewer 3jdb raised the same concern and suggested that we "rethink the threat model".
  • In response, the revision made the setting without the target's checkpoints (N=1) the main threat model.
  • In the first follow-up, Reviewer xF92 found the idea "interesting and potentially valuable, particularly the (N=1) setting", noted that N=1 "is now presented as the paper's primary threat model", and wrote that the main reservation "is not about the proposed idea" but about why and under what conditions it works. We responded to each of the five points and added N=1 analyses to the PDF.
  • The latest comment notes that these analyses "address several of my questions", but does not identify which remains open. It states instead that making N=1 the primary focus is a major shift that requires rewriting the Abstract, Introduction, Related Work, Method and Results, and thus a new submission.

We believe the revision does not change the core claim of the paper, because N=1 is a scenario already covered by the threat model in the main text of the original submission, and its results were already reported in its appendix.

1. The scenario was in the threat model of the original main text. The threat model paragraph of the original Section 5 reads: "Ghost-LiRA can also be applied by external adversaries who either obtain a leaked intermediate checkpoint or, in its absence, train a single model under the standard assumption of per-sample MIAs and use its checkpoints as the OUT proxy (Appendix J.4)."

2. The results were already reported in the appendix of the original submission. The original Appendix J.4 reported these results and concluded: "Ghost-LiRA therefore does not depend on properties specific to the target model's own training trajectory, and extends to external adversaries under the standard assumptions of per-sample MIAs at the cost of training one model." The revision moves this result into Figure 4, and the cost table of the original appendix into Table 1. Likewise, the N=1 row added to the SeqMIA comparison in Appendix H takes its numbers unchanged from the original Appendix J.4.

3. The core claim and the motivation are therefore unchanged, and the five sections needed only local revisions.

  • The use of the target model's own checkpoints was the main setting of the original submission, not its core claim. The core of the paper is the finding that vulnerable samples tend to be memorized late in training, so intermediate checkpoints can behave like OUT models on these samples (Section 4, Figure 2). This finding does not depend on whose trajectory the checkpoints come from, and the N=1 results add support for it.
  • The revision promotes N=1 to the main setting and adds analyses in Appendices G.4, J.1, and J.2 that further validate the existing claims. The N=1 rows of Table 1 summarize the cost of these results. The five sections have already been revised accordingly: the cost and performance sentences in the Abstract and Introduction, the threat model and notation in the Method, Figure 4 and Table 1 in the Results, and Appendices B.1 and H for the Related Work. We also added the clarifications requested by the reviewers. Because the claim did not change, local revisions were sufficient.
  • The motivation (the inverted signal of population-based MIAs and the cost of per-sample MIAs), the late-memorization results (Figure 2), and all original N=0 results, including the main and unlearning results, are unchanged. Sections 2 and 3, the Conclusion, the title, and the method sentence of the Abstract were not modified.

We therefore see the revision as the response to what Reviewer xF92 named the biggest drawback. The revised paper makes the same claim as the original submission, supported by results that were already in it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.