acceptodds
Under review as a conference paper at ICLR 2027

Opting Out Should Not Be Detectable: Fresh-Identity Indistinguishability for Identity Unlearning

Abstract

Unlearning is evaluated by what a model can still retrieve, not by whether the deletion request is visible. We define fresh-identity indistinguishability (FII) relative to an adversary class and test for its violation with the paired detection-AUC gap between a released encoder and its same-seed retrain oracle on identical identities and splits: a positive gap witnesses a violation. At 100 opted-out Market-1501 identities, an adversary holding no training image reaches AUC 0.802 +/- 0.020 against the oracle's 0.622 +/- 0.081 (gap +0.180, 5/5 seeds), naming 17% of requesters at a 1% false-accusation rate; one unknowingly holding some training images reaches 0.968 +/- 0.008 against 0.717 +/- 0.019 (gap +0.252, TPR 0.588). The mark replicates on ResNet-18 and ConvNeXt-T encoders and on CUHK03-NP, and a detector fit on one opt-out set reads it off disjoint identities (0.801 against 0.591 on their oracle, 20 of 20 pairs), off other architectures and off the other dataset, and it stays detectable through a retrieval service that returns only top-10 ranks. It is a tighter-than-never-seen cluster: forgotten identities remain more self-similar than people the model never saw. Across 39 operating points of five unlearning methods, none keeps retain mAP at or above 0.95, brings forget mAP to at most 0.02 above the oracle's and passes the mixed-adversary FII test; the methods that preserve utility best, neggrad_plus and finetune, leak on every seed under the mixed adversary. Our evidence is three academic benchmarks, not face retrieval.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, privacy, person re-identification, membership inference, unlearning evaluation, representation learning, retrieval

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XwXeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper investigates whether machine unlearning can reveal which individuals have requested to be removed from a model's training data. The authors introduce Fresh-Identity Indistinguishability (FII), a privacy criterion that measures whether forgotten identities can be distinguished from never-seen identities more easily than in a model retrained from scratch. They develop a detection attack based on embedding similarity statistics and evaluate it under two settings, where the attacker either has access to some training images or only uses previously unseen images. Experiments on Market-1501, CUHK03-NP, and Tiny-ImageNet, using different model architectures, show that forgotten identities can leave detectable traces in the embedding space. The authors also evaluate five unlearning methods across 39 configurations and find that none simultaneously achieves strong forgetting, preserves model utility, and eliminates the detection signal under their evaluation criteria. Overall, the paper highlights an additional privacy risk in machine unlearning that is not captured by conventional forgetting metrics.

Critical strengths
  1. Interesting research question: The paper raises an important privacy concern that is often overlooked in machine unlearning: whether the act of requesting data deletion can itself be detected from a released model.

  2. New evaluation perspective: The proposed Fresh-Identity Indistinguishability (FII) criterion provides a useful perspective on evaluating unlearning methods beyond conventional forgetting effectiveness and model utility.

  3. Well-motivated evaluation baseline: Comparing detection performance against a retrained model is a reasonable choice, as it helps distinguish residual training signals from differences that may exist even without exposure to the forgotten identities.

Critical weaknesses
  1. Limited evaluation of unlearning methods: The paper evaluates only four existing unlearning methods and one newly introduced diagnostic baseline. This relatively narrow selection makes it difficult to determine whether the observed privacy leakage is a general limitation of machine unlearning or specific to the methods studied.

  2. Unclear interpretation of FII: The paper defines FII using the detection-AUC gap relative to a retrained model, but the retrained model itself can distinguish target identities from never-seen identities. This raises questions about whether the proposed metric fully captures the intended notion of deletion-request privacy.

  3. No effective mitigation strategy: While the paper demonstrates that existing unlearning methods can leave detectable traces, it does not propose a practical solution to eliminate these signals while maintaining model utility. As a result, the contribution is primarily diagnostic, with limited guidance on how to improve existing methods.

Questions for authors

Generalizability of the findings: The experiments focus primarily on person re-identification models. What evidence supports the applicability of the proposed privacy concern to other model types or unlearning settings?

4 comments

(1/2)

Thank you for the review, and for being direct about presentation. We took it seriously: the abstract now reports four AUCs rather than a dozen figures, the introduction states the claim in a single paragraph on its first page, and the per-seed detail of the abstract and introduction has moved into the tables. Point by point:

W1, the range of methods. The submission had four published methods and our own diagnostic baseline (SCRUB, one of the four, collapsed at 100 identities and is in Table 3 as such); the revision widens that to ten. It adds SSD (Fisher dampening, no gradient descent), Bad-Teacher, SalUn, random-label fine-tuning, SCRUB at its published lower rates, and a re-implementation of De-ReID, all under the identical protocol on the submitted checkpoints (Appendix Q): ten methods and 72 operating points. Every added method, at the operating point the paper's rule selects, leaks on 5/5 seeds under the mixed adversary, with gaps from +0.09 to +0.23 against the original's +0.25, and so do finetune and NegGrad+. Appendix R repeats the exercise for eight methods on a ViT-B/16: every operating point that keeps retain mAP above 0.95 leaks under the mixed adversary on 3/3 seeds. The leakage is not specific to the original methods.

W2, the interpretation of FII. You are right that the retrained model itself separates the targets from never-seen identities (0.72 against a 0.48 null floor), and that is exactly why FII is defined relative to the oracle rather than to chance. Retraining is the exact-unlearning reference, the thing a deletion request is honoured against, so a criterion that retraining fails cannot be asked of approximate methods; the paired gap charges a method only for what it leaves beyond retraining. Appendix N adds the complementary fact: against retained training identities the released original sits at chance (0.49) while the oracle separates them at 0.96, so what the detector reads is "was trained on", and exact retraining is the one procedure that removes it. Both facts are now on the first page: the FII paragraph of the introduction says why the oracle, not chance, is the reference, and the claim paragraph says what the mark is.

W3, mitigation. The paper tests the zero-cost post-hoc defences on the released embedding (nulling the direction that separates forgotten from never-seen identities, Table 7; Gaussian noise and PCA truncation, Table 16) and one unlearning baseline of our own, artifact_aware. That baseline is the one method whose selected operating point sits at the criterion's margin: its gap is within 0.02 of zero under both adversaries, with 1/5 seeds rejecting under the paired bootstrap (the mixed rejection above the oracle, the fresh one below), and it pays for that in retrieval on never-seen identities, test mAP 0.66 against the original's 0.74 and the oracle's 0.72 (Appendix P). In a single-seed control at 20 identities (Section 9, Table 18), the same procedure applied to never-seen identities instead moves the real requesters' gap by -0.33 without seeing a forget image, so its non-leak is not specific to the opt-out identities. Across the ten methods the gap shrinks only where retrieval has been paid for: the methods that leave forget mAP at the original's level (SSD at alpha >= 5, SCRUB at its published rates, NegGrad+, finetune: forget mAP 0.96 to 0.97 in Appendix Q) leak at +0.22 to +0.25, and the smaller gaps belong to operating points that pay in never-seen retrieval (standard-test mAP 0.63 to 0.68 against the original's 0.74, Appendix Q). The contribution is the criterion and the audit, and the audit carries the guidance the data support: a deletion procedure has to move the opt-out identities to where never-trained ones sit, and FII is the test it should be scored against. Appendix Q is what that scoring looks like for ten methods.

(2/2)

Q1, generalisability. The released-against-oracle gap replicates in every setting we have: Tiny-ImageNet as a class-level proxy with no cameras and an oracle at chance; ResNet-18 and ConvNeXt-T beside the ResNet-50 (3/3 seeds each under both adversaries); a ViT-B/16 (3/3 under the mixed adversary by every test, 3/3 under the fresh one by the permutation and bootstrap tests, Appendix R); and CUHK03-NP (3/3 under the fresh adversary; its mixed row is saturated for oracle and released model alike, as Table 2 notes). A detector fitted on one architecture or dataset reads the others above their oracle (Table 13; Table 28 for the ViT). The statistics it uses are cosine similarities among the adversary's own images, and the same battery, less its camera statistic, runs on Tiny-ImageNet (Section 4); from that study Appendix D concludes that the signal is not a property of person re-identification, of metric learning with a BNNeck head, or of Market-1501's camera structure. On the unlearning setting, the gap persists when the 100 identities are deleted one or ten at a time instead of in one batch (Appendix S: +0.19 to +0.27 under the mixed adversary at the 100th request, interval excluding zero in 20/20 runs).

Ethics flag. The ethics statement covers the data (public benchmarks, no face recognition, no real erasure request, services simulated over our own galleries) and the dual-use question: the detector needs a labelled opt-out set and is released for reproducibility and defensive auditing. We will answer any further question the AC or the ethics reviewer raises.

Thank you again. If a particular section still reads badly, tell us which and we will fix it while the discussion is open.

Thank you for the detailed rebuttal and clarifications. The responses have adequately addressed my main concerns regarding the evaluation coverage, the interpretation of FII, and the scope of the paper's contributions. I'd like to increase my score.

Thank you for reading the rebuttal and for raising your score. Your comments on the evaluation coverage and on the interpretation of FII shaped the revision, and your note on presentation led to the rewritten abstract and introduction. Any further suggestion is welcome, and we will carry it into the final version.

Reviewer_nmG6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces fresh-identity indistinguishability (FII) relative to an adversary class and test for its violation with the paired detection-AUC gap between a released encoder and its same-seed retrain oracle on identical identities and split. It helps to detect the residual privacy leakage after unlearning. The experiments are conducted across various models (ResNet-50 , ResNet-18, ConvNeXt-T), datasets (Market-1501, CUHK03-NP, Tiny-ImageNet) and unlearning methods (fine-tuning, NegGrad+, Gradient Ascent, Artifact-Aware).

Critical strengths
  • The paper presents evaluation across different adversary settings: fresh and mixed. Former uses images withheld from training and latter, possess some images used during training.
  • The paper proposes a retraining-based evaluation criterion to demonstrate the efficacy of their paired AUC-gap evaluation by comparing the released model to a same-seed retraining oracle. Therefore, indicating that unlearning can leave detectable traces, specifically tighter identity clusters, which can reveal prior training.
Critical weaknesses
  • The transferred detector is validated only on tight clusters, where the detector is fitted on released originals, where forgotten identities are clustered more tightly than strangers, so it learns that tight cluster means opt out. The person-ReID specific De-ReID method removes a person by shifting the embeddings along the variations of that person’s own appearance. This probably changes the shape and position of the cluster rather than leaving it intact. The shifted opt-out can be regarded as never-seen for the transferred detector on such a model, so the attack may fails. The paper must show experiments to evaluate the transferred detectors on De-ReID-unlearned encoders and show AUC against or acle with two sided gap.

  • The authors should report the standard test-split mAP and Rank-1 accuracy on the held-out identities before and after unlearning, and report these metrics in the utility-forgetting trade-off analysis. This will tell whether the unlearning methods maintain generalization to unseen identities or only retain performance on identities seen during training, giving a more realistic assessment of their practical utility for person ReID.

  • The authors must perform an experiment to show whether the supervised detector remains effective when identities are deleted sequentially, one at a time, rather than through the deletion of multiple identities together. This would evaluate whether the attack generalizes to realistic, incremental erasure requests as happens in a real-world setting.

  • The results mentioned in the paper are conducted only on CNN backbones and on simple person ReID datasets. It limits the generalizability of the findings of the paper. The authors must evaluate a ViT (TransReID) or multimodal ReID model (DIFFER, CLIP-ReID), and a ReID-specific unlearning method (De-ReID), and a larger or clothes-changing ReID dataset (MSMT17, LTCC) to determine whether the opt-out signature persists across architectures, unlearning strategies, and appearance variations.

Questions for authors
  • The transferred detector is validated only on tight clusters, where the detector is fitted on released originals, where forgotten identities are clustered more tightly than strangers, so it learns that tight cluster means opt out. The person-ReID specific De-ReID method removes a person by shifting the embeddings along the variations of that person’s own appearance. This probably changes the shape and position of the cluster rather than leaving it intact. The shifted opt-out can be regarded as never-seen for the transferred detector on such a model, so the attack may fails. The paper must show experiments to evaluate the transferred detectors on De-ReID-unlearned encoders and show AUC against or acle with two sided gap.

  • The authors should report the standard test-split mAP and Rank-1 accuracy on the held-out identities before and after unlearning, and report these metrics in the utility-forgetting trade-off analysis. This will tell whether the unlearning methods maintain generalization to unseen identities or only retain performance on identities seen during training, giving a more realistic assessment of their practical utility for person ReID.

  • The authors must perform an experiment to show whether the supervised detector remains effective when identities are deleted sequentially, one at a time, rather than through the deletion of multiple identities together. This would evaluate whether the attack generalizes to realistic, incremental erasure requests as happens in a real-world setting.

  • The results mentioned in the paper are conducted only on CNN backbones and on simple person ReID datasets. It limits the generalizability of the findings of the paper. The authors must evaluate a ViT (TransReID) or multimodal ReID model (DIFFER, CLIP-ReID), and a ReID-specific unlearning method (De-ReID), and a larger or clothes-changing ReID dataset (MSMT17, LTCC) to determine whether the opt-out signature persists across architectures, unlearning strategies, and appearance variations.

6 comments

(1/2)

Thank you for four concrete requests. The revised PDF answers them with new experiments (Appendices P to T), and we take them in the order you raised them.

W1, De-ReID and the transferred detector. De-ReID has no released code, so Appendix T implements the objective of Section 3 of the paper (all four loss terms, the published weights, batch composition, optimiser and augmentation pool) and adapts it to our ResNet-50 protocol: LoRA on layer 4, frozen BatchNorm, the triplet margin detached (a per-anchor reading that the paper's wording permits), and a learning rate of 3e-5 or below. We changed only what the literal transposition broke: transposed literally, the ViT recipe collapses CNN retrieval through the triplet term's data-dependent margin, and an ablation in Appendix T locates that. The paper labels it accordingly, as a protocol-adapted implementation of the variation-guided identity shift.

With that in place, here is how your prediction fares. At the operating point the paper's rule selects (retain mAP 0.990, forget 0.932), De-ReID leaks at +0.220 under the mixed adversary on 5/5 seeds and at +0.132 under the fresh one on 5/5. At the stronger setting (retain 0.949, forget 0.87; it sits just under the rule's 0.95, which is why it is not the operating point) the mixed gap is +0.197 on 5/5. The transferred detector, fitted on the released originals of other seeds and applied without refitting, reads the De-ReID encoders above their oracle on 20 of 20 disjoint-identity pairs in three of the four cells: 0.907 against 0.568 (mixed) and 0.683 against 0.591 (fresh) at the operating point, and 0.833 against 0.568 (mixed) at the stronger setting. The one cell where your prediction holds is the fresh adversary at the stronger setting: there the transferred detector reads 0.614 against 0.591, positive on 12 of 20 pairs and within 0.03 of the oracle, and the encoder has given up retain mAP to get there (0.949 against 0.990 at the operating point). So shifting a person along their own variations does not make them look never-seen to a detector trained on other people, except where retrieval is already being given up. On the ViT-B/16 originals, the architecture De-ReID was designed for, the same procedure placed as the paper places it gives no configuration above the retain-mAP rule at any learning rate from 1e-5 to 3e-4: the LoRA runs collapse, and the whole-block runs either collapse or hold retain mAP at 0.56 to 0.73. We report that in Appendix T rather than a ViT row.

W2, test-split mAP and Rank-1. Appendix P reports test mAP and Rank-1 on the 750 Market-1501 test identities, which no model trained on, for the paper's encoders, with each identity's lowest-camera images as queries against the full test gallery: original 0.739 / 0.854, retrain oracle 0.716 / 0.845, NegGrad+ 0.751 / 0.856, finetune 0.700 / 0.831, artifact_aware 0.661 / 0.807, grad_ascent 0.561 / 0.759. For every run on the submitted checkpoints, Appendix Q adds a "std" column with test mAP under the standard query-set protocol (so its values differ from Appendix P's): originals 0.736, oracles 0.714, NegGrad+ 0.749, finetune 0.743, SCRUB at 0.001 0.746, SSD 0.677, Bad-Teacher 0.646, SalUn 0.643, random-label 0.626, De-ReID 0.656. The methods that keep forget mAP at the original's level also keep never-seen retrieval there (0.743 to 0.749 against 0.736); the methods that forget more pay for it on never-seen people as well, and the column puts that cost next to retain and forget mAP for every configuration.

W3, sequential deletion. Appendix S deletes the 100 identities one request at a time (and, in a second condition, ten at a time) from the submitted checkpoints, drawing each request's retain budget from everyone not yet requested, with total compute equal to one batch run. Every checkpoint is scored on all identities deleted so far. At the 100th request the paired gap on the 100 deleted identities is +0.19 to +0.27 under the mixed adversary and +0.07 to +0.28 under the fresh one, with the bootstrap interval excluding zero in all 20 runs (5 seeds, 2 methods, 2 groupings), the same range as batch deletion. Forget mAP on the deleted identities stays at 0.93 to 0.98 along the way, close to the batch operating points of finetune and NegGrad+ in Table 3 (0.921 and 0.971): the per-request step modifies the encoder as little as the batch run does, which is the under-modification result of Section 7 seen from another angle.

(2/2)

(W3, continued)

Because the final-state retrain is not the exact counterfactual at an intermediate request, we also trained a prefix oracle for every seed at k = 25, 50 and 75, retrained from scratch without only the first k requesters (15 models). Against it, on the 40 equal-compute rows, the gap is +0.17 to +0.38 (mixed, interval excluding zero on 40/40) and +0.06 to +0.31 (fresh, 35/40), within 0.05 (mixed) and 0.06 (fresh) of the final-state reading on every row.

We also ran two larger budgets, because the natural objection to the equal-compute condition is that nothing was given the chance to forget. Applying the batch operating point to every request (on the replicate family) destroys NegGrad+: it is non-finite or at chance retrieval by the 25th single request and by the fifth grouped request on 5/5 seeds. finetune survives that budget with retain mAP swinging between 0.52 and 0.94 and standard-test mAP of 0.36 to 0.66 against 0.75, and its gap is within 0.09 of zero (interval excluding zero in 2 of 30 rows). In between, at ten batch runs of total compute, NegGrad+ holds retain mAP 0.95 through the 50th request on 5/5 seeds (through the 75th on one) with forget mAP still at 0.93 to 0.99 and a mixed gap of +0.13 to +0.31 (interval excluding zero on 10 of those 11 checkpoints), then falls below 0.95 and collapses by the 100th request on every seed; finetune at that budget falls below retain 0.95 by the 25th to 50th request on 5/5 seeds and collapses on 3/5, and the two seeds that reach the 100th end at retain mAP 0.66 to 0.73 and standard-test mAP 0.45 to 0.49. On the checkpoints below the 0.95 line the mixed gap is -0.07 to +0.22. Across the three budgets the picture is the same: the mark does not fade while the encoder is useful, and the budget large enough to move the deleted identities is the budget that destroys retrieval.

W4, a ViT, De-ReID, larger datasets. Appendix R trains ViT-B/16 originals and oracles on three seeds: TransReID's backbone without its side-information embedding, jigsaw branch or overlapping patches, so that only the architecture changes. The released original reads 0.925 against the oracle's 0.691 under the mixed adversary (+0.234, 3/3 by every test) and 0.790 against 0.623 under the fresh one (+0.167, 3/3 by the permutation and bootstrap tests, 2/3 by DeLong on every draw). The ResNet-50 detector reads the ViT's opt-outs at 0.949 against 0.581 on its oracle under the mixed adversary (0.773 against 0.598 under the fresh one; 12/12 disjoint pairs each), and the ViT's detector reads the ResNet-50's at 0.981 against 0.565 (fresh: 0.792 against 0.570). Running the methods on the ViT with their own sweeps, every operating point that keeps retain mAP above 0.95 leaks under the mixed adversary on 3/3 seeds, from NegGrad+ at +0.080 (retain 0.964) to finetune at +0.214 and SSD at +0.202; SCRUB, whose sweep-selected operating point collapses the ResNet-50 at 100 identities, survives on the ViT and leaks at +0.132. De-ReID is covered under W1. MSMT17 and LTCC are distributed under licence agreements that did not clear within the rebuttal window, so CUHK03-NP remains the second dataset, with detector transfer in both directions (Table 13); a multimodal model (CLIP-ReID, DIFFER) is not part of this revision.

Thank you again. The transferred-detector and sequential experiments in particular made the paper better, and we are glad to run any further variant you would find informative while the discussion is open.

As the discussion period closes on 18 November (AoE), we wanted to check whether the new experiments address your four requests:

  • De-ReID and the transferred detector (Appendix T): at the operating point, the detector fitted on other seeds' originals reads the De-ReID encoders above their oracle on 20 of 20 disjoint-identity pairs under both adversaries.
  • Utility on never-seen people: test mAP and Rank-1 on the 750 Market-1501 test identities (Appendix P), and test mAP beside retain and forget mAP for every configuration (Appendix Q).
  • Sequential deletion, one and ten identities at a time, with prefix oracles (Appendix S): at the 100th request the interval excludes zero in 20/20 runs.
  • A ViT-B/16 backbone with its own method sweeps (Appendix R): gap +0.234 (mixed) and +0.167 (fresh), 3/3 seeds under the paired bootstrap.

If any of these would be more useful in another form, we are glad to run it before the deadline. Thank you again for the concrete requests; they improved the paper.

Reviewer_xVZfOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies whether machine unlearning in identity-retrieval models leaves a detectable mark, i.e., whether a person who requested deletion remains distinguishable from a person the model never saw. The authors define fresh-identity indistinguishability (FII) relative to an adversary class and test for violations using the paired detection-AUC gap between the released encoder and a same-seed retrained oracle. A logistic detector built on ten similarity and dispersion statistics is evaluated under a "fresh" adversary, who holds no training images, and a "mixed" adversary, who unknowingly holds some. On Market-1501 with 100 opted-out identities, the fresh adversary reaches an AUC of 0.802 against 0.622 for the oracle. The effect replicates on other encoders and on CUHK03-NP, transfers across identities, architectures, and datasets, and survives a top-10 retrieval interface. Across benchmarks of five unlearning methods, including NegGrad+ and SCRUB, none simultaneously preserves utility, matches oracle-level forgetting, and passes the FII test.

Critical strengths
  1. The paper studies an important and under-explored question: whether the act of removal itself is visible in the released model.
  2. Using a retrained oracle [1], rather than chance, as the reference is the right comparison, and the fresh adversary uses images that were never part of training or unlearning.
  3. The experimental study is extensive: transfer tests, restricted top-10 access, a 39-configuration sweep, a seed-stability study, and ghost-unlearning controls.
  4. The authors report limitations, including a corrected data-loader bug, the limits of the Tiny-ImageNet proxy and low-FPR estimates, and the fact that the evidence comes from academic benchmarks rather than face retrieval.

[1] L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot. Machine Unlearning. IEEE Symposium on Security and Privacy, 2021.

Critical weaknesses
  1. The paper is hard to read: the abstract and main text are dense with numbers, the main claim is hard to extract, and many conclusions rely on appendices that mix corrected and earlier experiments, with some inconsistencies (e.g., the Tiny-ImageNet ordering, top-10 significance, and Table 5 p-values).
  2. The headline 0.802/0.968 AUCs are for the original, non-unlearned model, i.e., essentially user-level membership inference, which has already been studied on Market-1501 [2]. Prior work on the privacy risks of unlearning [3] is not cited. The contribution is mainly the retrain-calibrated audit rather than detecting deletion per se.
  3. The label-permutation test checks feature-label independence, not equality of two non-zero AUCs; with oracle AUCs of 0.62-0.72, the "significant on 5/5 seeds" claims need a valid paired test.
  4. The unlearning baselines are limited. SCRUB's learning rate of 0.01 is attributed to the published setting, but the SCRUB paper uses 0.0005 [4], and the collapse conclusions depend on this choice. Only gradient-based methods are evaluated: second-order identity unlearning, e.g., L-CODEC [5], which unlearns individual identities from re-ID models on Market-1501, is neither cited nor compared, and retrieval-specific methods (De-ReID, TIU-ReID) are cited but not evaluated. The "none of the methods" conclusion is therefore narrow.

[2] G. Li, S. Rezaei, and X. Liu. User-Level Membership Inference Attack against Metric Embedding Learning. ICLR Workshop on PAIR2Struct, 2022. [3] M. Chen, Z. Zhang, T. Wang, M. Backes, M. Humbert, and Y. Zhang. When Machine Unlearning Jeopardizes Privacy. ACM CCS, 2021. [4] M. Kurmanji, P. Triantafillou, J. Hayes, and E. Triantafillou. Towards Unbounded Machine Unlearning. NeurIPS, 2023. [5] R. Mehta, S. Pal, V. Singh, and S. N. Ravi. Deep Unlearning via Randomized Conditionally Independent Hessians. CVPR, 2022.

Questions for authors
  1. Can the authors use a valid paired test for the AUC gap (e.g., one that preserves identity-level dependence across seeds and draws)?
  2. Can the detector distinguish requesters from retained (non-requesting) training identities, rather than only from never-seen identities?
  3. How is the 1% FPR threshold selected without using the test negatives, and what is the uncertainty of the reported TPR?
  4. Can the authors evaluate at least one identity-unlearning method beyond the gradient-based ones De-ReID, or TIU-ReID) under the same protocol?
  5. Can the authors comment on how the findings are expected to change with larger galleries or face-recognition models?
6 comments

(1/2)

Thank you for a careful read. Your point about the permutation test was right, and it sent us back through every table; we also took the comments on positioning and baselines as the occasion to widen the study considerably. Point by point below; appendix letters refer to the revised PDF.

W1, readability and the three inconsistencies. We rewrote the abstract and the introduction. The abstract now reports four AUCs rather than a dozen figures, the introduction states the claim in a single paragraph on its first page, and the per-seed detail of the abstract and introduction has moved into the tables. The three inconsistencies you found are fixed. Section 7 now says that Tiny-ImageNet reproduces the under-modification result and that the ordering cannot be tested there, which is what Appendix D had said all along. The rank-only service claim now reads 3/3 seeds under the mixed adversary and 2/3 under the fresh one, everywhere it appears. Table 5's caption now says what its pooled null is (the 10,000 single-draw gaps of the same 2,000 replicates, smallest attainable p 0.0001), that the table is the three-seed audit of the first protocol, and that the five-seed run is tested in Appendix M.

W2, user-level membership inference and Chen et al. Yes. On the original model, what our detector does is user-level membership inference, and the new Appendix N makes that concrete: set against retained training identities instead of never-seen ones, the released original sits at chance (AUC 0.49) while the retrain oracle separates them at 0.96. The mark it reads is "was trained on". The claim paragraph of the introduction and Section 2 now say this explicitly. What is new is the question asked of that signal, and three things that user-level MI, as it is usually run, does not provide: a criterion for a deletion procedure stated against the exact-retraining counterfactual rather than against chance (the oracle itself fails the absolute version, so chance is the wrong reference for an approximate method); an audit that applies that criterion to released encoders after the procedure has nominally removed the person, with the oracle scored on the same identities and draws and with an adversary who holds no training image; and what that audit finds across ten methods and 72 operating points, namely that utility-preserving unlearning leaves the membership signal detectable (in Appendix Q every configuration that keeps retain mAP at or above 0.95 has a mixed gap of at least +0.09 on every valid seed) and the only operating points that remove it pay for it in retrieval on everyone. Chen et al. (CCS'21) is now cited and positioned in Section 2: their adversary queries the model before and after deletion and reads the deleted sample's membership from the change in posteriors, and retraining is among the procedures their attack degrades. Our adversary holds a single released encoder, the fresh one no training image at all, and retraining is the reference rather than the target.

W3 and Q1, a valid paired test. Appendix M re-tests every scored entry of Tables 1 to 3 on the identical out-of-fold scores with two tests that keep the pairing: an identity-level paired bootstrap (2,000 replicates; the forget identities, which the five draws share, are resampled once and applied to all draws, the matched negatives are resampled per draw, and both encoders are scored on the same resample) and DeLong's test for correlated ROC curves on each draw, with an across-seed t-test on top. For the headline numbers, the fresh original's gap of +0.180 is significant on 5/5 seeds under the bootstrap, and under DeLong on every draw on 3/5 seeds and on at least one draw on 5/5; the mixed original (+0.252), and finetune and NegGrad+ under the mixed adversary, are 5/5 by every test; the across-seed t-test gives p = 0.003 (fresh) and p < 0.001 (mixed). The bootstrap returns the same count as the permutation test on every 5/5 and 4/5 cell. Two entries move: artifact_aware at its operating point goes from 0/5 to 1/5 under each adversary (the mixed rejection above the oracle, the fresh one below), and CUHK03-NP's mixed gap of +0.014, a row saturated for oracle and released model alike, is rejected on 3/3 seeds by the bootstrap and on none by DeLong. The captions of Tables 2 and 3 report both, and Section 3 states what the bootstrap changes, so the two tests never disagree silently.

(2/2)

W4, SCRUB and the baselines. You are right about SCRUB's learning rate. Every operating point in the submission, SCRUB's included, was selected on the 20-identity sweep of Appendix G, and the text now gives that provenance in place of "as published". The 100-identity frontier sweep (Table 15) already contained SCRUB at 0.003 (2/3 seeds survive, retain 0.992, mixed gap +0.158); the revision adds 0.001 and 0.0005 on the submitted checkpoints. Both survive on all five seeds with retain mAP 0.999 and forget mAP 0.971 to 0.972, the original's level, and leak at +0.227 and +0.244 under the mixed adversary (+0.138 and +0.167 under the fresh one). At its published rates SCRUB leaves the encoder essentially as it was, and the mark it carries is the original's.

On the wider question of baselines: L-CODEC is cited and positioned in Section 2. Its paper removes a single randomly chosen identity per experiment, one sample per Hessian step at about three minutes a step, and reports fewer than ten supported removals at small epsilon, so a 100-identity deletion lies outside the regime it evaluates. The non-gradient family is covered by SSD (selective synaptic dampening), run under the full protocol: for alpha >= 5 it leaves the encoder essentially unchanged (retain 0.999, gap +0.244 to +0.252, 5/5 seeds); at alpha = 3 it already costs retain (0.987) and still leaks at +0.207 on 5/5; at alpha <= 2 it collapses on three to five of five seeds. De-ReID we re-implemented from the paper, since no code is released; Appendix T documents the implementation and the adaptation it needed on a CNN, and our reply to reviewer nmG6 gives its numbers in full (at its operating point it leaks at +0.220 under the mixed adversary on 5/5 seeds). TIU-ReID is a single-target adapter on a jointly fine-tuned TransReID with no 100-identity release, so there is no single released encoder for the protocol to score; we note that its own discriminator still separates the target at AUC 0.75 after suppression, which is the quantity we measure.

Q2, requesters against retained identities. Covered above and in Appendix N: under a symmetric training-image protocol the released original separates requesters from retained identities at AUC 0.49, the retrain oracle at 0.96, finetune at 0.63 and NegGrad+ at 0.60. The mark reads "was trained on", and exact retraining is the one procedure that removes it; that is why the criterion compares against never-seen identities and against the oracle.

Q3, the 1% FPR. It is the retrospective ROC read-off, the convention in the membership-inference literature, so its threshold comes from the test negatives; the paper reports it with its per-seed span, and Appendix O adds two readings of the same scores: an identity-level bootstrap interval (mixed adversary, seed 0: TPR 0.57, interval [0.41, 0.72]) and a prospective threshold fixed at the 99th percentile of the training-fold negatives. Under the mixed adversary that threshold attains an FPR of 3.0% on the held-out negatives for both encoders (seed average, nominal 1%), with TPR 0.69 for the released original against 0.11 for the oracle; under the fresh adversary the TPR is 0.30 against 0.20, at 2.9% and 3.5%. Both readings establish the ordering of released model and oracle, and that ordering is the claim.

Q4. Covered under W4. Q5. Table 9 shows the deletion headroom (original forget mAP minus the oracle's) more than doubling as the gallery grows to 2.3 times the standard one (0.13 to 0.26), with no plateau: the larger the gallery, the more the released model's advantage on the people it trained on stands out against retraining, so a larger gallery gives a deletion procedure more to remove, not less. Face retrieval is outside the paper's scope by design.

We are grateful for the push on the statistics in particular; the paper stands on firmer ground for it. If any of the above would be clearer with a different cut of the data, we are happy to run it while the discussion is open.

As the discussion period closes on 18 November (AoE), we wanted to check whether our response and the revised PDF address your concerns. In brief:

  • Paired test: every scored entry is re-tested with an identity-level paired bootstrap and DeLong's paired test (Appendix M). The fresh gap of +0.180 is significant on 5/5 seeds under the bootstrap, the mixed gap of +0.252 on 5/5 by every test, and the across-seed t-test gives p = 0.003 (fresh) and p < 0.001 (mixed).
  • Requesters against retained identities (Appendix N): released original 0.49, retrain oracle 0.96.
  • Chen et al. (CCS'21) and L-CODEC are cited and positioned in Section 2.
  • Baselines: SCRUB at its published rates, SSD, Bad-Teacher, SalUn, random-label and De-ReID under the same protocol (Appendices Q and T): ten methods, 72 operating points.
  • The 1% FPR now has an identity-level bootstrap interval and a prospective threshold (Appendix O).
  • The abstract and introduction are rewritten, and Section 7, the rank-only claim and Table 5's caption are updated as you suggested.

If anything remains unclear, or a different analysis would help, we are glad to provide it before the deadline. Thank you again for your time.

Comments on the paper

2 comments

We thank the reviewers. All three rate soundness 3; the requests concern presentation, the significance test, and the coverage of unlearning methods and backbones. The revision (10-page main text; Appendices M to T are new) does the following, and each per-reviewer reply points into it.

  1. Readability. The abstract reports four AUCs; the introduction states the claim in one paragraph on its first page; the per-seed detail of the abstract and introduction moved to the tables.
  2. Paired significance tests (xVZf W3). Every scored entry is re-tested with an identity-level paired bootstrap and DeLong's paired test on the identical out-of-fold scores (Appendix M). The main-text tables keep the permutation count and state beside it (Section 3, captions of Tables 2 and 3) what the bootstrap changes: the same count on every 5/5 and 4/5 cell; artifact_aware at its operating point moves from 0/5 to 1/5 under each adversary (gaps +0.015 mixed, the rejection above the oracle, and -0.019 fresh, the rejection below it), and CUHK03-NP's saturated mixed gap of +0.014 is rejected on 3/3.
  3. Ten methods, 72 operating points, same protocol, the submitted checkpoints. Added: SSD (Fisher dampening, no gradient descent), Bad-Teacher, SalUn, random-label, SCRUB at the published lower rates, and a re-implementation of De-ReID (Appendices Q, T). Every added method, at the operating point the paper's rule selects, leaks on 5/5 seeds under the mixed adversary: SSD +0.207, Bad-Teacher +0.118, SalUn +0.101, random-label +0.092, SCRUB at 0.001 +0.227, De-ReID +0.220 (gap to the same-seed oracle; original +0.252). No configuration of any method meets the three-way target.
  4. A transformer. ViT-B/16 originals and oracles, three seeds (Appendix R): released 0.925 against oracle 0.691 (mixed, +0.234, 3/3 by every test), 0.790 against 0.623 (fresh, +0.167, 3/3 by the permutation and bootstrap tests, 2/3 by DeLong on every draw); the ResNet-50 detector transfers to the ViT and back on 12/12 disjoint-identity pairs under both adversaries.
  5. Never-seen-identity utility for every encoder (Appendix P; standard query protocol for every run in Appendix Q and for the ViT in Appendix R).
  6. Requesters against retained identities (Appendix N): released original 0.49, retrain oracle 0.96. The mark reads "was trained on", which exact retraining alone removes; this is why the criterion is stated against the never-seen reference and the retrain oracle.
  7. Sequential deletion, 100 requests one (or ten) identities at a time on the submitted checkpoints at equal total compute, scored on every identity deleted so far (Appendix S): the gap at the 100th request is +0.19 to +0.27 (mixed) and +0.07 to +0.28 (fresh), interval excluding zero in 20/20 runs; against a prefix oracle retrained without only the first k requesters (k = 25, 50, 75) the gap on the equal-compute rows is +0.17 to +0.38 (mixed, 40/40), within 0.05 of the final-state-oracle gap on every row. Applying the batch operating point to every request is reported as the control: NegGrad+ reaches chance retrieval or non-finite weights by the fifth grouped request on 5/5 seeds, and finetune's gap closes only where retrieval has been paid for (standard-test mAP 0.36 to 0.66 against the original's 0.75), the pattern of Section 7. A third arm at ten times the equal-compute budget sits between them: NegGrad+ holds retain mAP 0.95 through the 50th request on 5/5 seeds with the gap intact (+0.13 to +0.31, 10 of 11 checkpoints), then collapses by the 100th; finetune falls below retain 0.95 by the 50th request on every seed and collapses on 3/5.
  8. Text revisions (xVZf W1, W2, W4). Section 7 now states that Tiny-ImageNet reproduces the under-modification result and that the ordering cannot be tested there; every operating point is stated as selected on the 20-identity sweep (Appendix G); the rank-only (top-10) claim carries its per-adversary seed counts (3/3 mixed, 2/3 fresh); Table 5's caption states the pooled null; Chen et al. (CCS'21) and L-CODEC are cited and positioned in Section 2; an AI-disclosure section is added.
AuthorsOpenReview ↗

Summary of the discussion and the revisions

We thank the reviewers and the AC; this note gathers what the discussion produced. Every number below is in the revised PDF (uploaded 19 Nov); nothing here is new.

The claim. In identity retrieval, utility-preserving unlearning leaves a mark: a person who opted out stays distinguishable from a person the model never saw, by more than a model retrained without them allows. On Market-1501 the released original, before unlearning, reads 0.802 against the retrain oracle's 0.622 for an adversary holding no training image and 0.968 against 0.717 for one who unknowingly holds some (Section 5); after unlearning, finetune and NegGrad+ keep mixed gaps of +0.192 and +0.225 on 5/5 seeds at retain mAP 0.959 and 0.999 (Section 7, Table 3).

Where the discussion stands. Ratings are 2, 2 and 3; all three reviewers rate soundness 3. XwXe found the concerns adequately addressed and raised the rating to 3. Each point raised by xVZf and nmG6, including their 18 Nov comments, has a reply.

What the reviewers asked for, and where it is

  1. A valid paired test (xVZf). The identity-level paired bootstrap is now the main-text test, with DeLong's test beside it for every entry (Sections 3 to 5 and 7; Appendix M, Tables 19 and 20). The headline gaps, +0.180 and +0.252, are significant on 5/5 seeds; across seeds, p = 0.003 and p < 0.001. Three counts move, each stated where it appears; Tiny-ImageNet and the simulated top-10 services keep the permutation, labelled so.
  2. More unlearning methods (all three). Ten methods at 72 operating points on the submitted checkpoints, including the non-gradient SSD (Appendix Q). At the operating point the paper's rule selects, every added method leaks on 5/5 seeds under the mixed adversary: SSD +0.207, Bad-Teacher +0.118, SalUn +0.101, random-label +0.092, SCRUB at 0.001 +0.227, our De-ReID re-implementation +0.220 (original +0.252). No configuration of any method meets the criterion. SCRUB's submitted rate is now stated as sweep-selected (Sections 7 and 8); at its published rates it leaves forget mAP at the original's level.
  3. De-ReID in its authors' released code (nmG6, xVZf), run unchanged on three ViT-B/16 originals (Appendix T.1, Table 32). It forgets far more than retraining (forget mAP 0.05 to 0.27 against the oracle's 0.793) and leaks at +0.18 to +0.27 (mixed) and +0.23 to +0.33 (fresh), 3/3 seeds at every checkpoint from epoch 40. As nmG6 predicted (W1), a detector fitted on released originals falls to the oracle's level once the release passes the oracle; one fitted on a release-unlearned shadow model reads the opt-outs above the oracle on every pair at every checkpoint.
  4. A transformer (nmG6, XwXe). ViT-B/16: +0.234 (mixed) and +0.167 (fresh), 3/3 seeds; a ResNet-50 detector reads the ViT's opt-outs at 0.949 against 0.581 (mixed) (Appendix R).
  5. Sequential deletion (nmG6, XwXe). 100 identities deleted one or ten at a time at equal compute: once all are deleted the gap is +0.19 to +0.27 (mixed) and +0.07 to +0.28 (fresh), excluding zero in 20/20 runs; against prefix oracles retrained without only the first k requesters, +0.17 to +0.38 (mixed, 40/40) (Appendix S).
  6. What FII measures (XwXe, xVZf). Against retained training identities the released original sits at chance (0.49) and the oracle at 0.96: the mark reads "was trained on", which exact retraining removes, so the criterion is stated against the oracle, not chance (Appendix N). Section 2 positions user-level membership inference, Chen et al. (CCS'21) and L-CODEC.
  7. Utility on never-seen identities (nmG6, Appendix P); the 1% FPR with an interval and a prospective threshold (xVZf, Appendix O); larger galleries (xVZf, Appendix F, Table 9); mitigation (XwXe): the post-hoc defences tested leave the attack in place until retrieval is destroyed (Appendix I).
  8. Presentation (xVZf, XwXe). Abstract and introduction rewritten, the three inconsistencies xVZf found fixed, the AI-disclosure section expanded. XwXe's ethics flag: the ethics statement covers the data and dual use.

Not covered. Multimodal re-identification (CLIP-ReID, DIFFER), clothes-changing data (LTCC) and a larger dataset (MSMT17), asked by nmG6: the multimodal run needs originals and oracles trained from scratch, and the LTCC and MSMT17 licences did not clear in time. TIU-ReID and L-CODEC remove one identity per run, outside a 100-identity deletion (reply to xVZf).

The revisions. 14 Nov: Appendices M to T and the rewritten front matter. 18 Nov: De-ReID's released code, Appendix T.1. 19 Nov: the paired bootstrap as the main-text test (no new experiments).

For the final version: the CLIP-ReID run we plan (reply to nmG6), the scripts of the De-ReID release run, and the training and unlearning harness, checkpoints and manifests (reproducibility statement).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.