acceptodds
Withdrawn from ICLR 2027

Erase, Don’t Distort: Machine Unlearning through Representation Realignment

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Machine unlearning is often carried out by deliberately increasing the loss on the forget set to suppress the model’s response to targeted samples. While effective at inducing forgetting, this can leave a distinct statistical footprint that is vulnerable to membership inference attacks, a phenomenon known as the Streisand effect. The core issue is that loss maximization specifies what should be erased without controlling how the underlying representations evolve, often driving forgotten samples into abnormally high-loss regions that are easy to distinguish from naturally unseen data. We instead formulate unlearning as a controlled subspace realignment process that selectively modifies forget-specific dimensions while preserving the latent structure shared by retained data. To realize this formulation, we first isolate a low-dimensional subspace that captures the dominant discrepancy between forget and retain representations, while constraining its orthogonal complement to remain close to the original model. Within this subspace, we then realign the forget distribution toward a nearby retain distribution by matching both its centroid and its dispersion, thereby avoiding the abnormal concentration induced by destructive forgetting. This design confines forgetting to the representation directions where it is needed while preserving shared structure elsewhere. Comprehensive evaluations across CIFAR-10, CIFAR-100, SVHN, Tiny ImageNet, and ImageNet32-200 demonstrate a strong privacy–utility trade-off, driving membership inference performance close to the random-guess regime () while matching or surpassing representative baselines in retain utility.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
machine unlearning, privacy, membership inference attacks, representation learning, subspace alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DBGsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes QF-Entanglement, a representation-based method for class unlearning that aims to avoid the statistical artifacts caused by aggressive loss maximization. It identifies an active feature subspace from forget–retain discrepancies, moves forget representations toward a nearby retain-class centroid, and matches their linear entropy while preserving the orthogonal complement and training on retained data. The analysis provides a conditional utility bound and relates entropy matching to effective-rank preservation. Experiments use ResNet-18 across five image-classification datasets, with evaluations of forgetting, retained accuracy, membership inference, and representation similarity to retrained models. The reported results suggest a favorable privacy–utility trade-off against the evaluated baselines.

Critical strengths

The method addresses a meaningful limitation of loss-maximization approaches: suppressing forget accuracy can create statistical artifacts without reproducing the behavior of a retrained model. Combining targeted representation changes with preservation of shared features is a reasonable approach to balancing forgetting and retained utility.

The experiments include retrained references, membership inference at low false-positive rates, and representation probes alongside classification accuracy. The reported results, particularly on CIFAR-100, provide encouraging evidence that the method can suppress the forgotten class while preserving utility and approaching retrained behavior on several measures.

Critical weaknesses

The proposed alignment target is not sufficiently justified as a proxy for unlearning. A model retrained without the forget class need not map its samples to a single neighboring retain-class distribution. Matching that class’s centroid and linear entropy may suppress the original classification behavior while preserving information about the forgotten data. The paper does not establish why this target should reproduce retrained behavior, and matching a scalar entropy statistic is insufficient to support its broader claims of distributional indistinguishability and protection against membership inference.

The contribution over existing representation-based methods remains unclear. DUCK already moves forget embeddings toward nearby incorrect class centroids, while MUDA aligns forget representations with the retain feature space. Neither is included in the comparisons, although both directly relate to the paper’s central idea. Comparisons with POUR and TRW, together with the neighboring-class attack MIA-NN, are also missing. Without these comparisons, it is difficult to determine whether the proposed combination provides a meaningful improvement over existing alignment methods or avoids the privacy leakage that motivates the paper.

Questions for authors

Clarification requests:

  1. What evidence supports using a neighboring retain class as the target distribution for forgotten samples? Which classes are selected in the reported experiments, and how closely does this alignment reproduce the representations and predictions of models retrained without DfD_f?
  2. Linear entropy is invariant to covariance scaling, so matching it does not control total variance or guarantee a nonsingular covariance. What additional assumptions justify Corollary 1’s conclusions about likelihood anomalies and membership inference?

Additional evidence needed to assess the current claims:

  1. How does the method compare with DUCK and MUDA, which already use centroid movement or representation alignment, and with POUR and TRW? These comparisons would help establish the contribution of the proposed combination.
  2. Does the reported privacy improvement persist under MIA-NN or another attack that examines predictions on neighboring retained classes? This is particularly relevant because the method explicitly redirects forgotten representations toward such classes.
1 comment

Response to Reviewer DBGs

We thank Reviewer DBGs for highlighting key representation-level baselines and neighbor-aware audits.

W1 & Clarification Q1, Q2. Alignment Target and Variance Control

  1. Target Justification & Fidelity: As detailed in G1, a retrained model naturally projects deleted classes onto nearest visual neighbors (e.g., CIFAR-10 Airplane →\to Bird / Ship, Figure 1(b)). Dynamic selection arg⁡min⁡c∈Ccand∥fˉdisc−μcsub∥2\arg\min_{c \in \mathcal{C}_{cand}} \lVert\bar{f}_{disc} - \mu_c^{sub}\rVert_2 mirrors this inductive bias, verified by linear probing (Δ=+0.20%\Delta = +0.20\% vs. Retrain; Appendix B.5) and layer-wise CKA (≥0.9837\ge 0.9837 in Layers 1–3; Appendix B.6).
  2. Variance Control: Please see G2. Theorem 1 bounds effective rank; batch normalization and LCE(Dr)\mathcal{L}_{\mathrm{CE}}(D_r) preserve non-vanishing trace (Appendix B.6), and Corollary 1 explicitly assumes sub-Gaussianity and non-vanishing trace (Section 4.3).

W2 & Additional Evidence Q1. Comparison with DUCK, MUDA, POUR, and TRW

All four representation-level baselines were evaluated on CIFAR-100 (seed 42; Appendix B.8, Table 17):

Table R3: CIFAR-100 Representation Unlearning Baselines (Seed 42)

MethodRetain Acc (%) ↑Forget Acc (%) ↓Retain Test (%) ↑MIA (%) (Target: ~50%)
Pretrained Baseline99.98%100.00%77.06%49.00%
Retrain (Gold Reference)99.97%0.00%76.47%50.00%
QF-Entanglement (Ours)99.96%0.00%75.79%50.00%
DUCK (Cotogni et al., 2024)95.53%1.40%68.00%48.00%
MUDA (Seo et al., 2025)99.88%0.00%74.49%53.00%
POUR (Le et al., 2026)99.96%0.00%75.09%49.00%
TRW (Wang et al., 2025)99.49%1.40%70.88%52.00%
DELETE85.73%3.00%61.26%47.00%
GDR93.13%0.00%66.89%43.00%
Gradient Ascent97.20%0.00%69.58%46.00%
SalUn97.54%0.00%70.18%46.00%
SCRUB98.57%30.80%71.03%52.00%
SSD77.46%0.00%56.38%47.00%

DUCK and TRW leave residual forget accuracy (1.40%1.40\%) and degrade retain test utility (68.00%68.00\%, 70.88%70.88\%); MUDA shows higher MIA deviation (53.00%53.00\%); QF-Entanglement achieves complete erasure (0.00%0.00\%), exact chance MIA (50.00%50.00\%), and retain test utility (75.79%75.79\%) closest to Retrain (76.47%76.47\%).

Additional Evidence Q2. Resistance Under MIA-NN

Evaluations under MIA-NN on CIFAR-10 (Appendix B.7, Table 15) confirm that neighbor-class alignment introduces no detectable prediction signature:

Table R4: CIFAR-10 MIA-NN Target Attack Evaluation

MethodMIA-NN Target Acc (%) (Target: ~50%)
Pretrained Baseline45.40%
Retrain (Gold Reference)50.40%
QF-Entanglement (Ours)49.40%
DELETE50.30%
GDR47.90%
Gradient Ascent49.40%
SalUn49.20%
SCRUB51.10%
SSD49.80%

Revision: Added Tables 14, 15, and 17 in Appendix B.7–B.8 and updated the Bibliography.

We hope these baseline comparisons and neighbor-aware audits address your questions, and we would deeply appreciate your consideration of a higher score.

References:

  • Cotogni, M., Bonato, J., Sabetta, L., Pelosin, F., & Nicolosi, A. (2024). DUCK: Distance-based unlearning via centroid kinematics. arXiv:2312.02052.
  • Le, A., Peng, C., Liu, Y., & Noble, J. A. (2026). POUR: A provably optimal method for unlearning representations via neural collapse. CVPR.
  • Seo, S., Kim, D., & Han, B. (2025). Revisiting machine unlearning with dimensional alignment. IEEE/CVF WACV.
  • Wang, Y., Ebrahimpour-Boroojeny, A., & Sundaram, H. (2025). On the necessity of output distribution reweighting for effective class unlearning. arXiv:2506.20893.
Reviewer_UmZwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is about class unlearning in image classifiers, and it starts from an observation I agree with. If you force the forget set into very high loss, the model does forget, but those samples now look unlike anything the model has never seen, and that difference is exactly what a membership inference attack picks up on. To avoid this, the proposed method works in feature space instead of on the loss. It finds a low dimensional subspace where forget and retain features differ, keeps everything outside that subspace close to the original model, and inside it moves the forget features toward the nearest retain class by matching their centroid and spread. The hope is that forget samples then look like ordinary unseen data of a neighboring class, so the attack drops to chance while retain accuracy survives. Experiments cover five classification benchmarks with ResNet-18 and about ten baselines.

Critical strengths

(1) The framing is right. Zero forget accuracy is neither necessary nor sufficient for unlearning, because the real target is how a model retrained without the forget data would treat those samples, and such a model would simply send them to neighboring classes with ordinary confidence. Turning that into centroid and spread matching inside a forget specific subspace, with the rest of the feature space anchored, addresses both the privacy signature and the collateral damage by design rather than by tuning.

(2) The supporting evidence is more complete than usual. Multi-seed runs, a wide sweep of the main coefficient, probing against a retrained model, and an attack table at low false positive rates are all things a reviewer would normally have to ask for.

Critical weaknesses

(1) The membership inference number carries the paper, and I cannot trust it as reported. To begin with, it is described only as a LiRA true positive rate with an ideal value of 50 percent, which is not how LiRA results are normally reported, and a true positive rate near 50 percent is not by itself a definition of indistinguishability. Nothing is said about how many shadow models were trained, whether the attack was run online or offline, what the non-member pool is for a class that has been removed entirely, or how many samples were evaluated. Then the numbers themselves raise flags. The retrained model lands on exactly 50.00 on every dataset, two of the main tables contain only integer values, and the low false positive rate table reports rates at a threshold that would correspond to half a sample for a class of 500 images. At that resolution the one or two point margins over the best baseline are noise. On top of this, the unlearning aware attack of Hayes et al. (2024), which is the standard audit for inexact unlearning, is not run, and the original model is missing from every table, so there is no reference for how detectable the forget set was to begin with.

(2) The paper also never shows that the problem it solves is actually there. The motivation is that aggressive unlearning methods leave a conspicuous statistical footprint, but in the main tables the baselines sit within a few points of 50 percent as well, so there is no footprint to see. Showing it would be simple. A histogram of forget set loss against the loss of truly unseen test data, drawn for a loss maximizing baseline and for this method, would make both the effect and the fix visible, and a projection of the feature space would show whether forget samples move into the neighboring cluster or just collect somewhere of their own. Neither is in the paper. There is, moreover, a threat the paper does not consider. All forget samples are pulled toward a single retain class, which a retrained model would never do, so an attacker who notices that images of the forgotten class are systematically predicted as one particular neighbor has a signal that the 50 percent number does not capture.

(3) Finally, the theory and some baselines are overstated. The main theorem follows in two lines from the definition of effective rank, the corollary argues with Gaussian likelihoods without assuming Gaussian features, and a Wasserstein bound referred to in the appendix is never actually stated, so words like guaranteed should go. On the empirical side, retain accuracy above the retrained model on CIFAR-10 is left unexplained, and the fine-tuning baselines on ImageNet32-200 fall far enough below the retrained model to look undertuned.

Questions for authors

(1) Please specify the membership inference protocol exactly, and explain why the retrained model is exactly 50 percent everywhere and how the low false positive rate values were obtained from a class of 500 images.

(2) Add loss histograms, a log scale ROC curve, and a projection of forget features before and after unlearning. Also report the Hayes et al. attack with confidence intervals on at least one dataset, and add the original model to every table.

(3) Since all forget samples are anchored to one retain class, would simply looking at the predicted labels of forget images distinguish your model from a retrained one? A few forget images with their predictions before and after unlearning would help here.

(4) How does retain accuracy exceed the retrained model on CIFAR-10, and how were the fine-tuning baselines configured on ImageNet32-200?

(5) Which of the subspace threshold, the candidate pool size and the scaled coefficient on ImageNet32-200 were tuned, and how sensitive are the results to them?

3 comments

Response to Reviewer UmZw

We thank Reviewer UmZw for the detailed audit suggestions. All requested visualizations and audits have been added to Section 5.1 (Figures 1–2), Appendix B.4, and Appendix B.7 (Tables 12–15, Figure 4).

W1 & Q1. MIA Protocol, Resolution, U-MIA, and Pretrained Baselines

  • U-MIA with 95% CI (Hayes et al., 2024): On CIFAR-100 (Appendix B.7, Table 14), destructive baselines distort confidence statistics (e.g., GDR 66.00%, GA 59.00% on MaxProb U-MIA), whereas QF-Entanglement matches Retrain across both point estimates and 95% confidence intervals:

Table R2: CIFAR-100 Population U-MIA (95% CI and Mean ± SD, in %)

MethodLoss AccLoss 95% CI (Mean ± SD)MaxProb AccMaxProb 95% CI (Mean ± SD)
Pretrained Baseline58.00[51.00, 66.00] (58.56 ± 3.70)45.00[36.00, 50.50] (43.23 ± 3.68)
Retrain (Gold Reference)54.00[46.00, 60.00] (53.24 ± 3.66)51.00[43.50, 57.50] (50.41 ± 3.60)
QF-Entanglement (Ours)55.00[47.00, 62.00] (54.55 ± 3.65)49.00[42.00, 57.00] (49.28 ± 3.74)
DUCK53.00[46.00, 60.00] (53.40 ± 3.60)50.00[42.00, 56.00] (49.11 ± 3.67)
MUDA58.00[50.50, 64.00] (57.38 ± 3.57)47.00[39.50, 53.00] (46.21 ± 3.59)
GDR40.00[33.00, 48.00] (40.17 ± 3.88)66.00[58.50, 72.50] (65.50 ± 3.60)
Gradient Ascent45.00[37.50, 52.00] (44.51 ± 3.64)59.00[51.00, 66.00] (58.59 ± 3.76)
SalUn45.00[39.00, 53.50] (46.06 ± 3.69)55.00[47.00, 61.00] (54.20 ± 3.64)
  • Why Retrain = 50.00%, Integer Tables, and 500-Image Low-FPR Quantization: Under N=1N=1 Retrain-anchored LiRA, self-calibration yields zero residual log-likelihood ratio on Retrain, placing its median split at 50.00% by definition. On CIFAR-100 (100 test samples/class) and Tiny-ImageNet / ImageNet32-200 (50 test samples/class), the 1:1 split ∣Din∣=∣Dout∣=∣Dtestforget∣\lvert D_{in}\rvert = \lvert D_{out}\rvert = \lvert D_{test}^{forget}\rvert yields 1% or 2% step granularity (hence integer values). Similarly, in CIFAR-100 full-cohort low-FPR LiRA (500 training samples/class; Appendix B.7, Table 12), FPR below 1% is quantized in 0.20% steps (1/5001/500), causing [email protected]% and [email protected]% to plateau. We therefore added CIFAR-10 log-scale ROC and low-FPR audits (5,000 train / 1,000 test samples/class; Figure 2, Appendix B.7 Table 13) and Pretrained rows to Tables 1–4, 7, 14, 15, and 17.

W2 & Q2. Visual Proof of the Streisand Footprint and Realignment Fix

All requested plots on CIFAR-10 (Class 0: Airplane) are in Section 5.1:

  • Loss Histogram (Figure 1(a)): Pretrained forget loss concentrates near zero (μ=0.00,σ=0.03\mu=0.00, \sigma=0.03 vs. unseen μ=0.15\mu=0.15); Gradient Ascent shifts forget loss to a wide tail (μ=11.51,σ=5.14\mu=11.51, \sigma=5.14); QF-Entanglement aligns smoothly with unseen data (μ=5.11,σ=0.79\mu=5.11, \sigma=0.79 vs. μ=5.48,σ=1.46\mu=5.48, \sigma=1.46).
  • Feature Projection (Figure 1(b)): Pretrained forms an isolated cluster; Gradient Ascent scatters outliers; QF-Entanglement merges forget samples into the neighbor cluster (Bird), matching Retrain.
  • Log-Scale ROC (Figure 2 & Appendix B.7, Tables 12–13): Across FPR∈[10−3,100]\mathrm{FPR} \in [10^{-3}, 10^0] on CIFAR-10, QF-Entanglement tracks Retrain (0.51910.5191 vs. 0.50000.5000 AUC; 0.14%0.14\% vs. 0.10%0.10\% [email protected]% FPR), while GA (0.54%), SalUn (0.38%), and RF-Entanglement (0.74%) leak at the tail. On CIFAR-100, QF-Entanglement yields 0.20% [email protected]% FPR (vs. 2.60% SalUn, 1.80% GA).

W3 & Q3. Theoretical Calibration & Neighboring Prediction Leakage

  • Theory: Removed "guaranteed" and "Wasserstein bound" phrasing in Section 4.3 and Appendix A, and stated sub-Gaussian and non-vanishing trace conditions explicitly in Corollary 1.
  • Neighbor Leakage: Under MIA-NN (Table R4; Appendix B.7, Table 15), attack accuracy stays at chance (49.40% vs. 50.40% Retrain). As shown in Figure 4 (Appendix B.7), both Retrain and QF-Entanglement prioritize Bird (Class 2: 37.8% vs. 73.6%) while Monte Carlo pooling (Ccand\mathcal{C}_{cand}) distributes residual predictions across adjacent classes (Cat 16.3%, Frog 3.5%, Truck 2.7%), preventing deterministic label collapse.

Q4. CIFAR-10 Retain Utility & ImageNet32-200 Fine-Tuning

  • Retain Utility > Retrain: The 10-class pretrained backbone retains richer generic visual primitives than a 9-class retrained model; brief retain fine-tuning regularizes features (Kurmanji et al., 2023; Section 5.2).
  • ImageNet32-200 FT: Executed with SGD (lr=0.01, momentum 0.9, 3 epochs), yielding 0.38% forget and 58.74% retain test accuracy (weaker learning rates leave 52.15% forget accuracy).

Q5. Hyperparameter Sensitivity on ImageNet32-200 and CIFAR-10

Across all datasets, τ=0.97\tau=0.97 and ∣Ccand∣=5\lvert\mathcal{C}_{cand}\rvert=5 are fixed without per-dataset tuning; on ImageNet32-200 (βanchor=0.03\beta_{anchor}=0.03), we swept λq∈{0.5ln⁡2,1.0ln⁡2,2.0ln⁡2}\lambda_q \in \lbrace 0.5\ln 2, 1.0\ln 2, 2.0\ln 2\rbrace:

  • λq=0.5ln⁡2\lambda_q = 0.5\ln 2: Forget 0.00%, Retain Test 63.34%, MIA 50.00%
  • λq=1.0ln⁡2\lambda_q = 1.0\ln 2 (Default): Forget 0.00%, Retain Test 62.87%, MIA 50.00%
  • λq=2.0ln⁡2\lambda_q = 2.0\ln 2: Forget 0.00%, Retain Test 63.36%, MIA 60.00%

Performance is stable across [0.5ln⁡2,1.0ln⁡2][0.5\ln 2, 1.0\ln 2], while λq=2.0ln⁡2\lambda_q = 2.0\ln 2 over-regularizes entropy and increases MIA exposure. Appendix B.4 further verifies stability on CIFAR-10 across β∈[0.01,10.0]\beta \in [0.01, 10.0] and λq∈[0.5ln⁡2,1.5ln⁡2]\lambda_q \in [0.5\ln 2, 1.5\ln 2] (0.00%–0.42% forget, >93.80% retain test, 49.80%–50.40% MIA).

We sincerely hope these new visualizations, confidence intervals, and protocol details address your concerns, and we would greatly appreciate your consideration in raising your score.

References:

  • Hayes, J., et al. (2024). Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. IEEE SaTML.
  • Kurmanji, M., et al. (2023). Towards unbounded machine unlearning. NeurIPS.

Thank you for the thorough response.

What keeps me at my score is what those additions show. The MIA is an offline LiRA calibrated on a single retrained model, which puts Retrain at exactly 50 by construction and gives one or two point resolution on cohorts of 50 to 100 samples, so the margins over the better baselines in the main tables are at the level of the quantization. Under U-MIA the method sits inside the confidence intervals of Retrain and DUCK, and in the new Table 17 POUR reaches the same forget accuracy, the same retain accuracy and MIA within one point of yours, with a 0.7 point difference in test accuracy on a single seed. The gap to existing representation level methods is therefore small, and the paper should say so.

Two of the new results also cut the other way. In the loss histogram the forget set is about half as spread out as truly unseen data (sigma 0.79 against 1.46), which is itself a footprint. And Figure 4 shows the method sending 73.6 percent of forget images to Bird where Retrain sends 37.8 percent, which is the label concentration I asked about. MIA-NN at chance does not rule that signal out, since it does not test the prediction histogram directly. Please describe the calibrated protocol in the main text with U-MIA beside it, report POUR and DUCK in the main tables over seeds, and discuss the two points above. I keep my score.

Reviewer_YgbeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes QF-Entanglement, a representation-level approach to machine unlearning. It identifies a forget-related subspace through differential PCA, preserves representations in the orthogonal complement, and aligns forgotten features with a nearby retain distribution using centroid and linear-entropy matching. Experiments on five image classification benchmarks compare the method with existing unlearning approaches in terms of forgetting effectiveness, retain accuracy, and membership inference resistance. The paper also includes ablation studies and representation-level analyses.

Critical strengths
  1. The paper addresses an important issue in machine unlearning: aggressively suppressing forgotten samples may leave detectable statistical artifacts. Aligning forgotten representations with retained data, rather than simply maximizing their loss, offers a reasonable alternative.
  2. The evaluation is fairly extensive, covering five datasets and several representative unlearning methods. The component ablations in Table 5, multiseed results in Table 7, and additional representation and membership inference audits provide useful evidence beyond standard forgetting accuracy.
Critical weaknesses
  1. The theoretical privacy guarantee is not sufficiently justified. Theorem 1 bounds the difference in effective rank, but this does not establish distributional indistinguishability or protection against likelihood-based membership inference. In particular, uniformly shrinking the covariance eigenvalues leaves linear entropy unchanged while making the feature distribution increasingly concentrated. Therefore, the conclusion in Corollary 1 that entropy matching prevents likelihood anomalies does not follow from the current analysis. This weakens one of the paper's central theoretical claims.
  2. Section 5.1 describes MIAtrue as a true-positive rate but treats 50% as the ideal random-guess level, which is appropriate for balanced attack accuracy rather than TPR. The construction of the attack evaluation, particularly the selection of non-member samples in class unlearning, also needs clearer explanation. Moreover, low-FPR LiRA results are provided only for CIFAR-100 (Table 11), while four of the five benchmarks rely on single-seed results. These limitations make it difficult to assess how consistently the reported near-random MIA scores reflect successful unlearning.
Questions for authors
  1. Could the authors clarify the exact definition of MIAtrue, including the attack threshold and the construction of member and non-member samples? Why is 50% considered the ideal value if the metric represents a true-positive rate?
  2. Since linear entropy depends on normalized covariance statistics, what prevents the forget representations from undergoing a uniform reduction in variance while still satisfying the entropy-matching objective?
1 comment

Response to Reviewer Ygbe

We thank Reviewer Ygbe for the thoughtful theoretical and evaluation feedback.

W1 & Q2. Theoretical Scope and Scale Invariance

Please see G2. Theorem 1 strictly bounds effective-rank discrepancy (∣Reff(Σf)−Reff(Σr)∣≤ϵe(1−SL(ρr))(1−SL(ρf))\lvert R_{\mathrm{eff}}(\Sigma_f) - R_{\mathrm{eff}}(\Sigma_r)\rvert \le \frac{\epsilon_e}{(1-S_L(\rho_r))(1-S_L(\rho_f))}), while shared batch normalization and LCE(Dr)\mathcal{L}_{\mathrm{CE}}(D_r) maintain non-vanishing feature scale (Tr(Σ)=18.5290\mathrm{Tr}(\Sigma)=18.5290, Reff=20.38R_{\mathrm{eff}}=20.38 on CIFAR-100; Appendix B.6). Under sub-Gaussian representations with non-vanishing trace (Corollary 1), bounded rank and centroid alignment constrain expected log-density anomalies. Section 4.3 and Appendix A have been revised to remove absolute "guarantee" phrasing.

W2 & Q1. MIA Definition, 1:1 Balance Justification, Low-FPR LiRA, and Multi-Seed Stability

  1. Protocol & 50% Baseline: The reported metric is Balanced Attack Accuracy (not raw TPR) under Global-Variance Offline LiRA (Carlini et al., 2022), calibrated against an N=1N=1 Retrain anchor using the cohort median threshold on a 1:1 split (∣Din∣=∣Dout∣=min⁡(∣Df∣,∣Dtestforget∣)\lvert D_{in}\rvert = \lvert D_{out}\rvert = \min(\lvert D_f\rvert, \lvert D_{test}^{forget}\rvert)), where DoutD_{out} consists of unseen test samples of the forgotten class. Under unbalanced splits, a trivial majority classifier inflates accuracy and invalidates the 50%50\% baseline; hence, 1:1 balancing is standard in unlearning audits (Chen et al., 2021; Chundawat et al., 2023; Kurmanji et al., 2023; Fan et al., 2024).
  2. CIFAR-10 Low-FPR LiRA & Multi-Seed Results: Beyond CIFAR-100 (Appendix B.7, Table 12), we added CIFAR-10 full-cohort ROC-AUC and low-FPR LiRA (Figure 2; Appendix B.7, Table 13), tracking Retrain closely (0.51910.5191 vs. 0.50000.5000 AUC; 1.12%/0.46%/0.14%1.12\%/0.46\%/0.14\% TPR at 1%/0.5%/0.1%1\%/0.5\%/0.1\% FPR). Three-seed CIFAR-10 trials (Appendix B.3, Table 8) confirm minimal variance:

Table R1: CIFAR-10 Multi-Seed Stability

SeedRetain Test Acc (%) ↑Forget Acc (%) ↓MIA Accuracy (%) (Target: ~50%)
Seed 4294.18%0.00%50.30%
Seed 12393.52%1.26%49.90%
Seed 202793.80%0.00%50.00%
Mean ± Std93.83 ± 0.27%0.42 ± 0.59%50.07 ± 0.17%

Revision: Updated Section 5.1 (Balanced Attack Accuracy definition), Figure 2, Appendix B.3 (Table 8), and Appendix B.7 (Table 13).

We hope these clarifications and additional audits resolve your concerns, and we would be grateful if you could reconsider your score.

References:

  • Carlini, N., et al. (2022). Membership inference attacks from first principles. IEEE S&P.
  • Chen, M., et al. (2021). When machine unlearning jeopardizes privacy. ACM CCS.
  • Chundawat, V. S., et al. (2023). Can bad teaching induce forgetting? Unlearning in deep networks using an incompetent teacher. AAAI.
  • Fan, C., et al. (2024). SalUn: Empowering machine unlearning via gradient-based weight saliency. ICLR.
  • Kurmanji, M., et al. (2023). Towards unbounded machine unlearning. NeurIPS.

Comments on the paper

1 comment

General Response to All Reviewers

We thank Reviewers Ygbe, UmZw, and DBGs for their constructive feedback. Below, we address the three shared questions regarding (1) neighboring retain alignment (Section 5.1, Appendices B.5–B.7), (2) linear entropy scale invariance (Section 4.3, Appendices A and B.6), and (3) MIA protocol standardization (Section 5.1, Appendices B.3 and B.7).

G1. Rationale of Neighboring Retain Alignment

When a class (e.g., Airplane) is omitted during retraining from scratch, shared convolutional primitives naturally map its inputs onto semantically adjacent retain classes (Bird, Ship) with moderate confidence rather than random noise. To eliminate statistical footprints, unlearned representations should mirror this natural overlap. Figure 1(b) (Section 5.1) confirms that QF-Entanglement embeds forgotten samples into the neighboring retain cluster without outlier islands, matching the retrained manifold. White-box audits further verify this alignment: binary linear probing matches Retrain within +0.20%+0.20\% (Appendix B.5), layer-wise CKA remains ≥0.9837\ge 0.9837 across Layers 1–3 (Appendix B.6), and the neighbor-aware MIA-NN attack remains at chance (49.40%49.40\% vs. 50.40%50.40\% Retrain; Table R4).

G2. Linear Entropy and Ambient Trace Preservation

Linear entropy SL(ρ)=1−∥ρ∥F2S_L(\rho) = 1 - \lVert\rho\rVert_F^2 on ρ=K/(Tr(K)+ϵ)\rho = K / (\mathrm{Tr}(K)+\epsilon) is indeed invariant to uniform scaling (Σ→cΣ\Sigma \to c\Sigma). Accordingly, Theorem 1 bounds continuous geometric dimensionality (the effective rank Reff=(TrΣ)2/Tr(Σ2)R_{\mathrm{eff}} = (\mathrm{Tr}\Sigma)^2/\mathrm{Tr}(\Sigma^2)), precluding directional anisotropy or low-rank collapse. Corollary 1 explicitly conditions on sub-Gaussian representations and a non-vanishing ambient trace (Tr(Σf)≥δ>0\mathrm{Tr}(\Sigma_f) \ge \delta > 0), maintained in practice by batch normalization and LCE(Dr)\mathcal{L}_{\mathrm{CE}}(D_r). On CIFAR-100 (Appendix B.6), Tr(Σ)\mathrm{Tr}(\Sigma) remains non-vanishing (18.529018.5290 vs. 26.248626.2486 original) and effective rank stays preserved (Reff=20.38R_{\mathrm{eff}} = 20.38 vs. 21.9121.91). In Section 4.3 and Appendix A, we removed overclaiming terms ("guaranteed", "Wasserstein bound") and stated these conditions explicitly.

G3. Standardized Privacy Auditing Protocols

Our primary MIA metric is Balanced Attack Accuracy under Global-Variance Offline LiRA (Carlini et al., 2022), calibrated against a single Retrain reference (N=1N=1) on a 1:1 balanced split (50%50\% forget members vs. 50%50\% unseen same-class test non-members). Evaluating 1:1 balanced cohorts where 50.00%50.00\% denotes chance indistinguishability is the established standard in machine unlearning (Chen et al., 2021; Chundawat et al., 2023; Kurmanji et al., 2023; Tarun et al., 2023; Fan et al., 2024). We complement this with Population U-MIA (Hayes et al., 2024; Table R2), MIA-NN (Wang et al., 2025; Table R4), CIFAR-10 log-scale ROC and low-FPR LiRA (Figure 2; Appendix B.7), and multi-seed CIFAR-10 results (Table R1).

References:

  • Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., & Tramèr, F. (2022). Membership inference attacks from first principles. IEEE S&P.
  • Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2021). When machine unlearning jeopardizes privacy. ACM CCS.
  • Chundawat, V. S., Tarun, A. K., Mandal, M., & Kankanhalli, M. (2023). Can bad teaching induce forgetting? Unlearning in deep networks using an incompetent teacher. AAAI.
  • Fan, C., Liu, J., Zhang, Y., Wong, E., Wei, D., & Liu, S. (2024). SalUn: Empowering machine unlearning via gradient-based weight saliency. ICLR.
  • Hayes, J., Shumailov, I., Triantafillou, E., Khalifa, A., & Papernot, N. (2024). Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. IEEE SaTML.
  • Kurmanji, M., Triantafillou, P., Hayes, J., & Triantafillou, E. (2023). Towards unbounded machine unlearning. NeurIPS.
  • Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2023). Fast yet effective machine unlearning. IEEE TNNLS.
  • Wang, Y., Ebrahimpour-Boroojeny, A., & Sundaram, H. (2025). On the necessity of output distribution reweighting for effective class unlearning. arXiv:2506.20893.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.