- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies machine unlearning in class-incremental learning (CIL). The authors observe that a CIL model encodes different samples with very different strength: classes from early tasks are usually weakly encoded while classes from the most recent task are strongly encoded. Applying a uniform unlearning update to all forget samples therefore over-removes weakly encoded samples and under-removes strongly encoded ones. To address this, the authors propose IMPRINT, which estimates the imprint of each forget sample using its logit margin and converts the margin into a per-sample weight through a sigmoid. The weighting is model-agnostic and plug-and-play, and can be applied on top of existing unlearning methods such as RL, SalUn, AMUN and BS. The appendix also gives the margin a geometric interpretation and a lower bound connecting it to parameter-space removal effort. Experiments on CIFAR-100, Tiny-ImageNet, ImageNet-1k, a ViT setting, iCaRL, Task-IL/Domain-IL and 20 Newsgroups show that IMPRINT achieves the smallest average gap to retraining among the compared methods, with negligible runtime overhead.
Critical strengths
-
The problem is concrete and well motivated. The variation in sample-wise imprint strength has direct empirical support: the class-level correlations in Figures 1 and 3, and the agreement with IMU's influence ranking in Table 2. This is a real issue that has not received explicit attention in the unlearning literature, and the paper gives a clear illustration of the failure case of uniform unlearning in Figure 2.
-
The supporting analysis is complete relative to the simplicity of the method. The appendix derives the connection between margin, decision-boundary distance and parameter-space removal effort, and backs it with measured correlations of 0.960 against the first-order removal effort and 0.959 against an empirical boundary-crossing search. The mean-matched uniform control in Table 16 isolates the effect of sample-dependent weighting from simply rescaling the loss. The method requires no additional data or reference model, and the overhead in Figure 6 is negligible. The improvement holds across backbones, datasets, buffer sizes, ten baselines and several continual-unlearning dynamics experiments in Section E.
Critical weaknesses
- The main text is somewhat imbalanced in coverage. Most headline experiments use class-wise forgetting on CIFAR-100 and Tiny-ImageNet; the ImageNet-1k and ViT results use FOSTER only, and random data forgetting is discussed briefly.
- The margin proxy assumes the logit scale is roughly comparable across tasks. Appendix B.7 audits the natural task-wise scale, finding a ratio of only 1.032, and runs a 4x stress test in Table 15; however, the stress test only applies a monotone, task-level scaling to the margin computation. Whether the ordering stays stable under other drift patterns, such as drift concentrated in a few classes rather than whole tasks, is not really covered. I suggest adding a short discussion of such failure modes.
- The sensitivity ablations over the temperature, the forgetting ratio and the CIL training budget are all run on CIFAR-100 with DER++ only. The proxy comparison does extend to Tiny-ImageNet, as shown in Table 11, but I suggest extending at least the temperature ablation to one more backbone to make the robustness claims easier to accept.
Questions for authors
- The temperature sweep in Table 17 is on CIFAR-100 only. How sensitive is IMPRINT to tau on Tiny-ImageNet and ImageNet-1k, where the logit distributions can be quite different from CIFAR-100?
- Section 3.5 mentions upsampling the retain set to match the effective size of the forget set. How many upsampling iterations are typically needed, and does the unlearning time scale badly when the forget set is much larger than the replay buffer?
- The sample-wise margin is recomputed at every training epoch. Have you observed any instability in this adaptive process, such as oscillating weights for strongly encoded samples, and does a coarser update frequency, say every few epochs, work about as well?
- The scale audit in B.7 is task-wise. Have you checked whether a per-class logit-scale drift, one not aligned with task boundaries, occurs in practice, and if so, whether the margin ordering survives it?
5 comments
Response to Reviewer Gmkd (2/3)
2. Controlled within-task perturbations.
We apply three class-dependent scaling patterns within each original ten-class task:
Table: Scaling assignments within each original ten-class task.
| Pattern | Scaling assignments within each original task |
|---|---|
| Balanced | Five classes receive ; five receive . |
| Sparse | One class receives ; nine receive . |
| Inverse sparse | One class receives ; nine receive . |
Each pattern has a maximum-to-minimum scaling ratio of and a geometric mean of . The sparse patterns distinguish one class from the other nine, directly testing variation concentrated within tasks.
For a sample with label , we modify its margin and weight as
\widetilde{\omega}_i = \sigma\left(\frac{\widetilde{m}_i}{\tau}\right).$$ Base forget/retain losses and evaluation predictions use the original logits; perturbations affect only IMPRINT's weighting signal. We report ranking agreement between unperturbed and perturbed margins alongside unlearning performance. **Table: Effects of class-dependent scaling.** | Condition | Spearman $\rho$ ↑ | Top-10% overlap ↑ | FA ↓ | RA ↑ | MIA ↑ | FT ↓ | RT ↑ | Avg. Gap ↓ | |---|---:|---:|---:|---:|---:|---:|---:|---:| | Retrain | — | — | 0.00 | 64.36 | 100.00 | 0.00 | 55.62 | — | | **Native IMPRINT** | 1.0000 | 100.00% | 0.02 | 62.40 | 99.86 | 0.01 | 55.51 | **0.48 ± 0.01** | | Balanced $4\times$ | 0.9204 | 45.83% | 0.08 | 61.64 | 99.75 | 0.04 | 54.64 | 0.81 ± 0.05 | | Sparse $4\times$ | 0.9709 | 66.44% | 0.04 | 62.03 | 99.81 | 0.01 | 54.92 | 0.66 ± 0.03 | | Inverse sparse $4\times$ | 0.9756 | 91.80% | 0.08 | 61.90 | 99.75 | 0.01 | 54.81 | 0.73 ± 0.15 | Global rank agreement remains high, but top-decile membership changes substantially under balanced and sparse scaling—by approximately one-third under sparse scaling—and is better preserved under inverse-sparse scaling. Avg. Gap increases under all three perturbations, while the other metrics remain close to native performance. These results support continued effectiveness under the tested distortions, not invariance of ranking or performance. **3. Scope and failure modes.** Positive class-dependent scaling preserves margin signs and within-class ordering but can reorder samples across classes. Independently scaling or shifting output-class logits can also change the strongest competitor or margin sign, potentially weakening raw-margin weighting; these more general distortions are outside our tests. We have added the class-wise logit-scale analysis and discussion of failure modes to Appendix B.8 (Tables 18 and 19). ### W3 & Q1: Temperature Sensitivity Across Datasets and CIL Methods We extend the sweep to $\tau\in\{0.25,0.5,1.0\}$ on Tiny-ImageNet (ResNet-18, DER++), ImageNet-1k (ResNet-18, FOSTER), and CIFAR-100 (ViT, FOSTER and DER++), considering five forgetting configurations. **Table: Temperature sensitivity: Tiny-ImageNet, ResNet-18, DER++.** | Method | τ | FA ↓ | RA ↑ | MIA ↑ | FT ↓ | RT ↑ | Avg. Gap ↓ | |---|---:|---:|---:|---:|---:|---:|---:| | Retrain | — | 0.00 | 56.78 | 100.00 | 0.00 | 44.01 | - | | RL | — | 0.00 | 25.02 | 99.99 | 0.00 | 26.80 | 9.80 ± 1.20 | | **RL+IMPRINT** | 0.25 | 0.08 | 47.34 | 99.61 | 0.01 | 39.67 | **2.85 ± 1.26** | | RL+IMPRINT | 0.5 | 0.17 | 42.98 | 98.95 | 0.03 | 36.04 | 4.60 ± 1.02 | | RL+IMPRINT | 1 | 0.05 | 31.94 | 99.30 | 0.00 | 26.83 | 8.55 ± 4.10 | **Table: Temperature sensitivity: ImageNet-1k, ResNet-18, FOSTER.** | Method | τ | FA ↓ | RA ↑ | MIA ↑ | FT ↓ | RT ↑ | Avg. Gap ↓ | |---|---:|---:|---:|---:|---:|---:|---:| | Retrain | — | 0.00 | 39.43 | 100.00 | 0.00 | 43.18 | — | | RL | — | 0.00 | 20.76 | 100.00 | 0.00 | 23.57 | 7.66 ± 4.15 | | RL+IMPRINT | 0.25 | 0.00 | 34.53 | 100.00 | 0.00 | 38.22 | 1.97 ± 0.31 | | RL+IMPRINT | 0.5 | 0.00 | 34.55 | 100.00 | 0.00 | 38.19 | 1.97 ± 0.35 | | **RL+IMPRINT** | 1 | 0.00 | 34.88 | 100.00 | 0.00 | 38.61 | **1.83 ± 0.21** | **Table: Temperature sensitivity: CIFAR-100, ViT, FOSTER.** | Method | τ | FA ↓ | RA ↑ | MIA ↑ | FT ↓ | RT ↑ | Avg. Gap ↓ | |---|---:|---:|---:|---:|---:|---:|---:| | Retrain | — | 0.00 | 91.63 | 100.00 | 0.00 | 89.00 | - | | RL | — | 0.00 | 77.93 | 100.00 | 0.00 | 75.61 | 5.42 ± 2.56 | | **RL+IMPRINT** | 0.25 | 0.08 | 89.86 | 99.97 | 0.19 | 87.79 | **0.66 ± 0.59** | | RL+IMPRINT | 0.5 | 0.04 | 89.08 | 99.98 | 0.09 | 86.94 | 0.95 ± 0.62 | | RL+IMPRINT | 1 | 0.01 | 85.85 | 99.99 | 0.01 | 83.61 | 2.24 ± 0.61 |Response to Reviewer Gmkd (3/3)
Table: Temperature sensitivity: CIFAR-100, ViT, DER++.
| Method | τ | FA ↓ | RA ↑ | MIA ↑ | FT ↓ | RT ↑ | Avg. Gap ↓ |
|---|---|---|---|---|---|---|---|
| Retrain | — | 0.00 | 80.14 | 100.00 | 0.00 | 73.51 | - |
| RL | — | 0.02 | 69.38 | 99.99 | 0.00 | 62.69 | 4.32 ± 0.97 |
| RL+IMPRINT | 0.25 | 1.15 | 75.42 | 99.84 | 0.42 | 69.05 | 2.18 ± 0.26 |
| RL+IMPRINT | 0.5 | 0.88 | 76.35 | 99.83 | 0.31 | 69.87 | 1.76 ± 0.31 |
| RL+IMPRINT | 1 | 0.31 | 74.93 | 99.90 | 0.05 | 68.58 | 2.12 ± 1.29 |
Temperature sensitivity varies across settings: smaller temperatures yield lower mean gaps on Tiny-ImageNet and ViT–FOSTER, ImageNet-1k shows little variation, and performs best on ViT–DER++. RL+IMPRINT achieves lower reported mean gaps than uniform RL at every tested temperature in all four settings, supporting as an effective shared default. We have added the temperature sweeps across datasets and backbones to Appendix C.1 (Table 22).
Q2: Retain-Set Upsampling and Computational Cost
We evaluate 10- and 30-class deletion on CIFAR-100 using RTX 3090 GPUs, a replay buffer capacity of 2,000, and batch size 256. Upsampling counts equivalent full passes through available retained samples per epoch. Elapsed times include setup and validation through the first epoch's validation checkpoint.
Table: Retain-set reuse and elapsed time on CIFAR-100.
| Number of forget classes | Forget samples | Forget/retain batches per epoch | Retain-set upsampling (×) | Elapsed time through first-epoch validation |
|---|---|---|---|---|
| 10 | 4,500 | 18 / 18 | 2.284 | 4.02 s |
| 30 | 13,500 | 53 / 53 | 8.917 | 10.34 s |
The two settings require approximately and retain-set reuse per epoch. Each forget minibatch is paired with exactly one retain minibatch, giving minibatch pairs for forget samples, batch size , and epochs. At fixed and , optimization workload scales linearly with forget-set size; a smaller retain set increases reuse without adding optimization steps or a nested loop. We have clarified retain-set upsampling in Section 3.5 and added the reuse and runtime measurements to Appendix C.4 (Table 29).
Q3: Stability of Adaptive Logit Margin
Native IMPRINT recomputes sample-wise margins and weights at every forget minibatch; The submitted version described epoch-wise updates. We examine stability and coarser refresh schedules on CIFAR-100 with DER++ and a replay buffer of 2,000.
1. Adaptive-weight stability.
We track fixed evaluation samples in the top quartile of pre-unlearning margins, representing initially strongly imprinted samples, and evaluate their weights at epoch checkpoints.
Table: Adaptive-weight stability for initially strongly imprinted samples.
| Diagnostic | Result |
|---|---|
| Samples with repeated margin-sign changes | 0.484% |
| Samples with repeated extreme weight switches (≤0.1 ↔ ≥0.9) | 0.000% |
| Mean adjacent absolute weight change | 0.00382 |
Repeated margin-sign changes are rare, no repeated extreme weight switches are observed, and mean adjacent-epoch changes are small. These diagnostics provide little evidence of persistent oscillations at the recorded epoch checkpoints.
2. Weight-refresh frequency.
For , refresh epochs begin at epoch 1 and recur every epochs. During each refresh epoch, margins and weights are recomputed at every forget minibatch using the native implementation and cached. Intervening epochs continue minibatch optimization using the corresponding cached weights. Thus, alternates native recomputation in epochs 1, 3, 5, … with cached weights in epochs 2, 4, 6, …. Results are averaged over three paired deletion requests.
Table: Effect of weight-refresh frequency on CIFAR-100.
| Method | FA ↓ | RA ↑ | MIA ↑ | FT ↓ | RT ↑ | Avg. Gap ↓ |
|---|---|---|---|---|---|---|
| Retrain | 0.00 | 64.36 | 100.00 | 0.00 | 55.62 | — |
| RL | 2.15 | 57.32 | 99.38 | 1.21 | 50.69 | 3.19 ± 0.35 |
| IMPRINT (native) | 0.02 | 62.40 | 99.86 | 0.01 | 55.51 | 0.48 ± 0.01 |
| IMPRINT () | 0.16 | 62.96 | 99.50 | 0.03 | 55.80 | 0.48 ± 0.12 |
| IMPRINT () | 0.02 | 61.88 | 99.87 | 0.00 | 54.70 | 0.71 ± 0.15 |
| IMPRINT () | 2.29 | 63.20 | 96.88 | 0.86 | 55.99 | 1.56 ± 0.21 |
Alternating native recomputation and cached epochs () matches the native mean Avg. Gap of ; and increase it to and . At , forgetting weakens despite relatively high retain performance. One possible explanation is that outdated weights become less aligned with the model's current imprint. These results support periodic recalibration.
We have clarified the weight-update frequency in Section 3.5 and added the stability diagnostics and refresh-frequency comparison to Appendix C.5 (Table 30).
Thank you again for your time and effort. We would be happy to address any remaining questions or concerns the reviewer might have.
Response to Reviewer Gmkd (1/3)
We sincerely thank the reviewer for the time, effort, and valuable feedback.
W1: Experimental Coverage
The revised manuscript now includes the following results.
1. ImageNet-1k plug-and-play comparison.
We compare RL, SalUn, AMUN, and BS with their IMPRINT variants on ImageNet-1k using ResNet-18 and FOSTER.
Table 7. ImageNet-1k plug-and-play results with FOSTER.
IMPRINT reduces the mean Avg. Gap for all four methods on ImageNet-1k.
2. ViT with DER++.
We evaluate IMPRINT on CIFAR-100 with ViT and DER++, extending the submitted paper's ViT–FOSTER analysis. We compare uniform RL and SalUn with their IMPRINT variants at the default τ=0.5 across five forgetting configurations.
Table: CIFAR-100 with ViT and DER++ (τ=0.5).
IMPRINT reduces the mean Avg. Gap and improves retained-data performance for both methods, extending the ViT evidence to another CIL strategy.
3. Random forgetting on Tiny-ImageNet.
We extend random forgetting from CIFAR-100 to Tiny-ImageNet with ResNet-18 and DER++. Random forgetting selects individual samples across classes, producing forget sets with diverse class memberships and imprint strengths.
Table: Random forgetting on Tiny-ImageNet with ResNet-18 and DER++.
IMPRINT reduces the average gap to retraining and improves retained-data performance, extending the evidence to sample-level deletion on another dataset. Together, these evaluations broaden coverage across the tested scenarios, datasets, CIL methods, and backbones. We have moved the ImageNet-1k plug-and-play comparison to Section 4.2 (Table 7) and added the ViT–DER++ and Tiny-ImageNet random-forgetting results to Appendix D.1 (Table 36) and Appendix D.2 (Table 37), respectively.
W2 & Q4: Robustness to Per-Class Logit-Scale Variation
We examine natural class-wise scale variation and controlled class-dependent distortions on CIFAR-100, using a replay buffer capacity of 2,000 and three forgetting configurations.
1. Natural variation and margin ordering.
We measure each sample's logit scale as the standard deviation of its output logits and define sc as the median scale among samples of class c. We compare raw margins with class-scale-normalized margins:
minorm=misyig,
where g is the geometric mean of class-level scales.
Table: Natural class-wise scale variation and margin ordering.
Class-wise scale variation occurs, with task means explaining only a small fraction. Overall ranking agreement remains high, although normalization changes some of the highest-ranked samples.