acceptodds
Under review as a conference paper at ICLR 2027

Long-Tailed Learning Strategies Matter in Machine Unlearning

Abstract

Real-world datasets often exhibit long-tailed distributions, where head groups contain many samples and tail groups contain few. Although long-tailed learning strategies are widely used, how they affect machine unlearning remains unclear. In this paper, we study how such strategies influence unlearning difficulty. We find that strategies improving tail performance often make unlearning harder, producing larger deviations from retraining. Moreover, the same degree of tail exposure does not guarantee similar forgetting behavior. Beyond these findings, we observe domain-specific characteristics in vision and language across CIFAR-10, CIFAR-100, Tiny-ImageNet, and long-tailed variants of TOFU and ELUDe, using resampling, augmentation, and loss reweighting under class-wise and random-sample forgetting. Our findings suggest that long-tailed learning strategies should be carefully considered when comparing unlearning methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
long-tail, machine unlearning, learning strategy

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uc9JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how long-tailed learning strategies affect subsequent machine unlearning. The authors conduct a systematic empirical study across vision and language tasks, examining resampling, data augmentation, and loss reweighting under different forgetting settings. The evaluation covers CIFAR-10, CIFAR-100, Tiny-ImageNet, and long-tailed variants of TOFU and ELUDe, involving multiple unlearning methods and strategy-matched retraining references. The experimental results suggest that improving tail performance often leads to larger deviations from retraining, while strategies with similar training exposure can exhibit different forgetting behaviors. The paper further identifies strategy-dependent effects under severe data imbalance, variations across target classes, and interactions between learning strategies and unlearning algorithms.

Critical strengths
  1. The paper investigates an important and relatively underexplored aspect of machine unlearning by examining how training-time long-tailed learning strategies influence subsequent forgetting. The findings highlight the importance of considering training configurations when evaluating unlearning methods.

  2. The authors evaluate multiple learning strategies and unlearning methods across vision and language benchmarks, with additional backbone robustness experiments. The use of strategy-matched retraining references and the analysis of strategy–method interactions provide useful empirical insights.

Critical weaknesses
  1. The novelty of the paper could be better clarified, particularly in relation to existing studies on how training strategies affect machine unlearning.

  2. The paper mainly presents empirical observations, while the underlying mechanisms explaining the observed differences remain insufficiently explored.

  3. The experimental settings may introduce certain confounding factors, making it difficult to fully attribute the observed differences to long-tailed learning strategies.

  4. The evaluation relies on a relatively limited set of metrics, which may not fully capture the effectiveness and reliability of machine unlearning.

  5. Although the experiments cover multiple datasets and models, further validation on more diverse and realistic long-tailed scenarios would strengthen the generalizability of the conclusions.

Questions for authors
  1. Could the authors further clarify the main contributions of this work compared with existing studies on training strategies and machine unlearning?

  2. To what extent might the observed differences be influenced by the hyperparameter settings of the unlearning methods?

  3. Could the authors provide further insights into the underlying factors that explain why different long-tailed learning strategies lead to different unlearning behaviors?

  4. How sensitive are the main conclusions to the choice of evaluation metrics? Would alternative metrics lead to similar observations?

  5. To what extent can the findings be generalized to more diverse datasets, model architectures, and real-world long-tailed distributions?

9 comments

We thank the reviewer for the thoughtful assessment and constructive feedback. Since the questions closely correspond to the critical weaknesses, we address each weakness [W#] together with its corresponding question [Q#] below.

[W1/Q1]

We appreciate the reviewer's request to clarify how our contribution differs from existing studies on training strategies and machine unlearning. Our novelty lies in systematically characterizing how preceding long-tailed learning strategies shape subsequent machine unlearning, rather than simply showing that training-time choices affect unlearning. Prior studies have investigated long-tailed recognition, privacy leakage, and the effects of training-time choices on unlearning. However, their primary research questions differ from ours:

StudyResearch focusDistinction from our work
Buda et al. (2018)Class imbalance and resampling in CNNsExamines classification rather than subsequent unlearning
Ahn et al. (2023)Class-adaptive augmentation for long-tailed recognitionFocuses on improving Tail recognition
Kaya & Dumitras (2021)Data augmentation and membership inferenceEvaluates privacy leakage rather than unlearning fidelity
Falcao & Cordeiro (2025)Data augmentation and image unlearningDoes not systematically examine imbalance-aware strategies across modalities
Wu et al. (2026)Learning-time knowledge encoding and LLM unlearningDoes not investigate long-tailed distributions or vision unlearning
Chen et al. (2025)Tail-class unlearning through boundary manipulationProposes a long-tailed unlearning method rather than comparing preceding learning strategies
OursLong-tailed learning strategies and subsequent unlearningSystematic evaluation across vision and language, covering imbalance severity, deletion settings, and strategy–method interactions

Our study identifies two central findings across vision and language, together with domain-specific observations:

  • Cross-modality findings: Improving Tail performance does not necessarily make subsequent unlearning easier, and the same Tail exposure does not guarantee similar forgetting behavior (Table 1; Figures 2–3).

  • Domain-specific findings: In vision, severe imbalance amplifies strategy-dependent differences in Tail forgetting, and improved Tail forgetting under CB Loss can coincide with larger Head discrepancies (Figure 6; Table 2). In LLMs, learning strategies redistribute forgetting discrepancies across Seen and Held-out Tail QA and change the relative behavior of unlearning methods (Figures 7–8).

These findings establish that unlearning behavior depends not only on the unlearning method, but also on how the deleted data were learned. Unlike prior studies focused on individual training interventions or new unlearning algorithms, our work systematically examines these interactions under long-tailed distributions across vision and language.

We further reproduce the main findings under strategy-specific HP selection (Appendix B.5) and extend the evaluation to financial and medical data (Section 5; Appendix B.7).

References

  1. Buda, M., Maki, A., & Mazurowski, M. A. (2018). A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, 106, 249-259.

  2. Ahn, S., Ko, J., & Yun, S. Y. (2023). Cuda: Curriculum of data augmentation for long-tailed recognition. arXiv preprint arXiv:2302.05499.

  3. Kaya, Y., & Dumitras, T. (2021, July). When does data augmentation help with membership inference attacks?. In International conference on machine learning (pp. 5345-5355). PMLR.

  4. Falcao, A. M., & Cordeiro, F. R. (2025, September). Data Augmentation Improves Machine Unlearning. In 2025 38th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (pp. 1-2). IEEE.

  5. Wu, R., Garov, K., & Chaudhuri, K. (2026, April). Learning-time encoding shapes unlearning in llms. In International Conference on Learning Representations (Vol. 2026, pp. 87040-87067).

  6. Chen, M., Gao, W., Wang, C., Liu, G., Abdelmoniem, A. M., & Peng, K. (2025, August). From Expansion to Retraction: Long-tailed Machine Unlearning via Boundary Manipulation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 (pp. 215-226).

[Q2/W3]

We agree with the reviewer that HP selection and potential confounding factors warrant further investigation. We selected unlearning HPs under Standard and fixed them across learning strategies to examine the effect of preceding training history under a common unlearning procedure, rather than independently optimized configurations. Similar fixed-protocol comparisons appear in Wu et al. (ICLR 2026) and Falcao and Cordeiro (SIBGRAPI 2025). Nevertheless, we agree that strategy-specific HP selection and potential confounding factors warrant further investigation.

We therefore conducted additional strategy-specific HP tuning on CIFAR-100-LT, Tiny-ImageNet-LT, TOFU-LT, and ELUDe-LT, covering four vision and five LLM unlearning methods. The selected configurations were evaluated on held-out conditions without retuning.

The two central cross-modality findings are reproduced after strategy-specific HP selection: improving Tail performance does not necessarily make subsequent unlearning easier, and matched Tail exposure does not guarantee similar forgetting behavior.

Representative results show that stronger Tail performance continues to be accompanied by larger forgetting discrepancies after tuning:

SettingStrategiesPre-unlearning Tail performance ↑Tuned forgetting gap ↓
CIFAR-100-LT, IF=50Standard / CB LossTail TA: 10.84 / 11.78%ΔT: 27.00 / 39.25 pp
Tiny-ImageNet-LT, IF=10Standard / ROSTail TA: 24.78 / 26.16%ΔT: 17.95 / 28.28 pp
TOFU-LTStandard / ROSSeen P: 0.452 / 0.958Seen Gap: 0.069 / 0.259
ELUDe-LTStandard / ParaphraseSeen P: 0.962 / 0.998Seen Gap: 0.107 / 0.252

The exposure-controlled comparisons also reproduce the second finding:

  • Vision: Tail ColorJitter and CB Loss use the same original training examples at the same sampling frequency, yet retain different Tail forgetting gaps after tuning (18.75 vs. 39.25 pp).

  • LLMs: Exposure-matched ROS and Paraphrase exhibit different Seen Tail Probability Gaps on TOFU-LT (0.259 vs. 0.126) and ELUDe-LT (0.199 vs. 0.252), preserving the original benchmark-dependent reversal.

The findings also persist under both Fixed and Tuned HP protocols in the additional financial and medical experiments. As detailed in our response to [Q5/W5], Credit Default reproduces the two central findings, while BloodMNIST-LT also reproduces the imbalance-dependent differences observed in vision (Section 5; Appendix B.7).

We additionally examined two potential confounding factors in the LLM experiments: unequal optimizer-update counts and differences in forget-input composition. Under the original epoch-based protocol, ROS and Paraphrase involve five times as many unlearning optimizer updates as Standard and Loss Weighting. We therefore conducted 120 additional runs across two benchmarks, three seeds, and five unlearning methods using two controls:

  • Step-matched: Preserves augmented forget inputs while matching optimizer-update counts to Standard.

  • Common-source: Uses identical original forget inputs and matched update budgets across strategies while retaining their distinct pre-unlearning checkpoints.

The central strategy-dependent differences persist under the stronger Common-source control, despite identical forget inputs and optimizer-update budgets.

BenchmarkStandardROSLoss WeightingParaphrase
TOFU-LT0.0490.3910.3060.209
ELUDe-LT0.1210.2670.2880.204

Seen Tail Probability Gap under Common-source, averaged over five unlearning methods and three seeds. Lower values indicate closer agreement with matched retraining.

Step matching reduces the Held-out Tail Probability Gaps of ROS/Paraphrase from 0.088/0.066 to 0.022/0.028 on TOFU-LT, and from 0.087/0.127 to 0.034/0.043 on ELUDe-LT. Thus, unequal optimizer-update budgets explain part of the original Held-out discrepancies, but do not account for the persistent strategy-dependent Seen Tail differences under matched inputs and budgets.

These experiments show that the main findings are not explained solely by Standard-selected HP transfer, unequal optimizer-update counts, or forget-input composition. Strategy-dependent unlearning differences persist after HP adaptation and under the stronger LLM controls.

Reviewer_qjtYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper explores the effect of long tailed learning strategy on machine unlearning. It tries to shed some light on how such strategies influence the unlearning difficulty. They empirically show that the strategies improving tail performance often make the unlearning harder and even produce large deviations from retraining. Further they also show that same degree of tail exposure does to guarantee similar forgetting behaviour. The authors perform multiple experiments across vision and language domains spanning, multiple benchmarks, unlearning strategies, and imbalance factor and try to establish their findings.

Critical strengths
  1. The authors explore a relevant but unexplored problem regarding the effect of long tailed learning strategies on machine unlearning and they try to properly shed sufficient insight behind the factors that influence this.
  2. The paper tries to sufficiently cover multiple strategies, domains, baselines and imbalance factors in order to establish their findings.
  3. The paper is well written with sufficient care given for clarity and is easy to read.
  4. The experiments and the corresponding results do justice within the span of the baselines and strategies considered.
Critical weaknesses
  1. The paper although tries to shed light on the effect of long tailed learning strategies on machine unlearning, it is not fully able to explain the problem, causes etc mechanistically. The whole paper remains like a long but interesting diagnostic paper.
  2. The choice of long tailed learning strategies of the authors, specifically that in the vision domain falls short by not including many very important works in long tailed learning literature. The authors include random oversampling, information augmentation, etc but the algorithmic coverage is limited to CBLoss and omits LDAM-DRW, Logit Adjustment, Decoupled training like cRT, Bayesian approaches like Prior2Posterior, etc. In short part from CB-Loss everything is data-level sampling or augmentation. Increasing the coverage to include at least a few of these works would significantly improve the resulting insights which can be gained.
  3. Mixup, Cutmix, Color Jitter, Rotation etc are not exactly long tailed learning strategies, and the motivation behind using these as major long tailed strategies is not completely understood from the manuscript. Further as the strategies like Mixup and cut mix generally blend samples and thus the effect of forget set and retained samples may be coupled. It is not clear whether these methods were chosen deliberately by the authors.
  4. The reference to the class wise results Figure5 would be more apt if mentioned earlier in the section 4.1, around lines 313-314.
  5. Line 456-457 claims that "On TOFU-LT, NPO has the largest Tail Probability Gap under Standard.." while the largest Tail probability gap is 0.55 user Loss weighting compared to the 0.024 value of Standard.
Questions for authors

Check the Critical Weakness section.

4 comments

We appreciate the reviewer's insightful assessment of the strengths and weaknesses of our paper. Your feedback provides valuable insights that help us improve the quality and clarity of our work. Below, we provide a detailed response addressing each of the weaknesses [W#] raised with all references aligned to the main paper:

[W1]

We thank the reviewer for recognizing the interest of our empirical findings. We believe that systematically identifying and characterizing previously underexplored phenomena constitutes an important scientific contribution, even without proposing a new algorithm or establishing a complete causal mechanism. Our study is empirical by design: we investigate how preceding long-tailed learning strategies affect subsequent machine unlearning.

Influential empirical studies have similarly advanced deep learning research by systematically examining existing methods and challenging prevailing assumptions:

StudyMain empirical contributionIndexed citations
Jiang et al. (ICLR 2020)Systematic evaluation of generalization measures1047
Toneva et al. (ICLR 2019)Characterization of example-forgetting dynamics1417
Liu et al. (ICLR 2019)Reassessment of assumptions underlying network pruning2413
Buda et al. (Neural Networks 2018)Systematic investigation of class imbalance in CNNs4617

Similarly, our study systematically identifies two central cross-modality findings: improving Tail performance does not necessarily make subsequent unlearning easier, and the same Tail exposure does not guarantee similar forgetting behavior. We further characterize strategy-dependent forgetting across deleted classes and unlearning methods in vision, and Seen versus Held-out Tail discrepancies in LLMs. These findings show that unlearning behavior depends not only on the unlearning method but also on how the deleted data were learned.

Nevertheless, we agree that deeper mechanistic understanding would further strengthen our empirical findings. We therefore conducted additional exploratory analyses to investigate potential factors underlying the observed strategy dependence across vision and language:

  • Vision — Pre-unlearning gradient geometry. We examined Forget–Utility gradient alignment and the relative magnitudes of Tail and Head forget gradients on CIFAR-100-LT and Tiny-ImageNet-LT. ROS and CB Loss consistently exhibit lower Forget–Utility gradient cosine similarities and smaller Tail/Head gradient-norm ratios than Standard, Global CutMix, and Tail ColorJitter across both datasets.
StrategyCIFAR-100 CosineTiny-ImageNet CosineCIFAR-100 Tail/Head RatioTiny-ImageNet Tail/Head Ratio
Standard0.3000.44318.055.66
ROS−0.213−0.0110.441.81
CB Loss−0.2620.2480.492.21
Global CutMix0.1110.37712.454.21
Tail ColorJitter0.2470.57222.4113.95

These differences are already visible before unlearning begins. They suggest that gradient alignment and group-specific gradient magnitudes may contribute to strategy-dependent unlearning behavior, although these diagnostic measurements do not establish a causal mechanism.

  • LLMs — Training-history dependence. We examined how different ways of reinforcing Tail learning relate to subsequent forgetting. ROS repeats source QA, Loss Weighting increases their contribution to the training objective, and Paraphrase introduces linguistically diverse variants. Our additional Common-source control uses identical original forget inputs and matched optimizer-update budgets while retaining distinct pre-unlearning checkpoints. Under this control, substantial differences in Seen Tail Probability Gap persist:
BenchmarkStandardROSLoss WeightingParaphrase
TOFU-LT0.0490.3910.3060.209
ELUDe-LT0.1210.2670.2880.204

Matching optimizer-update counts reduces some Held-out discrepancies, but the strategy-dependent Seen Tail differences remain even under identical forget inputs and update budgets. Thus, unequal updates and forget-input composition alone do not explain the observed strategy dependence.

These exploratory analyses provide additional insights into the factors underlying strategy-dependent unlearning. The vision diagnostics reveal differences in pre-unlearning gradient geometry, while the LLM controls show that strategy-dependent differences persist under matched forget inputs and optimizer-update budgets. Although these analyses do not establish a complete causal mechanism, they offer complementary evidence for understanding the empirical findings of our paper.

References

  1. Jiang, Y., Neyshabur, B., Mobahi, H., Krishnan, D., & Bengio, S. (2019). Fantastic generalization measures and where to find them. arXiv preprint arXiv:1912.02178.

  2. Toneva, M., Sordoni, A., Combes, R. T. D., Trischler, A., Bengio, Y., & Gordon, G. J. (2018). An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159.

  3. Liu, Z., Sun, M., Zhou, T., Huang, G., & Darrell, T. (2018). Rethinking the value of network pruning. arXiv preprint arXiv:1810.05270.

  4. Buda, M., Maki, A., & Mazurowski, M. A. (2018). A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, 106, 249-259.

[W2]

We appreciate the reviewer's suggestion to broaden the coverage of algorithmic long-tailed learning methods. Our initial selection emphasized individual augmentation strategies to examine their effects on subsequent unlearning separately. This choice was motivated by long-tailed learning methods such as CUDA (Ahn et al., 2023), which adapt augmentation strength across classes. By isolating Tail-specific and global augmentations, we aimed to understand their distinct effects on unlearning and provide insights for designing long-tailed learning methods that account for future deletion requests.

Nevertheless, we agree with the reviewer that broader algorithmic coverage would strengthen our analysis. We therefore added training-time Logit Adjustment (LA; Menon et al., 2021) alongside the existing CB Loss, covering two distinct imbalance-aware approaches:

  • CB Loss: Reweights class contributions according to the effective number of samples.

  • LA: Adjusts training logits using class priors without resampling.

We extended the CIFAR-100-LT experiments to include LA under both random-sample and class-wise forgetting, with strategy-specific unlearning HP selection. The results continue to show distinct unlearning behavior across imbalance-aware learning strategies, as shown below.

StrategyRandom-sample ΔT Forget ↓Random-sample ΔH Forget ↓Class-wise ΔTail ↓
Standard27.00 ± 4.2410.45 ± 0.917.50 ± 3.54
CB Loss39.25 ± 2.479.17 ± 0.764.00 ± 0.71
LA (Added)37.00 ± 4.9515.09 ± 3.879.50 ± 4.95

All values are in percentage points under strategy-specific selected HPs, averaged over four unlearning methods. Random-sample forgetting uses CIFAR-100-LT (IF = 50), while class-wise forgetting uses IF = 10 and held-out deleted classes 14 and 83. The class-wise LA evaluation includes one flagged collapse among eight runs.

Despite using the same original training examples without resampling, CB Loss and LA exhibit different forgetting discrepancies under both deletion settings. On Credit Default and BloodMNIST-LT, these two strategies also show distinct unlearning behavior, further extending the comparison to financial and medical applications.

Given the computational demands and limited rebuttal period, we prioritized these representative evaluations while extending the experiments to broader configurations. Detailed results are provided in Appendix B.5 and D, with the financial and medical experiments in Section 5 and Appendix B.7. We will incorporate additional completed and verified results into the final manuscript if accepted.

References

  1. Ahn, S., Ko, J., & Yun, S. Y. (2023). Cuda: Curriculum of data augmentation for long-tailed recognition. arXiv preprint arXiv:2302.05499.
  2. Menon, A. K., Jayasumana, S., Rawat, A. S., Jain, H., Veit, A., & Kumar, S. (2020). Long-tail learning via logit adjustment. arXiv preprint arXiv:2007.07314.

[W3]

We thank the reviewer for pointing out the distinction between general augmentation operations and long-tail-specific learning strategies. We agree that Mixup, CutMix, Rotation, and ColorJitter are not inherently long-tail-specific algorithms. However, many established long-tailed learning methods build upon these basic operations and adapt them to imbalanced distributions. For example, Remix (Chou et al., 2020) and CurrMix (Jian et al., 2026) extend Mixup, while CMO (Park et al., 2022) and ConCutMix (Pan et al., 2024) build upon CutMix. Similarly, CUDA (Ahn et al., 2023) adapts augmentation strength across classes. We therefore examined these underlying operations separately to understand their effects on subsequent unlearning under long-tailed distributions. We also added training-time Logit Adjustment (LA) alongside CB Loss to broaden our coverage of imbalance-aware learning strategies, as discussed in [W2].

Two design choices further clarify our experimental setup:

  • Global versus Tail-only mixing. Preliminary experiments on CIFAR-10-LT (IF=10) showed that Tail-only Mixup and CutMix reduced Tail TA by 12.78 and 45.94 pp relative to Standard, while Head TA remained approximately 95–96%. Tail-only mixing applies mixed supervision to Tail examples while Head examples retain clean supervision. We therefore adopted standard global Mixup and CutMix to examine the effects of sample mixing without introducing this Tail-only asymmetry.

  • Forget–retain coupling. We agree that Mixup and CutMix can couple forget and retain examples during training. Our evaluation explicitly accounts for this through strategy-matched Exact Retraining. For each deletion request, we remove the original forget examples before constructing retraining batches, excluding them from both anchor and mixing-partner selection. Exact Retraining then applies the same augmentation policy to the retained data. Thus, the retraining reference excludes the deleted examples, while our evaluation measures how closely approximate unlearning approaches that reference.

We have clarified these design choices and the handling of mixed examples in Appendix A.1.

References

  1. Chou, H. P., Chang, S. C., Pan, J. Y., Wei, W., & Juan, D. C. (2020, August). Remix: rebalanced mixup. In European conference on computer vision (pp. 95-110). Cham: Springer International Publishing.

  2. Jian, Z., Chen, Y., Hu, B., Lv, W., Wang, S., Wu, J., ... & Wu, Q. (2026). CurrMix: Curriculum-Enhanced MixUp for Long-Tailed Visual Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7706-7716).

  3. Park, S., Hong, Y., Heo, B., Yun, S., & Choi, J. Y. (2022, June). The majority can help the minority: Context-rich minority oversampling for long-tailed classification. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 6877-6886). IEEE.

  4. Pan, H., Guo, Y., Yu, M., & Chen, J. (2024). Enhanced long-tailed recognition with contrastive cutmix augmentation. IEEE Transactions on Image Processing, 33, 4215-4230.

  5. Ahn, S., Ko, J., & Yun, S. Y. (2023). Cuda: Curriculum of data augmentation for long-tailed recognition. arXiv preprint arXiv:2302.05499.

[W4]

We thank the reviewer for this suggestion. We have moved the reference to Figure 5 earlier in Section 4.1, immediately after introducing the strategy–method differences, to better connect the aggregate and class-wise results.

[W5]

We thank the reviewer for pointing out the ambiguity in our description of Figure 8(a), which was intended to compare unlearning methods within each learning strategy, and apologize for the lack of clarity. Our statement compares the five unlearning methods within each learning strategy, rather than across strategies. In Figure 8(a), NPO has the largest Tail Probability Gap under Standard (0.024), but the smallest under ROS (0.017) and Paraphrase (0.009). The Loss Weighting value (0.055) belongs to a different learning-strategy setting. We have clarified the comparison scope in Section 4.2.

Reviewer_GBb6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how long-tailed learning strategies affect machine unlearning across vision and language benchmarks. It compares multiple long-tailed learning strategies, including resampling, augmentation, and loss reweighting. The results suggest that training strategies influence unlearning discrepancies and can change the relative performance of unlearning algorithms. Therefore, the authors claim that long-tailed learning strategies and training-data distributions should be explicitly considered when evaluating and comparing unlearning methods.

Critical strengths
  • The studied problem is interesting. The paper highlights the importance of long-tailed learning training strategies in the context of machine unlearning.

  • The conducted experiments cover multiple scenarios, including multiple datasets, methods, and backbones from both vision and language tasks..

  • The proposed strategy-matched retraining and exposure-matched comparisons are reasonable.

Critical weaknesses
  • The paper does not adequately explain the practical motivation for studying machine unlearning together with long-tailed learning. Medical imaging and financial fraud detection are mentioned, but no concrete scenario connects these applications to the studied deletion requests, especially deleting an entire tail class.

  • The influence of unlearning hyperparameters is not sufficiently examined. Hyperparameters are selected under standard training and reused across other strategies, so larger discrepancies may simply reflect unsuitable configurations. The main findings should be checked with an equal tuning budget for each strategy, otherwise there might be some strategies make unlearning harder. Also, the robustness is further limited by the fixed vision training seed and the single final tail entity in ELUDe-LT.

  • Several important details of LLM unlearning are not clearly defined. For example, it does not explicitly state the complete definitions of losses or optimization target. Moreover, how duplicated and paraphrased examples affect the number of unlearning updates also needs clarification.

  • The evaluation relies heavily on differences in aggregate accuracy or answer probability, which can hide example-level disagreement. The authors should verify whether the conclusions hold with example-level comparisons.

Questions for authors

See weaknesses.

5 comments

Your feedback provides valuable insights that help us improve the quality and clarity of our work. Below, we provide a detailed response addressing each of the weaknesses [W#] raised with all references aligned to the main paper:

[W1]

We appreciate the reviewer's suggestion to clarify the practical motivation for studying long-tailed learning together with machine unlearning. The central motivation is that data targeted for forgetting may have been repeatedly sampled, augmented, or assigned larger loss weights during training. These choices can improve Tail performance, but may also affect how closely subsequent unlearning approaches Exact Retraining. We consider two deletion settings with distinct practical motivations:

  • Random-sample deletion: In financial prediction, individual customer records may need to be removed when they are no longer authorized for model training. Similarly, medical imaging models may need to remove contributed images following data withdrawal or changes in permitted use. These scenarios motivate removing examples from both Head and Tail groups.

  • Class-wise deletion: Removing an entire Tail class is relevant when a class corresponds to a specific identity, as in person re-identification systems. An individual represented by only a few training images may request removal of all their contributed data. Chen et al. (KDD 2025) also investigate complete Tail-class unlearning in long-tailed settings, including person re-identification.

To further examine unlearning in these application domains, we conducted additional experiments on financial data (Credit Default) and medical images (BloodMNIST). We evaluated five and seven learning strategies, respectively, using four unlearning methods under random-sample and class-wise forgetting. Credit Default retains its naturally imbalanced distribution, whereas BloodMNIST-LT uses real medical images with constructed imbalance factors of 10, 50, and 100.

Our additional experiments reproduce both central cross-modality findings, the vision-specific observation that severe imbalance amplifies strategy-dependent differences in Tail forgetting, and the dependence of unlearning-method behavior on the preceding learning strategy observed in LLMs.

On Credit Default, ROS, CB Loss, and LA improve Tail Recall over Standard, yet exhibit larger discrepancies from strategy-matched Exact Retraining:

StrategyTail Recall (%) ↑ΔT Forget (pp) ↓
Standard41.751.91
ROS59.6112.32
CB Loss50.946.72
LA60.519.10

Results use Standard-selected unlearning HPs fixed across learning strategies. The main findings also persist under strategy-specific HP selection.

The additional experiments further reproduce the following observations:

  • Matched exposure: CB Loss and LA use the same original training examples at the same sampling frequency without resampling, yet exhibit different forgetting behavior on Credit Default and BloodMNIST-LT.

  • Increasing imbalance: On BloodMNIST-LT, the cross-strategy Tail forgetting range increases from 2.85 to 26.85 pp as IF increases from 10 to 100. The pattern also persists under strategy-specific HP selection.

  • Strategy–method interaction: On Credit Default, the unlearning method with the smallest Tail forgetting discrepancy changes across learning strategies. For example, SCRUB has the smallest gap under Standard, NegGrad under ROS, and SalUn under CB Loss. Strategy-dependent method differences remain after HP tuning.

Full results are provided in Section 5 and Appendix B.7. We are extending these experiments to broader configurations and additional models and will incorporate the completed and verified results into the final manuscript if accepted.

Reference

Chen, M., Gao, W., Wang, C., Liu, G., Abdelmoniem, A. M., & Peng, K. (2025, August). From Expansion to Retraction: Long-tailed Machine Unlearning via Boundary Manipulation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 (pp. 215-226).

[W2]

We agree with the reviewer that strategy-specific HP selection and additional robustness checks are important for validating our findings. We selected unlearning HPs under Standard and fixed them across learning strategies to isolate the effect of preceding training history under a common unlearning procedure, as in Wu et al. (ICLR 2026) and Falcao and Cordeiro (SIBGRAPI 2025). Nevertheless, we agree that strategy-specific HP selection is important for verifying our findings.

We therefore conducted additional strategy-specific HP tuning on CIFAR-100-LT, Tiny-ImageNet-LT, TOFU-LT, and ELUDe-LT, covering four vision and five LLM unlearning methods, with evaluation on held-out conditions without retuning. Both central cross-modality findings are reproduced after strategy-specific HP selection: stronger Tail performance does not necessarily imply easier unlearning, and matched Tail exposure does not guarantee similar forgetting behavior.

The following comparisons show that stronger Tail performance continues to be accompanied by larger forgetting discrepancies after tuning:

SettingStrategiesPre-unlearning Tail performance ↑Tuned forgetting gap ↓
CIFAR-100-LT, IF=50Standard / CB LossTail TA: 10.84 / 11.78%ΔT: 27.00 / 39.25 pp
Tiny-ImageNet-LT, IF=10Standard / ROSTail TA: 24.78 / 26.16%ΔT: 17.95 / 28.28 pp
TOFU-LTStandard / ROSSeen P: 0.452 / 0.958Seen Gap: 0.069 / 0.259
ELUDe-LTStandard / ParaphraseSeen P: 0.962 / 0.998Seen Gap: 0.107 / 0.252

The exposure-controlled comparisons also reproduce the second finding:

  • Vision: Tail ColorJitter and CB Loss use the same training examples at the same sampling frequency without oversampling, yet exhibit different Tail forgetting gaps after tuning (18.75 vs. 39.25 pp).

  • LLMs: Exposure-matched ROS and Paraphrase retain different Seen Tail Probability Gaps on TOFU-LT (0.259 vs. 0.126) and ELUDe-LT (0.199 vs. 0.252), preserving the benchmark-dependent reversal.

We further examined HP sensitivity on Credit Default and BloodMNIST-LT. As discussed in [W1], these experiments also reproduce the main findings. Importantly, the results persist under both Standard-selected HPs transferred across strategies (Fixed) and independently selected strategy-specific HPs (Tuned):

SettingFixed HPTuned HP
Credit Default — ΔT Forget (pp)
Standard1.911.91
ROS12.323.01
CB Loss6.722.04
LA9.103.66
Mixup1.261.15
BloodMNIST-LT — Cross-strategy ΔT range (pp)
IF=102.852.85
IF=5011.7713.25
IF=10026.8527.46

Results average four unlearning methods and two held-out deletion requests. The BloodMNIST-LT ranges include all seven learning strategies.

On Credit Default, ROS, CB Loss, and LA retain larger Tail forgetting gaps than Standard after strategy-specific tuning. Their exposure-controlled differences also persist. On BloodMNIST-LT, the cross-strategy forgetting range continues to increase with IF under both HP protocols. These results show that the main findings are not explained solely by transferring HPs selected under Standard.

We also address the remaining robustness concerns:

  • Vision training seed: We acknowledge the limitation of using a single training seed. However, our primary ResNet-18 experiments span three datasets, seven learning strategies, and six unlearning methods under class-wise and random-sample forgetting. Adding another independent training seed would require repeating the full training, strategy-matched Exact Retraining, and unlearning pipeline, approximately doubling the vision computational cost. Given the limited rebuttal period and GPU resources, we instead prioritized held-out forget-set seeds {123, 2024} and class-wise transfer from the HP-development class 28 to classes 14 and 83 on CIFAR-100-LT, where the main strategy-dependent differences persisted.

  • Tail-entity diversity: ELUDe-LT uses distinct entities for HP development (Justin Bieber) and final evaluation (Michael Jordan) across three SFT seeds. TOFU-LT provides complementary evidence from ten Tail authors, each with four Seen and sixteen Held-out QA pairs.

Detailed results are provided in Appendix B.5 and D, with the financial and medical extensions in Section 5 and Appendix B.7. We are extending these experiments to broader configurations and additional models and will incorporate the completed and verified results into the final manuscript if accepted.

References

Wu, R., Garov, K., & Chaudhuri, K. (2026, April). Learning-time encoding shapes unlearning in llms. In International Conference on Learning Representations (Vol. 2026, pp. 87040-87067).

Falcao, A. M., & Cordeiro, F. R. (2025, September). Data Augmentation Improves Machine Unlearning. In 2025 38th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (pp. 1-2). IEEE.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Additional Experiments

Across the three reviews, the main concerns involved our scientific contribution, mechanistic understanding, HP sensitivity, evaluation metrics, and generalizability. We conducted extensive additional experiments to address these concerns. Our main findings remain supported across additional datasets, model architectures, HP selection protocols, and controlled evaluations.

  • Scientific contribution and mechanistic understanding. Our study systematically characterizes how preceding long-tailed learning strategies shape subsequent unlearning across vision and language, with additional validation on financial and medical data. We also conducted exploratory analyses of pre-unlearning gradient geometry in vision and training-history dependence in LLMs to investigate potential factors underlying the observed differences.

  • HP sensitivity and confounding factors. We performed strategy-specific HP tuning across CIFAR-100-LT, Tiny-ImageNet-LT, TOFU-LT, and ELUDe-LT. The main cross-modality findings persist after HP adaptation, together with key domain-specific observations in vision and LLMs. We further conducted 120 additional LLM runs controlling optimizer-update counts and forget-input composition. Substantial strategy-dependent Seen Tail differences remain even under matched inputs and update budgets.

  • Example-level evaluation. We extended our evaluation to paired confidence differences, Prediction Disagreement, Jensen–Shannon divergence, and Paired MAE. These additional evaluations reinforce our central finding that preceding learning strategies shape subsequent unlearning, beyond aggregate metrics. Our within-class analysis further shows that which individual examples exhibit larger discrepancies from Exact Retraining depends on the preceding learning strategy.

  • Generalizability and experimental coverage. We added Credit Default and BloodMNIST-LT, completing 624 additional evaluation configurations under fixed and strategy-specific HP protocols. These experiments reproduce the main cross-modality findings, the vision-specific imbalance-dependent behavior, and strategy–method interactions previously observed in LLMs. We also evaluated additional model architectures and expanded our coverage of imbalance-aware learning strategies with Logit Adjustment.

Together, these additional experiments reinforce our central conclusion: subsequent unlearning depends not only on the unlearning method, but also on how the deleted data were learned.

The revised manuscript presents the additional application experiments in Section 5, controlled and example-level analyses in Appendices B.4–B.7, backbone evaluations in Appendix C, and HP configurations in Appendix D.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.