acceptodds
Under review as a conference paper at ICLR 2027

L2U: Learning What to Unlearn for Continually Adapting Cybersecurity Threat Detectors

Abstract

Machine learning models deployed on evolving data streams must continually adapt as the underlying data distribution changes over time. This challenge is particularly critical in cybersecurity, where shifts can occur naturally as malware, intrusion techniques, and adversarial behaviors evolve, or adversarially through poisoned samples designed to manipulate the model and enable future evasion. For example, benign network behavior observed in an earlier period may later resemble emerging malicious behavior, meanwhile an adversary may deliberately inject poisoned samples to influence how future attacks are classified. Continual Learning (CL) primarily focuses on incorporating new knowledge while preserving prior knowledge; however, indiscriminately retaining historical information can preserve redundant, obsolete, or harmful information that hinders future adaptation. We argue that effective adaptation should instead enable models to adaptively unlearn: learning what historical information to retain and what to remove as the data evolves. In this work, we propose a learning-to-unlearn (L2U) framework that learns which historical samples to unlearn while preserving useful prior knowledge. At each distribution shift, our mechanism evaluates historical data against the current data to determine what should be retained or unlearned, then performs unlearning as part of the adaptation process. We demonstrate the benefits of unlearning as part of the continual adaptation process in both non-adversarial settings, where it removes redundant or obsolete information, and adversarial settings, where it removes poisoned historical samples. Experiments across multiple continual adaptation scenarios show that L2U consistently outperforms CL baselines on new data while preserving performance on previously learned data.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual learning, Cybersecurity, Malware, Machine Unlearning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bidFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Learning to Unlearn (L2U), a framework that improves continual learning by selectively removing outdated, redundant, or poisoned information. The authors introduce two approaches: Heuristic-L2U, which uses predefined rules or limited supervision to select samples for unlearning, and Meta-L2U, which learns the selection strategy through meta-learning. The framework is evaluated under both natural distribution shifts and adversarial poisoning attacks. Experiments on cybersecurity and image datasets show that L2U generally outperforms existing continual learning methods, especially under poisoning attacks and limited model capacity.

Critical strengths
  1. The paper addresses an important limitation of traditional continual learning: preserving historical knowledge is not always beneficial, especially when previous data becomes outdated or poisoned. Introducing selective unlearning into continual adaptation is a meaningful direction, particularly for cybersecurity applications where data distributions frequently change.

  2. The authors evaluate L2U on multiple datasets, including malware detection, network intrusion detection, and image classification. The experiments cover natural distribution shifts, poisoning attacks, different unlearning strategies, and reduced model capacity. The results are particularly promising under poisoning attacks, where Meta-L2U achieves higher accuracy than existing continual learning baselines.

Critical weaknesses
  1. The main contribution of this paper is learning which samples to unlearn through meta-learning. However, as shown in Table 1, Meta-L2U only provides small improvements over Heuristic-L2U on several datasets. For example, the improvement is less than 0.3 percentage points on both EMBER 2018 and EMBER 2024. Meanwhile, Meta-L2U requires additional computational cost to train the selection strategy. It is therefore unclear whether the performance improvements justify the additional complexity of meta-learning.

  2. Although L2U achieves promising results under poisoning attacks as shown in Table 2, the evaluation mainly focuses on classification accuracy rather than directly measuring the success rate of the proposed long-range poisoning attack. In addition, the adaptive selection methods use a small number of labeled clean and poisoned samples to identify suspicious data, while the compared continual learning baselines do not necessarily have access to the same information. Therefore, it remains unclear how much of the improvement comes from the proposed unlearning framework itself.

Questions for authors
  1. In the poisoning experiments, L2U filters detected poisoned samples from the replay buffer, while MADAR does not. Could the authors provide comparisons where all methods use the same replay-buffer filtering and poisoning-related supervision?

  2. How can the authors distinguish the benefits of selective unlearning from those of simply removing suspicious samples and performing additional fine-tuning? Could the authors clarify whether the reported improvements reflect actual removal of the selected samples' influence, rather than the effects of data filtering or additional optimization?

  3. Meta-L2U requires substantial computation for meta-training and additional adaptation under adversarial drift. What is the total computational overhead compared with Heuristic-L2U and other baselines? Also, have you evaluated the learned selection strategy on distribution shifts not used during meta-training?

4 comments

We thank the reviewers for their insights and suggestions:

Q1/W2: Baselines with additional supervision: We provide three supervision-compatible baselines with the same poison supervision as L2U (labeled "Informed" below): the identical poison-aware detector, annotated set (60 / class), flagged set, and replay-buffer filtering. Every method is tested under independent poison, where each run crafts its own poisoned samples and test-time attacks, as in the original paper pipeline.

MethodMean AccTask 10 AccTask 1 Acc
MADAR57.6±4.850.3±3.666.5±7.0
Informed MADAR67.8±3.446.1±7.080.6±2.8
SSF80.9±8.037.7±32.296.3±2.8
Informed SSF74.4±6.048.5±36.584.6±24.4
DEDUCE71.5±2.996.2±0.574.2±3.4
Informed DEDUCE74.1±3.670.6±38.979.9±11.0
Heuristic-L2U (Approx. Unlearning)84.7±4.667.5±2.893.9±1.8
Meta-L2U (Approx. Unlearning)87.1±0.487.7±0.687.3±1.5

The detector improves MADAR and DEDUCE, but no informed baseline approaches L2U: the best reaches 74.4%, over 10 points below Heuristic-L2U (84.7%). Since all informed baselines receive the same detector, the remaining gap reflects how L2U uses the flagged samples (unlearning) rather than detection itself. Basic SSF's high Mean Acc reflects stability rather than adaptation: it learns from few labeled samples per task, so it retains older tasks but largely fails on recent ones (Task 10: 37.7%), also reflected in the table below.

W2: Attack Success Rate Evaluation: Thank you for pointing this out. We report the adversarial error rate at the final task: the percentage of attacked test samples from each task still misclassified by the final model. Lower indicates better resistance to poisoning.

MethodTasks 0–7Tasks 8–9Avg ↓
Naive38.3±7.913.8±16.433.4±9.1
Joint‡5.5±4.548.3±37.714.1±5.7
SI39.2±10.09.2±4.933.2±7.3
A-GEM23.3±6.18.9±5.420.4±4.0
SSF7.0±2.784.7±5.622.6±1.1
DEDUCE32.1±2.12.7±0.626.2±1.6
MADAR63.0±8.293.6±2.069.1±6.9
L2U (Dropped Rows)17.0±9.033.6±42.520.3±1.9
L2U (Flip Label)13.6±7.329.0±29.416.7±0.4
L2U (Approx.)6.2±1.442.2±21.313.4±5.3

Note:Joint conducts a full retrain on all clean data at every task, making it an upper-bound reference rather than a baseline.

Q2: Effect of Selective Forgetting: In the poisoned setting, we ablated unlearning, retaining a framework that only filters the data fed to our CL pipeline, fine-tuning with samples that are not flagged as perturbed/poisoned.

MethodMean AccTask 10 AccTask 1 Acc
L2U w filtering w/o Unlearning67.8±3.446.1±7.080.6±2.8
Heuristic-L2U: Approximate Unlearning84.7±4.667.5±2.893.9±1.8
Heuristic-L2U: Dropped Rows83.2±2.372.5±26.987.4±9.1
Heuristic-L2U: Flip Label83.4±4.673.4±14.985.9±7.5

We see a similar trend in the natural drift setting with a Gaussian Naive Bayes Classifier (GNB) on EMBER 2018.

MethodMean AccTask 0 Final AccAcc on Most Recently Added Families
Heuristic L2U, w/o data filtering and unlearning52.2±0.0251.155.0
Heuristic L2U, w/o unlearning52.2±0.151.055.1
Heuristic L2U, with Unlearning52.4±0.450.959.9

Q3/W1: Benefit of Meta-L2U: The primary advantage of Meta-L2U is generalizability, rather than superior performance. Heuristic-L2U relies on hand-designed rules, assuming that the donut's middle band is beneficial to forget. When that assumption does not hold, Meta-L2U serves as a reliable alternative that automatically learns what needs to be forgotten. Meta-L2U is meta-trained once offline (8 candidate scorers × 5 ES generations = 40 continual-learning runs), and at deployment the learned scorer requires only a forward pass to rank candidate samples. On EMBER 2018, Meta-L2U takes 4.92 GPU hrs on an NVIDIA RTX 6000 GPU, compared with 7.0 min for Heuristic-L2U, 73.9 min for DEDUCE and 1015.5 min (17 hrs) for MalCL. In the poisoned setting, Meta-L2U takes about 20.3 min on CPU, compared with 29.1 min for Heuristic-L2U, which trains all three unlearning lineages (Dropped Rows, Approximate Unlearning and Flip Label) in a single run. Thus, while Meta-L2U incurs additional training overhead, it offers the flexibility to adapt across data distributions without manually redesigning and validating selection heuristics for each setting. We meta-trained the learned selector on EMBER 2018 and tested it on EMBER 2024 (different distribution) without any further meta-training.

ExperimentMean AccTask 10 AccTask 1 Acc
Heuristic-L2U (EMBER 2018)80.3± 0.778.882.5
Meta-L2U (EMBER 2018)80.6± 0.778.982.6
Heuristic-L2U (EMBER 2024)70.9±0.266.074.7
Meta-L2U (EMBER 2024, meta-trained on EMBER 2018)71.1±0.266.174.5
AuthorsOpenReview ↗

Request for feedback

Dear Reviewer,

We have tried our best to address the concerns. Please let us know if you have any further concerns. We will do our best to address those. Your response will be greatly appreciated.

Thanks

I appreciate the authors' efforts and acknowledge that the rebuttal has addressed most of my concerns. I am inclined to increase my score.

Thank you. If there are any additional clarifications that we can provide, please do let us know!

Reviewer_EUjxOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript proposes a "learning-to-unlearn" (L2U) framework designed to improve Continual Learning (CL) in cybersecurity threat detectors. The authors argue that merely retaining historical knowledge is insufficient as data distributions shift; instead, models must actively unlearn obsolete, redundant, or harmful information. The framework evaluates historical data against current data during adaptation and employs two distinct selection strategies: a rule-based approach (Heuristic-L2U) and a meta-learned approach (Meta-L2U). The authors evaluate the framework under both natural distribution drift (evolving threats) and adversarial drift (long-range poisoning), demonstrating that L2U consistently outperforms traditional CL baselines by improving plasticity on new data while preserving performance on past tasks.

Critical strengths

High Relevance to the Cybersecurity Domain: The application of an active unlearning mechanism to Continual Learning is highly pertinent to cybersecurity. Because threat landscapes constantly introduce zero-day attacks and evolving malware, the conceptual shift from purely preserving knowledge to actively discarding obsolete or conflicting knowledge is a strong and practical contribution.

Dual-Threat Applicability: The paper successfully demonstrates the utility of the framework across two distinct and critical scenarios: non-adversarial settings (removing redundant data) and adversarial settings (mitigating long-range poisoning attacks). This dual evaluation significantly strengthens the real-world viability of the proposed method.

Critical weaknesses

Insufficient Differentiation from Natural Episodic Forgetting: Replay-based CL methods inherently perform selective maintenance of episodic memory, where less representative or outdated samples are naturally dropped to accommodate new data within a bounded buffer. The manuscript lacks a comparative analysis detailing precisely how the L2U unlearning mechanics outperform the natural attrition of data in standard replay-based CL. Additionally, the authors do not demonstrate whether a simpler baseline—such as fine-tuning on freshly filtered data—could achieve similar performance gains without a complex unlearning framework.

Algorithmic Novelty and Justification of the "Donut" Heuristic: The Heuristic-L2U approach heavily relies on the Isolation Forest (IF) anomaly detection algorithm, which is already utilized in the baseline MADAR framework for distribution-aware replay. Reusing IF limits the independent contribution of the L2U methodology. Furthermore, the "donut" selection strategy—which explicitly targets samples lying between the most anomalous and most prototypical boundaries—is presented as a rule-of-thumb without rigorous theoretical or empirical justification. It remains unclear why these "middle" samples universally represent redundant or harmful information across different distributions.

Marginal Value Proposition of Meta-Learning: Meta-L2U optimizes its sample selection strategy by exploiting seven distribution-aware features. However, the empirical results show only marginal performance improvements over the simpler Heuristic-L2U (e.g., a fraction of a percentage point difference on the EMBER 2018 and EMBER 2024 datasets). The paper fails to adequately justify the necessity and computational overhead of this meta-learning approach when a simple heuristic provides nearly identical results.

Questions for authors

How does the L2U selective unlearning mechanism specifically provide advantages over standard replay-based CL methods, wherein unimportant or older historical samples are naturally aged out of the bounded memory buffer?

Can you provide a more rigorous theoretical or empirical justification for the "donut" heuristic? Specifically, why are samples with median IF scores structurally considered the most redundant or harmful for continual adaptation?

Given the relatively minor performance improvements achieved by Meta-L2U over Heuristic-L2U in your natural drift evaluations, what is the primary justification for utilizing the computationally heavier meta-learning approach?

Have you considered or tested a baseline that simply fine-tunes the model on dynamically filtered datasets, and if so, how does the performance compare to the full L2U unlearning pipeline?

4 comments

Thank you for your comments and questions.

W1/Q1: Natural episodic forgetting vs. targeted forgetting: Aging samples out of the buffer only stops future replay; it does not remove what the model already learned from them. We therefore ran an additional experiment on EMBER 2018 with a classifier that cannot catastrophically forget: Gaussian naive Bayes (GNB), fitted incrementally (per-class statistics no other class can alter; Banayeeanzade et al., 2021). We fix the forget set (from Heuristic-L2U runs) and vary only how it is treated:

  • Excluded from buffer: Forget set excluded from the replay buffer, no unlearning
  • Random forgetting: Random forget set unlearned |Method|Mean Acc|Task 0 Acc|Acc on Most Recently Added Families| |---|---|---|---| |GNB, No Forget Set: Regular replay buffer|52.2±0.02|51.1|55.0| |GNB, Excluded From Buffer|52.2±0.1|51.0|55.1| |GNB, Random Forgetting|52.1±0.1|51.0|55.1| |GNB, Donut Forgetting|52.4±0.4|50.9|59.9|

Exclusion and random removal do not change accuracy; unlearning L2U's selected samples gains +4.9 on new families (all 3 seeds) while task-0 accuracy stays flat.

W2/Q2: Isolation Forest and the Donut heuristic: IF is a standard, efficient, unsupervised way to rank samples from typical to anomalous (Liu et al., 2008), widely used in intrusion detection (Marteau et al., 2017). In L2U, IF is a feature, not the selector: Meta-L2U uses a learned scorer over seven per-sample features, 2 of which are Isolation Forest scores. Donut keeps both tails of the IF ranking and forgets the middle band. The rationale draws on findings from prior work:

RegionGain if forgottenCost if forgotten
Coresmall: neighbors already represent itcollateral damage to similar samples (Zhao et al., 2024)
Edgelargevery large: often the only evidence of a subfamily (Feldman & Zhang, 2020)
Middlemoderate: partly memorized, so forgetting frees some capacity (Feldman & Zhang, 2020; Zhao et al., 2024)low: anchored on both sides by core and edge

Theoretical Assumption Let $(A(s))$ and $(R(s))$ denote new-task adaptability and old-task retention after forgetting a fixed number of samples at anomaly score (s) (0: inlier, 1: outlier). We define combined performance as $(B(s)=A(s)+\lambda R(s))$. We hypothesize that (i) forgetting core samples has limited impact because they are largely redundant, while (ii) forgetting peripheral samples can substantially hurt retention because they represent rare historical behaviors. If $(B(s))$ increases near the core, decreases near the tail, and is concave $((B'(0)>0), (B'(1)<0), (B''(s)<0))$, it has a unique interior maximum, favoring donut forgetting.

Our simple replay-buffer analysis on a data subset is consistent with (i)/(ii):

Buffer containsFirst task accuracyNew-family accuracy
Anomalous81.335.5
Representative70.349.2

We do not claim that donut is universally optimal: under adversarial drift it performs on par with random selection (See Table 3); this motivates Meta-L2U.

W3/Q3: Value of Meta-L2U: Meta-L2U's main benefit is generalizability: unlike the heuristic, it needs no hand-set assumption to be effective, making the computational overhead worthwhile. To showcase this we meta-trained the learned selector on EMBER 2018 and evaluated it on EMBER 2024 (different distribution) without re-training, and found it matches the heuristic.

ExperimentMean AccFinal Task AccFirst Task Acc
Heuristic-L2U (EMBER 2018)80.3±0.778.882.5
Meta-L2U (EMBER 2018)80.6±0.778.982.6
Heuristic-L2U (EMBER 2024)70.9±0.266.074.7
Meta-L2U (EMBER 2024, meta-trained on EMBER 2018)71.1±0.266.174.5

W1/Q4: Fine-tuning on filtered data: We fine-tuned SSF, MADAR, and DEDUCE on data filtered by our poison-aware classifier ("Informed"), giving them the same selector so that only the unlearning phase differs. Fine-tuning on filtered data improves the baselines, but none approach L2U, so the gains come from unlearning, not filtering.

MethodMean AccFinal Task AccFirst Task Acc
Informed MADAR67.8±3.446.1±7.080.6±2.8
Informed SSF74.4±6.048.5±36.584.6±24.4
Informed DEDUCE74.1±3.670.6±38.979.9±11.0
Heuristic-L2U (Approx. Unlearning)84.7±4.667.5±2.893.9±1.8
Meta-L2U (Approx. Unlearning)87.1±0.487.7±0.687.3±1.5

References:

Banayeeanzade, M., Mirzaiezadeh, R., Hasani, H., & Soleymani, M. Generative vs. discriminative: Rethinking meta-continual learning. Advances in Neural Information Processing Systems, 34,2021

Liu, F. T., Ting, K. M., & Zhou, Z.-H. Isolation forest. ICDM,2008

Marteau, P.-F., Soheily-Khah, S., & Béchet, N. Hybrid isolation forest – application to intrusion detection. arXiv:1705.03800,2017

Feldman, V., & Zhang, C. What neural networks memorize and why. NeurIPS,2020

Zhao, K., et al. What makes unlearning hard and what to do about it. NeurIPS,2024

AuthorsOpenReview ↗

Request for feedback

Dear Reviewer,

We have tried our best to address the concerns. Please let us know if you have any further concerns. We will do our best to address those. Your response will be greatly appreciated.

Thanks

Thank you for the detailed rebuttal and additional experiments. I have updated my rating 2. The additional experiments help address my concerns about the benefits of unlearning over replay-based CL and filtered-data fine-tuning. However, the theoretical justification for the donut heuristic remains limited, and the marginal improvements of Meta-L2U over Heuristic-L2U do not sufficiently establish the value of meta-learning. These concerns continue to affect my overall assessment.

We appreciate your engagement with our rebuttal. Below we expand on the two remaining concerns.

1. Theoretical justification for Donut. In our rebuttal we stated the conditions under which Donut is optimal; here we make this precise and derive what it implies. With B(s)=A(s)+R(s) as before:

Proposition: If A and R are twice differentiable with A''(s)0, R''(s)<0, >0, and B'(0)>0>B'(1), then B has a unique maximizer s(0,1), with B(s)>(B(0),B(1)). That is, forgetting the middle band strictly outperforms forgetting either the core or the tail.

Proof: B''=A''+R''<0, so B' is strictly decreasing. Since B'(0)>0>B'(1), by the intermediate value theorem B' has exactly one zero s(0,1), which is the unique global maximum of the strictly concave B.

Consequences: (i) At the optimum, A'(s)=-R'(s): Donut forgets where the marginal adaptation gain equals the marginal retention loss. (ii) Differentiating this condition gives ds/d=-R'(s)/B''(s)<0, so the more retention is valued, the closer to the core the forgotten band moves. In security, where old attack variants can reappear (large ), this favors a conservative band that protects the rare tail, which is what Donut implements.

The assumptions are grounded in prior work: forgetting the redundant core frees little (B'(0)>0; Zhao et al., 2024), and tail samples are often the only evidence of a rare subpopulation, so retention loss accelerates there (R''(s)<0, B'(1)<0; Feldman & Zhang, 2020). Empirically, the endpoints behave as predicted (anomalous buffer: higher retention; representative buffer: higher adaptation; see our previous reply), and forgetting the Donut band improves adaptation without retention loss in the GNB experiment.

These conditions characterize natural drift, the setting Donut is designed for. However, Donut does not work universally, and we report this explicitly as a finding (Section 5.3, Table 3). In natural drift, outdated samples tend to fall in the middle band, so Donut finds them. In adversarial drift, the attacker can place poisoned samples anywhere in the ranking, so a fixed band cannot target them. This is what motivates Meta-L2U.

2. Value of Meta-L2U. Under natural drift, Heuristic-L2U and Meta-L2U both perform well, and we do not claim Meta-L2U is more accurate there. Its value is generalizability: a single learned selector that works across settings without hand-designing a rule for each. This mirrors established work on meta-learned sample selection. Hand-designed weighting rules require opposite assumptions for different data problems, whereas a meta-learned weighting adapts to either without redesign (Ren et al., 2018; Shu et al., 2019). Similarly, learned optimizers are valued for working across many tasks without per-task tuning rather than for beating a tuned baseline on each one (Metz et al., 2022), and meta-learners are commonly evaluated by how well they generalize to unseen datasets (Triantafillou et al., 2020).

Our results follow this pattern. Meta-L2U matches the heuristic under natural drift, transfers to an unseen distribution without retraining, and remains strong where the fixed Donut rule does not apply (as seen in the adversarial experiments). Its one-time meta-training cost is amortized across these settings instead of redesigning and validating a selection rule for each.

Thank you again for your time and input!

References:

Ren, M., Zeng, W., Yang, B., & Urtasun, R. Learning to reweight examples for robust deep learning. ICML, 2018.

Shu, J., et al. Meta-Weight-Net: Learning an explicit mapping for sample weighting. NeurIPS, 2019.

Metz, L., et al. VeLO: Training versatile learned optimizers by scaling up. arXiv:2211.09760, 2022.

Triantafillou, E., et al. Meta-Dataset: A dataset of datasets for learning to learn from few examples. ICLR, 2020.

Feldman, V., & Zhang, C. What neural networks memorize and why. NeurIPS, 2020.

Zhao, K., et al. What makes unlearning hard and what to do about it. NeurIPS, 2024

Reviewer_ooTfOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies continual adaptation of cybersecurity threat detectors under distribution drift. It argues that continual learning should decide not only what to retain but also what to unlearn, and proposes L2U, a continual adaptation framework that incorporates selective unlearning into cybersecurity threat detection. At each task, L2U runs a four-stage pipeline consisting of distribution-aware continual adaptation with MADAR-style replay based on Isolation Forest ranking, selection of the forget set, an unlearning strategy, and evaluation after unlearning on the current task and past tasks. Experiments include class-incremental malware datasets, Tiny ImageNet, and poisoned domain incremental CICIDS 2017 and poisoned Tiny ImageNet.

Critical strengths
  • The paper applies selective forgetting systematically to cybersecurity threat detection under natural drift and long-range poisoning, and evaluates it within an integrated continual adaptation pipeline.
  • HEURISTIC-L2U provides an interpretable, distribution-aware selector, while META-L2U offers a learned adaptive counterpart driven by post-unlearning feedback. These two paths are complementary, making the observed results more likely to reflect the effect of selective forgetting itself rather than the idiosyncrasy of a particular selection rule or experimental setup.
Critical weaknesses
  • L2U filters replay samples with a perturbation detector and uses an informed selector with annotated clean and poisoned samples, while no baseline receives comparable supervision. Without a MADAR baseline with the identical detector and a detector-free L2U variant, the gains cannot be cleanly attributed to selective unlearning.
  • Donut forgets the middle band of the Isolation Forest ranking without using current-task data, so it supports removing redundant samples but not establishing obsolescence. If the candidate pool is the replay buffer, which already keeps both ends of the ranking, the middle band is largely absent from replay, and forgetting it may have little effect.
  • Several table numbers are inconsistent, undermining result credibility. For example, HEURISTIC-L2U Dropped Rows is 83.00 in Table 2 but 80.57 in Table 3; the text reports 84.04 as the lowest Meta-L2U result while the tables show 86.02; Table 4 and Table 12 also disagree on model-capacity results.
  • Evidence for the “learning” component is limited. META-L2U improves only slightly over HEURISTIC-L2U under natural drift, and on Tiny ImageNet HEURISTIC-L2U is only 0.03 above MADAR. Without a supervised selector baseline trained on the same features and annotation budget, learned selection does not clearly outperform supervised heuristics.
  • Selective forgetting has been explored in AFEC, DEDUCE, and SSF; long-range poisoning builds on existing delayed-effect poisoning work. The method largely combines MADAR-style replay, Isolation Forest ranking, heuristic selectors, a small Evolution Strategies scorer, and existing unlearning strategies, without a clearly new mechanism, objective, or theoretical insight.
Questions for authors

Please see the Weakness for details.

2 comments

Thank you for your comments and suggestions.

W1: Detector supervision and a detector-free variant: To test whether our gains come from L2U's unlearning rather than its poison classifier, we gave MADAR, SSF, and DEDUCE (for W5) identical supervision ( "Informed”): the same classifier, annotated set and flagged set, used to filter the replay buffer exactly as in L2U, with all else unchanged. We also tested L2U with no detector (forget samples chosen at random).

MethodMean AccTask 10 AccTask 1 Acc
Informed MADAR67.8±3.446.1±7.080.6±2.8
Detector-free L2U (Approx. U)69.9±2.174.1±4.955.2±1.6
Informed SSF74.4±6.048.5±36.584.6±24.4
Informed DEDUCE74.1±3.670.6±38.979.9±11.0
Heuristic-L2U (Approx. U)84.7±4.667.5±2.893.9±1.8
Meta-L2U (Approx. U)87.1±0.487.7±0.687.3±1.5

Some methods benefit from detector, but no informed baseline approaches L2U, and even detector-free L2U (random forgetting) exceeds Informed MADAR, so the gains come from unlearning, not detection.

W2: Donut candidate pool and obsolescence: To clarify, the candidate pool is the arriving task, not the replay buffer. The selector scores the incoming samples before the buffer is rebuilt;Some of the other baselines automatically exclude the middle band from the buffer. However, this may not be enough. The assumption here is, if they are not part of the replay buffer, they will be forgotten. Some models do not forget automatically (Banayeeanzade et al., 2021), and if we don't forget those points, conflicting data in later tasks can hurt performance. To test this, we ran experiments on Gaussian naive Bayes (GNB), which are robust to forgetting. Here we outline experiments that show the different effects of the forget set:

  • No forget set (regular replay buffer)
  • Excluded from buffer Forget set excluded from the replay buffer, no unlearning
  • Random forgetting Same sized random forget set unlearned |Method|Mean Acc|Task 0 Final Acc|Acc on Most Recently Added Families| |---|---|---|---| |GNB, No Forget Set|52.2±0.02|51.1|55.0| |GNB, Excluded From Buffer|52.2±0.1|51.0|55.1| |GNB, Random Forgetting|52.1±0.1|51.0|55.1| |GNB, Donut Forgetting|52.4±0.4|50.9|59.9|

W3: Table inconsistencies: The inconsistencies pointed out arose because we reported the statistics from different seeds. To ensure consistency, we ran some experiments using the exact same seeds as the other setups. We have updated our tables with consistent seeds; all our conclusions(with regards to capacity and poisoning) remain the same (See Tables 2, 3, 4 and 12).

W4: Evidence for the learning component: Meta-L2U’s main benefit is generalizability: unlike the heuristic, it needs no hand-set assumption to be effective. To showcase this we meta-trained the learned selector on EMBER 2018 and evaluated it on EMBER 2024 (different distribution) without re-training, and found it matches the heuristic.

ExperimentMean AccTask 10 AccTask 1 Acc
Heuristic-L2U (EMBER 2018)80.3±0.778.882.5
Meta-L2U (EMBER 2018)80.6±0.778.982.6
Heuristic-L2U (EMBER 2024)70.9±0.266.074.7
Meta-L2U (EMBER 2024, meta-trained on EMBER 2018)71.1±0.266.174.5

W5: Novelty: Our contribution is not a new unlearning operator but a new objective: learning which historical samples to forget to improve future adaptation while preserving prior knowledge, using post-unlearning performance as the training signal. Existing methods address forgetting at the parameter or task level (DEDUCE) or curate replay memory with fixed criteria (SSF). Other methods down-weight or exclude samples but cannot undo what the model has already learned. In cybersecurity, threats evolve and attackers adapt, so harmful samples are often identified only after being learned.To our knowledge, L2U is the first learned sample-level forgetting policy for continual learning under both drift and poisoning, and the policy transfers across tasks and datasets rather than being refit from scratch.It outperforms DEDUCE and SSF even when they receive the same poison supervision (W1). We further introduce a long-range poisoning threat model for continual intrusion detection. Unlike prior replay poisoning (Ariffin et al., 2026), our adversary shifts its perturbation distribution across tasks, combines limited training-time poisoning with stronger test-time evasion, and preserves valid network packet representations.

References: Mohammadamin Banayeeanzade, Rasoul Mirzaiezadeh, Hosein Hasani, and Mahdieh Soleymani. Generative vs. discriminative: Rethinking the meta-continual learning. In Advances in Neural Information Processing Systems 34 (NeurIPS 2021), pp. 21592–21604, 2021.

Azizi Ariffin, Afif Haris, Faiz Zaki, Hazim Hanif, and Nor Badrul Anuar. Adaptive intrusion detection system using transformer-based neural networks and continual learning approach with adversarial investigation. arXiv preprint arXiv:2608.04602, 2026.

AuthorsOpenReview ↗

Request for feedback

Dear Reviewer,

We have tried our best to address the concerns. Please let us know if you have any further concerns. We will do our best to address those. Your response will be greatly appreciated.

Thanks

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.