acceptodds
Withdrawn from ICLR 2027

Automatic Attack Discovery for Few-Shot Class-Incremental Learning via Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Few-shot class incremental learning (FSCIL) is a more realistic and challenging paradigm in continual learning to incrementally learn unseen classes and overcome catastrophic forgetting on base classes with only a few training examples. Previous efforts have primarily centered around studying more effective FSCIL approaches. By contrast, less attention was devoted to thinking the security issues in contributing to FSCIL. This paper aims to provide a holistic study of the impact of attacks on FSCIL. We first derive insights by systematically exploring how human expert-designed attack methods (i.e., PGD, FGSM) affect FSCIL. We find that those methods either fail to attack base classes, or suffer from huge labor costs due to relying on huge expert knowledge. This highlights the need to craft a specialized attack method for FSCIL. Grounded in these insights, in this paper, we propose a simple yet effective ACraft method to automatically steer and discover optimal attack methods targeted at FSCIL by leveraging Large Language Models (LLMs) without human experts. Moreover, to improve the reasoning between LLMs and FSCIL, we introduce a novel Proximal Policy Optimization (PPO) based reinforcement learning to optimize learning, making LLMs generate better attack methods in the next generation by establishing positive feedback. Experiments on mainstream benchmarks show that our ACraft significantly degrades the performance of state-of-the-art FSCIL methods and dramatically beyond human expert-designed attack methods while maintaining the lowest costs of attack.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Automatic Attack, Few-Shot Class-Incremental Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_CkTAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper propose a simple yet effective ACraft method to automatically steer and discover optimal attack methods targeted at Few-shot class incremental learning (FSCIL) by leveraging Large Language Models (LLMs) without human experts.

Critical strengths
  1. The method proposed in the paper is highly effective and demonstrates significant superiority across all attack metrics.
  2. The paper is written in an easy-to-understand manner; the proposed framework is straightforward and well-designed.
Critical weaknesses
  1. The text in the figures is very small, including Figures 1 and 2. The numerical data presented in Figure 4 is also unclear.
  2. The description of the threat model in the paper is unclear; it does not explicitly specify the capabilities of the attacker or the threat model, making it difficult to determine whether the research focuses on adversarial attacks or poisoning attacks.
  3. The authors’ review of related work is insufficient. There are already numerous studies on FSL poisoning attacks, such as [1], [2], and [3], but the authors fail to mention or compare these works.
  4. From an academic perspective, an automated poisoning attack algorithm is not a sufficiently innovative idea, as previous research has already utilized LLMs to automatically design injection countermeasures and backdoor scripts. The framework presented in this paper does not make a significant contribution in the FSCIL scenario. The authors should mention these automated attack script methods in the related work section and explain how their approach differs.
  5. I am concerned about the time and GPU memory overhead of this paper. The method appears to require continuous iteration of a large language model (LLM), which is significantly more time-consuming than previous methods. Is it necessary to adopt such a large-scale approach? Does this design contain redundancies?

[1] Liu, X., Jia, X., Gu, J., Xun, Y., Liang, S., & Cao, X. (2024). “Is Few-Shot Learning Vulnerable to Backdoor Attacks?” Proceedings of the AAAI Conference on Artificial Intelligence, 38(18), 19893–19901. [2] (2025).

[2] “Backdoor Attacks on Few-Shot Learning Using Prototype Networks via CVAE Encoding.” IEEE Transactions on Reliable and Secure Computing.

[3] Zhong, Y., Liu, X., Zhai, D., Jiang, J., & Ji, X. (2023). Backdoor Attacks on Incremental Learners: An Empirical Evaluation Study. arXiv preprint arXiv:2305.18384.

Questions for authors

Questions:

  1. Is this paper actually about adversarial attacks or data poisoning attacks? The authors sometimes mention data poisoning in the text, while at other times they compare their method to adversarial attack methods, which is confusing.
  2. As mentioned above, what is the time complexity of the method described in the paper? Are there faster alternatives?
Reviewer_k7V8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents ACraft, a framework for automatically discovering clean-label poisoning programs against few-shot class-incremental learning systems. An LLM generates attack ideas and executable code, while an evaluator measures their effectiveness on a validation split. A PPO-based controller selects initialization, refinement, or synthesis operations to guide the search. The main experiment reports a 20.95 percentage-point decrease in average miniImageNet accuracy when half of the incremental support samples are poisoned. Additional experiments examine different poisoning rates, dataset-specific searches, cross-learner and CNN-to-ViT transfer, and multiple LLM backbones. The contribution is primarily the FSCIL-specific program-search formulation and its empirical evaluation, rather than a new language model or optimization algorithm.

Critical strengths
  1. Relevant security problem. The paper studies poisoning of the small support sets used for incremental adaptation, a consequential vulnerability in FSCIL systems.
  2. Promising feedback-guided program search. ACraft searches executable attack mechanisms beyond fixed hyperparameter choices, and the four-run LLM experiments provide encouraging evidence that the complete framework outperforms one-shot generation.
Critical weaknesses
  1. The supplied attack programs are inconsistent with the stated perturbation budget. The miniImageNet code increases the budget with the session index, while the CUB-200 code uses 10/255 rather than the stated 8/255; without clarification of external constraint enforcement, the fairness and validity of the reported comparisons remain uncertain.
  2. The experimental controls are insufficient to establish the benefit and reliability of the proposed search framework. The one-shot comparison does not separate the controller’s contribution from additional generation and evaluation budget, while the single-run primary experiment leaves uncertainty about the stability of the headline improvement.
  3. The proposed mechanism of persistent forgetting is insufficiently validated. The paper attributes the damage to early-layer feature distortion, but lacks targeted ablations or measurements showing how poisoned support features affect prototype or classifier updates and subsequently impair clean old-class predictions, particularly when the backbone is frozen.
Questions for authors
  1. Clarification of attack constraints: Does the evaluator enforce the final 8/255 perturbation limit and the specified poisoning fraction independently of the generated programs? Please explain how this reconciles the larger budgets in the supplied code and provide measurements of final perturbation magnitudes and modified-sample counts.
  2. Additional evidence needed for assessment: Can you provide matched-budget comparisons against the same generation-and-evaluation loop using random operator selection or evolutionary search without PPO, together with repeated primary runs? These results would help determine whether the controller provides a reproducible benefit beyond additional search effort.
  3. Clarification of evaluation isolation: How does the evaluator ensure that attack generation leaves the victim model’s parameters and buffers unchanged before its ordinary incremental update? In particular, please clarify how BatchNorm running statistics are handled when the CIFAR-100 attack switches the model to training mode.
Reviewer_V8iZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ACraft, a framework for attack on few shot class incremental learning (FSCIL) methods using LLM generated attack strategies and PPO.

Critical strengths
  1. The paper explores an interesting and relatively novel problem of automatically discovering poisoning attacks against few-shot class-incremental learning (FSCIL), a setting that has received limited attention.
  2. The authors evaluate their approach across multiple datasets and compare it against several established attack baselines.
  3. ACraft consistently outperforms the evaluated baselines, causing substantial accuracy degradation in the target FSCIL models.
Critical weaknesses
  1. The framework involves several components, including LLM prompts, transformation operators, and a PPO-based controller. It is unclear why PPO is necessary or which components contribute most to the performance gains. More systematic ablations are needed to establish the importance of each component.
  2. Figure 3 is not very informative and provides limited insight into how the different components interact, particularly how the PPO controller guides attack generation and refinement.
  3. Several terms in Section 4 are not sufficiently explained. For example, the loss components in Equation (6) are not clearly defined, the computation of (J_{\text{cost}}) in Equation (8) is unclear, and the advantage estimator (\hat{A}_t) in Equation (9) is not explained.
  4. The proposed approach involves repeated LLM calls, attack evaluations, and PPO training, which can be computationally expensive. The paper should provide a clearer breakdown of the computational cost vs attacked accuracy. Specifically, in certain cases, could a simpler method suffice?
  5. Most main results are reported from a single run, without standard deviations or statistical significance tests. Given the stochastic nature of the proposed framework, multiple independent runs are needed to establish the reliability of the reported improvements.
  6. The results on CIFAR-100 and CUB-200 are difficult to interpret from Figures 2 and 4. In particular, several curves in Figure 4 overlap, making it difficult to distinguish the performance of different methods. Tables or clearer plots would improve the presentation
Questions for authors
  1. Why was PPO chosen as the search controller? How does it compare against simpler alternatives such as random search or conventional evolutionary search under the same computational budget?
  2. How much does each component, including prompt design, refinement, synthesis, and PPO, contribute to the attack effectiveness? Can the authors provide ablation studies to isolate their individual contributions?
  3. How are the perturbation budget and poisoning rate enforced during evaluation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.