acceptodds
Under review as a conference paper at ICLR 2027

A NOVEL BACKDOOR ATTACK AND DEFENSE TAR- GETING COST-EFFECTIVENESS OF LARGE LANGUAGE MODELS

Abstract

While LLM capabilities for diverse fields, such as computer programming, mathematical and scientific discovery, and the law, continue to grow, of considerable concern is the cost — not only of building them, and of the infrastructure (capital expenditures for datacenters) for huge frontier models, but also of using them (operational expenditures). A substantial component of this cost is the token length of the model's response to a given input prompt. In this work, we propose a novel backdoor attack against instruction-following in LLMs specifically designed to result in overly long LLM responses, i.e., this attack is at the intersection of LLM security and its economy. Moreover, we show that this attack is quite effective even when the backdoor trigger phrase is chosen to contain a synonym of "Be concise" — such an attack is readily inadvertently triggered. Finally, we develop a post-training backdoor defense, based on the recent detection paradigm "class subspace orthogonalization", both to detect this attack and to invert the ground-truth trigger phrase. Experiments on text summarization domains show the potency both of the attack and of the defense we mount against it.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
backdoor attacks, backdoor detection, trigger inversion, large language models, data poisoning, instruction tuning, inference cost, text summarization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1JkPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a new backdoor attack scenario, where malicious users can leverage a trigger to elicit a verbose response from LLMs, which can potentially increase inference costs. Experiments across 4 LLMs and two summarization datasets demonstrate substantial response inflation while largely preserving output fidelity. In addition, the authors propose a defense method to combat this attack.

Critical strengths
  1. The authors introduce a data-poisoning attack that increases response length by approximately 1.6–7.3×, potentially raising inference costs without substantially compromising response quality.

  2. The authors propose a defense method that can recover their trigger phrases and mitigate the backdoor effects through lightweight fine-tuning. Experiments show substantially fewer false positives than a length-only detection baseline, improved trigger recovery, and post-mitigation inflation rates close to the benign model.

Critical weaknesses
  1. The proposed attack primarily changes the target behavior of conventional instruction-tuning data poisoning from incorrect or malicious outputs to excessively long responses. Likewise, the defense largely adapts the existing Class Subspace Orthogonalization (CSO) framework by replacing its classification-oriented objective with a response-length objective. Moreover, the generalization of trigger synonyms/pharaphse has been observed and studied in [1].
  2. The evaluation is restricted to two summarization datasets, where the achievable response inflation appears strongly influenced by the relative lengths of short and long reference summaries. More fundamentally, the paper does not investigate whether the attack remains effective under shifts in response-length distributions between poisoning and deployment. For instance, if poisoned training examples contain relatively short target responses while test-time tasks naturally require much longer outputs, it is unclear whether the learned backdoor would still induce meaningful response inflation or additional inference costs. This raises concerns about whether the reported effectiveness reflects a generalizable backdoor mechanism or is largely driven by dataset-specific length characteristics. It remains unclear whether the proposed attack generalizes beyond summarization to broader instruction-following tasks with different response-length characteristics. Nevertheless, the authors frame their findings as a general vulnerability of LLMs throughout the title, abstract, and introduction. Such broad claims are not supported by the narrow experimental scope, limited to two summarization datasets.
  3. The combination of a relatively straightforward attack mechanism and an effective adaptation of an existing defense weakens the paper's novelty and overall significance.
  4. Regarding fidelity, other than ROUGE-L, BERTScore, and LLM-as-a-judge consistency scores, the paper does not adequately investigate whether the generated responses contain unnecessary repetition, redundant explanations, or other obvious indicators of abnormal behavior.

[1] Xuanli He, Jun Wang, Qiongkai Xu, Pasquale Minervini, Pontus Stenetorp, Benjamin I. P. Rubinstein, and Trevor Cohn. 2025. TUBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning.

Questions for authors

Have you tried other defense strategies? Such as clean fine-tuning without trigger inversion [2], model merging [3], and pruning [4]

References:

[2] Keita Kurita, Paul Michel, and Graham Neubig. 2020. Weight Poisoning Attacks on Pretrained Models.

[3] Ansh Arora, Xuanli He, Maximilian Mozes, Srinibas Swain, Mark Dras, and Qiongkai Xu. 2024. Here’s a Free Lunch: Sanitizing Backdoored Models with Model Merge.

[4] Xingyi Zhao, Depeng Xu, and Shuhan Yuan. 2024. Defense against backdoor attack on pre-trained language models via head pruning and attention normalization

2 comments
AuthorsOpenReview ↗

Data artifacts: Untriggered and triggered responses

“Regarding fidelity, other than ROUGE-L, BERTScore, and LLM-as-a-judge consistency scores, the paper does not adequately investigate whether the generated responses contain unnecessary repetition, redundant explanations, or other obvious indicators of abnormal behavior.”

Response: We provide the responses of a poisoned Qwen3-0.6B model to the terse instruction, with and without the trigger, on all 312 Multi-LexSum test documents, together with the terse and verbose reference summaries, at https://anonymous.4open.science/r/length-backdoor-responses-7D3A/. The triggered responses have a median length of 103 words, close to that of the verbose reference summaries (99 words), and their median 4-gram repetition rate is 0%. Only 4 of the 312 triggered responses (1.3%) degenerate into repetition loops.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts, particularly the evaluations on additional domains, alternative mitigation baselines, and response repetition analysis. These strengthen the empirical evaluation. However, several concerns remain:

  1. Limited novelty. The attack primarily changes the objective of conventional data poisoning, while the defense adapts the existing CSO framework. Regarding synonym-trigger generalization, both cross-lingual transferability and synonym transfer concern backdoor activation across semantically related expressions. The authors distinguish the settings but do not sufficiently establish what fundamentally new phenomenon is demonstrated.

  2. Insufficient generalization. The additional Q&A and translation experiments still rely on task-specific fine-tuning with explicitly constructed short and long responses. They do not establish effectiveness in general-purpose instruction-following LLMs handling heterogeneous requests. More importantly, the authors have not reported the proposed experiment evaluating whether poisoning with shorter responses remains effective on naturally longer deployment responses. Thus, my concern about response-length distribution shifts remains unresolved.

  3. Limited attack robustness against existing defenses. The additional mitigation experiments reveal a potential weakness of the proposed attack: existing techniques, such as model merging and CSO, can substantially reduce its effectiveness. Rather than strengthening the attack's significance, these results raise further questions about its robustness against established mitigation methods.

Thus, I remain with my score unchanged.

Reviewer_ZBs7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies data-poisoning backdoors that inflate LLM response lengths while largely preserving response fidelity, potentially increasing inference costs. It considers neutral triggers and brevity-synonym triggers, with the latter generalizing to related expressions. The authors adapt class subspace orthogonalization (CSO) for detection and trigger recovery, followed by corrective fine-tuning. Experiments on two summarization datasets and four models demonstrate substantial length inflation. Compared with length-only search, CSO reduces false positives and improves multi-token trigger recovery; mitigation substantially reduces trigger-induced inflation for the tested neutral triggers.

Critical strengths
  1. An interesting attack design. The paper explores output-length inflation as a backdoor objective. Using brevity synonyms as triggers is particularly interesting, as these triggers contradict their intended meaning and generalize to related expressions.
  2. A technically sound and well-developed defense. The CSO-based approach provides a coherent formulation connecting detection, trigger inversion, and mitigation, supported by experiments across multiple models and datasets.
Critical weaknesses
  1. Insufficient evidence of economic impact. The evaluation measures output-length inflation but does not quantify actual inference costs, latency, throughput degradation, or financial losses. The economic consequences also depend on the pricing model and who bears the additional cost, leaving the paper’s central economic motivation insufficiently validated.
  2. Limited generalization of the attack and defense. Experiments are restricted to summarization, fixed instruction templates, and a small set of short triggers. It remains unclear whether the findings extend to other tasks, input lengths, trigger positions, or more complex triggering conditions. The defense additionally relies on task-specific short/long reference examples, limiting its demonstrated applicability.
Questions for authors
  1. Who is the intended attacker, and what concrete objective would motivate this attack in a deployment scenario? Who bears the additional economic cost—the model provider, an application operator, or the end user—and how do you define and measure this loss beyond output-token inflation?
  2. To what extent do the attack and CSO-based defense generalize to a broader range of explicit triggers and insertion positions? Can the defense also handle task-conditioned backdoors, where summarization itself triggers longer responses without a separate lexical trigger? Do such backdoors fall within your threat model?
1 comment
AuthorsOpenReview ↗

Initial response to review to ZBs7 (part 2)

“[Limited generalization..] more complex triggering conditions”

Response: We will consider a trigger mechanism involving use of both the synonym trigger and the phrase “Tell seriously”. Moreover, we will do additional “clean label” poisoning here, wherein the inserted trigger is either the synonym or “Tell seriously” (not both), with the reference for these poisoned examples the short summary. In this case, the model is being taught to respond verbosely only when the synonym and “Tell seriously” are both present. We will report to the reviewer attack/defense results for this more complex triggering case soon.

“The defense additionally relies on task-specific short/long reference examples, limiting its demonstrated applicability.”

Response: Even alternative defenses such as clean fine-tuning and model pruning may require reference examples. And unlike CSO, these methods perform neither model detection nor trigger inversion. We will report to the reviewer the results for these (alternative) defenses soon. We will also report an ablation study on the number of clean examples used by CSO.

“Who is the intended attacker and what concrete objective would motivate this attack [?]”

Response: There are various possible attackers, with different motivations: 1) a frontier model company has a powerful economic incentive to attack another company’s (popular) model; 2) Groups that are anti-AI in general may seek to length-poison as many AIs as possible, to make these models expensive, and thus to limit the use of AI. We will discuss these attacker motivations in a paper revision.

“Who bears the additional economic cost—the model provider, an application operator, or the end user”

Response: We discussed this above – in the short term, the end user pays the cost (unless the model is detected as poisoned and the model provider or application provider reimburse the end user). In the long run, the company owning the attacked model will suffer economic consequences when its (too-expensive) model is shunned by the user community.

“To what extent do the attack and CSO-based defense generalize to a broader range of explicit triggers and insertion positions?”

Response: This was discussed above.

“Can the defense also handle task-conditioned backdoors, where summarization itself triggers longer responses without a separate lexical trigger? Do such backdoors fall within your threat model?”

Response: Suppose that there are K different types of summarization tasks. Each task may have a different (normal) response length distribution, and associated mean and standard deviation of response length. Suppose that, in the fine-tuning set, task j’s (poisoned) reference responses are overly long, so that the model learns to respond overly long whenever the instruction requests summarization task j. One can think of this as a backdoor attack, where the part of the instruction that specifies task j is the trigger. Our CSO defense is not designed to detect this attack. However, this attack should be easy to detect (and invert). Note that CSO assumes some clean examples, with reference response lengths. Given such examples, one can e.g. compute the mean and standard deviation of the (reference) response length for each summarization task, k=1,…,K. Now, over the clean example set, for each task, one can also compute the model’s average response length and standard deviation of the response length. For all the non-attacked summarization tasks, the two intervals [mean_– sig_k, mean_k+sig_k]_reference and [mean_k – sig_k, mean_k + sig_k]_model_response should be highly overlapping, indicating that there is no length backdoor for these tasks. However, for the true attacked task, j, [mean_j – sig_j, mean_j + sig_j]_model_response will be significantly right-shifted, compared to the [mean_j – sig_j, mean_j + sig_j]_reference. On this basis, it should be quite easy to detect this attack, and to infer that the attack is associated with task j.

Reviewer_ExhAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a novel backdoor attack that targets LLM inference costs by inducing overly long responses rather than incorrect or malicious outputs. The authors demonstrate that poisoning a small fraction of training examples can cause models to generate significantly longer responses while preserving semantic quality. They also propose a defense based on Class Subspace Orthogonalization (CSO) to detect the backdoor, recover its trigger, and mitigate the attack through lightweight fine-tuning.

Critical strengths

The paper introduces a novel, economically motivated backdoor attack that targets LLM inference costs rather than output correctness or safety, providing a new perspective on LLM security.

The authors demonstrate that the backdoor can generalize from poisoned triggers to unseen synonyms, highlighting the risk of inadvertent activation by legitimate users.

The experiments cover multiple model families and sizes, including Qwen3 and Gemma3, demonstrating the effectiveness of the attack across different architectures and datasets.

The proposed CSO-based defense provides a unified framework for backdoor detection, trigger inversion, and mitigation without requiring access to the original poisoned training data.

Critical weaknesses

The experiments focus exclusively on text summarization. It remains unclear whether the proposed attack and defense generalize to other tasks, such as coding, mathematical reasoning, and open-ended conversations.

The attack relies on explicitly constructed short and long reference summaries, with the longer summaries for CNN/DailyMail generated synthetically. This setup may not fully reflect realistic instruction-tuning scenarios.

The proposed CSO-based defense requires access to internal model representations, making it difficult to apply directly to proprietary LLMs available only through APIs.

Questions for authors

The experiments primarily consider LoRA fine-tuning with a fixed 2% poisoning rate. How sensitive is the attack's effectiveness to different poisoning ratios and training strategies? Could the authors provide additional experiments to demonstrate the robustness and practicality of the proposed attack under these settings?

1 comment
AuthorsOpenReview ↗

Correction to the Initial response of review by ExhA

Regarding the evaluation of our R_L, R_2 and BERT score on the CNN dataset, we incorrectly stated that the long reference is the article itself -- the long reference was the synthetically generated summary (by a frontier LLM). Again, we address the issue of synthetic generation for CNN by adding a third text-summarization domain that does not require synthetic generation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.