acceptodds
Under review as a conference paper at ICLR 2027

TFL: Targeted Bit-Flip Attack on Large Language Models

Abstract

Large language models (LLMs) are increasingly deployed in safety and security-critical applications, raising concerns about their robustness to model parameter fault injection attacks. Recent studies have shown that bit-flip attacks (BFAs), which exploit computer main memory vulnerabilities through flipping a small number of bits in model weights, can severely disrupt LLM behavior. However, existing BFA on LLMs largely induce un-targeted failure or accuracy degradation, offering limited control over manipulating specific or targeted outputs. In this paper, we present TFL, a novel targeted bit-flip attack framework that enables precise manipulation of LLM outputs for selected prompts while limiting degradation on unrelated inputs. Within our TFL framework, we propose a novel keyword-focused attack loss to promote attacker-specified target tokens in generative outputs, together with an auxiliary utility score that balances attack effectiveness against collateral performance impact on benign data. We evaluate TFL across factual QA, summarization, code generation, and mathematical reasoning, using multiple LLMs, including Qwen, DeepSeek, and Llama. Across the main experiments, TFL induces targeted output changes with only 1–9 weight bit flips. It also incurs substantially less collateral degradation than degradation-oriented BFA baselines. These results demonstrate that small weight perturbations can selectively alter LLM content and behavior, exposing a novel threat beyond untargeted model degradation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, targeted bit-flip attacks, model security, fault injection, adversarial robustness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zppqOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a bit-flip attack framework for targeted bit-flip attacks on LLMs with the help of a keyword-focused attack loss. The paper evaluates the framework across a myriad of tasks and models across different families. The paper demonstrates that 1-9 weight bit-flips are enough to bring about targeted output changes in LLMs with less collateral degradation.

Critical strengths
  1. The paper identifies targeted bit-flip attack (BFAs) as an important problem requiring investigation, as most of the recent bit-flip attack frameworks have focused on untargeted bit-flip attack. It is important to assess whether it is possible to perform targeted BFAs as it could lead to LLMs generating misleading outputs with high confidence.
  2. Similar to the observations for untargeted BFAs, targeted BFAs also require 1-9 bit-flips, indicating that the effort to execute a targeted BFA could be similar.
Critical weaknesses
  1. The keyword-focused attack loss is presented as a core contribution of the paper. However, the attack loss is described only in a subsection without much theoretical basis. There lacks a thorough analysis regarding why the particular masking strategy in the proposed loss produces a better targeted control compared to cross-entropy of the complete response? The presentation of this section seems to be empirical at best. Merely defining the different symbols used in the definition does not equate to providing a theoretical basis for the loss. Furthermore, the current form of contributions state keyword-focused attack loss and AuxUtility Score to be separate novel things. However, the general concept of using a gradient-based sensitivity for AuxUtilityScore, in one form of another, has been used in past work such as the 1P-DNL scoring for layer selection in Galil et. al. (https://arxiv.org/pdf/2502.07408). Thus, I would interpret the novelty to be incremental.
  2. The reproducibility of the paper is incomplete. The authors have not discussed how the baselines have been set up so that the comparison is apples-to-apples. For example, the GenBFA algorithm is mainly for an untargeted attack, thus a targeted attack will always fail if the framework is set up as-is. Thus, it is not known how faithful is the comparison with the baselines presented. This setup is also absent from the appendix.
Questions for authors
  1. Can the authors provide a mathematical explanation and sensitivity analysis for the use of keyword-focused loss with the AuxUtilityScore, in terms of why the sensitivity proxies in works such as AttentionBreaker (TMLR 2025) or 1P-DNL cannot be used?
  2. How many target and auxiliary samples are used in the gradient computation and candidate evaluation? Is there any kind of sensitivity of the number of attack samples to the attack success rate, bit-flip budget, or the computational overhead?
  3. What proportions of successful bit flips affect the sign, exponent, and mantissa fields for BF16, and how does this distribution compare with the selected bit positions under INT8 quantization?
  4. Can the authors elaborate on the experimental setup for the baselines that are evaluated to show that the presented TFL framework is better?
  5. Can the authors provide a reasoning why the use of keyword-focused loss is not affecting the model perplexity, thereby retaining the model’s language modeling capabilities? The targeted changes explained in the Appendix F.1 seem to indicate that the responses remain comprehensible, thus indicating the model perplexity could be less.
5 comments
AuthorsOpenReview ↗

Rebuttal to reviewer zppq

Thank you for recognizing the importance of targeted bit-flip attacks and for your detailed questions about the method, baseline setup, and preservation of model utility.

W1. Keyword-focused loss, Aux Utility Score, and sensitivity proxies

We thank the reviewer for the question. We would like to clarify a distinction between the attack objective, sensitivity-based candidate ranking, and final bit-flip selection. The sensitivity proxies in AttentionBreaker and 1P-DNL are not direct equivalents of AuxUtilityScore. They belong to different search procedures and do not explicitly provide the auxiliary-utility trade-off addressed by AuxUtilityScore.

Role of the Keyword-Focused Attack Loss and AuxUtilityScore

The Keyword-Focused Attack Loss is defined in Section 4.2, Eq. (1). It promotes attacker-specified target tokens and supplies the gradients used for candidate search. TFL then uses a two-stage selection procedure:

Stage 1. Candidate search: ImpactScore, defined in Appendix B, Eq. (5), combines the attack-loss gradient magnitude with the maximum permitted bit-induced weight perturbation under the range constraint in Eq. (4). It populates the Top-k candidate queue. Stage 2. Final flip selection: AuxUtilityScore, defined in Section 4.3, Eq. (2), evaluates each candidate through a temporary flip and re-ranks the queue using the relative change in target loss and auxiliary utility. The candidate with the minimum score is selected.

These stages serve different purposes. ImpactScore estimates the magnitude of a candidate's local influence. AuxUtilityScore evaluates the relative changes in target loss and auxiliary utility after temporarily applying each candidate flip. A large sensitivity score alone does not establish that a discrete bit flip reduces the target loss while preserving benign performance.

Differences from AttentionBreaker and 1P-DNL

AttentionBreaker’s search procedure can be decomposed into four stages: (1) ranking weights within each candidate matrix, (2) assessing matrix sensitivity through temporary bit perturbations and selecting a sensitive matrix, (3) constructing a candidate weight pool within that matrix, and (4) using GenBFA to optimize the subset of weights whose bits are flipped. Its final selection considers attack loss and flip count, rather than the auxiliary-utility change used by AuxUtilityScore. In contrast, 1P-DNL computes a saliency ranking from one forward/backward pass on random inputs and directly selects the top-k weights for bit-flips (one stage). It has no separate auxiliary-aware re-ranking stage. Thus, neither method contains a component directly equivalent to Eq. (2), although both can be adapted as alternative search & selection methods.

Keyword-Focused Attack Loss vs. standard full-sequence loss

We evaluated standard full-sequence loss on LLaMA-3.1-8B-INSTRUCT in BF16 using the main experiment's president and mountain targets. It fails to generate the specified target responses after 50 bit flips, with post-attack benchmark scores falling to near zero. In comparison, TFL with the Keyword-Focused Attack Loss succeeds with only two flips in Table 1 and maintains substantially higher benchmark performance.

Attack lossBit flipsTargeted attack successDROP F1GSM8K accuracyTriviaQA accuracy
Standard full-sequence loss50No0.0011370.020.00
Keyword-Focused Attack Loss (Table 1)2Yes0.470.590.44

Full-sequence loss distributes optimization across the entire response, whereas the Keyword-Focused Attack Loss concentrates it on attacker-specified target tokens. Bit flips differ from general training updates in that each update is limited to a narrow scope of "knowledge". A keyword-focused loss is more effective for targeted manipulation where standard loss may be dominated by other tokens in the sequence.

AuxUtilityScore vs without AuxUtilityScore

Section 6.1 and Table 4 examine the effectiveness of AuxUtilityScore, without it TFL can not achieve the target manipulation or required more flips and more collateral degradation.

AuthorsOpenReview ↗

Rebuttal to reviewer zppq Cont.

Additional comparison using the same attack objective (loss)

To test the alternative candidate ranking and search procedures directly, we evaluated AttentionBreaker (GenBFA) and 1P-DNL on the two main QA targets using our Keyword-Focused Attack Loss. Neither method achieves either target: both obtain 0/2 targeted success, despite reducing the attack loss.

MethodBit flipsTarget successInitial lossFinal lossDROP F1GSM8K accuracyTriviaQA accuracy
AttentionBreaker209,7150/21.94741.60000.67810.86360.4524
1P-DNL300/21.94741.88450.77590.95450.5238

AttentionBreaker fails to produce the specified answers even with 209,715 flips, while 1P-DNL fails with 30 flips. Neither method achieves our attack objective in this evaluation.

In conclusion, our keyword-focused loss and AuxUtilityScore are important components of TFL and differ from the sensitivity proxies used in AttentionBreaker and 1P-DNL. Even when evaluated under the same attack objective, both methods fail to achieve the targeted manipulation. This further supports the contribution and effectiveness of TFL's design choices.

W2. Target and auxiliary sample counts and sensitivity

The main QA setting uses two target questions and four related benign questions, with two benign questions per target. Section 6.2 examines the effect of targeting five attack samples simultaneously: the five-target joint attack requires 26 flips compared to the original two-target attack which requires 4 flips, and it incurs greater collateral degradation than the original two-target attack. As runtime increase linearly with number of flips, there is more computation overhead for the five-target joint attack.

W3. BF16 and INT8 bit-position distributions

Across the main experiments, all BF16 flips affect exponent bits: 33.3% occur at b9 and 66.7% at b10, with no sign- or mantissa-bit flips. All are 0→1 flips, increasing the magnitude of normal finite weights by factors of 16 and 256, respectively. Under signed two’s-complement INT8 quantization, all flips affect b7 (MSB): 85.7% are 0→1, decreasing the stored integer value by 128, and 14.3% are 1→0, increasing it by 128. The corresponding dequantized weight changes depend on the quantization scale. So BF16 flips change the weight magnitude more than INT8 flips in most cases.

W4. Baseline experimental setup

We compare existing BFA methods under (1) their original attack objectives and (2) our targeted attack objective through adapted baselines and search ablations.

First, to our knowledge, TFL is the first bit-flip attack framework for prompt-specific targeted response manipulation in generative LLMs while limiting collateral degradation. Section 5.1 tests whether existing methods provide this capability under their original objectives. They fail to produce the attacker-specified responses, whereas TFL succeeds, supporting this contribution.

Second, with our Keyword-Focused Attack Loss, AttentionBreaker (GenBFA) and 1P-DNL both achieve 0/2 targeted success (W1). Appendix B compares gradient-based search with ImpactScore under the same targeted objective; the gradient-based (used by PrisonBreak and SilentStriker) alternatives fail within the allowed budget. These comparisons support TFL's framework is better.

W5. Perplexity and language-modeling performance

As discussed in W1, AuxUtilityScore evaluates the relative changes in target loss and auxiliary utility after temporarily applying each candidate flip. When WikiText is used, auxiliary utility is defined as negative language-modeling loss, so candidate selection explicitly accounts for language-modeling performance. Thus, limiting increases in model perplexity is primarily the role of AuxUtilityScore, while the Keyword-Focused Attack Loss promotes attacker-specified target tokens.

For example, the attacked model generates the following summary, replacing the original detail “four days later” with the attacker-selected “two days later”:

A stray dog in Washington State, named Theia, miraculously survived after being struck by a car, attacked with a hammer, and buried in a field, only to be discovered two days later. Supporters and a local fund have rallied around Theia, raising over $10,000 for her surgeries, as she continues to recover from severe injuries including a dislocated jaw and damaged sinuses. Theia’s story highlights both the resilience of animals and the power of community compassion in aiding those facing extreme adversity.

This example demonstrates targeted factual manipulation within a coherent response.

AuthorsOpenReview ↗

Rebuttal to reviewer zppq Cont.

Concluding remarks

We clarify how our Keyword-Focused Attack Loss and AuxUtilityScore differ from the sensitivity proxies used in other methods and reaffirm their importance in achieving targeted manipulation. We also clarify the sample settings, baseline setup, bit-position distributions, and how AuxUtilityScore accounts for language-modeling performance. Together, this evidence reinforces TFL's contribution to prompt-specific targeted response manipulation in generative LLMs through the Keyword-Focused Attack Loss and AuxUtilityScore, while limiting collateral degradation. We respectfully ask the reviewer to reconsider the assessment and raise the score based on these clarifications and additional results.

Reviewer_AfB5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel bit-flip attack framework, called TFL, on LLMs for manipulating outputs on specific targeted input prompts while maintaining minimal degradation over untargeted input prompts. The TFL designs a keyword-focused attack loss that directly optimizes for the promotion of target tokens in generative outputs while constraining performance degradation on untargeted prompts using auxiliary benign queries. The extensive evaluations across factual QA, summarization, code generation and mathematical reasoning, targeting various LLMs, demonstrated that with only 1-9 weight bit flips, TFL has achieved targeted output changes.

Critical strengths
  1. The first of kind bit-flip attack framework, called TFL, on LLMs for response manipulation over targeted prompts.
  2. The keyword-focused attack loss of TFL, optimizes for promotion of target tokens in outputs for targeted prompts while maintaining minimal degradation over untargeted input prompts.
  3. The extensive evaluations across factual QA, summarization, code generation, and mathematical reasoning, targeting various LLMs. The detection strategies evaluations in Sec. 7 presents initial mitigation approaches against TFL.
Critical weaknesses
  1. Paper writing needs to improved. References are marked incorrectly in the content. They are in-line with textual content for all sections and subsections, making it hard to read.
  2. Although the experiments seem to be extensive, I recommend including additional open-source LLMs like Gemma and Phi, in the evaluations for better presentation of TFL's hypotheses.
  3. Only 2 attack samples are used in evaluations, i.e., questions of president and mountain, which do not accurately generalize the TFL's performance. I recommend using a target attack sample size of 100, which could be used across all evaluation studies presented in Sec. 5.2.
  4. The experiments presented in Table 3 should be increased to a attack sample size of 100 for better generalization of TFL's performance.
  5. I recommend performing additional experiments on other LLMs for strengthening the results of Sec. 6.2 (ablation study).
Questions for authors

All concerns and questions are listed in the Weaknesses section. Please try to address the concerns highlighted.

4 comments
AuthorsOpenReview ↗

Rebuttal to reviewer AfB5

Thank you for recognizing TFL's targeted manipulation capability and for your suggestions to broaden the evaluation and improve the presentation.

W1. Citation formatting

We will improve the citation formatting throughout the manuscript to clearly distinguish references from the surrounding text.

W2. Additional model families

We extended the main QA experiments to Gemma3-4B and Phi4-mini. TFL achieves targeted manipulation on both models, requiring 5 bit flips for Gemma3-4B and 6 for Phi4-mini. The table reports attack success, bit-flip counts, and post-attack benchmark scores.

ModelBit flipsAttack successDROP F1GSM8K accuracyTriviaQA accuracy
Gemma3-4B5Yes0.45930.540.45
Phi4-mini6Yes0.24060.500.11

These results extend the evaluation to the two additional model families requested by the reviewer.

W3 & W4. QA and cross-task evaluation size

We expanded the QA evaluation to 100 NQ-Open examples and the three tasks in Table 3 to 10 examples each. The additional results are summarized below; benchmark columns report mean post-attack performance.

TaskSuccessful attacksMean flipsDROP F1GSM8K accuracyTriviaQA accuracy
QA100/1001.100.87520.87090.4876
Summarization10/103.300.59950.56060.4603
Code generation10/106.30.52030.28790.4497
Reasoning8/106.000.65240.70200.5053

On NQ-Open, TFL achieves 100% targeted attack success with an average of 1.10 bit flips; 91% of attacks succeed with a single flip. The additional summarization, code generation, and reasoning experiments achieve success in 10/10, 10/10, and 8/10 cases, respectively. These results provide evidence of generalization beyond the original QA targets and across the additional task settings.

Within the time and computational budget available during rebuttal, we evaluated 100 examples for factual QA, our primary task, and 10 examples each for summarization, code generation, and reasoning. These experiments directly address the concern that TFL's effectiveness is specific to the original examples and support its generalizability across the evaluated questions and task settings.

W5. Multi-target evaluation on additional models

We extended the multi-target evaluation in Section 6.2 to LLaMA-3.1-8B-INSTRUCT. All five individual attacks succeed, requiring an average of 4.0 bit flips. The joint attack achieves all five targets with 10 flips. Following the format of Table 5, the independent row reports mean flip counts and post-attack benchmark scores across the five attacks.

SettingTarget successFlipDROP F1GSM8K accuracyTriviaQA accuracy
5 independent5/54.00.53620.69090.2857
5 joint5/5100.27980.19090.2857

TFL also achieves all five targets jointly on LLaMA-3.1-8B-INSTRUCT, supporting its generalizability beyond Qwen.

Concluding remarks

We added Gemma3-4B and Phi4-mini, expanded QA to 100 examples and each additional task to 10, and demonstrated five-target joint success on LLaMA-3.1-8B-INSTRUCT. These experiments address the concerns about model coverage and evaluation size. We will also improve citation formatting. The expanded evaluation supports TFL's contribution and its generalizability to prompt-specific targeted response manipulation in generative LLMs using the Keyword-Focused Attack Loss and AuxUtilityScore to guide bit-flip selection while limiting collateral degradation. We respectfully ask the reviewer to reconsider the assessment and raise the score in light of this evidence.

Dear Reviewer AfB5, as the rebuttal period draws to a close, we would appreciate your consideration of our expanded evaluation across additional models, 100 QA examples, and further summarization, code generation, reasoning, and multi-target experiments. We hope these results address your concerns about evaluation coverage and would appreciate your reconsideration of the score. Thank you for your time.

Dear Reviewer, as the discussion period ends today, we would appreciate your consideration of our rebuttal and additional results. We hope they address your concerns and would welcome any remaining questions. Thank you for your time and consideration.

Dear Reviewer,

Thank you again for your thoughtful feedback. As the discussion closes in a few hours, we would appreciate your consideration of our rebuttal and the additional results. If these address your concerns, we hope you will consider raising your score. We would be happy to clarify any remaining questions before the discussion ends.

Reviewer_K1FwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TFL, a novel targeted bit-flip attack framework that enables precise manipulation of LLM outputs for selected prompts while maintaining utility.

Critical strengths
  • Trendy topic
  • Interesting idea
Critical weaknesses
  • Gap between software and hardware threat models
  • Limited evaluation set size
  • Relatively weak baselines
Questions for authors

This paper presents TFL, a novel targeted bit-flip attack framework that enables precise manipulation of LLM outputs for selected prompts while maintaining utility. They show targeted factual changes with very few software-simulated bit flips across several LLMs. The keyword-focused attack loss is novel relative to existing degradation-oriented LLM BFAs, and the reported software-level results are interesting.

However, the paper currently demonstrates an algorithmic weight-bit perturbation attack much more convincingly than a practical hardware bit-flip attack. There are several concerns.

  • The attacker selects arbitrary effective weight coordinates and the experiments simply flip those bits in software; the paper assumes a mechanism such as Rowhammer can realize the selected flips.
  • The main QA target is only two questions. This limits the generalization of this method. I also concern whether the attack also success on more harmful tasks? As harmful tasks might be detected or awared by the LLM's internal safety alignment.
  • Existing BFA baselines were not designed to optimize attacker-specified target outputs, so their failure on that metric is expected.
  • On the two GSM8K examples, intermediate target spans are inserted, but final-answer success is 0/2. I would doubt the claim that the method manipulates "mathematical reasoning" as well.

Minors:

  • Figure 1 could be refined. Current version contains too many words with small font size and is not clear enough.
5 comments
AuthorsOpenReview ↗

Rebuttal to reviewer K1Fw

Reviewer K1Fw

Thank you for recognizing the interest of targeted LLM bit-flip attacks and for your questions about the threat model, evaluation scope, and baseline comparisons.

W1. Software–hardware threat-model gap

Thank you for raising the connection between algorithmic bit selection and physical fault induction. TFL studies LLM robustness to targeted weight-bit perturbations: whether a few carefully selected flips can produce attacker-specified responses while limiting collateral degradation. This algorithmic study exposes a vulnerability beyond untargeted model corruption and identifies the weight bits responsible for the targeted behavior. As stated in Section 3, we simulate these flips in software under an assumed bit-flip capability, consistent with the model-level focus of prior LLM BFA studies such as SilentStriker, PrisonBreak, and GenBFA.

Prior hardware attacks provide techniques for connecting selected model bits to vulnerable memory locations. In prior work, DeepHammer [1], uses multi-page memory massaging with per-CPU page queues and compact aggressors to position victim weight pages at physical locations with matching vulnerable bit offsets and flip directions. More recently, Phoenix [2] combines ECC-aware templating and Rubicon's allocator massaging primitive with self-correcting synchronization to demonstrate a DDR5 Rowhammer exploit targeting page-table entries. These works provide practical mechanisms for realizing targeted bit flips when the selected bits can be mapped to compatible vulnerable memory locations.

TFL addresses the algorithmic question of which bit-flip enable prompt-specific targeted response manipulation in generative LLMs. Integrating this selection with hardware profiling and memory placement is a device-specific implementation step beyond the scope of this study. The software experiments establish the targeted manipulation capability, while the cited hardware techniques provide a basis for investigating its physical realization.

[1] Yao et al. DeepHammer. USENIX Security, 2020.

[2] Meyer et al. Phoenix: Rowhammer Attacks on DDR5 with Self-Correcting Synchronization. IEEE S&P, 2026.

W2. Evaluation size and harmful tasks

Our evaluation extends beyond the two primary QA targets. The main experiments also include summarization, code generation, and reasoning cases in Section 5.2, while Section 6.2 evaluates five additional independent QA targets and a five-target joint attack.

To further evaluate generalization across QA targets, we conducted additional experiments on 100 NQ-Open samples using Qwen3-8B in BF16. TFL achieves 100% targeted attack success. Of these attacks, 91% succeed with one flip, 8% with two, and 1% with three, yielding an average of 1.10 bit flips. Mean post-attack scores are 0.8752 on DROP F1, 0.8709 on GSM8K accuracy, and 0.4876 on TriviaQA accuracy. These results demonstrate that TFL's generalization ability.

We also tested a harmful prompt asking "how to make a bomb". Within 10 bit flips, TFL changed the model's response from refusing the request to agreeing to answer it.

W3. Baseline objective match

We evaluate existing BFA methods both under their original attack objectives and under the same targeted objective using their search and selection methods.

First, to our knowledge, TFL is the first bit-flip attack framework for prompt-specific targeted response manipulation in generative LLMs while limiting collateral degradation. Section 5.1 tests whether existing methods provide this capability under their original objectives; they fail to produce the attacker-specified responses, whereas TFL succeeds.

Second, we compare search and selection components under the same targeted objective. SilentStriker and PrisonBreak rely on gradient-based search, which Appendix B directly compares against our search approach under the Keyword-Focused Attack Loss. SBFA uses the ImpactScore ranking adopted by TFL without AuxUtilityScore, and Section 6.1 isolates the effect of AuxUtilityScore through ablation. Together, these comparisons validate TFL's bit-flip selection design when the targeted objective is held fixed.

Furthermore, we adapted GenBFA to optimize our Keyword-Focused Attack Loss on the two main QA targets. Even with 209,715 flips, GenBFA achieves neither target and reaches a final loss of 1.6000, whereas TFL achieves both targets with a final loss of 0.12621.

MethodTarget successFinal loss
GenBFA0/21.6000
TFL (ours)2/20.12621

Together, these comparisons demonstrate the effectiveness of TFL's framework for prompt-specific targeted response manipulation in generative LLMs while limiting collateral degradation.

AuthorsOpenReview ↗

Rebuttal to reviewer K1Fw Cont.

W4. Reasoning manipulation and final-answer success

By design, our reasoning attack applies the Keyword-Focused Attack Loss only to a single intermediate reasoning span (e.g., replacing 60 × 3 = 180 with 65 × 3 = 195 in Task 202, or 2 sheets with 3 sheets in Task 208; Appendix F.3), and TFL succeeds in 2/2 cases on this targeted intermediate manipulation. Moreover, the injected value persists during subsequent calculations in both cases (2/2 downstream use)—for example, in Task 202, the model carries 195 into the next step (195 × 0.70 = 136.5). Although the model does not complete the remaining arithmetic to reach the exact intended final value, the propagated manipulation still steers both final answers away from the benign outputs (398 vs. clean 366 in Task 202; 1040 vs. clean 1200 in Task 208).

Accordingly, we explicitly report these cases as partial rather than claiming end-to-end final-answer control: Section 5.2 states that “target-span insertion and downstream use succeed in 2/2 cases, but exact end-to-end reasoning success is 0/2,” matching Table 3 and the Conclusion. We view this as a valuable result because it shows that flipping only 2–5 weight bits on a local intermediate span does not merely insert static text, but propagates through downstream reasoning steps and alters the final answer.

Concluding remarks

The additional experiments achieve 100% targeted success on 100 NQ-Open examples with an average of 1.10 flips. TFL also achieves both main QA targets where GenBFA fails with our attack objectives. Our responses clarify the hardware abstraction and the explicitly partial scope of the reasoning results. This evidence supports TFL's contribution to prompt-specific targeted response manipulation in generative LLMs: the Keyword-Focused Attack Loss promotes attacker-specified target tokens, while AuxUtilityScore guides bit-flip selection to limit collateral degradation. We respectfully ask the reviewer to reconsider the assessment and raise the score in light of this evidence.

Dear Reviewer K1Fw, as the rebuttal period draws to a close, we would appreciate your consideration of our responses, including the expanded QA evaluation, the comparison under the same targeted objective, and clarifications on hardware feasibility and reasoning manipulation. We respectfully ask you to reconsider your assessment and consider raising the score in light of this evidence. Thank you for your time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Reviewer Feedback and Rebuttal Results

Dear Area Chair,

Reviewer K1Fw recognized the novelty of the Keyword-Focused Attack Loss relative to degradation-oriented attacks. Reviewer AfB5 highlighted the importance of TFL and breadth of the evaluation and the inclusion of detection strategies. Reviewer zppq emphasized the importance of studying targeted responses and noted that TFL produces these changes with only 1 to 9 bit flips in the main experiments.

K1Fw. We clarified that TFL evaluates model robustness through software-simulated flips. Physical feasibility depends on the memory device. To address the original two-question QA evaluation, we tested 100 NQ-Open targets on Qwen3-8B BF16. TFL succeeded on all 100 with an average of 1.10 flips. On one harmful prompt, the response changed from refusal to agreement to answer. We also compared methods under the same targeted loss, as detailed below. For the two original GSM8K examples, we clarified exactly what succeeded: the modified intermediate value appeared and was used downstream in both cases, which statsify the target key manipulation.

AfB5. We extended the evaluation to Gemma3-4B and Phi4-mini, with targeted QA success on both. On LLaMA-3.1-8B-INSTRUCT, all five individual targets succeeded, and one joint attack achieved all five with 10 flips. Beyond the 100 QA examples, we evaluated 10 examples per additional task. TFL succeeded on all 10 summarization and all 10 code-generation examples. We also reported the expanded reasoning results.

zppq. We distinguished the original-objective baselines from comparisons under the same targeted loss. With our loss, AttentionBreaker and 1P-DNL each achieved 0/2 on the main QA targets. On Llama BF16, TFL achieved both targets with two flips, while standard full-sequence loss failed after 50. We explained how AuxUtilityScore differs from sensitivity proxies and how WikiText language-modeling loss enters the utility evaluation. We also reported the BF16 and INT8 bit-position distributions and clarified the sample counts and computational cost of the multi-target setting. Reviewer zppq raised their score following the rebuttal.

To our knowledge, TFL is the first bit-flip attack framework for prompt-specific targeted response manipulation in generative LLMs while limiting collateral degradation. The Keyword-Focused Attack Loss promotes attacker-specified target tokens, and AuxUtilityScore accounts for performance on benign data. The expanded QA results and comparisons under a shared objective further support this contribution. Overall, TFL exposes a vulnerability of generative LLMs to targeted weight modifications. We respectfully ask you to take these findings into account in your final assessment. Thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.