- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a bit-flipping attack applied to the training stage of AI/ML models. The key idea is to identify a "pair" of instructions (instead of weight values) that can be flipped such that the attack impact is internalized during training but in full swing during inference. Therefore, the attack will not be visible during training but only during inference time. This is demonstrated to be effective across several vision and LLM tasks. The results show that a small number of targeted instruction changes can lead to implementing such attacks.
Critical strengths
-
Interesting angle for attack research. The typical attacks on bit-flipping target inference rather than training, and weights instead of instructions.
-
An interesting find that a few targeted bit flips on instructions can achieve this desired effect.
Critical weaknesses
-
The attack is not evasive. It will be rather straightforward to detect that the training has been tampered with and to investigate based on what happens in the compromised model.
-
Threat model has strong assumptions, and the attack is hypothetical. The analysis, detection of paired instruction corruption and actual execution on the protected model architecture was not shown to be practical.
Questions for authors
This paper proposes a bit flipping attack applied to the training stage of AI/ML models. The key idea is to identify a "pair" of instructions (instead of weight values) that can be flipped such that the attack impact is internalized during training but in full swing during inference. Therefore, the attack will not be visible during training but only during inference time. This is demonstrated to be effective across several vision and LLM tasks. The results show a small number of targeted instruction changes can lead to implementing such attacks. I think the paper tackles an interesting problem but I have doubts about the basic premise and the practical impact of the attack.
Threat model and attack capability: This work assumes precise, persistent code corruption inside a protected training environment, including paired changes to different compute blocks in multiple attack configurations. While some of these fault capabilities may have been assumed in prior inference-time attacks, it is unclear if existing hardware fault-injection techniques can achieve the desired effects proposed in this paper. The experiments demonstrate the effects of modifying compiled binaries, but do not assess if such effects could even be achieved in the first place for the presumed protected environments.
Likewise, I am wondering how the attacker is observing the forward-backward calls and compute sequences of the protected victim in this threat model. I believe this is needed to figure out the specific bits to flip.
Pre-training vs. fine-tuning: I believe the paper claims training-time attacks, but it looks like the LLM part targets fine-tuning, not pre-training. Is that correct? Is this just because the pre-training cost is prohibitive to test the attack, or is the attack fundamentally limited to fine-tuning in an LLM scenario?
This attack seems to have a uniform impact on all decisions made. This would make it easier to detect the training errors and retrain with better protections (or inspect the training process carefully next time). Is it possible to make the attacker sneakier by affecting just a subset of decisions, purposefully?
There has been a line of Trojan-inspired model training attacks. Some even combined that with transient fault attacks (see below). How does your work compare to them? This could be added to the paper. Hommes, Steyn, Vincent Dankbaar, Tanguy Stekke, Xiaomeng Wang, Lisanne Weidmann, Senna van Hoek, Durba Chatterjee, Lejla Batina, and Zhuoran Liu. "Triggering Stealthy Feature Map Backdoors via Physical Fault Injection in Embedded Neural Networks." arXiv preprint arXiv:2607.09473 (2026).
How does memory encryption affect the attack?
4 comments
Response to Review jNaU Cont'd (2/3)
Q3-3. Pre-training vs. fine-tuning
Thank you for the comment, and we apologize for the confusion.
-
We use the term "training" to refer to the general process of updating model weights via optimization guided by gradients, which subsumes both pre-training (a.k.a., training from scratch) and fine-tuning (including LoRA and full-weight fine-tuning).
-
We evaluate fine-tuning cases in Appendix C because Section 5.2 already shows that our attack is effective in the training-from-scratch setting. As complementary studies, the LoRA and full-weight fine-tuning cases in Appendix C additionally demonstrate that fault internalization does not require updating the pretrained base weights and remains effective when the optimization has to re-adapt representations learned under the original computation to the altered computation.
-
Our attack is not limited to fine-tuning in an LLM scenario. During rebuttal, we have added attack experiments for the pre-training of a Qwen LLM on the FineWeb-Edu dataset. Fault internalization is consistently effective in this setting: the attacked LLM shows significantly worse PPL on the restored platform (see Table 3-1 below), while the attacked training loss curve substantially overlaps with that of benign training (Figure 13 in the updated paper), concealing the ongoing attack without abnormal training behaviors.
Table 3-1: Held-out loss and perplexity (PPL) under Benign, Attacked, and Restored execution for Qwen pre-trained on FineWeb-Edu.
| Metric | Benign | Attacked | Restored |
|---|---|---|---|
| Loss | 3.3923 | 3.3946 | 69.6423 |
| PPL | 29.73 | 29.80 |
Q3-4. The attack seems to have a uniform impact on all decisions made and purposely affect just a subset of decisions
Thank you for the suggestion. During rebuttal, we have added an experiment for attacking a 10-exit model trained from scratch on CIFAR-10, where only the prediction of the class "Truck" is attacked. Fault internalization remains effective and precise under this setting: (1) There is not any training abnormality, and the test accuracy of the class "Truck" and other unattacked classes are consistently high on the attacked platform. (2) Once the computation is restored on a clean platform, only the test accuracy of the attacked "Truck" class drops to zero, while the predictions of other unattacked classes are unaffected. Detailed results are reported in Table 3-2. In the updated paper, we also show the t-SNE visualization of the 10-exit model's logits (Figure 17 in Appendix C), where only the attacked class's logit cluster is drifted.
Table 3-2: Test accuracy (%) of the 10-exit CNN on CIFAR-10 under Benign, Attacked, and Restored execution.
| Class | Benign | Attacked | Restored |
|---|---|---|---|
| Truck (target) | 92.60 | 92.70 | 0.00 |
| Non-target average | 90.93 | 90.80 | 89.79 |
| Overall | 91.10 | 90.99 | 80.81 |
Response to Review jNaU Cont'd (3/3)
Q3-5. How our work compares to Hommes et al.
Thank you for mentioning this work. Hommes et al. is essentially an inference-time BFA, whereas fault internalization is a training-time attack. Hommes et al. assumes that the attacker tampers with the model supply chain and injects the backdoor. During the inference, the backdoor is triggered when there is an ongoing BFA. Compared with Hommes et al., our attack has the following advantages:
-
Our threat model is more practical in protected outsourced training, where the attacker is unable to tamper with the model supply chain (e.g., model training, checkpoint transmission).
-
BFA's effects are internalized into model weights by our attack, which persist even when there is no ongoing BFA. However, like prior BFAs, the attack effect in Hommes et al. disappears immediately when the underlying BFA is off.
-
Our attack is less detectable, because the model behaves normally when the attack is in progress, and the attack effects manifest only after the entire attack is already completed. Hommes et al., in contrast, cause the model to exhibit malicious behaviors immediately when the attack is launched.
We will surely add Hommes et al. and the discussion to our paper to make the positioning of fault internalization clearer.
Q3-6. How memory encryption affects the attack
Thank you for the question. We clarify this from two aspects:
-
As clarified earlier, the attacker is the host platform that provides the hardware and computation infrastructure, and the BFA instantiations are prepared in the offline analysis phase. Also, our attack does not require knowing the encrypted model weights, gradients, or intermediate computation results. Thus, the attacker does not need to invert the encrypted memory.
-
Our attack affects GPU-resident executable instructions and takes effect after the code is loaded (e.g., per the standards of NVIDIA H100's confidential computing). As a result, the induced bit flips directly alter native instruction encodings without corrupting ciphertext or causing attestation failures. Thus, memory encryption does not cause crashes when our attack is on. Moreover, we carefully select bit flips that preserve valid instruction encodings and execution semantics, allowing the training process to continue without crashes.
I thank the authors for the detailed response. The comments address my questions about LLM pre-training and whether the attack can selectively affect a single class. My main concern remains the practicality of the attack. The comparison to prior work addresses the number of bit flips required, but it remains unclear whether an attacker can reliably induce the specific, paired instruction changes in the protected training environment. I therefore maintain my score.
Response to Review jNaU (1/3)
Thank you for your efforts in reviewing our paper and providing helpful comments and suggestions. We hope you don't mind our long response, and we look forward to hearing your response.
Q3-1. Threat model, attack capability, and positioning of this paper
Thank you for the questions and comments. We clarify our threat model and the positioning of this paper below.
We use the standard threat model of attacks on protected outsourced training, where the attacker is the host platform that provides the computation infrastructure and hardware resources, such that information required by the pre-attack analysis is readily accessible. Nevertheless, the attacker cannot provide tampered infrastructure/hardware or directly modify the infrastructure/hardware due to authentication and attestation mechanisms. The remaining attack opportunity, as demonstrated in existing studies, is the bit-flip attack (BFA) that exploits hardware disturbances to flip bits without directly modifying them.
We directly follow the BFA techniques that have been established in existing work, and our BFA instantiations do not have any additional technical requirements. One key practical constraint of BFA is the bit-flip budget (i.e., the number of required bit flips, where a lower number is better). Prior techniques have successfully demonstrated flipping as many as 4K bits [R3-1]. All of the nine BFA instantiations presented in our paper require only 3-40 bits, which are significantly lower than this budget and show the high practicality of our BFA instantiations.
We do not aim to propose a new BFA technique, as it is out of the scope of an ML venue like ICLR. Instead, our goal is to understand how existing BFA techniques can be leveraged to attack ML tasks and how severe the results are. For the first time, we reveal that even in protected training environments and even when the training process behaves normally, the trained model weights can still be silently attacked with highly practical BFA instantiations. More broadly, our results also uncover an overlooked risk in the emerging silent data/bit corruption (SDC) issue in large-scale model training: although SDC is often caused by natural bit flips, we show that it does not necessarily cause training errors or instability, and it may alter the model's learned semantics without notable training abnormalities.
References:
[R3-1] GPUThor: Amplifying Rowhammer Attacks via Non-Uniform Patterns to Exploit ECC-Protected GPUs. In CCS 2026.
Q3-2. How the attacker is observing the forward-backward calls
Thank you for the question. We clarify that our attack consists of an offline profiling/analysis phase and an online attack phase:
During the offline profiling/analysis phase, since the attacker is the host platform that provides the computation infrastructure, the attacker naturally knows the implementations and can analyze the calls (but cannot directly modify or sabotage them per the threat model) following the procedure described in Section 4 of our paper. Attackers may also deploy their own training tasks (as simulated victims) to assist the analysis and validate their BFA instantiation. Such a profiling/analysis phase is common in existing BFA works.
During the online attack phase, the attacker picks one (or multiple) BFA instantiations to attack the protected training process. Our paper already provides BFA instantiations for different purposes (Section 4.2 and Appendix B) and shows the generality across computation functions, tasks, modalities, and training settings (Appendix C).
We will surely revise the paper to make the offline and online phases clearer to avoid confusion.