acceptodds
Under review as a conference paper at ICLR 2027

Not All Errors Matter: Semantic-Aware Silent Data Corruption Detection for LLMs

Abstract

Although Large Language Models (LLMs) exhibit strong inherent fault tolerance and mask the effects of most GPU soft errors, a small fraction of these errors can still cause severe semantic deviations in their outputs, leaving LLM inference vulnerable to Silent Data Corruption (SDC) with potentially serious consequences for safety-critical applications such as autonomous driving and robotics. Detecting these consequential faults is challenging because the magnitude of numerical anomalies alone does not reliably indicate their semantic impact. Our key observation is that features in adjacent LLM layers exhibit strong predictability during fault-free execution, while Significant SDCs tend to disrupt this relationship, increasing the discrepancy between a layer's predicted and observed features. Building on this observation, we propose SIEVE, a lightweight framework for detecting Significant SDCs through deviations from learned inter-layer predictions. SIEVE trains a Nominal Inter-Layer Predictor on clean execution traces to predict the current layer's projected self-attention features from those of the preceding layer. At inference time, it extracts statistical and geometric discrepancies between the predicted and observed features of the current layer, aggregates them across monitored layer pairs and decoding steps into a Discrepancy Profile, and uses a lightweight XGBoost classifier to identify Significant SDCs. Across three LLMs and three VQA benchmarks, SIEVE achieves 96.38% precision, 91.49% recall, and 93.78% F1 on the primary evaluation set, with a false positive rate of 0.11%. Compared with representative SDC detection baselines, SIEVE improves F1 by up to 11.38% while introducing an average end-to-end overhead of 6.65%.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Large language model, Silent data corruption, Semantic-aware detection, Fault injection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_MSQpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SIEVE, a system to detect the rare GPU soft errors that change what an LLM's answer means, which numerical thresholds handle poorly. It predicts each monitored layer's features from the layer before, then feeds the prediction errors to an XGBoost classifier. Across three vision-language models and three VQA benchmarks, it reaches 93.78% F1 with 6.65% added latency, beating Ranger and Dr.DNA by about 10–11 F1 points.

Critical strengths
  • The motivation is good: numerical deviation and semantic deviation may not be strongly correlated, and previous methods that rely only on numerical deviation fail.
  • The method consistently outperforms the baselines.
Critical weaknesses
  • The detector’s training data comes from fault injection, and the fault distribution after deployment may differ from the injected one.
  • The semantic labels come from a small 7B model, which could be unreliable.
  • The dataset is skewed: most executions contain errors. This may not be realistic and could inflate precision, recall, and F1.
Questions for authors
  • Would larger models be more fault-tolerant, for example with a much lower fraction of Significant SDCs?
  • Would the algorithm’s performance drop on larger models, either because Significant SDCs are rarer or because the XGBoost input grows larger and noisier?
  • How would the detector hold up if the deployment fault distribution differs from the injected one?
  • Is there a specific reason not to use a SOTA LLM as the judge to generate semantic labels?
  • What is the probability of a dual-bit fault in the experimental setup, and is it based on real statistics?
5 comments

Thank you for recognizing the motivation of our work and for raising these questions. We have addressed them below and hope the results help resolve your concerns.

Response to Weakness 1 / Question 3

Data distribution is a general concern, and the space of possible random faults is particularly large in LLMs. Under our double-bit fault model, we randomly select the injection location and the bits to flip. For a 16-bit tensor, a single element has (162)=120\binom{16}{2}=120 distinct bit pairs. A tensor with 10,000 elements therefore has 1,200,000 possible injections based on the element location and bit pair alone. We consider the results of our large-scale random fault injection experiments statistically informative and representative of detection performance across different fault distributions.

We also ran experiments that change the distribution of fault injection locations. We trained separate detectors using faults injected into the Attention or MLP modules, froze each detector and its threshold, and evaluated it on the test subset for the other module. As shown in Table R3, when the training fault source is switched from Attention to MLP, mean F1 on the Attention test subset is 88.90% and the false positive rate is 0.206%. On the MLP test subset, both detectors identify 16 out of 19 Significant SDCs. Compared with the detector trained on MLP faults, the detector trained on Attention faults achieves a higher mean F1 and a lower false positive rate. SIEVE thus retains a high detection rate for Significant SDCs under the two evaluated shifts in fault location.

Table R3. Cross-module detection under random double-bit injection. Results are for the prefill test subsets and are averaged equally across the nine tasks.

Source module (training) → Target module (test)F1RecallFalse positive rate
Attention → Attention93.26%88.52%0.010%
MLP → Attention88.90%85.82%0.206%
MLP → MLP82.49%91.67%0.153%
Attention → MLP94.07%91.67%0.015%

Response to Weakness 2 / Question 4

The judge applies a fixed rubric to two responses to the same input and determines whether the core answer has changed semantically. Given the requirements and scale of this annotation task, we selected Prometheus-7B v2.0. At the time, it was a state-of-the-art judge specifically trained to apply custom scoring rubrics. Its 7B size makes large-scale local annotation practical and lets us use a fixed model version and scoring prompt throughout the process. We also manually spot-checked a subset of the labels to verify that the judge's decisions followed the rubric.

Response to Weakness 3

As shown in Table A1 in the appendix, 90.90% of valid executions were subjected to fault injection, and 83.59% of those injections did not lead to an SDC. Significant SDCs accounted for 3.65% of the original test set (2,720 / 74,586). Although most executions received an injected fault, only a small fraction resulted in an SDC. SIEVE thus achieved high recall at a low prevalence of Significant SDCs without overstating recall.

Response to Question 1

Given our time and GPU constraints, we compared InternVL3-8B and InternVL3-14B, as shown in Table R4. Each model received 15,000 valid injections. For 8B, the proportions of SDCs and Significant SDCs were 5.25% and 3.89% (583 / 14,970), respectively. The corresponding proportions for 14B were 4.51% and 3.45% (517 / 14,973). For this task and injection setup, the proportion of Significant SDCs for 14B was only 0.44% lower.

Table R4. Mixed-injection results for InternVL3-8B and InternVL3-14B on LingoQA.

ModelSDC proportionSignificant SDC proportionPrecisionRecallF1False positive rate
InternVL3-8B5.25% (786 / 14,970)3.89% (583 / 14,970)70.65%82.28%76.02%1.25%
InternVL3-14B4.51% (676 / 14,973)3.45% (517 / 14,973)75.58%81.25%78.31%0.97%

Response to Question 2

The language decoder has 48 layers in InternVL3-14B and 28 layers in the 8B variant. For both models, we project the activations to 64 dimensions, monitor three layer pairs, and extract 12 summary statistics per pair. The XGBoost input therefore has 36 dimensions in both cases. With a fixed number of layer pairs and the same summary statistics, the XGBoost input dimension stays fixed even if the activation projection dimension increases.

As shown in Table R5, on the mixed-injection test subset for LingoQA, 14B achieved an F1 of 78.31%, 2.29% higher than the 8B score of 76.02%. Precision increased from 70.65% to 75.58%, the false positive rate decreased from 1.25% to 0.97%, and recall decreased slightly from 82.28% to 81.25%. With the same detector input dimension, SIEVE achieved comparable recall and higher F1 on the 14B model than on 8B in this comparison.

Response to Question 5

We did not set a fault rate in the experiment. Faults are rare in practice, and running the experiment for several days at a real-world fault rate would yield too few fault samples. We therefore inject one random double-bit flip per inference run in the fault-injection group, while the fault-free group receives no injection. After 55,000 valid injections, Significant SDCs accounted for 3.65% of the samples.

Thank you for the response. I have two follow-up questions.

Weakness 1 / Question 3: What if double-bit faults occur systematically rather than randomly, for example repeatedly in the same memory region or bit position? Would this affect the detector’s performance, given that the XGBoost classifier is trained on randomly injected faults?

Question 1: In general, are larger models more robust to soft errors, so that SDCs affecting the answer’s meaning occur less often? For example, would a model with 500B or 1T parameters be robust enough that such detection becomes unnecessary? Some discussion of how SIEVE’s value scales with model size would help clarify its practical relevance.

Reviewer_zUZeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SIEVE, a framework for detecting Significant SDC that causes semantic deviations in the final output during LLM inference. The authors demonstrate that the magnitude of numerical deviation does not necessarily correlate with the degree of semantic damage in the final output. Thus, unlike traditional numerical anomaly detection approaches, SIEVE aims to selectively detect Significant SDC that critically impacts the final output. SIEVE employs a Nominal Inter-Layer Predictor that learns the feature relationship between adjacent decoder layers. During inference, it aggregates the differences between predicted and actual features across multiple decoding steps to construct a Discrepancy Profile, which is then used by an XGBoost classifier to determine the presence of Significant SDC. Evaluated on 3 LLMs and 3 VQA datasets, SIEVE achieves a Precision of 96.38%, Recall of 91.49%, F1-score of 93.78%, and FPR of 0.11%, with a reported average end-to-end execution overhead of 6.65%.

Critical strengths
  • SIEVE addresses the necessity of identifying Significant SDC, defined as errors that actually impact output semantics, rather than relying solely on traditional numerical metrics. Using LLM-as-a-Judge to define Significant SDC is a sound approach.

  • SIEVE provides a lightweight detection mechanism based on inter-layer feature prediction without modifying the base LLM. This is a promising approach for efficiently detecting Significant SDCs.

Critical weaknesses
  • As the authors acknowledge in Appendix G, the method and evaluation are restricted to random dual-bit fault injections during the Prefill stage. In real-world LLM inference, faults can occur in various locations and types, including during the Decode stage. However, generalizing Prefill-only detection to broader inference scenarios is inherently flawed, rendering the claim that SIEVE is a general SDC detection framework inadequately justified.

  • Significant SDC is detected only after observing K decoding steps. It is questionable whether such delayed detection is practical, especially in safety-critical applications like autonomous driving emphasized in the paper. A delayed detection approach like this seems more suited for offline inference scenarios.

  • Additionally, most Significant SDCs in the test set involve trivially detectable NaN/Inf values (1,941 of 2,720), and the F1-score drops from 93.78% to 68.78% when these cases are excluded.

Questions for authors
  • SIEVE monitors three layer pairs: (6,7), (24,25), and (26,27). What are the selection criteria for these monitored layer pairs?
2 comments

Thank you for your detailed and constructive comments. We added experiments with mixed prefill and decode fault injection and address your questions below. We hope these responses address your concerns and that you will reconsider your overall assessment of the paper.

Response to Weakness 1

Prefill and decode process different numbers of tokens but use the same Transformer layers. Because the two stages share the main computations within each layer, we simplified the original experiment by injecting faults only during prefill. During generation, SIEVE detects errors using discrepancies between features from adjacent layers. The same detection procedure can therefore handle faults from both stages without prior knowledge of the injection stage.

Following your suggestion, we added experiments that mix prefill and decode injections. For each task, we ran one fault-free inference pass over 5,000 inputs and then ten rounds of random double-bit fault injection. As shown in Table R1, SIEVE achieved the highest F1 on all nine tasks. Its nine-task mean F1 was 74.29%, exceeding Ranger (57.02%) and Dr.DNA (52.70%) by 17.27% and 21.58%, respectively. Table R2 reports precision, recall, and false positive rates under mixed injection.

We also observed that F1 was lower for faults injected during decoding than for those injected during prefill. One possible explanation is that prefill processes the entire input sequence at once, whereas each decoding step processes a single new token. For standard full attention with a context length of approximately LL, the computational complexity is O(L2)O(L^2) for prefill and O(L)O(L) per decoding step. Faults injected during prefill may propagate through subsequent computations and affect the activations of more tokens, producing more pronounced discrepancies between adjacent-layer features and making these faults easier for SIEVE to detect.

Table R1. Significant SDC detection F1 (%) by fault injection stage. Each cell lists the scores for Ranger, Dr.DNA, and SIEVE in that order.

ModelDatasetPrefill onlyDecode onlyPrefill + decode
LLaVAEarthVQA96.47 / 98.80 / 98.8047.58 / 27.55 / 58.0155.29 / 48.75 / 68.79
LLaVALingoQA94.00 / 94.23 / 98.0845.70 / 26.18 / 54.5556.75 / 44.85 / 66.49
LLaVAVQAv293.43 / 90.78 / 91.4353.62 / 40.85 / 54.8564.31 / 54.44 / 65.07
QwenEarthVQA60.00 / 72.73 / 92.3152.31 / 58.93 / 80.0052.59 / 59.39 / 80.47
QwenLingoQA91.67 / 74.07 / 100.0062.95 / 41.34 / 77.3564.26 / 42.99 / 78.55
QwenVQAv275.68 / 70.59 / 96.9749.04 / 56.57 / 77.0250.53 / 57.47 / 78.29
InternVL3EarthVQA89.66 / 89.66 / 93.3352.67 / 51.66 / 79.5654.55 / 53.70 / 80.41
InternVL3LingoQA78.57 / 75.86 / 100.0056.38 / 54.55 / 74.4657.32 / 55.52 / 75.93
InternVL3VQAv290.91 / 90.91 / 96.5554.87 / 54.29 / 72.4557.57 / 57.23 / 74.58
All modelsNine-task mean85.60 / 84.18 / 96.3852.79 / 45.77 / 69.8157.02 / 52.70 / 74.29

Table R2. Precision, recall, and false positive rate on the mixed-injection test executions.

ModelMethodPrecisionRecallFalse positive rateSignificant SDCs detected
LLaVARanger49.19%75.15%2.11%437/584
LLaVADr.DNA52.32%49.28%1.35%294/584
LLaVASIEVE67.22%67.50%0.96%397/584
QwenRanger43.82%77.66%2.84%478/625
QwenDr.DNA50.35%58.19%1.84%365/625
QwenSIEVE77.76%80.52%0.66%503/625
InternVL3Ranger47.70%69.37%2.79%557/804
InternVL3Dr.DNA48.88%64.39%2.47%517/804
InternVL3SIEVE71.90%83.00%1.21%664/804
All modelsRanger46.91%74.06%2.58%1472/2013
All modelsDr.DNA50.52%57.29%1.89%1176/2013
All modelsSIEVE72.29%77.01%0.95%1564/2013

Response to Weakness 2

SIEVE's detection window KK can be adjusted to the task's latency requirements. Figure A3 shows detection performance for different values of KK. As a reference for safety-critical applications such as autonomous driving, the LMDrive baseline in a closed-loop CARLA study updates its control commands at 10 Hz, or once every 100 ms [1]. With this update interval as a reference, SIEVE can produce a detection result within that interval. For systems that use the model output after the inference run is complete, the detection decision needs to be available before the downstream system uses that output.

References

  1. Li et al., Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Driving, 2026.

Response to Weakness 3

NaN/Inf cases are easy to detect. As shown in Table 2 of the paper, after excluding these cases, SIEVE's nine-task mean F1 drops from 93.78% to 68.78%, with a mean recall of 62.81%. On this subset, SIEVE still exceeds the mean F1 of Ranger (37.74%) and Dr.DNA (22.74%) by 31.04 and 46.04 percentage points, respectively, and achieves the highest F1 on seven of the nine tasks.

Response to Question 1

Detection performance varies across layer pairs. We tested a range of layer-pair combinations and found that these three pairs yielded the best detection performance among the configurations evaluated. Figure 6 shows that the 36D configuration monitoring (6, 7), (24, 25), and (26, 27) achieves the highest nine-task mean F1 among the evaluated configurations. Removing any one of the three pairs and then retraining and recalibrating the detector reduces mean F1. Table A4 shows that mean F1 also decreases in the evaluated configurations that retain the original three pairs and add further pairs. These results support using the original three pairs among the configurations compared.

Reviewer_7yGKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a lightweight, semantics-aware Significant SDC detection framework for LLMs. The key is to not rely on simply numerical evaluations, but through a learned method.

Critical strengths

SDC is important to address.

Critical weaknesses

SDCs are introduced through soft errors, mostly. A faithful fault injection would have to be performed at least at the RTL level. The paper uses software-level fault injection, which has been shown extensively decades ago to be not reflecting the impact of soft errors. This is because a bit flip in a logic gate, for instance, can effect different software/model parameters. So if one really wants to inject faults at the software level, one must build a mapping from faults in hardware structures to faults at the software, which this paper doesn't discuss.

Questions for authors

See weakness.

1 comment

Thank you for raising this concern. We have addressed it as follows. If our response satisfactorily resolves your concerns, we would appreciate it if you could reconsider your overall assessment of our paper.

Response to Weakness

RTL-level fault injection can characterize the propagation of hardware faults in greater detail. However, full RTL designs for commercial GPUs are generally unavailable, and RTL simulation is time-consuming. For example, a DSN 2021 study estimated that evaluating LeNet by injecting faults into the GPU scheduler alone would take more than 720 hours in its simulation setup [1].

In our paper, we use software-level injection similar to PyTorchFI [2]. We inject one random double-bit flip into a selected tensor to evaluate its effect on output semantics and SIEVE's ability to detect the resulting corruption. Software-level injection of this kind has been shown to provide some degree of fidelity. Recent reliability studies that use software-level injection include Dr. DNA (ASPLOS 2024), ReaLM (DAC 2025), FT-Transformer (SC 2025), FT2 (HPDC 2025), and STAC (DAC 2026) [3–7].

References

  1. Revealing GPUs Vulnerabilities by Combining Register-Transfer and Software-Level Fault Injection, DSN 2021.
  2. PyTorchFI: A Runtime Perturbation Tool for DNNs, DSML 2020.
  3. Dr. DNA: Combating Silent Data Corruptions in Deep Learning using Distribution of Neuron Activations, ASPLOS 2024.
  4. ReaLM: Reliable and Efficient Large Language Model Inference with Statistical Algorithm-Based Fault Tolerance, DAC 2025.
  5. FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention, SC 2025.
  6. FT2: First-Token-Inspired Online Fault Tolerance on Critical Layers for Generative Large Language Models, HPDC 2025.
  7. STAC: Spatial-Temporal Activation Contextualization for Resilient LLM Inference, DAC 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.