acceptodds
Under review as a conference paper at ICLR 2027

ReAnchor: Input-Conditioned Post-Backbone Representation Recovery under Soft Errors

Abstract

Transient soft errors in hardware can corrupt DNN weights or intermediate activations during inference and propagate to incorrect predictions. Many algorithm-level mitigation methods require fault-aware retraining or modifications to the backbone, limiting their use with already-trained models. Range-based mitigation is less intrusive, yet residual errors at the backbone–head interface may remain within calibrated ranges or persist after range projection. This motivates recovering task-relevant representations directly at the post-backbone interface. We propose ReAnchor for this setting. It first applies a fixed range projection to stabilize the terminal backbone feature and then combines the projected feature with sample-aligned multi-scale context extracted from the current input to recover a task-relevant representation for the original task head. Only the ReAnchor module is post-trained; the backbone and task head remain fixed, and inference requires no knowledge of the fault location, corrupted bit index, or fault rate. Across three common datasets and three different model architectures, ReAnchor largely preserves clean accuracy while its advantage becomes more apparent as the injected fault rate increases. At a fault rate of , ReAnchor achieves the highest mean top-1 accuracy in all nine dataset–backbone settings. Compared with the best competing method in each setting, including the unmitigated baseline and the four range-based methods, the average improvement is 31.4 percentage points.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Soft Errors, Fault-Tolerant DNN Inference, Representation Recovery, Range-Based Correction, Input-Conditioned Recovery

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5m2EOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper empirically investigates the limitations of existing range-based mitigation methods against soft errors in DNNs and introduces ReAnchor, a novel framework that combines fixed range projection with input-conditioned representation recovery. The experimental results demonstrate the superiority of ReAnchor over state-of-the-art range-based baselines across multiple datasets and backbone architectures. Comprehensive ablation studies further highlight the contributions of the key components of ReAnchor. Furthermore, ReAnchor introduces modest parameter and FLOP overhead, albeit with some additional inference latency, without requiring retraining of the pretrained backbone or task head. The authors also demonstrate that ReAnchor remains effective even when the recovery module itself is exposed to faults. Overall, I find this work commendable, and the minor weaknesses I have identified do not significantly undermine its contributions or warrant rejection. Therefore, I recommend accepting this paper. I appreciate the authors' efforts and the thorough experimental evaluation.

Critical strengths
  1. The presentation is neat and well-organized. The problem formulation is very clear.

  2. The ablation study is thorough, making the empirical results more compelling.

  3. The identification of the limitations of range-based mitigation methods is interesting and further motivates the development of ReAnchor.

  4. Substantial empirical improvements over various baselines across multiple fault-rate conditions.

  5. The final evaluation of the fault robustness of the ReAnchor module itself is commendable, addressing a potential weakness that other reviewers might raise.

Critical weaknesses
  1. There are several minor presentation issues, such as the insufficient spacing between the caption of Table 2 and the table itself. However, this issue can be resolved in the camera-ready version and is relatively minor.

  2. I would appreciate it if the authors could clarify the motivation for and necessity of introducing modulation parameters such as γl\gamma_l and βl\beta_l in Equation (7). Additionally, the input condition sequence Cˉ\bar{C} is introduced rather abruptly, and I found its role and purpose insufficiently explained.

Both concerns are minor and do not affect my acceptance recommendation. The authors should be able to clarify them during the rebuttal period.

Questions for authors

See weaknesses.

Reviewer_s3vhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an approach to handle soft errors in the weights of deep networks, which can occur due to tampering or faults in the hardware where the networks are stored. To tackle this issue, the paper presents an approach by rectifying the features computed only from the final head of the model without modifying its backbone. The experiments are performed using multiple small-scale datasets and models.

Critical strengths
  1. The paper is easy to read, and the idea is interesting and works best compared to the baselines.
  2. The proposed idea works well with the limited available knowledge of the model, as compared to the existing approaches, which aim to rectify multiple layers of the model.
Critical weaknesses
  1. The primary weakness is that the figures presented in the paper are hard to read, and the values provided there are difficult to understand due to the size and the font used. I suggest to have the modalities that can be easily understood rather than complicating it.
  2. The majority of the experiments are reported on CIFAR-10, and even though a few results are on other datasets, large-scale datasets such as ImageNet are missing from the experiments.
  3. The above concern also applies to the models selected as well: is the proposed method applicable and effective on current-era models, including large foundation models?
  4. The attack descriptions are not adequately provided.
  5. The comparisons provided in the paper are not with the state-of-the-art models. The most recent model with which the comparison was reported is from 2024.
  6. It is not clear why the task head has been kept protected. The final model that is going to be stored and utilized for the task includes the task head and is prone to errors. What happens if this head is also unprotected?
  7. The results reported in Table 1 show that the ECR values of 10−610^{-6} are drastically lower than the values presented for both 10−510^{-5} and 10−710^{-7}. The explanation for this interesting observation is missing.
  8. While the performance at 10−510^{-5} is significantly better, the MHR values are also significantly higher than at other error rates. How does it affect the practicality of the approach?
Questions for authors
  1. The experimental validation needs to be performed on large-scale datasets and large foundation models to reflect the true strength of the proposed work.
  2. The attack descriptions are not adequately provided.
  3. Comparisons with recent and SOTA algorithms are required, and comparisons must be provided across models and datasets.
  4. The impact of the proposed solution must also be reported when the task head is unprotected.
  5. The explainability of the result could be improved. For example, the values reported in Table 1 show that the ECR values of 10−610^{-6} are drastically lower than the values presented for both 10−510^{-5} and 10−710^{-7}. The explanation and this interesting observation interpretation are missing.
  6. Similarly, while the performance at 10−510^{-5} is significantly better, the MHR values are also significantly higher than at other error rates. How does it affect the practicality of the approach?
  7. Check the weaknesses section for detailed comments and any other missing questions.
Reviewer_tEnMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ReAnchor, a trainable module that mitigates transient inference faults in image classifiers while keeping the backbone and task head frozen. Motivated by the finding that clipping the backbone’s final features alone leaves most fault-induced errors unresolved, ReAnchor first sanitizes and clips these features to calibrated ranges, then recovers a representation for the existing head using MLP-Mixer-style blocks conditioned on information extracted from the original image. Only the recovery module is trained, with faults injected at a rate of 10−510^{-5}. Experiments evaluate simulated bit flips in weights and activations across CIFAR-10, CIFAR-100, and Tiny ImageNet-200 using ConvNeXt, EfficientFormerV2, and Swin backbones. ReAnchor achieves the highest mean accuracy in all nine settings at 10−510^{-5}, with an average gain of 31.4 percentage points over the best evaluated comparator, while reducing clean accuracy by only 0.11 percentage points on average on CIFAR-10. Further analyses examine component contributions, corrected and newly introduced errors, sensitivity to faults within the recovery module, and inference overhead.

Critical strengths
  1. The paper addresses a practical setting where the backbone and head remain frozen, the backbone runs once, and inference requires no fault information (Section 2.1). Table 3 reports 2.89% additional parameters and 0.86% FLOPs, alongside a 20.37% latency increase. These constraints make the approach relevant to deployed systems where backbone retraining or replacement is difficult.
  2. Section 3 explains why terminal range projection can leave prediction errors unresolved. It separates errors that remain within bounds, errors corrected by projection, and violations that remain incorrect. At 10−510^{-5} on CIFAR-100, 98.42–100% of fault-induced errors exhibit terminal range violations, yet projection corrects fewer than 1% of all fault-induced errors (Figure 1(b)). This distinction shows why identifying abnormal features does not necessarily provide enough information to restore predictions and gives a clear motivation for the recovery stage.
  3. The moderate-fault results provide meaningful evidence for the approach. At 10−610^{-6}, ReAnchor achieves the highest mean accuracy in seven of nine settings. On CIFAR-10/EfficientFormerV2, it exceeds the matched affine variant by 4.54 percentage points, with a paired 95% confidence interval of [2.80, 6.28] (Section 5.3). On Tiny ImageNet-200/ConvNeXt, accuracy improves from 37.1% to 71.6% (Table 9). These gains occur while the faulty backbone retains useful predictive information, and the affine comparison supports the full architecture in the tested setting.
  4. Several evaluation choices make the results easier to assess. Paired fault realizations reduce comparison variance, while the dependency controls in Table 2 examine the roles of input and backbone features. Reporting mitigation-induced harm alongside error correction reveals the cost of changing previously correct predictions. Section 5.4 also exposes the recovery module to faults. Together, these analyses clarify the method’s benefits and limitations and help identify the remaining questions.
Critical weaknesses
  1. The headline gain at 10−510^{-5} needs a clearer explanation. On CIFAR-10/EfficientFormerV2 at 10−510^{-5}, ReAnchor reaches 63.0% (Table 7), close to the input-only control’s 62.5% (Table 2). That control is reported only at 10−610^{-6}, but its clean and faulty accuracies are identical, so backbone faults do not appear to affect it. The close match suggests that the input pathway may account for much of the severe-fault benefit. However, near-chance backbone accuracy does not establish that its features contain no useful information, and ReAnchor performs substantially better under clean and moderate-fault conditions. The paper should clarify how its dependence on backbone features changes with fault severity.
  2. The dependency controls cover only CIFAR-10/EfficientFormerV2 at 10−610^{-6}, and their architectures and training procedures are insufficiently specified. Extending the existing input-only and matched affine comparisons to 10−510^{-5} and at least two additional dataset–backbone settings would help establish the contribution of the full architecture. A simple detector-based fallback would also be informative, provided its false-positive rate is reported: detecting range violations among fault-induced errors alone does not show whether the detector can reliably identify when switching predictions would help.
  3. The statistical reporting also needs clarification. Under Equation 15, the input-only control’s ΔAcc interval of [−13.88, −0.24] over 100 runs implies approximately 34 percentage points of variation, measured as the trial standard deviation. Because the control’s clean and faulty accuracies are identical, nearly all of this spread would have to come from run-to-run variation in the faulty baseline’s full-test-set accuracy around its 69.6% mean. The evaluation batch size is unspecified, so this does not establish a contradiction with the stated protocol. Reporting per-run accuracies, evaluation batching, and the precise resampling and aggregation procedures would help explain the wide interval and make the main comparisons easier to assess.
  4. The range-based comparisons depend on adaptations that remain difficult to evaluate. Appendix B.1 describes applying the methods to GELU backbones without specifying the resulting restriction functions or handling of negative values and NaNs. The 6–12 percentage-point losses for all four range methods on Tiny ImageNet-200/ConvNeXt at 10−710^{-7}, relative to the unmitigated baseline, make validation of these adaptations particularly important (Table 9). These losses do not prove an implementation problem, but the paper should explain their cause before drawing broader conclusions about the limitations of range-based mitigation.
Questions for authors
  1. How does the paper explain accuracy increasing with fault severity, for example, from 53.0% at 3×10−63\times10^{-6} to 60.6% at 10−510^{-5} on CIFAR-10/ConvNeXt? Could training the faulty branch at a fixed rate of 10−510^{-5}contribute to this behavior, and does training across several rates change the trend?
  2. Could an ImageNet-1k evaluation, including the matched input-only control, clarify how the contribution of backbone features changes on a more demanding classification task? Results beyond classification with a linear head would also help establish the scope of the recovery approach.
  3. Could the Table 2 controls be extended across fault rates, including 10−510^{-5}, and at least two additional dataset–backbone settings, with paired confidence intervals? Please specify each control’s architecture, size, and training procedure. A detect-and-fallback baseline, together with the frequency of Stage I violations among still-correct faulty predictions, would help establish whether simpler alternatives offer a comparable balance between clean and faulty accuracy.
  4. What are the evaluation batch size and exposed activation tensors, and how is per-forward fault resampling and restoration implemented? Please provide per-run accuracy distributions and explain the approximately 34-percentage-point trial standard deviation implied by the input-only control’s confidence interval. Are ECR and MIHR pooled or averaged across runs? For Swin-Transformer, Figure 1(c) and Table 8 are hard to reconcile. On CIFAR-100 at 10−610^{-6}, FT-ClipAct, FitAct, and ProAct recover 28.1–28.7% of the baseline error set. With a clean accuracy near 0.86 (Figure 3) and a faulty baseline of 0.478, this corresponds to more than 10 percentage points of accuracy, yet the net gains in Table 8 are only 0.7–0.8 points. The gap implies roughly 10 points of newly introduced errors, or a mismatch between the two analyses. Prediction-transition counts from shared trials, including corrected and newly introduced errors, would clarify how Table 1 and Figure 1(c) relate to the net accuracy gains in Tables 7–8.
  5. What restriction functions and insertion points are used for each range baseline, including the handling of negative GELU outputs, NaNs, and infinities? Is Ranger applied only at activation-module outputs, as Table 5 indicates, or are other layers also range-restricted? Please report the frequency of non-finite terminal features, each method’s clean accuracy, and whether the gains persist when the baselines receive the same terminal sanitization as Stage I. As a further extension, would combining ReAnchor with FT-ClipAct provide complementary benefits?
  6. How do 10−710^{-7}–10−510^{-5} relate to measured device-level fault rates, or are these intended as stress tests? If all 27.8M ConvNeXt backbone parameters are exposed, 10−710^{-7} implies an expected 2.8 weight flips per forward, before activation faults. How does performance change under isolated single-fault injection, and does the ordering of the range-based baselines change in that setting?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.