acceptodds
Under review as a conference paper at ICLR 2027

LoDiSS: Learning in Low-Dimensional Seeded Subspaces for Long-Term Test-Time Recovery of Floating-Point Neural Weights

Abstract

As machine learning models are increasingly deployed on standalone edge devices, their weights are exposed to random and adversarial bit flips that can accumulate over time. Most defenses target integer weights or tolerate corruption without repair, leaving floating-point deployment insufficiently protected. Prior real-valued weight recovery suffers from inefficient training and recovery due to redundant variables and group-wide optimization, while inadequate handling of rounding and nonfinite values can lead to unreliable recovery. We introduce LoDiSS, which learns equivalent recoverable weights in low-dimensional seeded subspaces with less training state. Shared pseudorandom transformations efficiently regenerate recovery constraints at test time, while calibrated iterative recovery handles rounding and nonfinite values and reconstructs only suspected weights, without training data or a clean model copy. Across 20 classification settings, LoDiSS retains approximately 100% of clean accuracy under repeated random faults and outperforms baselines under adversarial and recovery-aware adaptive attacks, with mean training and recovery speedups of and over the existing baseline. Together with evaluations across additional tasks, these results demonstrate reliable, efficient, and task-agnostic floating-point recovery for long-term standalone deployment.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Neural weight recovery, bit-flip robustness, floating-point neural networks, low-dimensional learning, seeded subspaces, edge deployment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_cB6NOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for reconstructing corrupted floating point model weights, using “Low-Dimensional Seeded Subspaces” to establish a baseline upon a healthy model, identify suspect weights during deployment, and adjust these weights back to the baseline to retain original model accuracy. This methodology has merit as it is compatible with floating point weights, where most recovery methods rely on integer-quantized weights, and it is faster and more efficient than existing floating-point weight recovery methods.

Critical strengths

The problem statement is well laid out; summarizing the concerns of current methodologies into three challenges, and then referring to those challenges while describing LoDiSS’s methodology, allows the reader to more easily understand what the authors are attempting to solve and why they constructed the tool in the way that they did.

The robustness of the methodology is well-established through multiple domain applications, problem datasets, and model deployments. Furthermore, the utility of the authors’ additions is demonstrated sufficiently in the Experimental Results.

Critical weaknesses

The figures and tables in the paper are difficult to read, mostly due to their size. Reading these elements to any degree of detail requires significant scrutiny, making the results difficult to glean from them aside from a baseline understanding that LoDiSS is seemingly better than other tested methods.

Section 3.2 is very mathematically dense, reading more like a proof than a methodology. While this information is important for replication, the delivery makes it difficult to follow exactly how each addition accomplishes the goal set by its subheading.

Questions for authors

I do not understand how Figure 2 shows LoDiSS retaining 100% of the model’s initial accuracy after fault handling, as I do not know what the model’s initial accuracy was before fault injection. Figure 4 in the supplemental material does not improve this understanding. Maybe adding a “pre-fault accuracy” to these figures would help convey this?

Reviewer_QEdWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces LoDiSS to repair floating-point neural-network weights damaged by bit flips. Their method learns the weights in a constrained subspace. It uses the resulting relationships to repair corruption without training data or a clean model copy. The experiments show substantial speedups over Real-ECC. Across 20 classification settings, the method retains nearly all of its clean accuracy over ten repair cycles at the lower tested random-fault rates. However, I still have serious concerns about how clearly the novelty is established and whether the theory adequately supports practical floating-point recovery. These leave me unconvinced by the broad reliability claims.

Critical strengths

I particularly appreciate the practical numerical safeguards in Section 3.2 and the encouraging recovery results across different tasks in Table 3.

Critical weaknesses
  1. I am not convinced about the contribution beyond Real-ECC. The introduction starts by emphasizing the limitations of integer-weight protection. However, Real-ECC already provides a framework for real-valued neural-weight recovery, as the authors also acknowledge on page 2. My own reading of Real-ECC confirms this as well. By no means I am undermining the fact that floating-point storage introduces additional difficulties, particularly rounding and nonfinite values, and that LoDiSS addresses these explicitly. But personally, it remains unclear how much of the improvement comes from the proposed representation and recovery procedure versus the numerical safeguards. The Real-ECC baseline already includes noise-tolerant recovery, but the experiments do not sufficiently isolate the effects of calibration, nonfinite handling, structured subspaces, and selective reconstruction. These points make it hard for me to see assess the real contribution here or rather its significance.

  2. I personally value theoretical guarantees that explain when a proposed method should work in practice. For this paper, I am slightly disappointed to not find the theory in the main text where the methods are described. But more importantly, I am not fully convinced that the guarantees adequately cover floating-point recovery. The main text explicitly claims that restoration depends on successful localization. However, Proposition 3 establishes localization for finite sparse errors under exact arithmetic, without clean numerical error. The numerical restoration analysis in Appendix C.6 then assumes that all corrupted positions have already been selected. This leaves a gap in establishing localization when rounding errors and finite and nonfinite corruption occur together. I also do not see evidence that the required coherence and error-size conditions hold in the evaluated settings. Finally, although the paper bounds individual reconstruction errors, it does not establish comparable stability over successive approximate repairs.

Questions for authors
  1. Since Real-ECC already provides real-valued recovery, which additions are most responsible for the improvement? I would particularly like to understand how much advantage remains when calibration and nonfinite handling are made comparable. A focused ablation would help me assess the contributions of the proposed representation and decoder beyond these numerical safeguards.

  2. How does Proposition 3 connect to the implemented recovery procedure when rounding errors and nonfinite values are present? I would find either an extended analysis or direct measurements of localization success useful here. Do the sufficient coherence and error-size conditions hold, or at least yield informative bounds, for the actual seeded frames used in the experiments?

  3. How should one choose the representation ratio in practice? Table 5 supports 0.8 among the tested choices, but I am still unsure how the balance between clean accuracy and recovery reliability changes under more severe or concentrated faults. I would also like to know whether imperfect repairs accumulate over longer trajectories, and whether the method can distinguish a successful repair from an incorrect reconstruction.

Reviewer_D8RxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The proposed method projects weights into recoverable subspaces, then performs test-time adaptation through compact training representation and recovery constraints. The proposed method is tested across multiple models and datasets, which shows reasonable performance against both random and two kinds of adversarial bit error injections.

Critical strengths

The proposed method is an interesting way to approach BFA defenses. While the methodology itself may not be new, in the context of BFA defense, there is utility in the proposed approach.

The authors ran a lot of experiments and evaluated against a wide range of datasets and architectures.

Critical weaknesses
  1. Novelty: The proposed method of subspace projection and test-time adaptation is surprisingly similar to prior approaches in [A]. While this work addresses internal noise to bit error, its methodological principle is identical.

  2. Experimental evaluation: The paper runs lot of benchmark and evaluation, however, most of dataset is somewhat similar such as CIFAR-10/100. I would prefer to see results on ImageNet to fully appreciate the improvement, especially for clean performance recovery against BFA. Does the recovery cost go up on larger benchmarks?

  3. ECC is already an expensive defense; the comparison to baseline is not fair. What happens if the defender applies some of the conventional software-based protections? Such as aggressive quantization or run-time weight recovery?

  4. Writing flow can be better as some of figure descriptions are not clear, e.g., Figure 3.

[A] NeurIPS 2025 paper: “Lifelong Test-Time Adaptation via Online Learning in Tracked Low-Dimensional Subspace”

Questions for authors

Refer to the weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.