- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a method for reconstructing corrupted floating point model weights, using “Low-Dimensional Seeded Subspaces” to establish a baseline upon a healthy model, identify suspect weights during deployment, and adjust these weights back to the baseline to retain original model accuracy. This methodology has merit as it is compatible with floating point weights, where most recovery methods rely on integer-quantized weights, and it is faster and more efficient than existing floating-point weight recovery methods.
Critical strengths
The problem statement is well laid out; summarizing the concerns of current methodologies into three challenges, and then referring to those challenges while describing LoDiSS’s methodology, allows the reader to more easily understand what the authors are attempting to solve and why they constructed the tool in the way that they did.
The robustness of the methodology is well-established through multiple domain applications, problem datasets, and model deployments. Furthermore, the utility of the authors’ additions is demonstrated sufficiently in the Experimental Results.
Critical weaknesses
The figures and tables in the paper are difficult to read, mostly due to their size. Reading these elements to any degree of detail requires significant scrutiny, making the results difficult to glean from them aside from a baseline understanding that LoDiSS is seemingly better than other tested methods.
Section 3.2 is very mathematically dense, reading more like a proof than a methodology. While this information is important for replication, the delivery makes it difficult to follow exactly how each addition accomplishes the goal set by its subheading.
Questions for authors
I do not understand how Figure 2 shows LoDiSS retaining 100% of the model’s initial accuracy after fault handling, as I do not know what the model’s initial accuracy was before fault injection. Figure 4 in the supplemental material does not improve this understanding. Maybe adding a “pre-fault accuracy” to these figures would help convey this?