- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PROTOQUANT, this is a method that unifies the arithmetic rings, fixed-point scales, input domain, lookup-table payload encodings, and truncation that need to be considered in FSS-based secure inference into an operator contract. This idea is very innovative, and I like it.
And also, PROTOQUANT first performs model compression and calibration, then freezes the contract and retrains through contract-aligned forward computation. In three BERT-based tasks, contract-aligned training recovered most of the accuracy, and in Softmax and GELU, it achieved substantial speedup and cost reduction compared to SIGMA.
Critical strengths
-
This paper has a good observation, which is insight. More compact numerical representations are used for operators in secure inference to save resources, and these compact numerical representations will not cause significant accuracy loss.
-
In the experiments, SST-2 achieved a Generic QAT accuracy of 82.57% and a PA-QAT accuracy of 91.02%, with significant recovery in the other two tasks, this also supported the real value of training.
-
Dealerless preprocessing is good. Unlike SIGMA and FuseFSS, which focus on the online inference phase, this paper also mentioned on the preprocessing of materials generated through two-party interaction. This is actually a very important aspect because it involves the establishment of a security model.
Critical weaknesses
-
Dealerless preprocessing is valuable, but the related protocol descriptions and security justifications are insufficient. While I consider dealerless preprocessing is one of the strengths of this paper, this is mainly because I acknowledge its inherent value. However, this paper does not explain more depth how the FSS keys, masks, and multiplication materials for different operators are specifically generated, which steps directly reuse existing protocols, and how these steps are combined with the truncation and numerical transformations described in this paper. It is recommended to supplement the preprocessing stage and corresponding composition arguments.
-
The experiments are insufficient. End-to-end experiments of the complete model and comparisons of latency and communication with stronger baselines are lacking. The paper mainly reports on the Softmax and GELU operators, but lacks end-to-end comparisons of model latency and communication. Furthermore, it should not be limited to BERT-base but should be extended to larger models, such as GPT-2, LLaMA-7B, etc. The baselines for comparison are also insufficient. To my knowledge, FuseFSS has also open-sourced its code, and FuseFSS should be a stronger baseline than SIGMA at the time, it also should be included in the experimental comparison.
-
The contribution of the core training mechanism has not been fully validated. Table 2 shows that the complete PA-QAT process outperforms Generic QAT, which is a positive result. However, PA-QAT simultaneously performs semantic alignment and input domain constraints, and existing comparisons do not distinguish the roles of these two. For example, if a standard QAT uses the same input domain constraints and aligns the LUTs and numerical scales, would such a large accuracy gap still happen? How much accuracy can be recovered by further simulating share-wise truncation and error correlation?
Questions for authors
-
Can you clarify the protocol and applicable conditions used in dealerless material generation, explain the ideal functionality corresponding to share-wise truncation, and how the complete execution chain inherits existing security guarantees?
-
As mentioned in Weakness 2, can the current experiments be expanded to include stronger baselines and end-to-end testing of the complete model?
-
If a standard QAT uses the same input domain constraints and aligns the LUTs and numerical scales, would such a large accuracy gap still occur? How much accuracy can be recovered by further simulating share-wise truncation and error correlation?
These questions and weaknesses are my main concerns and limit my current score. If the authors can fully address my concerns, I will increase my score.