- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CtrlFormer, a secure transformer inference framework based on FHE. To reduce avoid the feature explosion and low inference efficiency of division operations, authors propose an alternative bounded FHE-friendly Softmax for non-polynomial function. Besides, authors also introduce DSF to replace standard LayerNorm to reduce computational cost.
Critical strengths
Authors design non-polynomial activation function, XControl, that operates in element-wise to constrain exponential outputs and improve numerical stability. Such design can reduce likelihood of feature explosion in prior polynomial Softmax alternatives.
Critical weaknesses
-
Although authors claimed CtrlFormer can avoid feature explosion, there are lack of empirical experiments to report this statement.
-
In Table 7, authors omitted the baseline such as ARION and NEXUS. It is important to report end-to-end inference time of these two methods to justify how CtrlFormer can improve inference speed with less multiplicative levels.
Questions for authors
In the proposed DSF, how the mean is calculated? Is dynamically computed in token level on ciphertexts in inference stage or it is pre-computed during training.