- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper attempts to address a problem in FHE-based autoregressive decoding: typically, the model only advances one token at a time, while ciphertext computation is expensive, but can be parallelized. This paper packs different candidate states into ciphertexts for parallel processing, allowing multiple tokens to be processed simultaneously, thus it can accelerate secure inference. The paper designs several algorithms to accelerate FHE-based secure inference. Besides processing multiple states in parallel, it also constructs short CDFs hierarchically for discrete comparisons and keeps selection decisions private using encrypted selectors, fixed scans, and arithmetic combinations. Overall, this paper investigates a noteworthy problem in the field of secure inference and makes substantial contributions.
Critical strengths
-
Taurus makes a good observation: it finds that FHE-based ciphertext computation can handle multiple potential states simultaneously and can be used to improve efficiency. Experiments also verify this, compared to CacheMir, Taurus significantly reduces the time per token.
-
Beyond simply packing multiple candidates into ciphertexts, Taurus also implements a complete pipeline including encrypted sampling, candidate verification, and KV cache commit etc.
-
I also really like the ablation experiments in this paper; they clearly explain the functions of several key mechanisms.
Critical weaknesses
-
The main end-to-end experiments focus on a context length of 256 tokens, but as far as I know, using longer contexts might yield different results, so experiments with longer context lengths, such as 512 or 1,024 tokens, need to be added.
-
Although the paper presents meaningful ablation experiments, it does not isolate the contribution of packing different candidate states into ciphertexts within speculative decoding. If speculative decoding is used without Taurus’s packing strategy, how does the performance change?
-
The paper claims to protect the privacy of both parties, which I understand to mean protecting the user and the server. That is, the server cannot see the user's input and output, and the user cannot obtain model weights or intermediate outputs. However, the paper's security discussion mainly focuses on the semi-honest server, with limited discussion of what the client can learn. For example, does the client know how many tokens are generated in each round?
Questions for authors
- Can experiments with longer context lengths, such as 512 or 1,024 tokens, be conducted? And how does it compare to the baseline under longer context lengths?
- If speculative decoding is used without Taurus's packing strategy, how does the performance change?
- As mentioned in point 3 of the Weakness section, what information is available from the client's perspective?
- The paper has tested cryptographic generation for each prompt and observed no inconsistencies in the output tokens. I'm curious if further testing with longer, complete answers is possible, comparing Taurus's accuracy in mathematical problems or its code test pass rate against the baseline. I'd like to know if Taurus can maintain the quality of the final answer while increasing generation speed?