- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes MOSAIC framework to accelerate two-party (client and server) secure LLM inference on SPUs by leveraging public context. MOSAIC enables the server to precompute the KV cache for the public prefix in plaintext and integrates it into the 2PC process as secret shares. During the generation stage, both parties locally slice and update their cache shares based on public positions, thereby preserving full contextual attention while safeguarding the privacy of both the model weights and client privacy inputs.
Critical strengths
This paper proposes MOSAIC where the server first computes the KV cache for the public inputs in plaintext, then converts it into secret shares and combines them with the private inputs to calculate attention. Such configuration ensures context integrity without exposing model weights.
Critical weaknesses
MOSAIC’s core operation enables server to send the entire multi-layer public KV cache in the form of secret share with client. This operation can cost large communication time. Author should evaluate end-to-end runtime efficiency of MOSAIC.
MOSAIC requires the privacy input at end of the public input. In the case of privacy input in the middle of public inputs, authors provide solution to rewrite. However, it can eventually degrade the inference accuracy even in plaintext form.
Questions for authors
In practical scenario, with the public context increasing to 512, how do the cross-attention overhead and the cost of transmitting large portions of the KV cache evolve?