acceptodds
Under review as a conference paper at ICLR 2027

Less Work, Secure Inference: Accelerating Two-Party LLMs on SPU via Public Context

Abstract

Secure two-party computation (2PC) protects private inputs and model parameters during LLM inference. In SPU-based inference, decoding without key-value (KV) reuse recomputes historical states, while cache updates using secret write indices require secure comparisons and selections. Inputs to secure inference need not be entirely confidential and may combine public context with private data. We present , a framework for efficient two-party LLM inference on SPU. It moves public-context construction to plaintext computation on the server while securely processing private inputs and generated tokens. Connecting these computations requires preserving attention dependencies across public and private inputs. imports locally computed per-layer KV tensors as server-private inputs to 2PC and reuses them during inference, preserving full causal context. During generation, each party updates its cache shares locally at public positions determined by disclosed lengths and generation progress, avoiding secure comparisons and selections used to hide write positions. We implement on SPU for GPT-2, SmolLM2-135M, and SmolLM2-360M, and evaluate effectiveness, efficiency, component contributions, and robustness to input adaptation across six datasets. achieves a request-time speedup over Full2PC without KV reuse in the main GPT-2 configuration and matches native plaintext predictions on 95.45% of the 220 examples.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Privacy-preserving LLM inference, Secure two-party computation, KV caching

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gu29OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes MOSAIC framework to accelerate two-party (client and server) secure LLM inference on SPUs by leveraging public context. MOSAIC enables the server to precompute the KV cache for the public prefix in plaintext and integrates it into the 2PC process as secret shares. During the generation stage, both parties locally slice and update their cache shares based on public positions, thereby preserving full contextual attention while safeguarding the privacy of both the model weights and client privacy inputs.

Critical strengths

This paper proposes MOSAIC where the server first computes the KV cache for the public inputs in plaintext, then converts it into secret shares and combines them with the private inputs to calculate attention. Such configuration ensures context integrity without exposing model weights.

Critical weaknesses

MOSAIC’s core operation enables server to send the entire multi-layer public KV cache in the form of secret share with client. This operation can cost large communication time. Author should evaluate end-to-end runtime efficiency of MOSAIC.

MOSAIC requires the privacy input at end of the public input. In the case of privacy input in the middle of public inputs, authors provide solution to rewrite. However, it can eventually degrade the inference accuracy even in plaintext form.

Questions for authors

In practical scenario, with the public context increasing to 512, how do the cross-attention overhead and the cost of transmitting large portions of the KV cache evolve?

Reviewer_R1RaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MOSAIC, a two-party secure LLM inference framework on SPU. It computes public-prefix KV caches locally and imports them into secure computation. It also uses public cache indices to reduce secure update overhead. The authors report up to 4.41× speedup over Full2PC without KV reuse.

Critical strengths

1.The paper addresses a practical efficiency problem in secure LLM inference. The proposed approach is reasonable and preserves causal attention dependencies under the stated assumptions. 2.The implementation and evaluation are relatively comprehensive, including efficiency measurements, ablation studies, and numerical consistency analysis.

Critical weaknesses

Limited novelty and an insufficient baseline. The main techniques rely on public-prefix computation, KV caching, and public-index updates, which are relatively straightforward or already well-established. More importantly, the reported 4.41× speedup is compared against Full2PC without KV caching. The authors' own ablation shows that KV caching alone provides a 2.87× speedup. Although a cached Full2PC result is provided, its timing boundaries differ from MOSAIC. Therefore, the actual contribution of public-context optimization remains unclear.

Limited experimental validation and unresolved correctness issues. The largest evaluated model has only 360M parameters, and the main efficiency experiments use GPT-2 on a single host. This limits the evidence for practical LLM inference. More concerningly, Appendix B.3 reports inconsistent predictions between caching configurations for SmolLM2-135M-Base, together with an incorrect RoPE configuration. These issues remain unresolved and weaken confidence in the implementation.

Questions for authors

1.Can the authors provide a fair end-to-end comparison against Full2PC with KV caching and public-index updates, using identical timing boundaries? How much speedup comes specifically from public-prefix computation? 2.What causes the prediction inconsistencies reported in Appendix B.3? Do these results remain valid after correcting the RoPE configuration?

Reviewer_RVwfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work accelerates two-party secure LLM inference on SPU through MOSAIC, a framework which locally computes public context KV tensors and then integrates them into the secure computation protocol as private inputs. In MOSAIC, each party also updates their relevant shares of the KV cache with slicing and concatenation at public positions, avoiding expensive secure protocols. The experiments show significant speedups and reductions in communication cost.

Critical strengths
  1. I find the public position idea, and the idea of locally computing KV tensors and importing them as private inputs, to be quite compelling. The experimental results are also very strong, particularly the >4x speedup for GPT-2 and the 84% reduction in communication cost.
Critical weaknesses
  1. My main concern with acceptance is that the >4x speedup number is against a fairly weak baseline (no KV caching). The authors do compare MOSAIC against Full2PC with KV caching, but they use different timing boundaries for these, so it is difficult to see how much MOSAIC really improves over the more realistic baseline.

  2. Some of the consistency results are not great, such as the 65% consistency with native plaintext for MOSAIC on CoLA, vs 90% on compatible plaintext. There is also a fairly large drop in MCC here. The authors should explain why this degradation happens and how to prevent it, as this is a large drop in inference quality.

Questions for authors
  1. Can the authors include a comparison of MOSAIC against Full2PC baseline with KV caching, with the same timing boundaries? I am willing to raise my score to accept if the benefits of MOSAIC persist here.

  2. Why does SmolLM2-360M have such low prediction consistency? Is it because of numerical instability?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.