RESAD: Round-Efficient Secure Autoregressive Decoding Scheme for Transformer Inference

Secure Transformer inference for generative large language models can provide protection for both user inputs and model parameters. Existing secure transformer inference schemes based on secure multi-party computation suffer from high latency during secure autoregressive decoding. This latency is amplified by sequential token generation: each token requires a secure inference pass, and the communication rounds of secure protocols are accumulated over many decoding steps. In this paper, we propose RESAD, a round-efficient secure autoregressive decoding scheme for Transformer-based models. To alleviate the performance bottlenecks introduced by nonlinear functions in secure decoding, we design low-round secure computation protocols, with a focus on Softmax and GeLU. For Softmax, a novel bucket-based maximum approximation algorithm is proposed to reduces secure maximum computation from logarithmic-round to constant-round. We further introduce a novel interval lookup-table-based approximation protocol which is used not only for secure evaluation on the exponential and reciprocal functions in few rounds, but also for the secure evaluation of GeLU and LayerNorm. These techniques substantially reduce the interaction overhead of nonlinear functions in secure autoregressive decoding. These techniques are integrated into an end-to-end secure Transformer inference scheme optimized for autoregressive decoding. Experiments on Transformer models show that, compared with the state-of-the-art efficient scheme, RESAD reduces communication rounds by 56.4%-63.2%, online communication by 28.9%-51.4% and end-to-end runtime by 24.4%-43.3% in WAN settings and by 14.5%-16.4% when input 1024th tokens under LAN, while maintaining comparable accuracy. We further implement and evaluate RESAD on TinyLlama, demonstrating its applicability to LLaMA-style models.