DP-LAIR: Differentially Private LLM Inference with Adaptive Routing and Latent Perturbation

Differential privacy (DP) provides a principled framework for protecting sensitive data, where its applications in machine learning have largely focused on training-time privacy. However, Language-model APIs currently can expose information from both protected training records and query time context. Thus, We study the joint protection of these two sources under a practical trusted-server, API-only deployment model, where internal model states remain hidden and private information becomes externally observable only through the released transcript. We propose Differentially Private Latent Adaptive Inference and Routing (DP-LAIR), a unified framework for protecting both training-time and runtime private information at this interface. DP-LAIR introduces a router that directs tokens that can be generated from public information to a public path. For tokens requiring private context, their hidden-state residual is compressed into a latent representation, clipped and perturbed with Gaussian noise, and mapped back to the output space by a decoder. The router and decoder are trained with differential privacy to protect training-time data, while runtime privacy is adaptively accounted for using data-dependent leave-one-out sensitivity. Experiments across multiple datasets and models support DP-LAIR’s protection of both training data and query time context while demonstrating good generation utility.