DepthRouter: Privacy-Preserving LLM Inference via Joint Depth-Axis Pruning

Secure multi-party computation (MPC) protects both the user's prompt and the provider's weights during large language model (LLM) inference, but makes every executed layer expensive. Reducing model depth could lower the cost of private LLM inference, but removing blocks can degrade answer quality, while input-dependent selection introduces additional privacy and computation requirements. DepthRouter addresses this design problem by constructing pruned model variants with LoRA-based quality recovery and studying query-level selection among them. Variants skip FFN and attention sub-blocks under public, static schedules chosen with an MPC cost model, and adapters placed next to the skipped sub-blocks are distilled from the unpruned model. On Llama-3.1-8B with MPCache, a static variant that skips $12$ FFN sub-blocks retains $51.70$ of $54.71$ average F1 on LongBench QA at a $22.8$% estimated reduction in MPC cost. Selection among variants shows clear headroom: an oracle gains $+4.7$ F1 on QA at $9$% lower estimated cost and $+7.0$ accuracy points on TREC, while our learned quality estimator matches, but does not yet exceed, the best fixed variants.