Towards Practical FHE-Federated LLMs Training: FHE-Compatible Compression Techniques and Versatile Advanced Aggregation

Federated learning (FL) enables collaborative LLM training on decentralized data, but plaintext updates remain vulnerable to privacy leakage, such as gradient inversion. Fully Homomorphic Encryption (FHE) can secure aggregation, but scaling FL-FHE to LLMs introduces two bottlenecks. First, ciphertext expansion exacerbates communication overhead. Prior work applies Top-k sparsification to compress updates, but it conflicts with FHE’s SIMD packing, forcing a trade-off: coarse ciphertext-level sparsification discards more information, whereas element-wise sparsification requires Top-k unions that sacrifice compression efficiency. Second, costly non-linear FHE operations limit aggregation to linear FedAvg, excluding the advanced aggregators essential for statistical heterogeneity. In this work, we present **FUSE** (FHE-compatible Unified Sparse-Encode), a unified, novel FHE-native compression suite. It features (i) **FUSE-SPA** that resolves the SIMD-sparsification conflict by reframing Top-k selection as encrypted importance voting, achieving fine-grained sparsification at exact cardinality under a selection fidelity bound; and (ii) **FUSE-RDX**, which avoids pruning-induced information loss by preserving full density through radix encoding under an aggregation-aware radix condition. For statistical heterogeneity in FL-FHE, we also design a homomorphic Norm Bounding circuit that reformulates costly non-linear comparison into polynomial arithmetic, extending secure aggregation beyond linear FedAvg. Experiments demonstrate that FUSE achieves communication reductions of 3.46× for GPT-2 and 3.87× for Llama-7B, yielding end-to-end WAN latency speedups of 3.24× and 3.88× with < 1% perplexity degradation relative to FHE baselines.