FHE-Agent: Automating CKKS Configuration for Practical Encrypted Inference via an LLM-Guided Agentic Framework

Fully homomorphic encryption (FHE), particularly CKKS, enables privacy-preserving machine learning services, but configuring encrypted inference requires balancing security, numerical quality, and latency. A deployment must jointly choose cryptographic parameters and packing settings whose effects extend across the network and its compiler-derived bootstrap schedule. We present FHE-Agent, a large language model (LLM)-guided agentic framework that automates CKKS configuration search through coordinated multi-agent LLM roles and a deterministic FHE tool suite. Proposal agents develop initial configurations and global or layer-local revisions; selection and assessment agents interpret tool evidence and measured outcomes through shared history. A three-phase workflow progresses from initial proposals to a verified starting configuration and then feedback-guided refinement. Implemented on Orion and Lattigo, it uses static and offline analyses to guide candidate selection, while encrypted execution verifies outcomes and deterministic checks preserve the best result. Recorded searches produce verified configurations on LeNet, AlexNet, and ResNet20, including a 604.79 s ResNet20 configuration. Operation-level profiles attribute most first-to-best time reductions to bootstrapping in AlexNet and ResNet20 and to linear transforms in LeNet; paired changes show that removing a bootstrap auxiliary prime slowed inference by 14–18%, while fixed-global BSGS overrides reduced the targeted layers' time. In same-LLM LeNet pilots with three LLMs, compared with one-shot and repeated independent generation, direct measured feedback, and a general-purpose agent, FHE-Agent is the only method whose attempts all pass under every LLM. These results concern configuration quality in recorded runs, not repeated-deployment speedups or search cost.