HELITE-RESNET: A CKKS-FRIENDLY RESIDUAL NETWORK FOR END-TO-END ENCRYPTED FACIAL EXPRESSION RECOGNITION

Since facial expression recognition (FER) processes sensitive biometric data, privacy concerns arise. Secure multiparty computation and trusted execution environments are common but require multiparty interaction and trusted hardware, respectively. Fully homomorphic encryption instead enables non-interactive, end-to-end encrypted inference. Unlike exact-integer FHE schemes, CKKS supports approximate arithmetic on real and complex numbers and SIMD packing. However, CKKS-based residual networks still incur substantial computational and memory overheads. Prior work optimizes ciphertext packing, convolution mapping, and polynomial activations, while global costs from stage topology and cross-resolution residual paths remain underexplored. We propose HELite-ResNet, a CKKS-friendly residual network for end-to-end encrypted FER. The proposed framework is the first CKKS-based residual-network framework for encrypted FER. Guided by encrypted execution cost, HELite-ResNet reconfigures ResNet20 stages and proposes an FHE-friendly HELite Transition to reduce homomorphic computation overhead. Knowledge distillation maintains performance without deployment overhead. Compared with the fastest FIDESlib-ResNet20 baseline, HELite-ResNet reduces average encrypted inference latency by 28.6% across CIFAR-10, FERPlus, and RAF-DB, whilst the encrypted accuracy is even slightly higher with the identical CKKS parameters and peak GPU memory usage.