EEL: Calibrating Early-Exit Neural Networks via Hierarchical Self-Supervised Learning

Early-exit neural networks reduce inference latency by enabling dynamic termination at intermediate exit branches. However, standard supervised training typically prioritizes the final-layer objective, leaving shallow exits with underdeveloped representations and poor calibration. To address this, we propose a Siamese hierarchical Self-Supervised Learning (SSL) paradigm for producing early-exit neural networks, that treats intermediate exits as independent representation endpoints. By applying SSL regularization symmetrically across network depth, our framework promotes useful representations at all exits independently of task-specific classification heads. The resulting multi-branch encoder is then fine-tuned with early-exit classification heads to obtain the final early-exit network. Evaluation across convolutional and Vision Transformer backbones testing both contrastive and non-contrastive objectives demonstrates that the proposed approach improves the calibration of shallow exits, while maintaining a comparable efficiency trade-off.