REAP: Rare-event Adaptation for Deep Learning

Rare but important samples are critical to model reliability, yet their low frequency creates a persistent rare-sample underfitting problem: standard training is dominated by common patterns, causing rare samples to be systematically underrepresented and poorly learned. Existing importance sampling strategies can increase the exposure of informative samples during post-training, but they do not modify the model architecture to help pretrained models adapt to rare samples, motivating an architectural solution for more effective rare-sample adaptation. To address this challenge, we propose REAP, a rarity-aware adapter that can be embedded into pretrained backbone networks during post-training to facilitate adaptation to rare samples. REAP is applicable to multi-stage hierarchical backbones and introduces lightweight residual adaptation, enabling a smooth transition from pretrained representations to post-training. Furthermore, we propose an implicit routing mechanism, where continuous spatial gates modulate adapter residuals through element-wise multiplication. This design enables smooth and stable adaptation without requiring additional supervision signals or rule-based preprocessing, thereby supporting fully end-to-end post-training. REAP can also be naturally combined with existing importance sampling methods, yielding complementary benefits from architectural adaptation and informative-sample sampling. Extensive experiments with a SegFormer backbone across semantic segmentation, medical image classification, and regression tasks demonstrate the effectiveness of REAP. Compared with the corresponding no-Adapter baselines, REAP improves tail IoU on ADE20K by 10.64%, increases classification accuracy on ISIC by 3.64%, and reduces the PHM08 score on the FD001 dataset by 7.12%, while preserving overall performance, contributing to safer and more trustworthy AI systems.