Enhancing few-shot samples for Deep Imbalanced Regression via Frobenius norm

Deep Imbalanced Regression (DIR) aims to learn a regression model where the training label distribution is extremely imbalanced. Recent studies in DIR have shown that aligning feature ordinality with that of labels can effectively enhance the prediction outcomes. However, these gains are largely confined to many-shot samples, while medium- and few-shot regions benefit little. This suggests that preserving feature-label ordinality alignment alone may be insufficient. Empirically, we further observe that feature intensity, measured by the Frobenius norm, also exhibits a clear ordinal structure primarily shaped by well-fitted many-shot samples. Specifically, samples whose feature intensities deviate from this structure consistently exhibit higher test errors, whereas those that better conform to it tend to achieve lower errors. Therefore, this motivates us to investigate the feature intensity to further improve the model performance. In this paper, we propose Frobenius-norm Enhanced Inference (FEI), a plug-and-play post-training pipeline that requires only a three-layer MLP to distill the feature-norm prior from a pretrained model and exploits the learned ordinal property to improve performance on medium- and few-shot samples. Furthermore, we theoretically prove that this constraint leads to a tighter generalization bound under mild assumptions in DIR. Extensive experiments on real-world datasets show that FEI improves medium- and few-shot performance by over 4% and 6%, respectively, at the cost of less than 2% on many-shot samples, which further demonstrates the effectiveness of our method.