Activation-Distribution Prototypes for Robust Learning under Noisy Supervision

Deep neural networks trained with noisy annotations may eventually memorize corrupted supervision because standard objectives treat all labels as equally reliable. We introduce a distribution-aware framework for estimating supervision reliability from intermediate network activations. Rather than representing a supervised unit only as a point in feature space, we summarize its activation behavior through sparsity, mean, standard deviation, skewness, and kurtosis. High-confidence predictions are used to construct class-specific activation-distribution prototypes. Reliability is then estimated by comparing the descriptor of each unit with the prototype of its observed label and the nearest competing prototype. When the activation distribution is more consistent with a competing class, the corresponding supervision is smoothly downweighted according to the resulting disagreement margin. This yields an online reliability signal complementary to prediction confidence, task loss, and point-wise feature representations. Experiments across image classification and semantic segmentation, synthetic and structured corruption, real-world label noise, and CNN and ViT architectures demonstrate consistent robustness gains. Direct reliability evaluation further shows that nactivation-distribution disagreement separates corrupted supervision more neffectively than confidence-, loss-, feature-prototype-, and Mahalanobis-based alternatives. These results show that class-specific activation-distribution modeling provides a practical and informative signal for learning under imperfect supervision.