Retrieval-Calibrated One-Step Categorical Denoising for Learning with Severe Label Noise

Learning with severe label noise requires a model to recover useful supervision without prematurely committing to potentially incorrect labels. Recent label-diffusion methods address this problem by progressively denoising corrupted label states, but it remains unclear which part of the diffusion formulation is actually responsible for their effectiveness: iterative reverse generation, or learning from label states with systematically varying reliability. We investigate this question through RC-Cat, a retrieval-calibrated categorical denoising framework that separates corruption-aware training from iterative generative inference. RC-Cat first converts each noisy annotation into a sparse candidate distribution using reliability-calibrated neighborhood evidence, thereby preserving uncertainty over the unknown clean class. It then corrupts sampled categorical label states across different noise levels and trains a shared timestep- conditioned denoiser to reconstruct the candidate distribution. Controlled experiments reveal a clear mechanism. On CIFAR-100 with severe PMD-70 noise, introducing categorical corruption improves accuracy from 60.29% to 64.78%, while exposing the corruption timestep further raises it to 71.29%. In contrast, adding the exact categorical posterior regularizer produces no measurable improvement, and iterative reverse inference does not outperform direct prediction from the terminal corruption level. When the representation and candidate supervision are held fixed, RC-Cat exceeds a parameter-matched soft-target classifier by 5.96 percentage points; the advantage remains 2.53 points when externally pretrained CLIP features are replaced by a target-data-only SimCLR representation. Improvements also extend to composite synthetic noise and CIFAR-100N, although they become smaller on Animal-10N and negligible on Clothing1M. These results suggest that the useful inductive bias of label diffusion can arise primarily from corruption-aware learning: exposing a classifier to label states of varying reliability and explicitly informing it of their corruption level. From this perspective, iterative label generation is not necessary to obtain the observed denoising benefit, enabling a one-step categorical formulation that is substantially smaller and faster than a protocol-matched iterative diffusion baseline.