Suspicious Labels, Useful Inputs: Cross-Fitted Soft Retention for Few-Shot Text Regression

When training labels come from multiple raters, an unreliable label does not imply that its associated text is useless. Existing noisy-label methods often treat a high-loss example as both unreliable supervision and an input that should be removed; under limited data, these decisions need not coincide. We separate supervision reliability from input utility: a held-out predictor, decoupled from student training, estimates text–label compatibility, and this signal is converted into a continuous supervision weight so that unreliable labels have less influence while their text remains available to the student. We show theoretically that the held-out construction removes the direct self-influence of an example's label on its own residual and give conditions under which residual ranking is better than random ranking. Numerical experiments on two real multi-rater text datasets show that continuous soft retention outperforms both equal weighting and hard deletion with privileged knowledge of the corrupted examples; shuffling the correspondence between weights and examples removes the gain. The results suggest that, in limited-data noisy-label learning, label reliability should not directly determine whether an input is discarded.