Epistemic Learning from Imprecise Annotation

Imprecise annotations may support several plausible labelling distributions, yet learning methods often discard this ambiguity into a single predictive distribution. This can obscure what the annotation evidence leaves unresolved. We introduce *epistemic learning from credal supervision*, a framework that uses convex sets of plausible labelling distributions, called credal sets, as supervision and learns sets of compatible predictive distributions. We instantiate the framework with a *pessimistic–optimistic credal classifier* (POCC), which combines a shared backbone with two classification heads trained to minimise worst-case and best-case losses over the available supervisions. Their outputs define a predictive credal set whose spread provides an uncertainty score. We also show how credal labels can practically be obtained through a simple relaxation of existing probabilistic labels, reducing commitment to their precise probability assignments. This construction admits closed-form inner optimisation under cross-entropy loss, enabling efficient training. Assuming the supervision sets contain the true conditional label distributions, and other regularity assumptions, we establish a finite-sample generalisation bound for the averaged predictor with an explicit penalty for supervision imprecision. We evaluate POCC using human annotator disagreement and teacher predictions, alongside label smoothing as a controlled proxy for annotation imprecision. Across these settings, POCC achieves a favourable balance of predictive accuracy, calibration, and uncertainty-based selective classification against competitive baselines.