Learning from Partially Corrected Labels: A Framework for Human-in-the-Loop Data Curation with Statistical Guarantees

Large language models rely heavily on high-quality data, making annotation and curation central to their development. As models become more capable, curation is increasingly shifting from labeling from scratch toward reviewing, correcting, or rejecting existing outputs. These workflows naturally produce before/after correction records, a structure largely overlooked in conventional learning from noisy labels. We formalize this setting as *Learning from Partially Corrected Labels* (PCL), where every instance has a potentially noisy label and a subset additionally reveals its correction. We show that this additional observation fundamentally changes identifiability under instance-dependent label noise: while the noise process is generally non-identifiable from noisy labels alone, partial correction pairs make the underlying correction process identifiable under mild audit conditions. Building on this result, we develop a learning framework with finite-sample guarantees, achieving an $\mathcal{O}(1/\sqrt{n_{\mathrm{e}}})$ correction-estimation rate with $n_{\mathrm{e}}$ observed corrections. We evaluate PCL across image and text settings, including human-noise benchmarks and historical curation data, and further translate the approach into a real annotation pipeline through a prospective user study. Our results show that correction histories provide reusable supervision for improving downstream learning and data curation, suggesting a promising direction for future human-AI annotation systems.