Understanding Semantic Generalization in Knowledge Tracing through Difficulty Supervision

Knowledge tracing (KT) predicts students' responses from interaction histories, but new questions have no training responses. Knowledge components (KCs) can be shared across questions, while pretrained question semantics add limited or mixed gains beyond them. A privileged-information experiment shows that empirical item difficulty can be highly predictive. However, measuring empirical difficulty requires student responses, which are unavailable for newly introduced questions. We therefore study whether difficulty can link response patterns from training questions to the content of unseen ones. To study this transfer, we adapt a pretrained semantic encoder with discrete difficulty classification and continuous correctness regression, then use its frozen representations for KT. We also test generated solutions as an augmentation to question content. Across three datasets and six KT backbones, adaptation improves unseen-item prediction, with controls linking the gains to content–difficulty correspondence. Probes find more accessible correctness information, while exposure experiments show that the readout advantage diminishes as item-specific responses accumulate. Together, these results show how difficulty supervision transfers information from observed student responses through question content to unseen items.