In-Context Semi-Supervised Learning via Discrete Diffusion

In-context learning (ICL), which enables a frozen model to adapt to new tasks using labeled demonstrations, has recently attracted considerable attention. However, in many practical settings, labeled demonstrations are scarce and costly to obtain, whereas unlabeled data are abundant. This motivates in-context semi-supervised learning (IC-SSL), where a model leverages both labeled and unlabeled examples together within the context. We propose Diffusion In-Context Learner (DICL), which treats the unknown labels in IC-SSL as discrete latent variables and learns a conditional diffusion process over them. Specifically, at inference time, a bidirectional Transformer iteratively denoises missing labels while keeping the input features and observed labels fixed. During this process, DICL progressively commits its most confident predictions, allowing previously inferred labels to serve as pseudo-labeled context for subsequent predictions. Moreover, the denoising budget provides a natural inference-time compute knob, allowing a single frozen model to trade additional computation for progressively refined predictions. Experiments on in-context classification of Gaussian mixtures and in-context wireless symbol detection consistently demonstrate the effectiveness of DICL.