Token-Adaptive Concrete Score Distillation (TA-CSD)

Knowledge distillation for autoregressive language models commonly applies a fixed temperature transform to logits before computing a divergence. We begin with an analytical observation: for any objective that depends on logits solely through pairwise differences, including softmax divergences over a fixed candidate window and concrete-score objectives, the centering component of logit standardization is mathematically inert. Consequently, standardization operates entirely as a per-token temperature ${\tau }_t^S$, collapsing the question of whether to standardize into which estimator of that per-token scale to use. The conventional plug-in choice, the student's head standard deviation ${\hat{\sigma }}_t^S$, is blind to teacher confidence, and because concrete-score gradients scale inversely with temperature, plug-in standardization silently acts as an effective learning rate modifier. We empirically quantify this effect in language model distillation, offering a candidate explanation for conflicting reports on logit standardization in the literature. Token-Adaptive Concrete Score Distillation (TA-CSD) predicts the per-token temperature ${\tau }_t^S$ via a lightweight multi-layer perceptron (MLP) gate conditioned on detached supervisory and predictive uncertainty features. In practice, the gate is trained jointly with the student under score distillation with detached features, while bilevel soft-ECE optimization provides a theoretical and diagnostic foundation. Under its learned scale, TA-CSD remains effective-learning-rate neutral (measured ratio $0.989\pm 0.050$, with mean $\tau =1.14$), so its differences from unscaled distillation cannot be attributed to step size. Evaluating across multiple model families and seeds with paired example-level bootstrap tests reveals a dissociation between calibration error and task capability. Plug-in standardization lowers top-1 calibration error only nominally ($0.0565$ vs. $0.0666$ for CSD, not significant) while hedging (flattening the predictive distribution), losing $33\%$ of decisive ($>0.9$) token predictions, degrading mathematical reasoning on GSM8K by $12.3$ percentage points ($36.69\%\to 24.37\%$), and significantly worsening proper score ($\Delta =+0.0075,p<0.001$). While on-policy rollouts (GKD) achieve strong calibration, they alter the training distribution and demand over $12\times$ more training compute. Among offline cached distillation methods, TA-CSD avoids this harm: our clamped formulation attains the best point-estimate Brier score ($0.1628\pm 0.0030$) and protects GSM8K reasoning ($36.36\%$, vs. $24.37\%$ for SWS), while a residual gate reaches smECE $0.0522$ at the cost of reasoning ($29.39\%$).