acceptodds
Under review as a conference paper at ICLR 2027

Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation

Abstract

Quantifying uncertainty in Large Language Models (LLMs) is essential for mitigating hallucinations and enabling risk-aware deployment in safety-critical tasks. However, estimating Epistemic Uncertainty (EU) via Deep Ensembles is computationally prohibitive at the scale of modern models. We propose a framework that leverages the small draft models to efficiently estimate token-level EU, bypassing the need for full-scale ensembling. Theoretically grounded in a Bias-Variance Decomposition, our approach approximates EU via Jensen-Shannon divergence among drafts (variance proxy) and KL divergence between the draft mixture and the target (bias proxy). We additionally bound finite-reference EU transfer error by teacher-specific predictive mismatch, distinguishing disagreement preservation from predictive-mean approximation. To support accurate estimation, we introduce Online Stochastic Distillation (OSD) to efficiently approximate target aggregation and the Data-Diverse Drafts (DDD) strategy to enhance draft diversity for better target approximation. Crucially, our approach achieves Hallucination Detection performance competitive with heavy perturbation-based methods like TokUR while replacing repeated target-model uncertainty passes with draft and proxy scoring, offering a practical solution for uncertainty-aware LLM deployment.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
epistemic uncertainty, llm uncertainty, efficient uncertainty, uncertainty quantification, hallucination detection, distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2SfsOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Authors provide a way to estimate epistemic uncertainty by training draft models which are trained on disjoint partitions of the training dataset, along with a distillation setup to predict the mean of the target predictive distribution. The estimates are tested against predicting Gaussian-induced uncertainty in a sandbox environment, as well as hallucination detection in GSM8K and ASQA. Theory is provided to show that the estimate is grounded.

Critical strengths
  • The proposed method is straightforward, although understanding the actual algorithm is difficult in the current flow (see weaknesses).
  • The connection between the proposed decomposition and bias/variance is elegant.
Critical weaknesses
  • Overall, the paper is difficult to follow and poorly written. In the theory section there are multiple incidents of missing or undefined notation, and although I could piece together what was meant in most cases this should not be the job of the reader. Even in the first equation in 2.1 although I inferred that this is decomposition of total uncertainty into aleatoric and epistemic, the presentation is extremely unclear. If I were a practitioner attempting to use this method I would come away from this paper frustrated and less likely to want to attempt to reproduce these results.
  • Comparisons with other methods need much more context. Neither semantic entropy or KLE are discussed in any detail but just tossed into a results table. KLE is not even cited in the references. I would consider these types of omission grounds for desk rejection.
  • No discussion is provided around why a token-based approach is outperforming semantic methods like semantic entropy which take the entire output into account. This result feels not very believable and many more ablations need to be performed.
  • No discussion around why hallucination detection is even a reasonable task for an epistemic uncertainty paper, given that it's unclear that hallucinations are purely epistemic.
  • Unclear how the scales of the Gaussian noise were selected to generate the ground truth in the sandbox experiment. I remain unconvinced that using a simple Gaussian to generate targets is warranted, and the appendix section attempting this justification is near indecipherable. It seems that there is an attempt to use the Laplace approximation to justify use of Gaussians to estimate the predictive posterior but given that there are very few implementation details on what the scale of these Gaussians are in the main text this entire explanation feels very jumbled and confusing.
  • Table 2 results are very unconvincing. In many categories the proposed method does quite poorly relative to baselines.
Questions for authors

Please see the comments in weaknesses above. All of these have to be addressed, but the main issue is that the paper basically needs to be rewritten with a much clearer narrative and much more intuitive discussion of the results before I would consider it acceptable at any conference, let alone ICLR. Currently the paper feels like a raw dumping of results without much context.

In general, it feels like LLM usage was very heavy in preparation of this manuscript, given that there is a lot of technical output without much actual insight or explanation. Can authors confirm exactly how much LLM usage was involved in preparation of this manuscript and explain the insights in every single piece of theory that was presented?

5 comments

Response to Reviewer 2Sfs

We thank the reviewer for the detailed feedback. The central question is whether a lightweight draft ensemble and a learned predictive-mean proxy can approximate a specified teacher family's token-level uncertainty while providing a useful detection–latency trade-off. We assess this through reference fidelity and downstream detection, with computational measurements reported separately. Submitted Table 2's 1×3 configuration reports a GSM8K mean AUROC of 0.768 versus 0.765 for TokUR. Separately, ASQA profiling measures 0.783 versus 0.890 s per batch, with substantially larger latency differences relative to Semantic Entropy and KLE. Below, we clarify the theory, report the component and length controls, and specify where this trade-off holds. We also acknowledge the missing KLE citation and will add it.

1. Clarity of Notation and the Insights Behind the Theory

The theory addresses a practical transfer question: which aspects of a teacher ensemble must the drafts approximate to preserve its uncertainty? Matching only the predictive mean can erase disagreement, whereas teacher-specific predictive fidelity provides a sufficient condition for finite-reference EU transfer. We will introduce the notation and organize the results around that distinction.

Definitions and uncertainty decomposition. At generation step tt, fix the shared prefix ht=(x,y<t)h_t=(x,y_{<t}). Let pθ,t=pθ(⋅∣ht)p_{\theta,t}=p_\theta(\cdot\mid h_t) denote a teacher model's next-token distribution over the vocabulary, let πT\pi_T denote the specified teacher-parameter reference distribution, and let pˉT,t=Eθ∼πT[pθ,t]\bar{p}_{T,t}=\mathbb{E}_{\theta\sim\pi_T}[p_{\theta,t}]. The uncertainty decomposition is

H(pˉT,t)=Eθ∼πT[H(pθ,t)]+I(Yt;θ∣ht).H(\bar{p}_{T,t})=\mathbb{E}_{\theta\sim\pi_T}[H(p_{\theta,t})]+I(Y_t;\theta\mid h_t).

Here, HH denotes categorical entropy and YtY_t is the next-token random variable. The left-hand side is total predictive uncertainty. The first term on the right is the aleatoric component, measuring uncertainty within individual teacher predictions. The second is the epistemic component relative to πT\pi_T, measuring predictive variation across the teacher reference. Its interpretation depends on the choice of πT\pi_T.

The practical draft score. For KK draft predictions, define qmix,t=1K∑k=1Kqk,tq_{\mathrm{mix},t}=\frac{1}{K}\sum_{k=1}^{K}q_{k,t}, and let pmix,tp_{\mathrm{mix},t} denote the learned proxy for the teacher predictive mean. All distributions are evaluated at the same prefix and over a common vocabulary. Our full score is

u^tfull=JSD⁡(q1,t,…,qK,t)+KL⁡(qmix,t∥pmix,t).\widehat u_t^{\mathrm{full}}=\operatorname{JSD}(q_{1,t},\ldots,q_{K,t})+\operatorname{KL}\left(q_{\mathrm{mix},t}\Vert p_{\mathrm{mix},t}\right).

The first term measures disagreement among drafts; the second measures mismatch between their mean prediction and the proxy reference. A scored draft prediction may itself be a group-averaged prediction, as specified by the ensemble configuration.

What each theoretical result contributes.

  • Lemma 2.1: Reference-relative EU equals the expected divergence of individual teacher predictions from their predictive mean. This connects uncertainty to predictive disagreement.
  • Proposition 2.2: The draft score decomposes exactly into disagreement and mean mismatch. This algebraic identity does not, by itself, guarantee accurate EU estimation or establish that adding the KL term always reduces estimation error.
  • Theorem 2.4: Approximating each teacher's predictive distribution with its corresponding draft prediction provides a sufficient condition for preserving finite-reference EU. Approximating only the teacher predictive mean is insufficient.
  • Proposition 2.5: Disagreement within groups of drafts assigned to the same teacher contributes additional variation. It must be distinguished from disagreement between teacher-associated groups.
  • Proposition 2.6: OSD provides an objective for approximating the teacher predictive mean. Preserving teacher-family disagreement requires a separate condition.
  • Appendix A.5: The finite-ensemble bias-correction interpretation requires additional ideal-sampling assumptions. It is not an unconditional interpretation of the practical score.

We will reorganize the theory around these questions, define notation before its first use, and place each result's intuition and assumptions alongside its statement. We will also distinguish the token-level definitions from the operational aggregation used for downstream detection. Summing scores along a generated response does not automatically produce sequence-level mutual information.

2. Context and Citation for Semantic Entropy and KLE

The missing KLE reference is an omission, and we apologize for it. We will add Nikitin et al., Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities, NeurIPS 2024, and expand the discussion of both semantic baselines.

How the baselines differ. Semantic Entropy groups sampled responses by semantic equivalence and measures uncertainty across those groups. KLE uses a semantic-similarity kernel and von Neumann entropy to represent finer relationships between responses. Our method instead measures differences between conditional next-token distributions at shared prefixes. Semantic Entropy, KLE.

These methods are relevant comparisons because they use uncertainty to identify unreliable generations. Table 2 evaluates their downstream detection utility; it does not assume that they estimate an identical mathematical quantity.

Settings for the additional evaluation below. The saved configurations use ten auxiliary responses per question, sampled at temperature 1.01.0 and top-p=0.95p=0.95, for both semantic baselines. Evaluated target responses were sampled separately at temperature 0.70.7 and top-p=0.9p=0.9, with a maximum of 2,048 generated tokens. Semantic Entropy uses empirical cluster frequencies, with normalized-answer matches and pairwise SAME/DIFFERENT judgments from the Qwen3-8B-Instruct checkpoint. KLE uses microsoft/deberta-large-mnli probabilities, combining entailment and half the neutral probability, symmetrized across directions. It uses a heat kernel with parameter 0.30.3, an unnormalized graph Laplacian, and unit-trace kernel normalization; the NLI pairs exclude the question prompt. Calibration and evaluation settings are specified in Response 3. These settings describe the additional Table A evaluation. We will consolidate the baseline details and timing boundaries in the manuscript.

3. Token-Level Scoring, Semantic Baselines, and Additional Ablations

The comparison asks whether token-distribution scoring offers a useful alternative to methods requiring sampled responses and semantic processing. In submitted Table 2, the measured 1×3 evaluation is much faster than Semantic Entropy and KLE; its ASQA mean AUROC is above Semantic Entropy's and below KLE's. The appropriate interpretation is a measured detection–latency trade-off. The additional 2×3 analysis below separately tests component contribution and response-length effects; it is not a new timing experiment.

What token-level scoring observes. "Token-level" describes the resolution of the score: each distribution is conditioned on the complete available prefix, and the response score aggregates across generated positions. One hypothesis is that local predictive disagreement or mismatch captures instability associated with an incorrect response. Semantic methods instead measure variation across the meanings of sampled responses. Our experiments do not establish which mechanism explains their relative performance, and summing token scores can introduce a response-length effect.

Completed post-processing analysis. We compared Full EU, JSD-only, and KL-only using both sum and mean aggregation, alongside length-only and the saved UQ baselines. For a response with TT retained scored positions, the aggregations are

ssum=∑t=1Tu^t,smean=1T∑t=1Tu^t.s^{\mathrm{sum}}=\sum_{t=1}^{T}\widehat u_t,\qquad s^{\mathrm{mean}}=\frac{1}{T}\sum_{t=1}^{T}\widehat u_t.

We also fitted mean-score-plus-length controls. Semantic Entropy and KLE were counted once per response, rather than summed across their repeated token rows.

Additional evaluation protocol. Table A uses saved Llama-3.1-8B-Instruct responses scored by six Llama-3.2-1B-Instruct drafts in the 2×3 configuration, with a generation budget of 2,048 tokens and generation seeds 0–2. Original labels and length-capped responses are retained. Length is the number of retained scored positions. We used a shared 80/20 question-grouped split across seeds, so all responses to the same question remain in one partition. Every method is evaluated on identical test responses within each dataset. The held-out sets contain 792 responses to 264 GSM8K questions and 569 responses to 190 ASQA questions.

Each row uses a separate logistic regression with training-only standardization, fitted to pooled training responses across seeds. Score direction is determined from training data only. This additional analysis differs from submitted Table 2, which used the 1×3 configuration, in ensemble configuration and evaluation protocol; numerical comparisons should be made within each table.

Table A. Length and Score-Component Ablation

Values are AUROC mean ± sample SD across three fixed-checkpoint generation repetitions, not independent retrainings. AUROC uses predicted error probabilities; ASQA error means hallucinated. Mean-score-plus-length rows use two-feature classifiers.

MethodGSM8K AUROC ↑ASQA AUROC ↑
Full EU (sum)0.716 ± 0.0120.618 ± 0.019
Full EU (mean)0.673 ± 0.0250.572 ± 0.048
JSD-only (sum)0.691 ± 0.0220.520 ± 0.038
JSD-only (mean)0.603 ± 0.0060.580 ± 0.029
KL-only (sum)0.715 ± 0.0120.619 ± 0.019
KL-only (mean)0.673 ± 0.0260.574 ± 0.049
Length-only0.642 ± 0.0370.566 ± 0.028
TokUR (sum)0.704 ± 0.0090.609 ± 0.032
TokUR (mean)0.654 ± 0.0350.574 ± 0.042
Semantic Entropy0.816 ± 0.0050.586 ± 0.017
KLE0.667 ± 0.0370.685 ± 0.012
Full EU (mean) + length0.716 ± 0.0070.609 ± 0.029
KL-only (mean) + length0.716 ± 0.0080.610 ± 0.030

Selected Paired AUROC Differences

Intervals are 95% percentile intervals from 2,000 paired question-cluster bootstrap samples, with each question resampled jointly across methods and seeds. The statistic is the mean per-seed test AUROC. The fitted classifiers and split remain fixed, and the intervals are not adjusted for multiple comparisons. The selected contrasts address component contribution, aggregation, and information beyond length.

First − secondGSM8K difference [95% CI]ASQA difference [95% CI]
Full EU (sum) − KL-only (sum)0.0005 [-0.0001, 0.0013]-0.0011 [-0.0022, -0.0001]
Full EU (mean) − KL-only (mean)0.0000 [-0.0011, 0.0010]-0.0022 [-0.0039, -0.0006]
Full EU (mean) − Full EU (sum)-0.0428 [-0.0820, -0.0069]-0.0455 [-0.1055, 0.0126]
[Full EU (mean) + length] − Length-only0.0736 [0.0237, 0.1259]0.0423 [-0.0405, 0.1203]
[Full EU (mean) + length] − [KL-only (mean) + length]-0.0002 [-0.0010, 0.0007]-0.0017 [-0.0031, -0.0002]

Interpretation. The length-controlled analysis identifies useful error-related information on GSM8K. Adding Full-EU mean to length improves AUROC over length alone by 0.0736, with 95% interval [0.0237, 0.1259]; KL-only gives a comparable improvement. Semantic Entropy has the highest mean GSM8K AUROC (0.816), and KLE leads on ASQA (0.685). The corresponding Full-EU-plus-length versus length-only interval on ASQA includes zero. These results establish the scope of the detection signal under this additional protocol; its cost–performance balance requires its own matched timing measurements.

Component contribution. Adding disagreement to KL-only changes sum-score AUROC by 0.0005 on GSM8K, with an interval containing zero, and by -0.0011 on ASQA. Mean aggregation likewise gives no consistent incremental detection benefit. We will therefore distinguish the algebraic decomposition from an empirical claim that both terms are needed for detection. This component result limits the claim for the combined score; the separate comparison with expensive uncertainty estimators concerns the cost and utility of draft/proxy scoring as a whole.

4. Why Hallucination Detection Is Relevant to Epistemic Uncertainty

Hallucination detection is an operational use of an uncertainty score, rather than a ground-truth measurement of EU. A model may consistently give an incorrect answer with low disagreement, and high uncertainty need not imply factual error. We therefore evaluate reference-EU fidelity and downstream detection as separate questions.

The two experiments answer separate questions. Table 1 measures fidelity to the uncertainty of a specified teacher family. Table 2 evaluates whether the resulting score helps identify incorrect or hallucinated responses. Downstream detection performance does not establish that the teacher reference is a calibrated Bayesian posterior.

We will make this distinction explicit and qualify the introduction's stronger language connecting EU to hallucinations. We will also clarify the operational labels: GSM8K evaluates final-answer correctness, whereas ASQA uses judge-assigned hallucination labels. Under the submitted ASQA policy, non-hallucinated responses include supported, abstained, and incomplete responses, so non-hallucination should not be equated with strict answer correctness. These evaluations provide evidence of practical utility within their respective protocols.

5. Gaussian Perturbation Scale and Its Posterior Interpretation

Thank you for highlighting this ambiguity. We agree that the perturbation-scale selection and its relationship to the posterior interpretation need to be explained more clearly.

Reviewer_oVcgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework for efficient uncertainty estimation in LLMs using lightweight small draft models, the paper studies EU, but the framework can be applied to any singals. It combines Online Stochastic Distillation to approximate the teacher's predictive mean estimation with Data-Diverse Drafts to preserve predictive disagreement (EU). The estimated EU is decomposed into draft disagreement and predictive-mean mismatch using KL divergence. Experiments on GSM8K and ASQA evaluate EU estimation accuracy, hallucination detection, and computational efficiency.

Critical strengths
  • Practical motivation: using lightweight models to avoid heavy test-time sampling to reduce the inference cost of uncertainty estimation is a reasonable direction. And the paper has prefectly reached the goal with the designed framework.

-The method achieves competitive downstream tasks, like hallucination detection performance, with much lower runtime than several sampling-based baselines.

Critical weaknesses
  • The methodological and theoretical novelty are really limited. Most theoretical results follow from established KL decomposition identities and information-theoretic inequalities. Uncertainty-preserving distillation has also been extensively studied (e.g., Ensemble Distribution Distillation and Hydra). The proposed DDD primarily relies on disjoint data partitioning, a conventional ensemble diversity strategy, without introducing a convincing new mechanism for preserving teacher epistemic uncertainty.

  • Insufficient experimental validation. The EU reference is constructed using artificial Gaussian parameter perturbations which need to be validated. The reference EU is only estimated by three low-rank Gaussian perturbations, and Table 1 mainly measures distillation fidelity to a synthetic family. The teacher uncertainty should also be validated on downstreaming tasks (like tTable2).

The comparisons for EU estimation mainly involve conventional distillation strategies or UQ methods, lacking stronger Bayesian UQ baselines. Furthermore, experiments cover only two datasets, and downstream improvements over existing methods are marginal or inconsistent. The efficiency claim is weak: runtime is about 12% lower than TokUR, but peak VRAM is about 45% higher, and the trained drafts are never shown to remain usable for speculative decoding.

Questions for authors
  1. Can you report LpairL_{pair}, and the draft-side JSD versus the teacher JSD, separately from the full score? This would show whether DDD actually preserves teacher disagreement or whether the gains come from the mean-mismatch term.

  2. Does the advantage of DDD hold with a larger reference family (e.g., M≥10M \ge 10) or with a more posterior-like reference such as a LoRA ensemble or a Laplace approximation?

  3. Can the authors provide stronger UQ comparisons? Why not directly distill from the best UQ estimation methods?

8 comments

Response to Reviewer oVcg

Thank you for the detailed questions. Our contribution concerns approximating a specified teacher family's token-level uncertainty with lightweight draft distributions and a learned predictive-mean proxy, and assessing the resulting detection–latency trade-off. The framework separates predictive-mean approximation from transfer of teacher-associated variation. The additional experiments directly examine disagreement fidelity, fresh-reference generalization, and downstream error detection. We first answer those questions, then relate the submitted 1×3 detection results to its ASQA profiling measurements in Response 6. The expanded 2×3 results identify how the method's behavior varies across tasks and model pairs.

1. Separating Paired Fidelity, Draft Disagreement, and the Full Score

Thank you for suggesting this decomposition. We separately report teacher-to-draft paired fidelity, draft JSD alone, and the full EU score. These measurements distinguish teacher-specific predictive fidelity, agreement with teacher disagreement, and the behavior of the combined estimator.

Definitions. At an evaluated prefix hth_t, let pi,tp_{i,t} be the predictive distribution of saved teacher ii, and let qij,tq_{ij,t} be the distribution of draft jj assigned to that teacher. For M=3M=3 teacher groups with S=2S=2 drafts per group, define

qˉi,t=1S∑j=1Sqij,t.\bar q_{i,t}=\frac{1}{S}\sum_{j=1}^{S}q_{ij,t}.

qmix,t=1MS∑i=1M∑j=1Sqij,t.q_{\mathrm{mix},t}=\frac{1}{MS}\sum_{i=1}^{M}\sum_{j=1}^{S}q_{ij,t}.

We compute paired fidelity as

Lpair=1∣T∣∑t∈T1M∑i=1MKL⁡(pi,t∥qˉi,t).L_{\mathrm{pair}}=\frac{1}{|\mathcal T|}\sum_{t\in\mathcal T}\frac{1}{M}\sum_{i=1}^{M}\operatorname{KL}\left(p_{i,t}\Vert\bar q_{i,t}\right).

where T\mathcal T contains the evaluated token positions. This is teacher-to-corresponding-draft-group KL; it differs from the global mean-mismatch term. The reference and the two estimators are

utref=JSD⁡(p1,t,…,pM,t).u_t^{\mathrm{ref}}=\operatorname{JSD}(p_{1,t},\ldots,p_{M,t}).

u^tJSD=1MS∑i=1M∑j=1SKL⁡(qij,t∥qmix,t).\widehat u_t^{\mathrm{JSD}}=\frac{1}{MS}\sum_{i=1}^{M}\sum_{j=1}^{S}\operatorname{KL}\left(q_{ij,t}\Vert q_{\mathrm{mix},t}\right).

u^tfull=JSD⁡(q1,t,…,qK,t)+KL⁡(qmix,t∥pmix,t).\widehat u_t^{\mathrm{full}}=\operatorname{JSD}(q_{1,t},\ldots,q_{K,t})+\operatorname{KL}\left(q_{\mathrm{mix},t}\Vert p_{\mathrm{mix},t}\right).

Here, K=MS=6K=MS=6 indexes the individual drafts after flattening the teacher–draft groups, and pmix,tp_{\mathrm{mix},t} is the learned predictive-mean proxy. We compare each estimator with teacher JSD on identical prefixes and token masks, reporting Spearman correlation and RMSE. Draft JSD is computed over all six drafts; it is not the JSD of only the three group-averaged distributions.

Evaluation protocol. Tables A–D use Llama-3.1-8B → Llama-3.2-1B in the 2×3 configuration. Values are means ± sample standard deviations across five generation seeds using fixed trained checkpoints, not five independent retrainings. The teacher perturbation scale is σT=0.1\sigma_T=0.1; IDD and FDD use draft initialization noise σD=0.1\sigma_D=0.1. The evaluated positions include prompt and completion positions, with length-capped responses retained. All methods share the generated sequences within each dataset and generation seed. ASQA uses ASQA-trained modules; GSM8K, Leg Counting, Color Cube Rotation, and MATH-500 use GSM8K-trained modules. Leg Counting, Color Cube Rotation, and MATH-500 therefore test cross-task transfer without dataset-specific distillation.

Method labels. IDD denotes Initialization-Diverse Drafts, DDD denotes Data-Diverse Drafts, and FDD denotes Fully-Diverse Drafts. The notrain baseline uses the draft ensemble without draft distillation.

Table A. Full-score fidelity to the saved three-teacher reference, with paired fidelity.

DatasetMethodSpearman ↑RMSE ↓LpairL_{\mathrm{pair}} ↓
ASQAnotrain0.7960 ± 0.00880.4945 ± 0.01670.2048 ± 0.0050
ASQAGKD0.8149 ± 0.01150.3506 ± 0.01130.1669 ± 0.0040
ASQAMiniLLM0.7976 ± 0.01100.4464 ± 0.01010.2115 ± 0.0039
ASQAIDD0.8186 ± 0.01060.3697 ± 0.00950.2016 ± 0.0047
ASQADDD0.8221 ± 0.01040.3669 ± 0.00950.1979 ± 0.0048
ASQAFDD0.8212 ± 0.01080.3695 ± 0.00950.2011 ± 0.0047
GSM8Knotrain0.8477 ± 0.01330.3779 ± 0.01130.1441 ± 0.0136
GSM8KGKD0.8758 ± 0.01480.3270 ± 0.01510.1315 ± 0.0133
GSM8KMiniLLM0.8708 ± 0.01490.3692 ± 0.01030.1475 ± 0.0137
GSM8KIDD0.8899 ± 0.01760.2866 ± 0.01940.1480 ± 0.0161
GSM8KDDD0.8885 ± 0.01750.2900 ± 0.01850.1434 ± 0.0157
GSM8KFDD0.8894 ± 0.01750.2855 ± 0.01940.1467 ± 0.0161
Leg Countingnotrain0.8325 ± 0.02210.4118 ± 0.03530.1558 ± 0.0397
Leg CountingGKD0.8616 ± 0.02270.3753 ± 0.03430.1458 ± 0.0393
Leg CountingMiniLLM0.8648 ± 0.02150.3975 ± 0.02940.1569 ± 0.0389
Leg CountingIDD0.8874 ± 0.02640.3643 ± 0.04900.1727 ± 0.0517
Leg CountingDDD0.8869 ± 0.02550.3629 ± 0.04610.1646 ± 0.0490
Leg CountingFDD0.8871 ± 0.02620.3643 ± 0.04850.1720 ± 0.0514
Color Cube Rotationnotrain0.8098 ± 0.01450.4012 ± 0.02000.1899 ± 0.0119
Color Cube RotationGKD0.8208 ± 0.01320.3609 ± 0.01790.1795 ± 0.0112
Color Cube RotationMiniLLM0.8178 ± 0.01280.3872 ± 0.01770.1931 ± 0.0120
Color Cube RotationIDD0.8252 ± 0.01710.3237 ± 0.01550.2065 ± 0.0126
Color Cube RotationDDD0.8265 ± 0.01620.3190 ± 0.01600.1971 ± 0.0121
Color Cube RotationFDD0.8255 ± 0.01680.3235 ± 0.01580.2048 ± 0.0126
MATH-500notrain0.8151 ± 0.00860.4225 ± 0.01230.1810 ± 0.0109
MATH-500GKD0.8268 ± 0.00870.3590 ± 0.01220.1665 ± 0.0105
MATH-500MiniLLM0.8215 ± 0.00850.3952 ± 0.01160.1779 ± 0.0104
MATH-500IDD0.8373 ± 0.01080.3504 ± 0.01350.2043 ± 0.0136
MATH-500DDD0.8345 ± 0.01070.3485 ± 0.01410.1942 ± 0.0131
MATH-500FDD0.8366 ± 0.01080.3482 ± 0.01390.2022 ± 0.0137

Table B. Draft-JSD-only fidelity to the same saved three-teacher reference.

DatasetMethodSpearman ↑RMSE ↓
ASQAnotrain0.7779 ± 0.01580.0163 ± 0.0003
ASQAGKD0.8032 ± 0.01370.0176 ± 0.0004
ASQAMiniLLM0.7978 ± 0.01090.0182 ± 0.0004
ASQAIDD0.8039 ± 0.00980.0171 ± 0.0004
ASQADDD0.8112 ± 0.00840.0179 ± 0.0004
ASQAFDD0.8040 ± 0.00820.0176 ± 0.0004
GSM8Knotrain0.8291 ± 0.01250.0154 ± 0.0003
GSM8KGKD0.8690 ± 0.01700.0154 ± 0.0003
GSM8KMiniLLM0.8662 ± 0.01670.0162 ± 0.0003
GSM8KIDD0.8823 ± 0.01530.0143 ± 0.0004
GSM8KDDD0.8824 ± 0.01510.0152 ± 0.0005
GSM8KFDD0.8798 ± 0.01490.0142 ± 0.0004
Leg Countingnotrain0.8178 ± 0.02430.0167 ± 0.0014
Leg CountingGKD0.8491 ± 0.02670.0165 ± 0.0016
Leg CountingMiniLLM0.8512 ± 0.02480.0169 ± 0.0016
Leg CountingIDD0.8530 ± 0.02470.0164 ± 0.0022
Leg CountingDDD0.8576 ± 0.02610.0167 ± 0.0026
Leg CountingFDD0.8532 ± 0.02400.0163 ± 0.0022
Color Cube Rotationnotrain0.7993 ± 0.01970.0157 ± 0.0010
Color Cube RotationGKD0.7994 ± 0.02100.0179 ± 0.0011
Color Cube RotationMiniLLM0.8055 ± 0.01660.0184 ± 0.0011
Color Cube RotationIDD0.8100 ± 0.01700.0165 ± 0.0011
Color Cube RotationDDD0.8083 ± 0.01920.0188 ± 0.0012
Color Cube RotationFDD0.8081 ± 0.01940.0165 ± 0.0011
MATH-500notrain0.8029 ± 0.01350.0231 ± 0.0007
MATH-500GKD0.8025 ± 0.01720.0245 ± 0.0007
MATH-500MiniLLM0.7987 ± 0.01580.0253 ± 0.0007
MATH-500IDD0.8201 ± 0.01380.0239 ± 0.0006
MATH-500DDD0.8159 ± 0.01450.0253 ± 0.0008
MATH-500FDD0.8139 ± 0.01420.0238 ± 0.0007

Interpretation. The GSM8K ranking result is present in the disagreement component itself: DDD's JSD-only Spearman correlation is 0.8824±0.01510.8824\pm0.0151, compared with 0.8690±0.01700.8690\pm0.0170 for GKD. Its full-score correlation is 0.8885±0.01750.8885\pm0.0175, compared with 0.8758±0.01480.8758\pm0.0148 for GKD. Thus, this observed ranking difference is not solely due to the added mean-mismatch term. GKD has the lowest mean LpairL_{\mathrm{pair}} across all five datasets. We therefore report ranking fidelity and paired predictive fidelity as distinct outcomes.

This decomposition makes the transfer behavior measurable rather than inferring uncertainty preservation from draft diversity alone. Against the saved reference, JSD-only has substantially lower RMSE than the full score. Accordingly, the full score is not a uniformly better reference-EU estimator. The downstream component comparison in Response 5 answers the separate question of whether adding disagreement improves detection.

2. Robustness to a Larger, Freshly Sampled Reference Family

Thank you for raising sensitivity to the reference family. We evaluate the same fixed trained drafts against a fresh ten-member reference comprising the unperturbed target and nine newly sampled low-rank Gaussian perturbations at σT=0.1\sigma_T=0.1. These new perturbations were not used during draft training, and no draft retraining is performed for the reference comparison. The reference-noise seed is fixed independently of the generation seed and distillation method. The full-score and JSD-only predictions are unchanged; only the teacher-JSD evaluation reference changes.

Table C. Full-score fidelity to the fresh ten-member reference.

DatasetMethodSpearman ↑RMSE ↓
ASQAnotrain0.8353 ± 0.01260.4858 ± 0.0169
ASQAGKD0.8415 ± 0.01360.3424 ± 0.0113
ASQAMiniLLM0.8287 ± 0.01220.4380 ± 0.0101
ASQAIDD0.8446 ± 0.01330.3611 ± 0.0096
ASQADDD0.8482 ± 0.01360.3585 ± 0.0096
ASQAFDD0.8471 ± 0.01290.3610 ± 0.0095
GSM8Knotrain0.8654 ± 0.01380.3700 ± 0.0111
GSM8KGKD0.8866 ± 0.01760.3191 ± 0.0150
GSM8KMiniLLM0.8838 ± 0.01740.3614 ± 0.0102
GSM8KIDD0.8987 ± 0.01980.2799 ± 0.0190
GSM8KDDD0.8972 ± 0.02030.2834 ± 0.0181
GSM8KFDD0.8982 ± 0.01970.2787 ± 0.0191
Leg Countingnotrain0.8496 ± 0.01950.4042 ± 0.0350
Leg CountingGKD0.8719 ± 0.02100.3679 ± 0.0340
Leg CountingMiniLLM0.8765 ± 0.01950.3901 ± 0.0291
Leg CountingIDD0.8983 ± 0.02670.3574 ± 0.0480
Leg CountingDDD0.8968 ± 0.02540.3561 ± 0.0452
Leg CountingFDD0.8980 ± 0.02660.3574 ± 0.0474
Color Cube Rotationnotrain0.8456 ± 0.01630.3914 ± 0.0200
Color Cube RotationGKD0.8487 ± 0.01470.3510 ± 0.0179
Color Cube RotationMiniLLM0.8468 ± 0.01490.3775 ± 0.0177
Color Cube RotationIDD0.8530 ± 0.01210.3138 ± 0.0154
Color Cube RotationDDD0.8532 ± 0.01300.3092 ± 0.0159
Color Cube RotationFDD0.8538 ± 0.01160.3136 ± 0.0157
MATH-500notrain0.8380 ± 0.00750.4123 ± 0.0119
MATH-500GKD0.8420 ± 0.00950.3491 ± 0.0120
MATH-500MiniLLM0.8379 ± 0.00880.3854 ± 0.0113
MATH-500IDD0.8526 ± 0.01070.3405 ± 0.0132
MATH-500DDD0.8483 ± 0.01090.3388 ± 0.0138
MATH-500FDD0.8517 ± 0.01080.3384 ± 0.0135

Table D. Draft-JSD-only fidelity to the fresh ten-member reference.

DatasetMethodSpearman ↑RMSE ↓
ASQAnotrain0.8222 ± 0.00990.0246 ± 0.0004
ASQAGKD0.8416 ± 0.01800.0266 ± 0.0005
ASQAMiniLLM0.8330 ± 0.01970.0289 ± 0.0005
ASQAIDD0.8329 ± 0.01320.0254 ± 0.0005
ASQADDD0.8364 ± 0.01630.0283 ± 0.0005
ASQAFDD0.8289 ± 0.01260.0255 ± 0.0005
GSM8Knotrain0.8564 ± 0.01090.0233 ± 0.0004
GSM8KGKD0.8821 ± 0.01660.0255 ± 0.0005
GSM8KMiniLLM0.8827 ± 0.01400.0265 ± 0.0005
GSM8KIDD0.8941 ± 0.01610.0224 ± 0.0006
GSM8KDDD0.8970 ± 0.01340.0252 ± 0.0008
GSM8KFDD0.8898 ± 0.01600.0225 ± 0.0006
Leg Countingnotrain0.8444 ± 0.01870.0236 ± 0.0017
Leg CountingGKD0.8573 ± 0.02560.0263 ± 0.0024
Leg CountingMiniLLM0.8622 ± 0.02260.0269 ± 0.0024
Leg CountingIDD0.8657 ± 0.02160.0237 ± 0.0026
Leg CountingDDD0.8676 ± 0.02260.0267 ± 0.0036
Leg CountingFDD0.8630 ± 0.02270.0239 ± 0.0026
Color Cube Rotationnotrain0.8460 ± 0.01180.0266 ± 0.0009
Color Cube RotationGKD0.8259 ± 0.01590.0307 ± 0.0012
Color Cube RotationMiniLLM0.8419 ± 0.01000.0315 ± 0.0012
Color Cube RotationIDD0.8454 ± 0.00920.0279 ± 0.0010
Color Cube RotationDDD0.8481 ± 0.01100.0319 ± 0.0013
Color Cube RotationFDD0.8395 ± 0.00740.0280 ± 0.0011
MATH-500notrain0.8405 ± 0.01270.0334 ± 0.0008
MATH-500GKD0.8161 ± 0.02000.0378 ± 0.0011
MATH-500MiniLLM0.8151 ± 0.01540.0391 ± 0.0011
MATH-500IDD0.8417 ± 0.01270.0349 ± 0.0009
MATH-500DDD0.8347 ± 0.01410.0389 ± 0.0012
MATH-500FDD0.8330 ± 0.01450.0351 ± 0.0009
Reviewer_JVptOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper estimates token-level epistemic uncertainty of a large LLM from a few small draft models. The score has two parts: the JSD among the drafts, read as variance, and the KL between the draft mixture and a distilled proxy of the target mean, read as bias. The authors show the decomposition is an exact identity and give a bound under which the draft JSD approximates the EU of a finite family of teachers, provided each draft is close to its own teacher. In practice the proxy is trained with Online Stochastic Distillation and the drafts on disjoint data partitions, which they call DDD. The reference for evaluation is three copies of the target with Gaussian noise added to the weights. On GSM8K and ASQA, DDD matches this reference better than the other distillation variants and than GKD and MiniLLM. On one Llama pair, the summed score detects wrong answers about as well as TokUR on GSM8K and worse than TokUR on ASQA.

Critical strengths
  1. The theory is honest about what it proves. The decomposition is an identity, the bound in Theorem 2.4 is conditional on per-teacher fidelity, and the paper says in plain terms that fitting the mean alone gives zero disagreement. I rarely see this stated so clearly in draft-disagreement work.

  2. DDD improves agreement with the reference in every pair and dataset in Table 1, including the 30B target.

Critical weaknesses
  1. The main evaluation does not necessarily examine epistemic uncertainty. Section 3.4 states that each draft group is optimized to mimic one of those copies, and the only teacher-generated datasets described in Appendix B.1 come from the same three copies. Table 1 therefore measures how well a draft imitates a teacher it was fitted to. To my understanding at least, the paper never tests whether the score is high when the target is wrong and low when it is right, apart from Table 2, where the full score is within noise of its bias term alone.

  2. The numbers in Table 1 do not match the definition of the reference. A JSD over three members is at most log 3 per token. The reported RMSE reaches 1.07 for DDD and 2.84 for untrained drafts. Either the scores are summed over sequences, which the paper does not state, or the estimate is off by more than the whole range of the target. Table 5 reports RMSE of 0.01 in a different setting with no explanation of the scale. The baselines are also not comparable: IDD and FDD drafts receive noise of 0.5 against 0.1 for the reference, and the paper does not say how GKD and MiniLLM, which train one student, become a family of six.

Questions for authors
  1. Was each DDD draft trained on text generated by one specific member of the reference family? If yes, each draft group is scored against the teacher it was trained to imitate, and the comparison with GKD and MiniLLM is not like for like.
  2. Is the RMSE in Table 1 computed on token-level scores or on sequence sums? How can it exceed log 3 if the former?
  3. Can you report hallucination detection for the other three pairs and in the 2x3 setting of the main results? One pair with a standard deviation of 0.10 on ASQA does not support a conclusion.
6 comments

Response to Reviewer JVpt

Thank you for the detailed questions. Our practical objective is to approximate teacher-family token-level uncertainty using a lightweight draft ensemble and a learned predictive-mean proxy, replacing repeated perturbed-target scoring passes. The contribution is assessed through reference fidelity and detection at the measured computational cost. Below, we directly address fresh-reference generalization, teacher assignment, initialization-noise sensitivity, the RMSE definition, and expanded 2×3 detection. The submitted 1×3 cost–performance comparison is discussed separately in Response 6, so its timing evidence is not conflated with the expanded configuration.

1. Reference Generalization and Hallucination Detection

We address the two questions with separate evidence: fixed trained drafts are evaluated against fresh reference members to test transfer beyond the training perturbations, and their scores are evaluated against target-response labels to test practical error detection. Table 1 measures fidelity to the specified reference, while label-based detection is an additional empirical criterion.

Generalization to fresh reference members. We evaluate the fixed trained drafts against a ten-member reference comprising the unperturbed target and nine newly sampled Gaussian perturbations. These new perturbations were not used for draft training. This tests generalization beyond the particular perturbed teachers used for distillation, within the same perturbation construction; it does not establish that this construction is an exact Bayesian posterior.

Additional datasets and transfer protocol. Leg Counting and Color Cube Rotation are procedurally generated tasks from Reasoning Gym, evaluating arithmetic counting and spatial state tracking, respectively. We additionally evaluate on MATH-500. For all three datasets, we reuse the GSM8K-trained draft adapters and the GSM8K-trained predictive-mean proxy pmixp_{\mathrm{mix}}, without dataset-specific distillation or fine-tuning. These are cross-task transfer evaluations. ASQA uses the ASQA-trained modules.

Fresh-reference EU estimation: Llama-3.1-8B → Llama-3.2-1B, 2×3 configuration. Values are means ± sample standard deviations across five generation seeds using fixed trained checkpoints. The fresh reference uses σT=0.1\sigma_T=0.1; IDD and FDD use draft initialization noise σD=0.1\sigma_D=0.1. Metrics pool evaluated prompt and completion token positions, including length-capped responses. This table reports the full score against the fresh reference.

DatasetMethodSpearman ↑RMSE ↓
ASQAnotrain0.8353 ± 0.01260.4858 ± 0.0169
ASQAGKD0.8415 ± 0.01360.3424 ± 0.0113
ASQAMiniLLM0.8287 ± 0.01220.4380 ± 0.0101
ASQAIDD0.8446 ± 0.01330.3611 ± 0.0096
ASQADDD0.8482 ± 0.01360.3585 ± 0.0096
ASQAFDD0.8471 ± 0.01290.3610 ± 0.0095
GSM8Knotrain0.8654 ± 0.01380.3700 ± 0.0111
GSM8KGKD0.8866 ± 0.01760.3191 ± 0.0150
GSM8KMiniLLM0.8838 ± 0.01740.3614 ± 0.0102
GSM8KIDD0.8987 ± 0.01980.2799 ± 0.0190
GSM8KDDD0.8972 ± 0.02030.2834 ± 0.0181
GSM8KFDD0.8982 ± 0.01970.2787 ± 0.0191
Leg countingnotrain0.8496 ± 0.01950.4042 ± 0.0350
Leg countingGKD0.8719 ± 0.02100.3679 ± 0.0340
Leg countingMiniLLM0.8765 ± 0.01950.3901 ± 0.0291
Leg countingIDD0.8983 ± 0.02670.3574 ± 0.0480
Leg countingDDD0.8968 ± 0.02540.3561 ± 0.0452
Leg countingFDD0.8980 ± 0.02660.3574 ± 0.0474
Color cube rotationnotrain0.8456 ± 0.01630.3914 ± 0.0200
Color cube rotationGKD0.8487 ± 0.01470.3510 ± 0.0179
Color cube rotationMiniLLM0.8468 ± 0.01490.3775 ± 0.0177
Color cube rotationIDD0.8530 ± 0.01210.3138 ± 0.0154
Color cube rotationDDD0.8532 ± 0.01300.3092 ± 0.0159
Color cube rotationFDD0.8538 ± 0.01160.3136 ± 0.0157
MATH500notrain0.8380 ± 0.00750.4123 ± 0.0119
MATH500GKD0.8420 ± 0.00950.3491 ± 0.0120
MATH500MiniLLM0.8379 ± 0.00880.3854 ± 0.0113
MATH500IDD0.8526 ± 0.01070.3405 ± 0.0132
MATH500DDD0.8483 ± 0.01090.3388 ± 0.0138
MATH500FDD0.8517 ± 0.01080.3384 ± 0.0135

The fixed drafts retain reference-ranking information on fresh perturbation members: on GSM8K, DDD's mean Spearman correlation is 0.8972, versus 0.8866 for GKD. This result addresses dependence on the particular training perturbations within the chosen family. The table does not establish uniform DDD superiority.

Association with target errors. We also report the mean and sample standard deviation of sequence-mean EU separately for correct and incorrect target responses. For each response, we first mean-pool the trained-draft token-level EU over completion positions. We then compute the group statistics across responses pooled over three generation seeds, after truncation filtering. The reported SD describes variation across responses, not variation across seed averages. For ASQA, the two groups are non-hallucinated and hallucinated; the non-hallucinated group includes incomplete and abstained responses and should not be interpreted as strictly correct answers.

Model PairDatasetCorrect / non-hallucinated: Mean ± SDIncorrect / hallucinated: Mean ± SD
30B → 1.7BGSM8K0.1930 ± 0.08100.2564 ± 0.0950
30B → 1.7BASQA0.5744 ± 0.26110.5595 ± 0.2625
30B → 1.7BLeg Counting0.2205 ± 0.06070.2435 ± 0.0736
30B → 1.7BColor Cube Rotation0.4188 ± 0.06400.4045 ± 0.0630
30B → 1.7BMATH5000.2428 ± 0.10810.3142 ± 0.1203
8B → 1BGSM8K0.1241 ± 0.04600.1586 ± 0.0589
8B → 1BASQA0.3135 ± 0.12290.3626 ± 0.1496
8B → 1BLeg Counting0.1202 ± 0.03890.1168 ± 0.0403
8B → 1BColor Cube Rotation0.2968 ± 0.06700.2936 ± 0.0696
8B → 1BMATH5000.1148 ± 0.06680.1477 ± 0.0659
3B → 1BGSM8K0.1129 ± 0.03950.1471 ± 0.0468
3B → 1BASQA0.2368 ± 0.08690.2570 ± 0.0948
3B → 1BLeg Counting0.1130 ± 0.04400.1079 ± 0.0390
3B → 1BColor Cube Rotation2.0978 ± 0.91512.0138 ± 0.9896
3B → 1BMATH5000.0989 ± 0.04900.1213 ± 0.0561

Incorrect responses have higher mean EU on GSM8K and MATH-500 for all three reported model pairs. This direction is not uniform across ASQA and the procedural tasks. We therefore interpret these statistics as a task-dependent association, rather than a universal rule that every incorrect response receives higher uncertainty.

The complete 2×3 detection results and the Full/KL-only/JSD-only ablation are provided in Response 5. Within each target-model and dataset setting, these three scores are evaluated on the same generated responses with the same test-response IDs and labels. Their detection performance is assessed separately from the descriptive sequence-mean statistics.

Incremental value of disagreement. The reviewer correctly identifies a separate component question: whether JSD adds detection value beyond KL-only. Neither the original table nor the expanded matched comparison establishes a consistent benefit. We will limit that claim while retaining the independently assessed contributions of reference-ranking transfer and the measured cost–performance trade-off. A mathematical decomposition and its transfer conditions do not require both components to improve every downstream classifier.

2. Teacher Assignment and Construction of the Baselines

Yes, each DDD draft group is trained using text generated by its assigned teacher member. Importantly, GKD and MiniLLM also use the three reference teachers, with two separately trained students per teacher, yielding the same six-student, 2×3 configuration. Teacher-specific supervision is therefore shared across these methods. Their students collect stochastic rollouts during distillation and receive feedback from their assigned teacher; these are six separately trained students, not six sampled responses from one trained student.

Our purpose is to compare distillation strategies for preserving the predictive variation of the specified teacher family. The methods differ in their training procedure, including the rollout distribution, but use the same teacher-to-student allocation and ensemble size. We will explicitly document this extension of GKD and MiniLLM and avoid attributing the observed differences solely to on-policy versus off-policy training.

3. Controlling the Draft Initialization Noise

We tested the draft-initialization choice directly, holding the teacher reference at σT=0.1\sigma_T=0.1. The teacher scale defines the reference family and the draft scale controls initialization diversity, but the numerical draft setting must be evaluated empirically.

We therefore conducted an initialization-noise sensitivity experiment for IDD on GSM8K, using Llama-3.1-8B → Llama-3.2-1B with the 2×3 configuration. We fixed the reference teacher family at σT=0.1\sigma_T=0.1 and evaluated the draft-noise settings on the same generated sequences. Values are means ± sample standard deviations across five generation seeds using fixed trained checkpoints, rather than five independent retrainings.

Draft noise σD\sigma_DFull-score RMSE ↓Spearman ↑
0.050.2840 ± 0.01910.8888 ± 0.0181
0.10.2866 ± 0.01940.8899 ± 0.0176
0.50.5800 ± 0.04120.7889 ± 0.0404
1.03.8224 ± 0.19920.3102 ± 0.1378

Reducing σD\sigma_D from 0.5 to 0.1 substantially improves both RMSE and Spearman correlation. The 0.05 and 0.1 settings give similar results, with different metrics favoring different means. The fresh-reference comparison in Response 1 uses the matched σD=0.1\sigma_D=0.1 setting for IDD/FDD. This directly addresses the baseline-setting concern: the updated evidence includes the stronger low-noise baselines and supports a metric- and dataset-dependent comparison, rather than relying on the original 0.5 setting to argue for DDD.

4. Token-Level RMSE and the ln⁡3\ln 3 Bound

Thank you for highlighting this ambiguity. Table 1 reports token-level RMSE, not RMSE over sequence sums:

RMSE=1N∑t∈T(u^t−utref)2.\mathrm{RMSE} = \sqrt{\frac{1}{N}\sum_{t\in\mathcal T} \left(\widehat u_t-u_t^{\mathrm{ref}}\right)^2}.

Here, T\mathcal T indexes all evaluated token positions across responses, and N=∣T∣N=|\mathcal T|.

The reference utrefu_t^{\mathrm{ref}} is the JSD of three teacher distributions and is indeed bounded by ln⁡3\ln 3. However, our full estimator is

u^t=JSD⁡(q1,t,…,qK,t)+KL⁡(qmix,t∥pmix,t).\widehat u_t = \operatorname{JSD}(q_{1,t},\ldots,q_{K,t}) + \operatorname{KL}\left(q_{\mathrm{mix},t}\Vert p_{\mathrm{mix},t}\right).

The additional KL mismatch term is not bounded by ln⁡3\ln 3. Consequently, neither the estimator nor its RMSE is constrained by the reference's range. Values exceeding ln⁡3\ln 3 are therefore mathematically possible without sequence summation. They nevertheless indicate substantial absolute estimation error, which we acknowledge. We will clarify the aggregation formula and the distinct ranges of the reference and estimator in the revision.

5. Expanded 2×3 Hallucination and Error-Detection Results

We report the 2×3 results for Qwen3-30B → Qwen3-1.7B, Llama-3.1-8B → Llama-3.2-1B, and Llama-3.2-3B → Llama-3.2-1B. For mathematical and procedural tasks, the labels indicate answer correctness; for ASQA, they indicate hallucinated versus non-hallucinated responses. Leg Counting, Color Cube Rotation, and MATH-500 use the GSM8K-module transfer protocol described in Response 1.

Detection protocol. The reported tables use the combined sequence-aggregation setting. For each token-level signal, seven aggregation features—mean, sum, maximum, median, top-k mean, 95th percentile, and threshold-exceedance proportion—are supplied to a logistic regression classifier. Feature standardization and the classifier are fitted on the training split. AUROC is computed from predicted error probabilities; ECE and Brier score use predicted correctness probabilities, with ASQA correctness interpreted as non-hallucination. These classifier results are distinct from the raw sequence-mean statistics in Response 1 and do not by themselves establish that higher raw EU always predicts an error.

Within each target-model and dataset setting, Full, KL-only, and JSD-only use identical test responses and labels. Values are means ± sample standard deviations across three generation seeds using fixed trained modules; they are not confidence intervals or independent training repetitions. Draft Disagreement denotes JSD-only, and Target Mismatch denotes KL-only.

Main Benchmark Results

Qwen3-30B → Qwen3-1.7B

GSM8K and ASQA

MethodGSM8K AUROC ↑GSM8K ECE ↓GSM8K Brier ↓ASQA AUROC ↑ASQA ECE ↓ASQA Brier ↓
MSP0.861 ± 0.0450.044 ± 0.0130.051 ± 0.0100.608 ± 0.0460.054 ± 0.0170.241 ± 0.008
Total Entropy0.828 ± 0.0610.032 ± 0.0110.050 ± 0.0110.658 ± 0.0160.090 ± 0.0500.224 ± 0.001
Self-Certainty0.829 ± 0.0260.032 ± 0.0030.050 ± 0.0080.617 ± 0.0080.085 ± 0.0230.239 ± 0.002
Semantic Entropy0.679 ± 0.0220.030 ± 0.0100.046 ± 0.0010.607 ± 0.0340.054 ± 0.0320.241 ± 0.004
KLE0.778 ± 0.0100.029 ± 0.0080.053 ± 0.0060.777 ± 0.0260.106 ± 0.0080.198 ± 0.008
Eccentricity0.542 ± 0.0760.001 ± 0.0000.057 ± 0.0030.650 ± 0.0640.084 ± 0.0120.235 ± 0.012
TokUR0.777 ± 0.0230.038 ± 0.0080.054 ± 0.0060.542 ± 0.0240.046 ± 0.0320.249 ± 0.001
Draft Disagreement0.827 ± 0.0260.033 ± 0.0100.054 ± 0.0090.529 ± 0.0300.045 ± 0.0160.249 ± 0.002
Target Mismatch0.827 ± 0.0570.031 ± 0.0020.054 ± 0.0070.545 ± 0.0420.064 ± 0.0240.248 ± 0.004
Full EU(Ours)0.831 ± 0.0600.036 ± 0.0100.054 ± 0.0080.543 ± 0.0390.053 ± 0.0230.248 ± 0.004

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.