acceptodds
Under review as a conference paper at ICLR 2027

Semantic Uncertainty Probes Across Language Models

Abstract

Semantic entropy is a useful measure of language-model uncertainty, but computing it requires generating and comparing multiple answers. Semantic entropy probes instead predict this uncertainty from a single forward pass, but they are normally trained separately for each model using expensive semantic entropy labels. We study whether a probe trained on one model can be reused on another. Our method learns a lightweight linear map between paired source and target hidden states, without using target-side semantic entropy labels, and then applies the fixed source probe in the aligned target space. Across 21 ordered source–target pairs spanning different model families and scales, transferred probes recover much of the performance of probes trained directly on the target model and sometimes match or outperform them. To understand these results, we partition prediction outcomes according to source-probe errors, transfer-induced changes, and disagreement between source and target uncertainty labels. On average, transfer-induced changes are more often corrective than harmful, while large source–target disagreement limits performance. These findings support the practical reuse of semantic entropy probes and suggest that language models encode partially shared and linearly recoverable uncertainty signals.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, hallucinations, linear probing, uncertainty quantification, semantic entropy

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RTEBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propose a probe transfer method for semantic entropy for LLM to avoid retraining. It's contribution include an affine map fitted by ridge least squares. and a decomposition of transfer outcomes, transferred error = E xor F xor D. It reports corrective versus harmful flips and the net accuracy change. It evaluates AUROC of the transferred probe of 21 ordered pairs from six open source models on 3 dataset.

Critical strengths
  1. A clean accounting of transfer outcomes, separating source-probe error, transfer flips and label disagreement (Eq. 6 to 12) with sufficient exp supports and negative cases are shown.
  2. A label-free, cheap method. the label-free layer choice nearly matches oracle layer choice with about 11.7 to 11.9 times lower cost.
Critical weaknesses
  1. There might be selection on the evaluation data or an unequal tuning budget issue. No seeds or intervals appear in the text or table. The ridge strength λ\lambda = 10⁴ was selected by transfer AUROC on the NQ test set (Appendix B.1) and reused elsewhere, while native probes keep default regularisation (Table 3).
  2. It is unclear if the transfer uses the target model’s own uncertainty, or only the fact that some questions are hard for every model. The Δ\Deltaacc is only compared with the source probe’s prediction, not with a target-trained probe or a majority-class guess, and I found no shuffled-pair control or pseudo-label baseline.
Questions for authors
  1. λ\lambda = 10⁴ was chosen on the NQ test set. Can you choose λ\lambda on a held-out split, tune the native probe's regularisation the same way, and give AUROC with 95% CI over test items?
  2. Can you add native-target and majority-class accuracy rows to the decomposition tables, and state the class balance of the high v.s. low labels? Can you also add a shuffled-pair control, a pseudo-label baseline ?
  3. How were the 2000 inputs per dataset drawn (which split, which seed)? Are the inputs and the 500 test items the same for every model and pair?
1 comment
AuthorsOpenReview ↗

Response to Reviewer 3

We thank the reviewer for the careful review and for recognizing our error decomposition. We address each point below.

On λ selection, native regularization (W1, Q1). Yes. In the submitted version, λ was chosen from a 5-value grid by transfer AUROC on the NQ test set and reused for all settings. We have now re-selected it on NQ without test labels, using two procedures: a label-free one (held-out reconstruction error on the non-test inputs, choosing the target layer and λ jointly, as in our layer selection) and validation AUROC on a 70/30 split of the non-test inputs. In both cases, the map is refit on all 1500 non-test inputs and evaluated once on the test set. Both procedures select λ = 10⁴, the value used in the paper; the label-free procedure selects it in all 21 pairs.

We also tuned the native probe's regularization by stratified cross-validation within its own training labels, at every budget, and now report AUROC with 95% paired-bootstrap CIs over the 500 test questions (general response, points 1–2). Transfer remains close to the native probe with default regularization, but tuning substantially strengthens the native probe at larger budgets: at n = 1500, it reaches .825 [.805, .846] AUROC, against .773 [.751, .794] for transfer. We will revise our claims to reflect this gap. At smaller budgets, tuning does not consistently improve the native probe. The practical benefit of transfer is that it needs no target SE labels, and it compares favourably at matched labeling cost (125 labels).

On confidence intervals (W1). We resample the 500 test-question indices with replacement (2,000 times), jointly across all pairs and methods. These intervals quantify uncertainty over test questions, rather than variability across training runs.

On shared difficulty vs. target-specific uncertainty (W2). Source and target SE labels agree on most inputs (about 71–75% on average), and this is where transfer helps. On inputs where they disagree, corrective and harmful flips largely cancel. We therefore interpret our results cautiously: the evidence for beneficial transfer comes primarily from inputs on which the models' uncertainty labels agree, and we will state this explicitly.

On accuracy baselines and class balance (Q2). We added these (NQ, n = 1500, mean over 21 pairs):

Accuracy (%)
Majority class60.5
Transferred probe (0 target labels)68.2
Native target probe, default C72.1
Native target probe, tuned C76.0

High-entropy labels make up 58–63% of the test labels for four of the five target models and 41% for Qwen3-8B. The majority class is chosen from the 1500 non-test labels. The transferred probe beats the majority class in 18 of 21 pairs. The native probe, trained on 1500 target labels at the same layer, is more accurate, consistent with our AUROC results. We will add these rows to the decomposition tables.

On the shuffled-pair control (Q2). To test whether transfer relies on question-level correspondence, we refit the alignment map on deliberately mismatched pairs, keeping everything else fixed (same layers, λ = 10⁴ and source probe). Target states were randomly permuted relative to source states over the 1500 non-test questions, with 5 seeds. On NQ (n = 1500, mean over 21 pairs, 95% bootstrap CIs over test questions), transferred AUROC falls from .773 [.751, .794] to .498 [.490, .507], i.e. chance level, and every pair's mean lies between .47 and .53. Transfer therefore depends on how the same question is represented in both models, not on their overall statistics.

On the pseudo-label baseline (Q2). We train a probe on the target model's hidden states (same layer as the transferred probe) using the source model's SE labels as pseudo-labels, with regularization tuned on those labels. On NQ, transfer outperforms this baseline at every evaluated budget. At n = 1500, the baseline's mean AUROC is .763 [.741, .785], compared with .773 [.751, .794] for transfer; the difference is +.010 [+.006, +.014] (95% paired-bootstrap CIs over test questions). This supports an advantage over directly training a target probe on source SE labels. We will add the full results for all budgets to the revision.

On data (Q3). We follow the data-generation code of Han et al. (2024b). For each dataset, 2,000 questions are sampled without replacement (seed 20) from an evaluation split: the NQ-Open validation split, the SQuAD v2.0 validation split, and, for TriviaQA, a held-out 20% split (seed 20). The 2,000 questions are then split once, with a fixed seed, into 500 test and 1,500 training questions. All models use the same 2,000 questions, prompt and split, so every pair shares the same 500 test questions; we verified that the question ids are identical across all six models.

Reviewer_mCzHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors look into whether a semantic entropy probe trained on one model can be reused on another model without computing semantic entropy labels for the second model. In their experiments both models are run on the same 1500 unlabeled prompts. A ridge regularized affine map from target to source hidden states is fit on these pairs. The frozen source probe is then applied to the mapped target states. The target layer is chosen by an alignment loss. They report 21 ordered source target pairs for five 8-14B models and 2 Llama models on TriviaQA, SQuAD and NQ,. The transferred probes often approach, and sometimes exceed, the AUROC of probes trained on target SE labels. Alignment on a different dataset degrades results, though. A decomposition of binary outcomes shows a net accuracy gain under same dataset alignment. They reports a 12x wall clock saving over generating target these SE labels.

Critical strengths
  • Semantic entropy approaches are expensive, and an approach to could mitigate these costs are valuable. The authors use many models and datasets that make their paper's findings more robust.

  • The label-free layer selection looks to work well and is a useful practical finding.

Critical weaknesses
  • There are baselines missing the make it uncertain the alignment adds value. These are examples I would have liked to have seen in the paper.

    • (a) A target probe trained on the source semantic entropy labels for the same 1500 prompts. These labels should already exist.
    • (b) A cost matched target SEP ( like 125 target semantic entropy labels, matching the ~12× cost ratio) with tuned regularization. The latter is emphasized to differentiate from Figures 3 through 5.
    • (c) A target probe trained on the correctness of the target's own answer. This is also presumably already generated to extract second to last token states.
    • (d) Single pass scores. Length-normalized log-likelihood, token entropy, P(True).
  • The papers decomposition points to shared question difficulty rather than target specific uncertainty. The practical motivation is a bit questionable. Alignment requires in domain prompts run through the target model. The cross dataset alignment degrades results. Is question difficulty being shared our the models’ actual uncertainty. Where source and target labels disagree, the transferred probe agrees with the target only about 48%/41%/42% of the time for TriviaQA/SQuAD/NQ in Tables 4/7/10 respectively, and it is closer to the source model's labels.

Questions for authors
  • Was the regularization strength lambda selected on the Natrual Questions test set? If so can you rerun without doing that?
  • Transferring steering vectors across model families with learned mappings between models has been shown [1, 2]. It has also been shown that probes on one model's hidden states can predict another model's uncertainty [3]. Can you position your paper against these?

[1] Y. Huang, C. Huang, D. Feng, W. Lei, J. Lv. Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts. ACL 2025. arXiv:2501.02009. [2] N. Oozeer, D. Nathawani, N. Prakash, M. Lan, A. Harrasse, A. Abdullah. Activation Space Interventions Can Be Transferred Between Large Language Models. ICML 2025. arXiv:2503.04429. [3] G. Ahdritz, T. Qin, N. Vyas, B. Barak, B. L. Edelman. Distinguishing the Knowable from the Unknowable with Language Models. ICML 2024, PMLR 235:503–549.

1 comment
AuthorsOpenReview ↗

Response to Reviewer 2

We thank the reviewer for the careful and constructive review, and for recognizing the practical value of reducing SE costs, the breadth of models and datasets, and our label-free layer selection. We address each point below.

On additional baselines.

(a) We added this baseline: a probe trained on the target's hidden states (same layer as the transferred probe) using the source model's SE labels, with regularization tuned on those labels. On NQ, transfer outperforms this baseline at every evaluated budget. At n = 1500, its mean AUROC is .763, compared with .773 for transfer. This supports an advantage over directly training a target probe on source SE labels.

(b) The native target probe in Figures 2–5 is trained on n = 50 to 1500 target labels, covering the cost-matched regime (~125 labels). We now report this directly (general response): with 125 target labels, the native probe reaches .690 (tuned C) or .712 (default C) on NQ, against .773 for transfer with no target labels. We also added cross-validated regularization for the native probes at all budgets.

(c, d) Our goal is to make SEPs reusable across models, not to re-establish SEPs as an uncertainty method. Han et al. (2024b) already compared SEPs with accuracy probes and with single-pass scores such as log-likelihood, token entropy and P(True). We take SEPs as given and ask whether an existing SEP can be reused on a new model without new SE labels. In addition, correctness probes require gold answers, which are unavailable for deployment queries, while SE labels need only the model’s own samples.

On shared difficulty vs. target-specific uncertainty. We agree with this reading of the decomposition, and our analysis supports it. On inputs where source and target labels disagree, transfer does not recover the target’s uncertainty: the transferred probe agrees with the target on only 48%, 41% and 42% of them, as the reviewer notes, and we do not claim a target-specific signal there. However, these inputs are a minority: source and target SE labels agree on about 71–75% of inputs on average. Shared question difficulty is therefore not separate from the target’s uncertainty but its largest component, which is why the transferred probe approaches native AUROC against the target’s own labels. We will state this explicitly in Section 4.2 and add strong source–target disagreement as a limitation.

On the practical motivation: alignment requires only unlabeled inputs, with no sampling or SE labels. Once deployed, the target model’s own queries can serve as in-domain alignment data, passed once through both models. Cross-dataset alignment does reduce accuracy, but the drop is much smaller in AUROC (Figures 3–5).

On λ selection. Yes. In the submitted version, λ was chosen from a 5-value grid by transfer AUROC on the NQ test set and reused for all settings; the submitted results were fairly stable around this choice (Table 3). We have now re-run the selection on NQ without test labels, using two procedures: a label-free one, which chooses the target layer and λ jointly by held-out reconstruction error on the non-test inputs (as in our layer selection), and one using validation AUROC with target labels from the non-test split only. In both cases, the map is refit on all 1500 non-test inputs and evaluated once on the test set. Both procedures select λ = 10⁴, the value used in the paper. Details are in the general response.

On positioning against [1, 2, 3]. We thank the reviewer and will add these works to our related work. Our contribution is not cross-model alignment itself, but its use for label-free transfer of a model-specific uncertainty predictor. [1] and [2] transfer steering vectors through learned mappings between models (e.g., for concepts or safety behaviors). These are behavioral interventions that write a direction into the target model. We instead transfer a predictive probe, whose target, semantic entropy, is model-dependent: the same input can have different labels for source and target. [3] is closer: it trains a new probe on one model’s hidden states, using labels from a larger reference model, to predict that model’s token-level confidence and separate epistemic from aleatoric uncertainty. In contrast, we reuse an existing SEP trained for the source model to predict the target model’s own sequence-level semantic entropy, without retraining the probe and without target labels. Our error decomposition then characterizes when this transfer helps and when it cannot. Finally, [3] reports that uncertainty is strongly correlated across models, which is consistent with our observations.

Reviewer_t3LmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether semantic entropy probes (SEPs), which predict a model's uncertainty from its hidden representations, can be transferred across different language models. Specifially, they propose learning a linear mapping between the hidden representations of a source and target model, and then applying the source probe (previously trained) to the target model. They evaluate the approach model pairs and show that transferred probes can achieve performance comparable to probes trained directly on the target model, without requiring target semantic entropy labels during training. They also analyze how differences between the source and target models' uncertainty affect transfer.

Critical strengths
  • Semantic entropy is a popular method, but is useful due to having to generate the labels needed for training SE probes. The paper addresses an interesting and practical problem by studying whether uncertainty probes can be reused across models.
  • The proposed method is simple, and I think it's interesting that linear alignment can transfer uncertainty related information across models from different families, without retraining the original probe.
  • The paper includes a fairly extensive evaluation across different model pairs and datasets. I also appreciate that the authors study cases where transfer can improve or hurt the predictions, rather than only reporting aggregate performance.
Critical weaknesses
  • One concern I have is that semantic entropy is inherently dependent on the model being evaluated. Different models can have very different response distributions (https://arxiv.org/pdf/2604.17112), and therefore different uncertainty levels, on the same input. However, the proposed alignment objective tries to reconstruct the source model's hidden representations from the target model's, rather than directly predicting the target model's uncertainty. I am not sure why this should work when the models have substantially different semantic entropies. While Section 4 studies disagreement between binary entropy labels, I think the paper could benefit from a more direct analysis of performance as a function of the difference between source and target semantic entropy.
  • I think some important baselines are missing. For example, one simple baseline would be to apply the source probe directly to the source model on the same questions, and evaluate how well its predictions correlate with the target model's entropy labels. This would help understand whether aligning the representations shows anything about the target model's uncertainty, or whether the results mostly come from the two models having similar uncertainty scores. It would also be useful to compare against training a target probe with a much smaller number of target entropy labels.
  • The main method uses a standard linear alignment between model representations, which has already been studied for transferring probes across language models. While applying this approach to semantic entropy is interesting, I would have liked to see more evidence that the method offers something beyond existing probe transfer approaches.
  • The performance seems quite sensitive to the dataset used for alignment. For example, on TriviaQA, the average improvement in target classification accuracy from transfer is higher when alignment is performed on TriviaQA, but decreases when using NQ and SQuAD for alignment. This seems like a significant limitation, since in practice we might not have unlabeled examples that closely match the deployment distribution.
  • In Appendix B.1, the authors mention that the regularization parameter (λ=104\lambda=10^4) is selected based on transfer AUROC on the NQ test set. However, this requires access to the target model's semantic entropy labels. The results are also quite sensitive to this parameter (e.g. on NQ, AUROC changes a lot for different values of λ\lambda at the largest probe budget). I think this is an important issue, and the authors should evaluate the method with hyperparameters selected independently of the target test labels.
Questions for authors

See weaknesses above.

1 comment
AuthorsOpenReview ↗

Response to Reviewer 1

We thank the reviewer for their constructive feedback. We address each concern below.

On the model-dependence of semantic entropy. We agree that semantic entropy depends on the model. However, the models we considered agree most of the time: on average, source and target SE labels match on 71–75% of test inputs, depending on the dataset. The disagreement rate is about 25% on TriviaQA, 26% on SQuAD and 29% on NQ (the sum of the D_only, ED_only, FD_only and EFD columns in Tables 4, 7 and 10).

On inputs where source and target labels disagree, corrective and harmful flips have a small net effect. This decomposition alone does not establish whether the corrective flips reflect target-specific uncertainty information. We therefore interpret these results cautiously: the evidence for beneficial transfer comes primarily from inputs on which the models’ uncertainty labels agree.

As a result, the more the models disagree, the worse the transfer. Across pairs, the disagreement rate is strongly correlated with transfer error on SQuAD and NQ. The clearest example is Llama-3.2-1B → Llama-3.1-8B on TriviaQA: its labels disagree on 39.4% of inputs, and its transfer error is 50.6%. We will revise Section 4.2 to state this explicitly and add strong source–target disagreement as a limitation.

On missing baselines.

Source probe on the source model, evaluated against target labels. This baseline is the reference in our error decomposition: Δacc is exactly the improvement of the transferred probe over it. In the submitted tables, its error against target SE labels is E_only + EF_only + D_only + FD_only, which gives 27.2%, 28.5% and 34.9% on TriviaQA, SQuAD and NQ. The transferred probe's error is 23.9%, 28.3% and 31.3%.

Following the reviewer's suggestion, we also evaluate it by AUROC against the target's SE labels (NQ; see general response; mean over 21 pairs, 95% paired bootstrap CIs over test questions):

nTransferredSource probe on source modelDifference
50.663 [.639, .686].641 [.618, .664]+.022 [+.009, +.035]
200.715 [.694, .739].690 [.668, .712]+.026 [+.012, +.038]
800.757 [.734, .780].717 [.695, .740]+.040 [+.027, +.053]
1500.773 [.751, .794].720 [.699, .742]+.052 [+.039, +.065]

Transfer outperforms this baseline at every budget, and in all 21 pairs from n = 400 upward. The gap grows with n: the baseline levels off, while the transferred probe keeps improving. This baseline also requires running the source model on every deployment query, whereas transfer needs only the target model.

Target probe trained on few target labels. The native target probe in Figures 2–5 is trained on n target labels, with n down to 50. We now also tune its regularization and add a cost-matched budget of 125 labels (see the general response). At matched cost, the transferred probe outperforms the native probe (.773 vs. .690–.712 AUROC on NQ, better in 17/21 pairs). With 1500 target labels and tuned regularization, the native probe is stronger, and we will revise our claims accordingly.

On novelty beyond existing probe transfer. We agree that the alignment method is standard, and we do not claim it as our contribution. Prior probe transfer work (Chen et al., 2025; Lee & Kondor, 2026) transfers probes for properties of the input text, such as concepts, toxicity or sentiment, whose labels are the same for both models. Semantic entropy is model-dependent: the source and target can have different labels for the same input. Our error decomposition is designed for this setting. Its disagreement term explains when transfer helps and when it cannot, and this term does not exist in prior settings. Our evaluation is also broader: we cover all 20 directed pairs among five independent model families, plus a cross-scale pair and cross-dataset alignment.

On sensitivity to the alignment dataset. We agree that performance drops when the alignment data does not match the evaluation dataset. However, alignment needs only unlabeled inputs. Once deployed, the target model’s own queries can be used as alignment data: they only need to be passed once through both models, with no sampling or SE labels. Before such inputs are available, mismatched alignment remains usable: the drop is much smaller in AUROC than in accuracy, with transferred probes typically staying within a few points of same-dataset alignment and well above chance (Figures 3–5).

On selecting λ. In the submitted version, λ was chosen from a 5-value grid by transfer AUROC on the NQ test set, then reused unchanged for all other datasets and alignment settings. We re-ran the selection without test labels, using two procedures: a label-free one (held-out reconstruction error, as in our layer selection) and one using validation AUROC with target labels from the non-test split only. Both select λ = 10⁴, the value in the paper; details are in the general response.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: new experiments (λ selection without test labels, tuned native baselines, 125-label baseline)

We thank all reviewers. Several reviews raised the selection of λ on the NQ test set and the untuned native baseline. We ran the following experiments on NQ (same-dataset alignment, all 21 pairs).

For the additional experiments, we regenerated the SE labels and hidden states on a different GPU setup, while keeping the same prompts, questions, and data splits. This may account for slight numerical variations relative to the submitted results, but the main findings remain consistent. All new comparisons use the same regenerated data, while results from the original experiments remain as reported in the submission.

1. λ selected without test labels. We used two procedures:

  • Label-free (main): for each pair, we choose the target layer and λ jointly by held-out reconstruction error on an 80/20 split of the 1500 non-test inputs, the same criterion as our layer selection. This uses no target SE labels.
  • Validation AUROC (variant): We split the 1500 non-test inputs into 70% fitting and 30% validation subsets. For each λ, we fit the alignment map and select the target layer using only the fitting subset. The target layer is chosen by our label-free best-to-align criterion: the lowest held-out reconstruction error. We select one λ by transferred AUROC on the validation subset, averaged over the 21 pairs. This variant uses validation target SE labels; the main label-free procedure uses none.

In both cases the map is refit on all 1500 inputs and evaluated once on the 500 test inputs. Both procedures choose λ = 10⁴, the value used in the paper. The label-free procedure picks it in all 21 pairs, at the same layers, by a clear margin: held-out error is 0.026–0.087 lower than for the next-best λ. Validation AUROC is .749 / .776 / .751 for λ = 10³ / 10⁴ / 10⁵.

Both revised procedures recover the submitted value of λ. This supports that choice under selection rules that exclude test labels, although it does not remove the limitation of the original test-based selection.

2. Native probes with tuned regularization. For each target model and budget n, we tune the native probe's regularization parameter C by stratified cross-validation within its n training labels only. Test AUROC on NQ, mean over the 21 pairs, with 95% CIs from a paired bootstrap over the 500 test questions (2,000 resamples):

nTransferred (0 target labels)Native, default CNative, tuned C
50.663 [.639, .686].666 [.642, .691].664 [.639, .690]
100.684 [.660, .709].698 [.674, .723].695 [.668, .723]
125–.712 [.688, .736].690 [.664, .716]
200.715 [.694, .739].732 [.710, .756].733 [.710, .758]
400.745 [.722, .769].751 [.729, .774].775 [.752, .798]
800.757 [.734, .780].772 [.751, .794].802 [.782, .824]
1500.773 [.751, .794].781 [.761, .802].825 [.805, .846]

Transfer remains close to the default-regularization native baseline, but tuning substantially strengthens the native probe at larger label budgets. At n = 1500, the tuned native probe achieves .825 AUROC, compared with .773 for transfer. We will revise our claims to reflect this gap. The practical benefit of transfer is its performance without target SE labels and its favourable comparison at matched acquisition cost. At smaller budgets, tuning does not consistently improve the native baseline.

3. Cost-matched comparison. Assuming an existing trained source SEP, the timings in Table 2 put transfer at approximately the cost of generating 125 target SE labels. At this budget, transfer achieves .773 AUROC on NQ, compared with .712 for the native probe with default regularization and .690 with tuned regularization. The tuned native probe reaches a similar mean AUROC with 400 target labels (.775). This cost equivalence is approximate and depends on the models and generation settings.

4. Revision. We will update all figures and tables with test-free λ selection, tuned native baselines, the 125-label cost-matched comparison and 95% confidence intervals.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.