acceptodds
Under review as a conference paper at ICLR 2027

HalluTracer: Pre-Decoding Truthfulness Prediction via Depth-Averaged Probe-Logit

Abstract

Internal-state probes enable truthfulness prediction before a large language model generates an answer. When detectors change both the layers they read and the rules used to combine them, the source of improved prediction becomes difficult to identify. We separate these choices and find that retaining more layers improves prediction even under fixed equal weighting. An exact Fisher-ratio decomposition explains why the additional benefit of linear reweighting is limited on these probe scores: information in layer-wise differences largely overlaps with that captured by the depth mean. Estimating additional weights can then offset this small benefit when the data used to fit them are limited. These findings motivate our proposed method HalluTracer, which averages layer-wise probe logits to predict truthfulness before decoding. Across six models and four benchmarks, HalluTracer achieves the highest AUROC in 23 of 24 model–benchmark pairs among the compared methods. The results support using evidence from across the network without requiring a correspondingly more flexible aggregation rule, clarifying the distinct roles of layer selection and weighting in pre-decoding detection.

Primary area:
interpretability and explainable AI
Keywords:
representative learning, mechanism analysis, Hallucination Detection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nP8qOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies pre-decoding truthfulness prediction, which quantifies how likely a given statement is to be false using the intermediate representations of an LLM before token generation. In particular, the authors ask where the benefit comes from when a detector reads several layers: from retaining more layers, or from learning how to weight them. They propose HalluTracer, which trains a logistic probe on every attention head of each layer at the last prompt token, keeps the best head per layer, and averages the resulting probe logits over all layers with equal weights. On the theoretical side, they compare this equal-weight average with the Fisher discriminant, the linear weighting of the layer scores with the highest signal-to-noise ratio (SNR), by decomposing the ratio of the two SNRs and relating it to AUROC under a Gaussian assumption. Experiments on six LLMs and four benchmarks show that HalluTracer outperforms the best single layer and achieves the highest AUROC in 23 of 24 model and benchmark pairs against five existing detectors.

Critical strengths

The paper shows that a very simple detector works well, which is practical knowledge for anyone who uses pre-decoding truthfulness scores. HalluTracer only averages layer-wise probe logits, so it requires neither choosing which layer to read nor learning aggregation weights, which is easy to adopt. The authors support this by broad experiments. HalluTracer is compared against single-layer selection, learned linear weighting with and without covariance shrinkage, and richer trajectory features, as well as against five existing detectors, across six models from three families, and the conclusion holds consistently.

Critical weaknesses

Although the method is practical and well supported empirically, I have two main concerns about how the paper supports and communicates its contribution.

  1. The theoretical analysis does not support the main conclusion strongly enough.

The analysis builds on the classical fact that the Fisher discriminant has the highest pooled SNR among all linear weightings of the layer scores. The paper decomposes the ratio between the Fisher SNR and the SNR of the equal-weight average into two sources of possible gain, the class separation carried by differences between layers, which the average ignores, and the covariance between the average and those differences. Under a Gaussian assumption on score differences, this ratio is then turned into a bound on the AUROC gap. Since the measured ratio is only 1.02 to 1.12, the paper concludes that equal weighting is sufficient and that learning weights for layer selection is not worthwhile.

My concern is that this analysis falls short of the role the paper gives it, namely a justification for equal weighting. The decomposition is an exact identity, so on its own it does not say when the ratio should be small. The only bound the paper gives in advance (Corollary D.4, which requires small between-layer separation and small cross-covariance) is far from tight on the evaluated data. With the measured values in Table 8, that bound allows ratios of 1.68 to 9.88, while the measured ratios are 1.02 to 1.12. The small ratio instead comes from the interaction term lying close to its Cauchy-Schwarz limit, which the paper observes on these benchmarks but does not derive. The analysis therefore describes why the gap was small here, but it does not tell the reader when equal weighting should be expected to be close to the best linear rule, or when learned weights would help.

The translation to AUROC is also loose. The paper does not justify the Gaussian assumption on score differences, and Corollary 3.3 accounts for its failure only through additive error terms. The paper's own measurements (Table 8) put these errors at 0.23 to 1.59 AUROC points per score, which is as large as or larger than the gaps they are meant to explain. The Fisher and average AUROCs differ by at most 0.72 points overall, and by at most 0.11 points on TrueFalse, where the error reaches 1.59 points. At this resolution, the bound cannot distinguish the two rules.

Finally, the conclusion itself is close to a known result. The paper cites Dawes (1979), Smith and Wallis (2009), and DeMiguel et al. (2009) for the observation that equal weights often match or beat estimated weights because the error in estimating the weights offsets the gain from optimizing them. HalluTracer's empirical findings confirm this effect for layer-wise probe scores, but it is unclear what the analysis adds beyond that confirmation.

  1. The core contribution and the problem setting are hard to identify.

After a careful reading, I understand the paper to argue that using all layers with equal weights, instead of selecting layers or learning weights, is sufficient, and to support this both theoretically and empirically. The method itself is simple, but this message was difficult to extract, and I was unsure what the central contribution is and what problem the paper is meant to solve.

(1) The paper does not tell one consistent story about its contribution.

  • The title is suggesting their method, HalluTracer, as the main point of the paper.
  • The abstract frames the problem as attributing gains between the choice of layers and the rule that combines them, rather than stating the task and the proposed detector.
  • The introduction presents the analysis first and introduces HalluTracer in a single sentence afterwards, and the contribution list does not include the method. It is unclear whether the paper intends to contribute a method, an empirical finding, or a theoretical result.
  • The theory is described both as explaining the results ("explains why the additional benefit of linear reweighting is limited") and as motivating the method ("These findings motivate our proposed method"), so its role in the argument is unclear.
  • The problem is framed as prediction before decoding, but three of the four benchmarks are statement truth classification, where the statement is already in the prompt and prediction before decoding holds by construction. Only TruthfulQA predicts errors in the model's own answers.

(2) Terms and notation appear before they are defined.

  • The abstract uses "these probe scores", "linear reweighting", and "depth mean" without definition (which was hard to interpret but minor).
  • The sentence "Estimating additional weights can then offset this small benefit when the data used to fit them are limited" appears without the setup it depends on.
  • Sec. 3.2 and 3.3 develop the theory for a generic score vector before Sec. 3.4 says what that vector is. The support I is introduced and then immediately fixed to all layers, and it is not explained why the normalized direction u_I = |I|^{-1/2} 1_I is needed.
  • Figure 1 on page 2 uses estimated quantities such as Σ̂^{-1} d̂ before any notation is introduced, and it does not show its own conclusion ("Mean ≈ Fisher > Single") because the projected class overlap is not displayed.
Questions for authors
  1. (Generalization under distribution shift) The probes, the head selection, and the calibration are all fitted on training folds of the same benchmark that is evaluated. Supervised truthfulness methods have been reported to be highly sensitive when the calibration data distribution shifts [1]. Have you examined the case where the calibration and test data come from different distributions, for example fitting on HaluEval2 and testing on TruthfulQA?
  2. (Numerical consistency) Several numbers for the same detector do not match across the paper. Could you clarify which setup produces each of the following?
    • HalluTracer in Table 2 differs from the depth mean in Table 1 (which Tables 6 and 10 reproduce exactly) by up to 0.5 AUROC, for example 84.53 vs 84.03 for Qwen2.5-32B on HaluEval2, and the Table 2 values are higher on TrueFalse for all six models. Some Table 2 margins over the strongest baseline are of the same size (e.g., 98.54 vs 98.19 for Qwen2.5-32B on TrueFalse).
    • The "Selected Head" column of Table 5, which is also the depth mean of selected-head logits, is 1.0 to 1.25 points higher than Table 1 on HaluEval2 for all five models (e.g., 0.832 vs 82.03 for Qwen2.5-7B), while its TrueFalse and HELM values match Table 1 within 0.2. Table 3 mentions a "pooled HaluEval2". Is Appendix B evaluated on different data?
  3. (Baseline fidelity) On TrueFalse, Fact-Probe reaches 97.73 AUROC on Mistral-Small-24B and 88.95 on Qwen2.5-7B but 52.66 to 61.98 on the other four models, while the other probing baselines stay above 93 on every model. HalluGuard, which scores a complete generated response, is near chance on all three statement benchmarks, and the paper does not say which response it scores there. Were these two baselines tuned and run in the setting they were designed for? If they are underperforming, the 23 of 24 result in Table 2 overstates the margin.

[1] Y. F. Bakman et al., "Reconsidering LLM Uncertainty Estimation Methods in the Wild," ACL 2025. https://aclanthology.org/2025.acl-long.1429/

6 comments
AuthorsOpenReview ↗

Response to nP8q (1)

Thank you for recognising that HalluTracer is simple, effective and easy to adopt, and that its practical value is supported by broad experiments. This was also our design goal: to understand what limits the benefit of learning layer weights and use that understanding to choose a simple aggregation rule.

W1. Contribution and theoretical justification.

Thank you for your question. On the evaluated scores, layer-wise differences carry class separation, but much of it overlaps with evidence already retained by the mean. Learned aggregation in detectors such as InternalInspector and SIVR and so on raises the question of how much learning the weights adds. We study this with fixed layer-wise scores, distinguishing the gains from retaining more layers and from learning their weights. The classical equal-weight literature motivates us to examine both the available gain and estimation cost.

What the decomposition adds. A small Fisher ratio alone leaves open whether layer-wise differences carry little separation or whether their separation overlaps with the mean. Theorem 3.1 and Eq. (7) distinguish these cases through the interaction term. Table 8 identifies the latter, with training-fold Fisher-ratio estimates of 1.02–1.12. The training-size experiment then measures the effect of estimating weights from fewer samples.

Corollary D.4 takes the worst case for this interaction, giving the loose range of 1.68–9.88 you noted. Appendix D.3 retains the interaction in the exact expression evaluated in Table 8. The overlap is an empirical finding on these benchmarks. We will make this distinction clearer in the paper.

What fitting weights achieves. Across 18 conditions and the training-set sizes tested, the largest gain from covariance shrinkage selected by test AUROC is 0.79 AUROC points over the mean. At one sixteenth of the training fold, unshrunk Fisher falls up to 9.2 points below the unchanged mean, while Fisher with the best tested shrinkage gains at most 0.58 points. These comparisons keep probes and test folds fixed. We report the best mean test AUROC among the shrinkage strengths tried. We also used five-fold splits grouped by source and fitted the probes and Fisher weights on separate data. Separate calibration improves Fisher by 0.02–2.18 points across nine conditions, but its largest advantage over the mean remains 0.071 points with the relative ridge fixed at 10−610^{-6} and no shrinkage. The support and weighting comparisons show that retaining more layers improves prediction, while learning linear weights adds little on the same scores.

AUROC approximation. §\S 4.3 reports agreement within 0.3 AUROC points on HaluEval2 and HELM between predicted and observed gaps. Corollary 3.2 assumes Gaussianity only for the one-dimensional class-score differences of the two directions. The SNR decomposition requires no Gaussian assumption. Corollary 3.3 keeps explicit approximation errors. For signed errors eFe_F and eMe_M for Fisher and the mean, the gap error is ∣eF−eM∣\lvert e_F-e_M\rvert, conservatively bounded by their absolute-value sum. On TrueFalse, where approximation errors can exceed the observed gap, we use measured AUROC to compare the two rules.

Q1. Generalization under distribution shift.

Yes. We tested transfer across benchmarks and observed a loss in performance. We used HaluEval2, TrueFalse and HELM because their ground-truth labels avoid the added uncertainty of LLM judging in our TruthfulQA evaluation. Across 12 transfers on Qwen-32B and Llama-3.1-8B, AUROC ranges from 71.9% to 94.2%. Using Table 1's depth-mean AUROCs as the in-domain reference, the largest drop is 16.9 points, from TrueFalse to HELM on Llama-3.1-8B. The smallest is 3.8 points, from HaluEval2 to TrueFalse on the same model. We will include the full results.

Q2. Numerical consistency.

We apologise for the inconsistency. Thank you for pointing this out. During our cluster migration and software environment updates, results from different runs were retained in different tables. We will use the Table 1 depth mean results as the reference and report the corresponding values consistently in Table 2 and the appendix. For Qwen-32B/HaluEval2, the reference AUROC is 84.03%.

Against the submitted baselines, the method with the highest AUROC in each condition remains unchanged. The main aggregation conclusions still hold. The corrected Fact-Probe comparison is reported below. We will also specify the HaluEval2 topic files and sample counts used in Appendix B and release the settings and results for each fold.

AuthorsOpenReview ↗

Response to nP8q (2)

W2. Main contribution and presentation.

Thank you for raising this. HalluTracer averages layer-wise probe logits for prediction before generation. Our contribution is to clarify whether learning additional layer weights is worth the estimation and tuning. On the evaluated scores, retaining depth helps, while fitting weights adds little.

Prediction of generated-answer errors. The added TruthfulQA analysis shows the same pattern for predicting errors in generated answers. We repeated the support and weighting analyses across six models using only activations available before generation. The depth mean improves on the single layer selected on the training data by 5.17–9.94 AUROC points. Five-fold mean AUROC increases from 2 to 4 to 8 to all layers on every model. The Fisher ratios estimated on the training data range from 1.073 to 1.221. With the paper's fixed ridge, Fisher has lower AUROC than the depth mean on five models. For each model, we tested 21 shrinkage strengths and chose the one with the highest five-fold mean test AUROC. The largest gain over the depth mean is 0.74 AUROC points. We will add these results and distinguish this task from classifying statements supplied in the prompt.

Presentation and notation. We will introduce the detector before the analysis and make this contribution clear in the abstract, introduction and contribution list. A probe logit is the decision score produced by a layer's fitted probe. The depth mean averages these layer-wise scores. The support II is the set of retained layers. It varies when comparing single-layer and full-depth readouts and stays fixed when comparing aggregation weights. Normalizing the uniform direction gives the orthonormal coordinates used in the decomposition. It is a positive rescaling of the mean, leaving SNR and AUROC unchanged. We will define these terms before using them. In Figure 1, we will explain the fitted quantities locally and show the projected class overlap so that the comparison between the three readouts is visible.

Q3. Baseline fidelity.

Fact-Probe. We apologise for the evaluation error. We fixed a token-indexing bug in our interface and reran all six models on the three statement benchmarks. Within each training fold, we selected among eight layers and two regularization values using validation data, then refitted the probe and evaluated it on the held-out fold. Corrected TrueFalse results are five-fold means ±\pm standard deviations (%).

ModelAUROCAUPRC
Llama-2-7B94.04±1.0894.04 \pm 1.0894.13±1.1194.13 \pm 1.11
Llama-3.1-8B97.34±0.4697.34 \pm 0.4697.23±0.2997.23 \pm 0.29
Qwen2.5-7B97.47±0.3797.47 \pm 0.3797.34±0.3297.34 \pm 0.32
Qwen2.5-14B98.12±0.1298.12 \pm 0.1298.28±0.1298.28 \pm 0.12
Qwen2.5-32B98.20±0.4298.20 \pm 0.4298.24±0.3898.24 \pm 0.38
Mistral-Small-24B98.45±0.2798.45 \pm 0.2798.34±0.3198.34 \pm 0.31

With the Table 1 depth-mean AUROCs and the corrected Fact-Probe results, HalluTracer has the highest AUROC in 20/24 conditions. Fact-Probe leads on TrueFalse for Qwen-7B, Llama-3.1-8B and Mistral, by 0.04, 0.04 and 0.23 points, respectively. With the current AUPRC entries, HalluTracer leads in 20/24, with Fact-Probe ahead on Qwen-14B/TrueFalse and Mistral/TrueFalse. We will replace the affected entries, update the reported win counts, and release all corrected results, code and settings.

HalluGuard. On the statement benchmarks, HalluGuard scores the model's generated response to a true/false classification prompt. We ran the evaluation on four PRO 6000 GPUs and tried to extend the response budget as far as memory allowed. For most runs, memory limited responses to fewer than 100 generated tokens. Running the complete method from the authors' repository on Qwen-32B with 1,024 generated tokens would require about 2 TB for the half-precision gradient list and stacked copy alone. We will describe the memory-constrained implementation and its response-length limitation clearly.

Thank you again for your detailed comments. We have uploaded the revision with a clearer presentation of the detector, its contribution, and the role of the analysis. We also aligned the repeated table values and included the corrected Fact-Probe results and transfer evaluation. We would appreciate your feedback on any remaining concerns.

If the revisions and additional results address your main concerns, we would be grateful if you could consider raising your rating.

Reviewer_VcL3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to combine per-layer internal-state probes into one truthfulness score computed before the model emits any answer token. At each layer, the method trains logistic probes on every attention head’s pre-output-projection slice at the last prompt token. It keeps the head with the best training AUROC, averages that probe’s logit over three bootstrap resamples, and so produces one score per layer. The proposed detector, HalluTracer, averages these per-layer logits across depth with equal weights and then applies a 1-D logistic calibration.

Critical strengths
  1. Separating support gain, potential reweighting gain, and estimation cost is a clean way to analyse multi-layer probing. It makes a question that is often muddled in this literature testable.

  2. The decomposition is a valid Schur-complement identity. The assumptions are stated precisely, the Gaussian-difference condition is applied only to scalar projections, and the approximation error k is reported rather than hidden.

  3. The paper includes a covariance-shrinkage path, a training-size sweep, median and trimmed-mean alternatives, head-selection ablations, a residual-stream readout, a logit-scale control, and a length-only baseline. Several of these are honestly reported even where they don’t favour the deployed design.

  4. Aggregation at deployment is O(m) and available at answer onset, which suits gating or retrieval-triggering pipelines.

Critical weaknesses
  1. The decomposition describes the gap but doesn’t explain it. Theorem 3.1 is an exact algebraic identity for dᵀΣ⁻¹d after rotating to the uniform direction. The authors acknowledge that the Fisher criterion, the binormal AUROC relation, and ratio-to-reference comparisons are classical. Because the identity holds for every (d, Σ), finding that t̂ sits at 87–96% of its Cauchy–Schwarz bound is a restatement of “R̂ ≈ 1 given θ̂, χ̂,” not a cause. The t = 0 counterfactual that giving ratios of 1.33–4.93 is also not meaningful, because θ, χ, and t are jointly determined by the same Σ and d.

  2. The single-layer comparison is not capacity-matched. “Single” is one 128-dim head probe, chosen by training pooled SNR; the chosen layer is unstable, with up to 5 distinct layers across folds. “Mean” is effectively an ensemble of 28–64 probes. The support gain is therefore partly an ensembling and variance-reduction effect, which is unsurprising. A fairer control is a single-layer probe on the full residual stream, or on all heads concatenated, with the layer chosen on a validation split.

  3. In three of the four benchmarks (HaluEval2, TrueFalse, HELM), the statement is in the prompt and the task is to classify a given statement as true or false. Reading at the last prompt token is then “pre-decoding” by construction. Only TruthfulQA tests prediction of the model’s own errors, which is the setting the title and abstract emphasise. However, all of the theory-facing analysis that include Tables 1, 8, 9, Figure 2 is run only on the 18 statement conditions, not on TruthfulQA.

Questions for authors
  1. See in Critical Weaknesses.

  2. How were HaluEval2 and HELM converted to prompt-embedded statements, and what prompt templates were used?

  3. Was the GPT-5.4 judge validated against human labels on TruthfulQA?

5 comments
AuthorsOpenReview ↗

Response to VcL3

Thank you for recognising that our analysis makes the aggregation question testable, with a valid decomposition and precisely stated assumptions. We also appreciate your recognition of the broad controls and their honest reporting, including results that do not favour the deployed design.

W1. What the decomposition explains.

The practical question is whether fitting aggregation weights is worthwhile. A small Fisher ratio can arise because layer-wise differences carry little class separation or because their separation overlaps with the mean. The decomposition distinguishes these cases, and Table 8 shows the latter. This identifies what limits the available gain on these scores. We agree that near-saturation of the Cauchy–Schwarz bound is an empirical finding and will state this explicitly.

We can construct the tI=0t_I=0 comparison by changing the class mean difference while keeping θI\theta_I, χI\chi_I, and the covariance fixed. In Eq. (4), take a=s=1a=s=1, C=I2C=\mathbf{I}_2, and b=(0.8,0)⊤\mathbf{b}=(0.8,0)^\top. Changing z\mathbf{z} from (1,0)⊤(1,0)^\top to (0,1)⊤(0,1)^\top preserves θI=0.64\theta_I=0.64 and χI=1\chi_I=1, but changes tIt_I from 0.80.8 to 00 and the Fisher ratio from 1.054 to 1.944. The covariance eigenvalues are 0.2, 1, and 1.8. This illustrates the role of the interaction at fixed covariance and χI\chi_I. Table 8 measures it on the observed scores, while comparisons on test data assess the benefit of fitted weights. We will describe this as a sensitivity analysis of the class mean difference at fixed covariance.

W2. Single-layer capacity and validation selection.

Following your suggestion, we gave the single layer probe all heads at that layer and selected the layer by validation AUROC within the training fold. We then refitted its probe before testing, retaining C=1C=1 and the original test folds.

ModelDatasetAll heads, single layerDepth mean
Qwen2.5-7BHaluEval275.30±0.8275.30 \pm 0.8282.0382.03
Qwen2.5-7BTrueFalse96.34±0.3996.34 \pm 0.3997.4397.43
Qwen2.5-7BHELM84.22±0.7884.22 \pm 0.7887.8587.85
Qwen2.5-32BHaluEval278.76±1.1178.76 \pm 1.1184.0384.03
Qwen2.5-32BTrueFalse97.94±0.3197.94 \pm 0.3198.5198.51
Qwen2.5-32BHELM87.07±1.0987.07 \pm 1.0988.5188.51

The depth mean leads in all six comparisons by 0.6 to 6.7 points. Using more layers therefore still helps after expanding the single layer input and choosing its layer on validation data. Ensembling and variance reduction may contribute to this gain. We will add this comparison to the revised manuscript.

W3. Aggregation analysis on TruthfulQA.

We repeated the layer-support and weighting comparisons on TruthfulQA to predict errors in the model's own answers. Using only activations from the prompt, averaging all layers improves AUROC over the single layer selected on the training data by 5.17–9.94 points across six models. Five-fold mean AUROC increases from 2 to 4 to 8 to all layers on every model.

Averaged over five folds, the Fisher ratios estimated on the training data range from 1.073 to 1.221. With the paper's fixed ridge, Fisher has lower AUROC than the depth mean on five models. For each model, we tested 21 shrinkage strengths and chose the one with the highest five-fold mean test AUROC. The largest gain over the depth mean is 0.74 AUROC points. These results support fixed averaging for predicting generation errors, with little benefit from fitting additional weights on these scores. We will add the Fisher-ratio estimates, layer-count curves and weighting results.

Q1&Q2. Prompt templates and GPT-5.4 and human labels.

We use the official prompt templates for all datasets. We will include the templates in the revision. LLM as judge in TruthfulQA is a standard evaluation practice, and we have checked again using GPT-5.6 and the results as same as GPT-5.4.

TruthfulQA's official evaluation of generated answers includes LLM-based judging. The benchmark provides GPT-judge (Lin et al., 2022), and later work uses GPT-4 in LoFiT (Yin et al., 2024) and GPT-4o in TSV (Park et al., 2025). Our use of a GPT judge follows this established evaluation practice. We chose GPT-5.4 as a strong GPT model available when we conducted the experiments. All detectors share its labels for each model's answers. The results use this automatic evaluation, with no separate comparison against human judgments. The three statement benchmarks use their released labels.

Thank you again for your suggestions. The revision includes the single layer control with all heads and the layer selected on validation data (Table 2), alongside the TruthfulQA analysis (Table 4). We also clarified the interpretation of the decomposition and added the exact statement prompts. We would welcome your feedback on the revised version.

Thank you for the response. I'm ok with the part of response. But I still have some concerns. The new all-heads single-layer baseline falls 4–5 points below the single-head baseline on HaluEval2, which suggests the untuned probe is overfitting rather than serving as a strong reference. In addition, how the statement benchmarks were constructed, judge agreement, baseline fairness, TruthfulQA significance, and cross-benchmark transfer.

I therefore keep my score.

Reviewer_1qXLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies layer selection versus linear aggregation weights for internal-state truthfulness probing, deriving a Fisher-ratio decomposition around the uniform depth mean and proposing HalluTracer, which averages bootstrap-fitted attention-head probe logits across all network layers prior to decoding. Empirical evaluation spans six open-weight LLMs across four datasets (HaluEval2, TrueFalse, HELM, and TruthfulQA).

Critical strengths
  • Decomposing multi-layer probing into layer support growth versus sample-estimated aggregation weights provides a clean analytical explanation for why learned multi-layer combiners can underperform simple uniform depth averaging under finite calibration samples.
  • Evaluating depth scaling and aggregation rules across six open-weight model backbones ranging from 7B to 32B parameters offers useful empirical insights into how truthfulness signals distribute across transformer layers.
Critical weaknesses
  • Positioning the method around pre-decoding truthfulness prediction is only partially aligned with the main experimental suite, as three of the four benchmarks (HaluEval2, TrueFalse, and HELM, covering all 18 settings in Table 1 and Figures 2–3) evaluate static truth classification on completed statements provided in the prompt rather than predicting errors in upcoming open-ended generation (Section 4.1, p. 6). Only TruthfulQA tests pre-decoding generation truthfulness, where the learned baseline SIVR-PD slightly outperforms HalluTracer on Qwen2.5-32B (81.29% vs. 80.97% AUROC in Table 2).
  • Fitting L2 probes to every attention head across bootstrap resamples to select the top head per layer (Section 3.4, p. 5) adds substantial fitting overhead, yet the paper's own readout ablation in Table 5 (p. 16) shows that directly probing the residual stream without head selection matches or outperforms the selected head in 14 of 15 conditions (including by 1.8–2.3 AUROC points on HELM).
  • Estimating the Fisher mean gap and covariance on the same training fold where the per-layer probes and head choices were already fitted (Appendix A.1, p. 14; Appendix D.7, p. 22) can introduce in-sample optimism into the plug-in Fisher baseline, and using ungrouped stratified five-fold splits on HELM and TrueFalse may allow passage or entity-template overlap across folds.
Questions for authors
  • Could the authors report the Fisher-ratio diagnostics, layer-support curves (Figure 2), and aggregation rule comparisons (Table 1) on TruthfulQA to show whether the same depth-averaging trends hold for pre-decoding generation?
  • Given that probing the residual stream outperforms attention-head selection in 14 of 15 settings in Table 5 while avoiding per-head bootstrap search, why was attention-head selection chosen as the default readout for HalluTracer?
  • Could the authors clarify why the five-fold AUROC values for the uniform depth-mean detector on HaluEval2 differ slightly across Table 1/Table 10 (e.g., 84.03% on Qwen2.5-32B), Table 2 (84.53%), and Table 5 (85.2%)?
3 comments
AuthorsOpenReview ↗

Response to 1qXL

Thank you for highlighting our "clean analytical explanation" of why learned aggregation can underperform depth averaging with finite calibration samples, and the "useful empirical insights" from our six-model evaluation. We have added the requested TruthfulQA analysis and controls for calibration reuse and source overlap.

W1 / Q1. Pre-decoding generation on TruthfulQA.

Yes. The requested TruthfulQA analysis shows the same pattern for predicting errors in generated answers: using more layers helps, while learning their weights adds little. All scores use activations from the prompt. Averaging all layers improves AUROC over the single layer selected on the training data by 5.17–9.94 points across six models. Five-fold mean AUROC increases from 2 to 4 to 8 to all layers on every model.

Averaged over five folds, the Fisher ratios estimated on the training data range from 1.073 to 1.221. With the paper's fixed ridge, Fisher has lower AUROC than the depth mean on five models. For each model, we tested 21 shrinkage strengths and chose the one with the highest five-fold mean test AUROC. The largest gain over the depth mean is 0.74 AUROC points. SIVR-PD's 0.32-point advantage on Qwen-32B in Table 2 remains. We will add the analysis and distinguish generation-error prediction from statement classification in §\S 4.1.

W2 / Q2. Default readout and fitting overhead.

Thank you for your question. The depth mean can use residual stream probes directly, avoiding head selection. Our default head readout keeps each probe's input at 128 dimensions. Table 5 shows that the residual stream readout matches or improves on it in 14 of 15 conditions. We will present this alternative clearly and separate probe fitting and head selection from the O(m)\mathcal{O}(m) aggregation cost.

W3. In-sample optimism and grouped splits.

Yes. Separate calibration improves Fisher in all nine grouped comparisons. Its largest advantage over the depth mean is 0.071 AUROC points. We fitted Fisher on either the probe training examples or a separate set with the same size and class counts. The probes and test examples are shared, and both fits use the original relative ridge of 10−610^{-6}, without shrinkage.

We addressed your overlap concern by repeating the experiment with grouped splits. HaluEval2 is grouped by original answer, HELM by title, and TrueFalse by identifiable subject, with duplicate text groups for the remaining examples. We checked that none of these groups crosses the probe training, calibration or test sets.

ModelDatasetDepth meanFisher, separate calibration
Qwen2.5-7BHaluEval281.14±1.5081.14 \pm 1.5081.13±1.1281.13 \pm 1.12
Qwen2.5-7BTrueFalse97.37±0.2497.37 \pm 0.2497.44±0.1697.44 \pm 0.16
Qwen2.5-7BHELM84.78±1.7684.78 \pm 1.7683.45±2.0883.45 \pm 2.08
Qwen2.5-32BHaluEval283.22±1.3683.22 \pm 1.3683.00±1.4383.00 \pm 1.43
Qwen2.5-32BTrueFalse98.39±0.1098.39 \pm 0.1098.40±0.2398.40 \pm 0.23
Qwen2.5-32BHELM85.80±1.3185.80 \pm 1.3184.94±1.0084.94 \pm 1.00
Mistral-Small-24BHaluEval283.69±1.5583.69 \pm 1.5582.96±1.5282.96 \pm 1.52
Mistral-Small-24BTrueFalse98.15±0.2098.15 \pm 0.2098.14±0.1898.14 \pm 0.18
Mistral-Small-24BHELM86.78±1.9686.78 \pm 1.9685.53±2.1085.53 \pm 2.10

Values are five-fold mean AUROC ±\pm standard deviation (%). The improvement from separate calibration ranges from 0.02–2.18 points. The Fisher-ratio estimates from the separate calibration data, averaged over five folds, range from 1.046 to 1.155.

In this calibration experiment, the mean's HELM AUROC is 1.86 and 2.12 points lower under grouped splits than under random splits for the 7B and 32B Qwen models, respectively. We will report this decrease alongside the comparison with Fisher.

Q3. Numerical consistency.

We apologise for the inconsistency. During our cluster migration and software environment updates, results from different runs were retained in different tables. We will use Table 1's depth mean AUROCs as the reference, including 84.03% for Qwen-32B/HaluEval2, and use these results consistently in Table 2 and the appendix. After this update and the corrected Fact-Probe evaluation, HalluTracer has the highest AUROC in 20/24 conditions. We will release the settings and results for each fold.

Thank you again for your feedback. We have uploaded the revised manuscript, including the TruthfulQA analysis (Table 4) and separate calibration with grouped splits (Appendix F.3). We also clarified the readout choice and made the depth mean results consistent across tables. We would appreciate your feedback on any remaining concerns.

Dear Reviewer 1qXL,

As the discussion period is drawing to a close, we wanted to briefly follow up on our responses and revised manuscript. In the revision, we incorporated all of your suggested updates: adding the TruthfulQA support and weighting analysis (Table 4, Appendix F.1), evaluating Fisher weighting with separate calibration data under source-grouped splits (Table 13, Appendix F.3), clarifying the default head readout alongside the residual-stream alternative (§ 3.4, Appendix B), and aligning all depth-mean results consistently across tables (Tables 1 and 3).

Please let us know if anything remains unclear or if there are any remaining questions we can address before the deadline. If these updates have resolved your concerns, we would be very grateful if you could consider raising your assessment and rating. Thank you again for your constructive review and time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Response and Revisions

We thank the reviewers, the Senior Area Chair, and the Area Chair for their feedback. We appreciate the reviewers’ recognition of the practical value of assessing hallucination risk before generation. HalluTracer forms a single score, the depth mean, by averaging probe logits across all layers at the final prompt token. How much does retaining more layers help, and how much more could learning their weights add?

With the averaging rule fixed, the depth mean of the default probes gives higher AUROC than the single layer selected on training data in all 18 statement conditions (Table 1). Since retaining the layers helps, we ask what a different weighting of those same scores could recover. Fisher’s rule gives the highest possible pooled signal-to-noise ratio (SNR) among linear weightings when the class means and covariances are known.

Our Fisher ratio decomposition measures what differences between layers add to class separation after accounting for their covariance with the depth mean. The differences help distinguish true from false statements, but much of that separation is already captured by the mean. Across the 18 conditions, training estimates put Fisher’s pooled SNR at 1.02–1.12 times that of the depth mean (Table 9). When score differences between independent examples from the two classes are Gaussian for both rules, the population Fisher ratio and the mean’s SNR determine the AUROC gap (§\S 3.3). Using the measured held-out SNRs of the depth mean and fitted Fisher score, the Gaussian formula predicts gaps below one AUROC point (§\S 4.3).

In practice, the class means and covariances must be estimated. With limited training data, the resulting error in the weights can offset the available gain. Shrinkage reduces that loss, but even when we selected the strength by mean test AUROC for each condition and training size, Fisher gained at most 0.79 AUROC points over the depth mean (§\S 4.3, Table 10). Since retaining layers helped and reweighting them added little, HalluTracer uses the depth mean without fitting aggregation weights or tuning covariance shrinkage.

In the revision, we strengthened the comparisons and clarified where the results apply.

  • Stronger controls. To test whether probe width or fixed regularisation limited the single layer comparison, we gave each layer’s probe the outputs of all its attention heads and selected its regularisation strength on validation data. On Qwen2.5-7B and Qwen2.5-32B, the selected layer nearly matches the mean of the same probes on HaluEval2 and TrueFalse, while averaging gains 1.5–1.6 AUROC points on HELM (Table 2). To test whether reusing probe training examples affected Fisher, we fitted its weights on separate calibration data with source groups kept apart. Separate calibration improved Fisher in all nine comparisons. Its largest lead over the depth mean was below a tenth of an AUROC point (Table 13).
  • Errors in generated answers. To see whether retaining more layers also helps predict errors in a model’s own answer, we tested both readouts on TruthfulQA using only the prompt. The depth mean improved on the selected single layer by 5.17–9.94 AUROC points across all six models (Table 4).
  • Corrected benchmark results. We aligned the reported depth mean AUROCs, reran Fact-Probe on the statement benchmarks with settings selected on validation data, and removed the residual stream comparison that combined separate runs. HalluTracer ranks first in 20 of 24 conditions by AUROC and in 20 of 24 by AUPRC (Table 3).

Because the main comparison fits each detector on the target benchmark, we also tested transfer. Applying HalluTracer unchanged to a different statement benchmark lowered AUROC by 3.8–16.9 points relative to fitting on the target (Table 14).

We hope our method and analysis contribute to more principled and interpretable approaches to pre-decoding hallucination detection.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.