- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies pre-decoding truthfulness prediction, which quantifies how likely a given statement is to be false using the intermediate representations of an LLM before token generation. In particular, the authors ask where the benefit comes from when a detector reads several layers: from retaining more layers, or from learning how to weight them. They propose HalluTracer, which trains a logistic probe on every attention head of each layer at the last prompt token, keeps the best head per layer, and averages the resulting probe logits over all layers with equal weights. On the theoretical side, they compare this equal-weight average with the Fisher discriminant, the linear weighting of the layer scores with the highest signal-to-noise ratio (SNR), by decomposing the ratio of the two SNRs and relating it to AUROC under a Gaussian assumption. Experiments on six LLMs and four benchmarks show that HalluTracer outperforms the best single layer and achieves the highest AUROC in 23 of 24 model and benchmark pairs against five existing detectors.
Critical strengths
The paper shows that a very simple detector works well, which is practical knowledge for anyone who uses pre-decoding truthfulness scores. HalluTracer only averages layer-wise probe logits, so it requires neither choosing which layer to read nor learning aggregation weights, which is easy to adopt. The authors support this by broad experiments. HalluTracer is compared against single-layer selection, learned linear weighting with and without covariance shrinkage, and richer trajectory features, as well as against five existing detectors, across six models from three families, and the conclusion holds consistently.
Critical weaknesses
Although the method is practical and well supported empirically, I have two main concerns about how the paper supports and communicates its contribution.
- The theoretical analysis does not support the main conclusion strongly enough.
The analysis builds on the classical fact that the Fisher discriminant has the highest pooled SNR among all linear weightings of the layer scores. The paper decomposes the ratio between the Fisher SNR and the SNR of the equal-weight average into two sources of possible gain, the class separation carried by differences between layers, which the average ignores, and the covariance between the average and those differences. Under a Gaussian assumption on score differences, this ratio is then turned into a bound on the AUROC gap. Since the measured ratio is only 1.02 to 1.12, the paper concludes that equal weighting is sufficient and that learning weights for layer selection is not worthwhile.
My concern is that this analysis falls short of the role the paper gives it, namely a justification for equal weighting. The decomposition is an exact identity, so on its own it does not say when the ratio should be small. The only bound the paper gives in advance (Corollary D.4, which requires small between-layer separation and small cross-covariance) is far from tight on the evaluated data. With the measured values in Table 8, that bound allows ratios of 1.68 to 9.88, while the measured ratios are 1.02 to 1.12. The small ratio instead comes from the interaction term lying close to its Cauchy-Schwarz limit, which the paper observes on these benchmarks but does not derive. The analysis therefore describes why the gap was small here, but it does not tell the reader when equal weighting should be expected to be close to the best linear rule, or when learned weights would help.
The translation to AUROC is also loose. The paper does not justify the Gaussian assumption on score differences, and Corollary 3.3 accounts for its failure only through additive error terms. The paper's own measurements (Table 8) put these errors at 0.23 to 1.59 AUROC points per score, which is as large as or larger than the gaps they are meant to explain. The Fisher and average AUROCs differ by at most 0.72 points overall, and by at most 0.11 points on TrueFalse, where the error reaches 1.59 points. At this resolution, the bound cannot distinguish the two rules.
Finally, the conclusion itself is close to a known result. The paper cites Dawes (1979), Smith and Wallis (2009), and DeMiguel et al. (2009) for the observation that equal weights often match or beat estimated weights because the error in estimating the weights offsets the gain from optimizing them. HalluTracer's empirical findings confirm this effect for layer-wise probe scores, but it is unclear what the analysis adds beyond that confirmation.
- The core contribution and the problem setting are hard to identify.
After a careful reading, I understand the paper to argue that using all layers with equal weights, instead of selecting layers or learning weights, is sufficient, and to support this both theoretically and empirically. The method itself is simple, but this message was difficult to extract, and I was unsure what the central contribution is and what problem the paper is meant to solve.
(1) The paper does not tell one consistent story about its contribution.
- The title is suggesting their method, HalluTracer, as the main point of the paper.
- The abstract frames the problem as attributing gains between the choice of layers and the rule that combines them, rather than stating the task and the proposed detector.
- The introduction presents the analysis first and introduces HalluTracer in a single sentence afterwards, and the contribution list does not include the method. It is unclear whether the paper intends to contribute a method, an empirical finding, or a theoretical result.
- The theory is described both as explaining the results ("explains why the additional benefit of linear reweighting is limited") and as motivating the method ("These findings motivate our proposed method"), so its role in the argument is unclear.
- The problem is framed as prediction before decoding, but three of the four benchmarks are statement truth classification, where the statement is already in the prompt and prediction before decoding holds by construction. Only TruthfulQA predicts errors in the model's own answers.
(2) Terms and notation appear before they are defined.
- The abstract uses "these probe scores", "linear reweighting", and "depth mean" without definition (which was hard to interpret but minor).
- The sentence "Estimating additional weights can then offset this small benefit when the data used to fit them are limited" appears without the setup it depends on.
- Sec. 3.2 and 3.3 develop the theory for a generic score vector before Sec. 3.4 says what that vector is. The support I is introduced and then immediately fixed to all layers, and it is not explained why the normalized direction u_I = |I|^{-1/2} 1_I is needed.
- Figure 1 on page 2 uses estimated quantities such as Σ̂^{-1} d̂ before any notation is introduced, and it does not show its own conclusion ("Mean ≈ Fisher > Single") because the projected class overlap is not displayed.
Questions for authors
- (Generalization under distribution shift) The probes, the head selection, and the calibration are all fitted on training folds of the same benchmark that is evaluated. Supervised truthfulness methods have been reported to be highly sensitive when the calibration data distribution shifts [1]. Have you examined the case where the calibration and test data come from different distributions, for example fitting on HaluEval2 and testing on TruthfulQA?
- (Numerical consistency) Several numbers for the same detector do not match across the paper. Could
you clarify which setup produces each of the following?
- HalluTracer in Table 2 differs from the depth mean in Table 1 (which Tables 6 and 10 reproduce exactly) by up to 0.5 AUROC, for example 84.53 vs 84.03 for Qwen2.5-32B on HaluEval2, and the Table 2 values are higher on TrueFalse for all six models. Some Table 2 margins over the strongest baseline are of the same size (e.g., 98.54 vs 98.19 for Qwen2.5-32B on TrueFalse).
- The "Selected Head" column of Table 5, which is also the depth mean of selected-head logits, is 1.0 to 1.25 points higher than Table 1 on HaluEval2 for all five models (e.g., 0.832 vs 82.03 for Qwen2.5-7B), while its TrueFalse and HELM values match Table 1 within 0.2. Table 3 mentions a "pooled HaluEval2". Is Appendix B evaluated on different data?
- (Baseline fidelity) On TrueFalse, Fact-Probe reaches 97.73 AUROC on Mistral-Small-24B and 88.95 on Qwen2.5-7B but 52.66 to 61.98 on the other four models, while the other probing baselines stay above 93 on every model. HalluGuard, which scores a complete generated response, is near chance on all three statement benchmarks, and the paper does not say which response it scores there. Were these two baselines tuned and run in the setting they were designed for? If they are underperforming, the 23 of 24 result in Table 2 overstates the margin.
[1] Y. F. Bakman et al., "Reconsidering LLM Uncertainty Estimation Methods in the Wild," ACL 2025. https://aclanthology.org/2025.acl-long.1429/
6 comments
Response to nP8q (2)
W2. Main contribution and presentation.
Thank you for raising this. HalluTracer averages layer-wise probe logits for prediction before generation. Our contribution is to clarify whether learning additional layer weights is worth the estimation and tuning. On the evaluated scores, retaining depth helps, while fitting weights adds little.
Prediction of generated-answer errors. The added TruthfulQA analysis shows the same pattern for predicting errors in generated answers. We repeated the support and weighting analyses across six models using only activations available before generation. The depth mean improves on the single layer selected on the training data by 5.17–9.94 AUROC points. Five-fold mean AUROC increases from 2 to 4 to 8 to all layers on every model. The Fisher ratios estimated on the training data range from 1.073 to 1.221. With the paper's fixed ridge, Fisher has lower AUROC than the depth mean on five models. For each model, we tested 21 shrinkage strengths and chose the one with the highest five-fold mean test AUROC. The largest gain over the depth mean is 0.74 AUROC points. We will add these results and distinguish this task from classifying statements supplied in the prompt.
Presentation and notation. We will introduce the detector before the analysis and make this contribution clear in the abstract, introduction and contribution list. A probe logit is the decision score produced by a layer's fitted probe. The depth mean averages these layer-wise scores. The support is the set of retained layers. It varies when comparing single-layer and full-depth readouts and stays fixed when comparing aggregation weights. Normalizing the uniform direction gives the orthonormal coordinates used in the decomposition. It is a positive rescaling of the mean, leaving SNR and AUROC unchanged. We will define these terms before using them. In Figure 1, we will explain the fitted quantities locally and show the projected class overlap so that the comparison between the three readouts is visible.
Q3. Baseline fidelity.
Fact-Probe. We apologise for the evaluation error. We fixed a token-indexing bug in our interface and reran all six models on the three statement benchmarks. Within each training fold, we selected among eight layers and two regularization values using validation data, then refitted the probe and evaluated it on the held-out fold. Corrected TrueFalse results are five-fold means standard deviations (%).
| Model | AUROC | AUPRC |
|---|---|---|
| Llama-2-7B | ||
| Llama-3.1-8B | ||
| Qwen2.5-7B | ||
| Qwen2.5-14B | ||
| Qwen2.5-32B | ||
| Mistral-Small-24B |
With the Table 1 depth-mean AUROCs and the corrected Fact-Probe results, HalluTracer has the highest AUROC in 20/24 conditions. Fact-Probe leads on TrueFalse for Qwen-7B, Llama-3.1-8B and Mistral, by 0.04, 0.04 and 0.23 points, respectively. With the current AUPRC entries, HalluTracer leads in 20/24, with Fact-Probe ahead on Qwen-14B/TrueFalse and Mistral/TrueFalse. We will replace the affected entries, update the reported win counts, and release all corrected results, code and settings.
HalluGuard. On the statement benchmarks, HalluGuard scores the model's generated response to a true/false classification prompt. We ran the evaluation on four PRO 6000 GPUs and tried to extend the response budget as far as memory allowed. For most runs, memory limited responses to fewer than 100 generated tokens. Running the complete method from the authors' repository on Qwen-32B with 1,024 generated tokens would require about 2 TB for the half-precision gradient list and stacked copy alone. We will describe the memory-constrained implementation and its response-length limitation clearly.
Thank you again for your detailed comments. We have uploaded the revision with a clearer presentation of the detector, its contribution, and the role of the analysis. We also aligned the repeated table values and included the corrected Fact-Probe results and transfer evaluation. We would appreciate your feedback on any remaining concerns.
If the revisions and additional results address your main concerns, we would be grateful if you could consider raising your rating.
Response to nP8q (1)
Thank you for recognising that HalluTracer is simple, effective and easy to adopt, and that its practical value is supported by broad experiments. This was also our design goal: to understand what limits the benefit of learning layer weights and use that understanding to choose a simple aggregation rule.
W1. Contribution and theoretical justification.
Thank you for your question. On the evaluated scores, layer-wise differences carry class separation, but much of it overlaps with evidence already retained by the mean. Learned aggregation in detectors such as InternalInspector and SIVR and so on raises the question of how much learning the weights adds. We study this with fixed layer-wise scores, distinguishing the gains from retaining more layers and from learning their weights. The classical equal-weight literature motivates us to examine both the available gain and estimation cost.
What the decomposition adds. A small Fisher ratio alone leaves open whether layer-wise differences carry little separation or whether their separation overlaps with the mean. Theorem 3.1 and Eq. (7) distinguish these cases through the interaction term. Table 8 identifies the latter, with training-fold Fisher-ratio estimates of 1.02–1.12. The training-size experiment then measures the effect of estimating weights from fewer samples.
Corollary D.4 takes the worst case for this interaction, giving the loose range of 1.68–9.88 you noted. Appendix D.3 retains the interaction in the exact expression evaluated in Table 8. The overlap is an empirical finding on these benchmarks. We will make this distinction clearer in the paper.
What fitting weights achieves. Across 18 conditions and the training-set sizes tested, the largest gain from covariance shrinkage selected by test AUROC is 0.79 AUROC points over the mean. At one sixteenth of the training fold, unshrunk Fisher falls up to 9.2 points below the unchanged mean, while Fisher with the best tested shrinkage gains at most 0.58 points. These comparisons keep probes and test folds fixed. We report the best mean test AUROC among the shrinkage strengths tried. We also used five-fold splits grouped by source and fitted the probes and Fisher weights on separate data. Separate calibration improves Fisher by 0.02–2.18 points across nine conditions, but its largest advantage over the mean remains 0.071 points with the relative ridge fixed at 10−6 and no shrinkage. The support and weighting comparisons show that retaining more layers improves prediction, while learning linear weights adds little on the same scores.
AUROC approximation. § 4.3 reports agreement within 0.3 AUROC points on HaluEval2 and HELM between predicted and observed gaps. Corollary 3.2 assumes Gaussianity only for the one-dimensional class-score differences of the two directions. The SNR decomposition requires no Gaussian assumption. Corollary 3.3 keeps explicit approximation errors. For signed errors eF and eM for Fisher and the mean, the gap error is ∣eF−eM∣, conservatively bounded by their absolute-value sum. On TrueFalse, where approximation errors can exceed the observed gap, we use measured AUROC to compare the two rules.
Q1. Generalization under distribution shift.
Yes. We tested transfer across benchmarks and observed a loss in performance. We used HaluEval2, TrueFalse and HELM because their ground-truth labels avoid the added uncertainty of LLM judging in our TruthfulQA evaluation. Across 12 transfers on Qwen-32B and Llama-3.1-8B, AUROC ranges from 71.9% to 94.2%. Using Table 1's depth-mean AUROCs as the in-domain reference, the largest drop is 16.9 points, from TrueFalse to HELM on Llama-3.1-8B. The smallest is 3.8 points, from HaluEval2 to TrueFalse on the same model. We will include the full results.
Q2. Numerical consistency.
We apologise for the inconsistency. Thank you for pointing this out. During our cluster migration and software environment updates, results from different runs were retained in different tables. We will use the Table 1 depth mean results as the reference and report the corresponding values consistently in Table 2 and the appendix. For Qwen-32B/HaluEval2, the reference AUROC is 84.03%.
Against the submitted baselines, the method with the highest AUROC in each condition remains unchanged. The main aggregation conclusions still hold. The corrected Fact-Probe comparison is reported below. We will also specify the HaluEval2 topic files and sample counts used in Appendix B and release the settings and results for each fold.