acceptodds
Under review as a conference paper at ICLR 2027

Digits Don't Lie: Detecting LLM Hallucinations through Activation Digit Distributions

Abstract

In large language models, wrong answers look as fluent as right ones, so the text is weak evidence about correctness, but the activations carry more. Many activation-based detectors read that evidence from thousands to tens of thousands of neuron-aligned coordinates, each tied to one neuron of one model, so the evidence is large and has no fixed meaning across models. We introduce DigitTrace, which summarizes how values are distributed, not which neurons produce them. Leading digits do this by describing magnitudes on a common relative scale. We reduce six activation streams at every fourth layer to their leading-digit frequencies, Benford-reference statistics, and activation moments, accumulating these during generation into a trace under a thousand coordinates. Against seven published detectors on three models and eight benchmarks under one protocol, DigitTrace comes within 0.55 AUROC points of the best of them while using one tenth as many coordinates, and it shows particular strengths on mathematics and science. The compact trace grows more informative during generation: on seven matched cohorts it gains 10.37 points from early to late, while a raw-state control barely moves, so later tokens add evidence at fixed size. The trace can be inspected, not only scored: in most settings, correct and incorrect answers favor different leading digits.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Hallucination Detection, Large Language Models, Internal Representations, Activation Statistics, Leading Digits, Benford's Law, Linear Probes

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_KEPpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DigitTrace, a compact activation-based approach for detecting incorrect LLM responses. Instead of relying on high-dimensional neuron-aligned representations, DigitTrace characterizes the distribution of activation magnitudes using leading-digit frequencies, Benford-reference statistics, and activation moments. These features are accumulated across generated tokens and extracted from six activation streams at selected transformer layers. The resulting representation is fed into an ensemble of sparse logistic regression probes to predict answer correctness.

Critical strengths
  1. A compact and interpretable alternative to neuron-aligned representations.
  2. Comprehensive experimental evaluation and analysis.
Critical weaknesses
  1. The independent contribution of leading-digit features is relatively limited.
  2. The empirical evidence does not sufficiently establish the distinctive advantage of leading-digit representations over alternative distributional summaries.
Questions for authors
  1. The central motivation emphasizes leading-digit distributions and Benford-inspired statistics. However, Table 2 shows that activation moments alone achieve 76.16 AUROC, compared with 76.68 for the complete DigitTrace representation.
  2. Although the paper demonstrates that leading-digit features are predictive, it remains unclear why decimal leading digits should be preferred over other compact distributional representations, such as logarithmic magnitude histograms or quantile-based statistics.
6 comments
AuthorsOpenReview ↗

Answers to Q1 and Q2

Thank you for recognizing DigitTrace as a new way of constructing representations, and for appreciating the breadth of our experiments. This is exactly the value we hope the paper offers: not just a performance number, but an in-depth, well-tested understanding of a new signal, built from extensive, rigorous, and transparently reported experiments. To answer your two questions, we analyzed our existing results in more detail and ran additional experiments on the same 50 splits.

1. Independent contribution of leading-digit features (Q1). Although the three feature families have similar average scores in Table 2, each carries different evidence.

  • By task. Leading-digit frequencies help most on open-ended QA and science multiple choice. On ARC-Challenge (averaged over the three models), moments alone reach 76.88 AUROC, and adding digits raises this to 80.15. Across the science group, digits add 1.77 points. The other two families have their own strengths. Reference statistics stand out for stability: they usually add a positive gain, and our case study finds them to be the most stably label-correlated coordinates (Section 6.2). Moments are strongest on math: on the three math datasets, moments alone average 84.84, only 0.41 below the best combination (85.25).
  • The gains are stable. All results above are averages over 50 random splits. Over all 24 main cells, the 95% confidence intervals for the gains from adding digits and from adding reference statistics both lie above zero. These gains are therefore reliable, not the product of a lucky split.
  • Configurations for different budgets. The full DigitTrace was fixed before we saw any test results. On principled grounds, we kept all three families so that the method would work reliably across diverse tasks and domains, and the results confirm this (see the overall robustness below). As discussed in Section 7, a single family can be used when resources are limited. Digit frequencies alone reach 75.63, which also supports the paper's core motivation: the leading-digit distribution itself carries substantial evidence about correctness.

2. Why decimal leading digits rather than log-magnitude histograms or quantiles (Q2). The log-magnitude histograms and quantile statistics you mention belong to the same family as leading digits. In fact, leading digits are a special case of a log-magnitude histogram. We chose this member of the family deliberately, because it comes with a principled reference that LLM activations in fact lie close to. Our reasons are as follows.

  • Fixed bin edges, no data needed. Leading digits fold all orders of magnitude together: 0.02, 2, and 200 all fall into the same bin. One set of edges therefore serves every activation stream, every layer, and every model. Quantile bins, by contrast, have edges estimated from data, with a separate set for each view. An unfolded log histogram first needs a value range, and activation ranges differ greatly across views and models, so the range also has to be set per view. Once edges depend on data, the same coordinate covers different value intervals in different views, which makes direct comparison across layers and models harder. Leading digits avoid this: each coordinate means the same thing in every model, so a probe trained on one model can be applied to another (see the transfer experiment below).
  • A ready-made theoretical reference: Benford's law. Benford's law is the only scale-invariant leading-digit distribution. Prior work finds that the weight matrices of trained LLMs are close to Benford's law, and we find the same for LLM activations (total variation about 0.033, Appendix F.8).
  • Deviations are directly readable. Because the edges are fixed, each bin has a plain meaning, such as "the share of activations whose leading digit is 1". This lets us say concretely what incorrect answers change: fewer activations lead with 1, and more lead with 5 and 9.
  • Why base 10. We use base 10 because it is the classical form of Benford's law, and its nine bins are intuitive and compact. With any other base above 2, the three reasons above still hold.
AuthorsOpenReview ↗

Additional Evidence

The fixed definition also makes cross-model transfer possible. In a new experiment, we applied a probe trained on Qwen3-8B directly to Qwen3-14B answers, and vice versa, without using any target-model labels. Over 16 cells (two directions × eight datasets), the transferred probe averages 63.87 AUROC. Supervised detectors such as MIND and SAPLMA take inputs whose size depends on the model, so they cannot be transferred this way. We therefore compare with HaloScope, which is not trained on correctness labels. Even though HaloScope is fitted directly on the target model’s own answers and uses target labels to choose its subspace size, score direction, and threshold, the transferred DigitTrace still outperforms it by 4.13 points on average.

Overall, DigitTrace is also highly robust. Across the 24 main cells (three models × eight datasets), it is the only one of the eight detectors that stays above chance in every cell, and it has the lowest mean standard deviation across the 50 splits (4.11). It also performs well on every kind of task: its worst rank is fifth, making it the only detector never placed in the bottom three, and its largest gap to the best detector in any cell (8.70) is the smallest of all eight.

In summary, leading digits are a deliberate choice within the family of log-magnitude binnings: their edges are fixed and data-free, they come with Benford's law as a reference, and every coordinate can be read directly. For detection, the three feature families carry different evidence and each contributes on different tasks, and together they make the full DigitTrace both accurate and stable. More broadly, DigitTrace offers a new, unconventional way to build representations, with competitive performance. We hope these new results and clarifications answer your questions, and we would be grateful if you would raise the score. We are happy to continue the discussion and hope to earn your further support.

Dear Authors,

Thanks for your rebuttal, my main concern about the effectiveness of DigitTrace remains:

Many baselines outperform DigitTrace, which makes DigitTrace not sound even though its performance is stable.

Yours, Reviewer

Reviewer_t3TyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors of this paper have introduced a new method, DigitTrace, to detect hallucinated generations of large language models (LLMs). Their method uses a readable, descriptive representation of activation functions sampled from 6 different streams of decoder blocks (every 4 layers) of LLMs. Applying linear probing to these compact, descriptive representations helps measure whether a hallucinated generation is ranked above the correct ones using AUROC. The paper’s results indicate that DigitTrace performs competitively with other deception approaches, while also being more memory- and compute-efficient than other methods. Additionally, the length of the generation affects the method’s accuracy, and as the model’s generation length increases, the detector's accuracy improves. The authors also provide further analysis to show where the detection signal originates in the descriptive representation.

Critical strengths
  1. Compared to other methods in the literature, DigitTrace is lightweight, and its descriptive representation is computed in one pass, so it does not have computational overhead or require extra compute. At the same time, DigitTrace shows competitive performance.
  2. The authors have done extensive experiments on two different model families and different benchmarks.
  3. Further analysis has been done to show where the detection signal of DigitTrace and its descriptive representation comes from.
Critical weaknesses
  1. Following the paper and its flow is a bit hard, specifically in the results section (5 and 6), because it's full of reported numbers while these results are presented in the tables and figures.
  2. While the paper has done its best to report experimental details, some choices are not justified.
  • Using GPT-4o mini for open-ended tasks as a judge for labels: can any other judge report the same results?
  • More importantly, selecting every fourth layer to sample streams is also not justified (why the authors selected every fourth layer)
  1. The results showing concentration of true answers activation on leading digits 1, 5, and 9 are not a general pattern in all of the models tested. So it needs more verification and investigation.

Main writing issues

  • The experimental setting (model and benchmarks used) of Table 2 is not mentioned.
  • Line 81: the sentence looks incomplete (“We evaluate on …”)
  • And as I mentioned, the flow is hard to follow.
Questions for authors
  • I would appreciate it if the authors could explain their choice of each of the four layers for sampling.
  • Considering differences in model architectures, is it possible to see different results in different architectures? I saw reports in Appendix F.14 for the Mistral model, and I encourage the authors (if possible) to report full results for this model too.
  • The authors have mentioned “meaning” as a superiority of DigitTrace, and they believe being defined by stats and streams makes their method different than methods based on neuron activations. But from the perspective they describe, “meaning’ the only thing that is different than neuron based methods is the descriptive representation they defined. I understand some stats (like moments) have meaning, but part of their built representation is leading digits, which do not have meaningful information. I would appreciate more elaboration on this.
  • Another question is about the claim that “digits only encode relative size”. This is because the leading digits completely ignore the magnitude of numbers, and from this perspective there is no difference between 0.14 and 0.000014. And because of this, the authors add statistical features to add information to the representation.
6 comments
AuthorsOpenReview ↗

Answers to the Weaknesses

Thank you for recognizing the value of our method, the breadth of our experiments, and our analysis of where the detection signal comes from. This is exactly the value we hope the paper offers: not just a performance number, but an in-depth, well-tested understanding of a new signal, built from extensive, rigorous, and transparently reported experiments. To answer your questions, we analyzed our results in more detail and ran additional experiments under the same 50-split protocol.

W1. Readability of Sections 5 and 6. Thank you for this helpful suggestion. Our original intent in writing the key numbers into the text was to let each claim be checked directly against a specific result. We will further improve readability in the next version.

W2. Justification of design choices.

  • Judge model. Yes, other judges give the same results. Only CoQA and TruthfulQA use a judge, and the other six datasets are labeled by deterministic rule-based checks. We used GPT-4o-mini because it is the judge used in OpenHalDet, the evaluation framework we follow (Appendix C). We re-judged all answers on the two open-ended datasets with two newer judges. Agreement is high:
\text{Judge pair} & \text{Agreement} \\\\ \hline \text{Ours (GPT-4o-mini) vs. GPT-4.1} & 95.8\\% \\\\ \text{Ours (GPT-4o-mini) vs. GPT-5.6 Luna} & 94.4\\% \\\\ \text{GPT-4.1 vs. GPT-5.6 Luna} & 95.5\\% \end{array}$$ - **Every fourth layer.** A stride of four **balances size and accuracy**. It was fixed before evaluation with reference to prior work, such as SAPLMA, one of our baselines. A new ablation over all 24 cells and 50 splits confirms the choice: $$\begin{array}{lccl} \text{Stride} & \text{Coordinates} & \text{Mean AUROC} & \text{Change vs. every 4th} \\\\ \hline \text{Every 2nd layer} & 1{,}536\text{ to }1{,}920 & 77.28 & +0.57\text{ on average, interval includes zero} \\\\ \text{Every 4th layer (ours)} & 768\text{ to }960 & 76.71 & \\\\ \text{Every 8th layer} & 384\text{ to }480 & 75.58 & -1.12\\ [-1.56, -0.69] \end{array}$$ Halving the stride doubles the size but gains only within noise. Doubling it costs accuracy. The pattern holds for all three models. **W3. Digits 1, 5 and 9.** First, the paper's finding is that incorrect answers shift leading-digit mass from 1 toward 5 and 9. It is reported as a pattern that holds in most settings, not as a universal law. To test how consistent it is, we ran a new test across cells. Of the 24 cells, the shift goes in the reported direction in 21, 18 and 19 for digits 1, 5 and 9, far more often than chance (sign test, p < 0.001, p = 0.01 and p = 0.003). In each of the three models, every digit also shifts in this direction on most datasets. The pattern is something the coordinates reveal, not an assumption the detector relies on. The probe is trained separately for each model and dataset, so it learns from the labels which digits signal errors in that setting, and detection still works in cells where the shift differs. **Main writing issues.** - **Table 2 setting.** Table 2 reports means over the 24 cells (three models × eight datasets) and 50 splits. - **Line 81.** In our version the sentence is complete, and no other reviewer flagged it: "We evaluate on Qwen3-8B, Qwen3-14B, and Llama-3.1-8B-Instruct across eight open-ended QA, math, science, and mixed benchmarks …". - **Flow.** We understand that the paper looks at activations from a less familiar angle, which can make a first read harder, and we will work to improve this!
AuthorsOpenReview ↗

Answers to Q1 to Q4

Q1. Every fourth layer. Please see W2. The new ablation shows it is an efficient trade-off between size and accuracy.

Q2. Other architectures. Following your suggestion, we completed Mistral-7B-Instruct-v0.3 on all eight datasets and added two models from further families, Granite-3.3-8B-Instruct (IBM) and GLM-4-9B-Chat (Zhipu AI). The experimental setup is the same as in the main evaluation. Mean AUROC over the eight datasets:

\text{Detector} & \text{Mistral} & \text{Granite} & \text{GLM} \\\\ \hline \text{DigitTrace} & 71.49 & 74.05 & 76.04 \\\\ \text{MIND} & 74.65 & 74.09 & 76.46 \\\\ \text{SAPLMA} & 74.77 & 74.11 & 75.72 \\\\ \text{SEP} & 73.90 & 73.33 & 74.24 \\\\ \text{PRISM} & 73.41 & 73.26 & 74.04 \\\\ \text{ICR Probe} & 68.53 & 68.49 & 72.05 \\\\ \text{HaloScope} & 55.76 & 55.60 & 58.85 \\\\ \text{Perplexity} & 55.85 & 61.65 & 43.75 \\\\ \hline \text{Projected raw (same size)} & 69.99 & 71.39 & 75.11 \end{array}$$ DigitTrace ranks second of eight detectors on GLM and third on Granite, within 0.5 points of the best detector in both, and fifth on Mistral, where the gap to the best remains moderate. In all three models it scores higher than a same-size projection of the raw states. **Q3. What leading digits mean.** A single digit has no semantics, but the leading-digit distribution of all activations in a view does. It describes where activation magnitudes sit within each order of magnitude, and three properties make it a meaningful coordinate system. - **Fixed definition.** Leading digits fold all orders of magnitude together (0.02, 2 and 200 fall into the same bin), so one set of bin edges serves every stream, layer and model, whereas a neuron index has no shared meaning across models. - **A theoretical reference.** Benford's law is the only scale-invariant leading-digit distribution. Prior work finds that the weight matrices of trained LLMs are close to Benford's law, and we find the same for LLM activations (total variation about 0.033, Appendix F.8). The reference statistics measure how far each view departs from it. - **Readable deviations.** Because each bin has a plain meaning, we can say concretely what incorrect answers change: fewer activations lead with 1, and more lead with 5 and 9. **Q4. "Digits only encode relative size".** Yes, exactly. Leading digits ignore the order of magnitude by design, which makes views from different layers and models comparable. The moments keep the scale and sign information that digits discard, so the two are complementary. For example, on ARC-Challenge (averaged over the three models), moments alone reach 76.88 AUROC, and adding digits raises this to 80.15. Your questions led us to add two newer judges, a layer-stride ablation, significance tests of the digit shift, a complete Mistral evaluation and two further models. Together, these results strengthen the paper’s central point: DigitTrace offers a new and unconventional way to build representations with competitive performance. We hope these results and clarifications answer your questions, and we would be grateful if you would raise your score. We are happy to continue the discussion and hope to earn your further support.
AuthorsOpenReview ↗

Follow-up and results on more benchmarks

We know your time is valuable, but as the discussion period is coming to a close, we wanted to follow up with you. If you feel that our point-by-point response has answered the questions you raised, we would be grateful if you would consider raising your score accordingly. In addition, we have run experiments on more benchmarks, which we share briefly below.

To further verify the generality of our method across diverse tasks and domains, we added seven mainstream benchmarks widely used to evaluate LLM reasoning, covering science, math, economics and business (GPQA-Ext, MMLU-Pro STEM, MMLU-Pro Econ, MMLU-Pro Biz, SciBench, OlympiadBench-Phys and AIME 2018–2025), and evaluated them under the same protocol as Table 1. These extend the evaluation to more types of tasks and domains, such as economics, business, olympiad-level physics and competition math.

\text{Method} & \text{GPQA-Ext} & \text{MMLU-Pro STEM} & \text{MMLU-Pro Econ} & \text{MMLU-Pro Biz} & \text{SciBench} & \text{OlympiadBench-Phys} & \text{AIME} & \text{Avg.} \\\\ \hline \text{Qwen3-8B} & & & & & & & & \\\\ \text{DigitTrace (ours)} & 66.16 & 81.23 & 71.78 & 79.65 & 75.72 & 65.85 & 91.80 & 76.03 \\\\ \text{MIND} & 59.43 & 81.48 & 75.53 & 82.80 & 75.18 & 59.72 & 89.22 & 74.77 \\\\ \text{SEP} & 64.64 & 77.90 & 70.61 & 78.78 & 72.12 & 57.06 & 79.63 & 71.53 \\\\ \text{SAPLMA} & 57.24 & 77.32 & 67.61 & 79.54 & 72.37 & 61.98 & 86.63 & 71.81 \\\\ \text{PRISM} & 51.00 & 79.01 & 71.34 & 82.65 & 75.49 & 59.04 & 83.23 & 71.68 \\\\ \text{ICR Probe} & 46.88 & 71.03 & 59.35 & 74.80 & 67.15 & 58.50 & 64.46 & 63.17 \\\\ \text{Perplexity} & 58.41 & 70.54 & 66.54 & 77.35 & 66.08 & 64.54 & 67.34 & 67.26 \\\\ \text{HaloScope} & 53.93 & 63.81 & 55.23 & 62.66 & 52.82 & 57.85 & 76.58 & 60.41 \\\\ \hline \text{Qwen3-14B} & & & & & & & & \\\\ \text{DigitTrace (ours)} & 67.92 & 74.62 & 70.71 & 83.34 & 70.09 & 59.12 & 90.32 & 73.73 \\\\ \text{MIND} & 67.31 & 81.48 & 71.14 & 82.07 & 75.28 & 57.15 & 94.19 & 75.52 \\\\ \text{SEP} & 66.30 & 74.22 & 63.45 & 78.17 & 68.75 & 58.07 & 76.76 & 69.39 \\\\ \text{SAPLMA} & 63.20 & 74.18 & 63.79 & 77.94 & 71.48 & 55.34 & 92.66 & 71.23 \\\\ \text{PRISM} & 56.58 & 71.54 & 60.83 & 75.33 & 69.88 & 61.60 & 81.43 & 68.17 \\\\ \text{ICR Probe} & 53.59 & 67.89 & 57.55 & 77.50 & 63.00 & 55.57 & 77.23 & 64.62 \\\\ \text{Perplexity} & 59.94 & 66.33 & 66.00 & 79.83 & 64.56 & 60.49 & 68.38 & 66.50 \\\\ \text{HaloScope} & 57.26 & 63.49 & 56.93 & 63.93 & 52.33 & 63.57 & 69.10 & 60.94 \end{array}$$ On Qwen3-8B, DigitTrace has the **highest average** of all eight detectors and ranks first on four of the seven benchmarks. **Science reasoning remains its strength**: on Qwen3-8B it ranks first on all three science benchmarks (GPQA-Ext, SciBench and OlympiadBench-Phys), and on GPQA-Ext it ranks first for both models. It also ranks first on competition math (AIME) for Qwen3-8B. On MMLU-Pro, it ranks second on STEM and Econ for both models and first on Biz for Qwen3-14B. Averaged over the two models, DigitTrace **ranks first or second on all seven benchmarks**, and it is in the top three in 13 of the 14 results. These results further support the paper's central point: DigitTrace builds representations from a new perspective and is competitive with the strongest detectors. They also show that correctness evidence need not be tied to individual neurons, since it can be read from how activation values are distributed. This opens a new direction for representations whose coordinates mean the same in every model and can be read directly, and we believe it will inspire further work.
Reviewer_z1kEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DigitTrace, a compact, online representation for detecting hallucinations from model activations. Whereas conventional activation-based detectors operate on thousands of neuron-aligned features that are model-specific and difficult to interpret, DigitTrace seeks to identify a smaller set of statistics that retain competitive detection performance while potentially improving portability and interpretability.

The method extracts 16 statistics from each of six activation streams at every fourth layer. These comprise nine leading-digit frequency features derived from the fractional part of (\log_{10}|a|), four statistics measuring agreement with the Benford distribution (MAD, Jensen-Shannon divergence, KL divergence, and Pearson correlation), and three moment-based features (mean, standard deviation, and mean absolute activation). Depending on model depth, this produces a representation of 768 to 960 features, which can be accumulated online with constant memory.

For classification, the authors introduce a factorized readout consisting of five random partitions, each divided into six disjoint coordinate groups. Each group is evaluated using an L1-regularized logistic regression probe, and the standardized scores are averaged. Conceptually, this approach resembles random-subspace bagging rather than a fundamentally new classification architecture.

The theoretical analysis considers an independent multinomial model, under which the log-likelihood ratio is linear in digit counts and AUROC increases proportionally to the square root of the number of observations. These results provide motivation for the representation, although the underlying statistical arguments are relatively standard.

The evaluation covers three models, eight datasets, 50 shared data splits, and seven OpenHalDet baselines. DigitTrace achieves a mean AUROC of 76.68, compared with 77.23 for MIND, 78.75 for a tuned full-raw-activation probe, and 75.73 for a same-size random projection of raw activations. The moments-only variant achieves 76.16, while a post hoc selected combination of digit features and moments reaches 76.92. A text-based Style7 baseline achieves 69.43.

Critical strengths

The evaluation protocol is unusually careful and substantially strengthens the empirical analysis. All detectors are evaluated using shared generations, labels, and 50 data splits, improving the comparability of the results. The authors also include dataset-level and leave-one-out robustness analyses. The uncertainty estimates resample evaluation cells rather than splits, which is an important methodological detail when interpreting the reported intervals.

A further strength is the authors’ willingness to report results that qualify their own method’s advantages. The tuned raw-activation probe outperforms DigitTrace, feature collection is approximately twice as slow as raw-activation collection, and cross-family transfer performs at chance. Science multiple-choice detection also fails under transfer. Rather than obscuring these limitations, the authors explicitly narrow the scope of their claims in Appendix H. This transparency makes the empirical conclusions more credible, although it also limits the method’s demonstrated practical advantages.

The readout ablations are well controlled and help disentangle the contributions of the ensemble components. Under a matched fitting budget, score averaging contributes a gain of 0.82, while partitioning contributes an additional 0.45. This analysis provides a clearer account of where the performance improvements originate than an evaluation of the full ensemble alone.

Finally, the fixed-cohort trajectory design is a sound methodological choice. By keeping the evaluated answers fixed as the observation cutoff changes, the authors avoid a potential confound in which the composition of the evaluation set changes across trajectory lengths. This makes the resulting comparisons over accumulated observations more interpretable.

Critical weaknesses

One question I had is how much the digit-based features contribute beyond simpler activation statistics. The moments-only variant achieves an AUROC of 76.16, which is close to the full DigitTrace representation at 76.68, while the digits-only variant achieves 73.91. These results suggest that the moments account for a substantial portion of the predictive signal. I would have liked to see a more detailed analysis of when the digit-based features provide additional value and whether that benefit is consistent across datasets.

I am also somewhat uncertain about the advantage of DigitTrace over a simpler representation. At a matched dimensionality, the improvement over a random projection of raw activations is 0.95 AUROC points, with a confidence interval of [−0.92, 2.79]. This suggests that the performance difference is relatively modest. The fixed interpretation of DigitTrace's features is a potentially useful property, but I would have liked to see more evidence that it leads to practical benefits, particularly given that cross-family transfer remains at chance.

I also think the relationship between the learned signal and response length or style deserves further investigation. Regressing out seven text-based counts reduces AUROC to 66.04, while length alone reaches 85.4 AUROC on SVAMP. These findings suggest that textual characteristics may account for a meaningful portion of the predictive signal. Additional controls could help clarify how much information DigitTrace captures beyond these characteristics and whether its performance reflects a more general signal of answer reliability.

Finally, I found the comparison with raw activations somewhat difficult to interpret because the two methods use different amounts of information. The raw probe uses all tokens across many layers, whereas DigitTrace uses only the last five tokens at five intermediate layers. This makes it challenging to isolate the contribution of the representation itself. Similarly, the trajectory analysis seems to show DigitTrace recovering from a lower initial performance level, rather than clearly surpassing the raw baseline. A more closely matched comparison would help clarify whether DigitTrace offers an advantage as more observations become available.

Questions for authors

See weaknesses.

6 comments
AuthorsOpenReview ↗

Answers to W1 and W2

Thank you for an exceptionally careful review, and for valuing our rigorous evaluation protocol and transparent reporting. This reflects what we hope the paper offers: not just a performance number, but a reliable answer to how much a model's activations reveal about whether its answer is correct. To address your four points, we analyzed our existing results in more detail and ran additional experiments on the same 50 splits.

W1. When do digit features add value, and is it consistent? Table 2 averages over all tasks, so the three families look close, but each carries different evidence and plays a different role. The full DigitTrace was fixed on principled grounds before we saw any test results. By design it keeps all three families, aiming at robustness and generalization across tasks, domains and models, and the results bear this out. In practice, when peak performance is not required, one can also choose one to three families as needed.

  • When digits help. Digit frequencies alone reach 75.63 AUROC overall, so the leading-digit distribution itself carries substantial correctness evidence. Their added value shows up in two places that the average hides: they add evidence where moments are weak, and they keep detection working in the hardest cases. Digits help most on open-ended QA and science multiple choice, where moments alone are clearly weaker. On ARC-Challenge, for example, adding digits raises Qwen3-8B from 75.11 to 80.45 and Qwen3-14B from 74.41 to 78.46. On TruthfulQA, it raises Llama-3.1-8B from 64.70 to 68.43.
  • The hardest case. On GPQA-Diamond with Llama-3.1-8B, all seven baselines and both raw-state controls (full raw and projected raw) fall below chance (the best reaches only 49.83), and moments alone stay at chance (50.40). DigitTrace reaches 53.38, the only detector above chance in this cell, and the difference comes from the digit features.
  • Consistency. All results above are averages over 50 random splits. Over all 24 main cells, the 95% confidence interval for the gain from adding digits lies above zero. The gain is therefore reliable, not the product of a lucky split. Its direction is also stable: incorrect answers almost always have fewer activations leading with 1, and usually more leading with 5 and 9.

W2. Advantage over a simpler representation, and the value of fixed meaning. Projected raw is a direct compression of the full raw states: it uses the value of every neuron and even keeps slightly more coordinates than DigitTrace. DigitTrace still scores 0.95 points higher on average, so its summary loses no correctness evidence, and it is clearly ahead on the tasks where digits help most. On open-ended QA it is 5.52 points higher on average, up to 8.70, and ahead in every case. On science it is 3.14 points higher on average, up to 9.03. Projected raw also falls below chance in one cell (44.35), while DigitTrace never does. What a projection cannot offer is fixed coordinate meaning, and this brings the following practical benefits.

  • Transfer within a model family. A probe trained on Qwen3-8B can score Qwen3-14B answers directly, and vice versa, without any target-model labels. Over 16 cells (two directions × eight datasets), it averages 63.87 AUROC, 4.13 points above HaloScope on average, even though HaloScope is fitted on the target model and tuned with its labels. On CoQA, the transferred probe reaches 68.64 and 68.01 in the two directions, close to probes trained with target labels (73.22 and 73.54). Detectors trained on hidden states, such as MIND and SAPLMA, as well as full raw and projected raw, cannot be transferred this way, because their input size changes with the model. Zero-shot transfer across model families remains a common limitation of activation-based detectors, and raw states are no exception. Still, because DigitTrace's coordinates mean the same in every model, such as "the share of activations whose leading digit is 1", findings like the digit shift above can be stated directly and compared across families. Projected coordinates cannot do this.

DigitTrace also adds value to other detectors. Fused with MIND, it reaches 79.48, and fused with the tuned full raw probe, 80.36, both above every single detector we evaluate. Its scores correlate only moderately with both (0.52 and 0.60), so DigitTrace supplies evidence that neuron-aligned detectors miss, at a cost of fewer than 1,000 extra coordinates.

AuthorsOpenReview ↗

Answers to W3 and W4, and Additional Evidence

W3. Length and style. DigitTrace is not a length or style shortcut. We first explain the two numbers you mention, then give the control you suggest.

  • 66.04 is a lower bound, not the share of text form. It is what remains after linearly removing the seven text counts (such as the numbers of characters, words and digits) from the DigitTrace score. This step removes every part of the score that correlates with these counts. Since correct and incorrect answers can differ in length, part of the genuine correctness evidence that DigitTrace reads from activations also correlates with length, and is removed as well. 66.04 therefore underestimates what DigitTrace reads beyond text form. A more direct test is already in the paper: adding Style7 (the probe on these seven counts) to DigitTrace gives no gain (−0.05), while adding DigitTrace to Style7 improves it in all 24 cells. In other words, DigitTrace contains what the text counts know, but not the other way around.
  • The 85.4 on SVAMP. This number comes from Appendix A.5 (Qwen3-8B), where DigitTrace reaches 90.18 in the same table. Among SVAMP answers of similar length, DigitTrace beats length by 13-16 points for every model, so even here it reads evidence that length does not carry.
  • New control: answers of similar length. We group test answers by length quartile, with edges taken from the development split, and evaluate only within each group, where length carries little information. Within groups, DigitTrace reaches 73.22 AUROC, against 56.76 for length and 63.96 for Style7. It beats length by 16.46 points [13.98, 19.00], ahead in all 24 main cells, and beats Style7 by 9.26 [7.02, 11.53], ahead in 23 of 24. Adding log-length to DigitTrace changes nothing (−0.33), because the trace already contains length. DigitTrace's signal thus goes well beyond length and text form.

W4. Information budget and trajectory.

  • What each method reads. The paper's raw probe reads only the last five generated tokens, averaged at five middle layers, but keeps every neuron: 20,480 to 25,600 coordinates per answer. DigitTrace reads all generated tokens and summarizes them, across more layers, in fewer than 1,000 coordinates. Raw therefore has more than 20 times as many coordinates as DigitTrace, while DigitTrace covers the tokens more completely.
  • Matched comparison. To isolate the representation itself, we let raw also cover all generated tokens: we averaged its hidden states over all generated tokens, projected them to DigitTrace's size, and fitted both with the same regularization grid, so fitting costs the same. As with projected raw above, this raw is still computed from the value of every neuron. The two are on par (76.68 for DigitTrace, 76.81 for raw), also when raw reads the same layers and streams as DigitTrace (76.86). So even with tokens and size matched, the predefined statistics lose no correctness evidence, while their coordinates stay readable.
  • Trajectory. You asked whether DigitTrace gains an advantage as more of the answer is observed. It does: its evidence grows much faster. On the fixed cohorts (the same answers scored at every cutoff), at constant size, it gains 10.37 AUROC points, while the raw of the matched comparison above gains only 3.8 to 5.3. The more of the answer is observed, the more DigitTrace gains over raw states.

Additional evidence. Overall, DigitTrace is highly robust. Across the 24 main cells (three models × eight datasets), it is the only one of the eight detectors that stays above chance in every cell, and it has the lowest mean standard deviation across the 50 splits (4.11). It also performs well on every kind of task: it is the only detector never placed in the bottom three, and it shows clear strengths on science and math (Table 1).

In short, digits add evidence that the other two families lack, consistently across datasets. At the same size, DigitTrace is as accurate as a compression of the raw states, while staying readable and transferable within a family, and its signal goes well beyond length and style. More broadly, DigitTrace offers a new, unconventional way to build representations while remaining competitive. We hope these results answer your questions and would be grateful if you would reconsider the score. We are happy to continue the discussion and hope to earn your further support.

Response to authors

Thank you for the thorough response and the new experiments. I apologize for getting back so late during the discussion period.

Two Remaining questions.

  1. Trajectory endpoint. The response reports that DigitTrace gains 10.37 AUROC points over the fixed cohorts, compared with 3.8–5.3 for the matched pooled raw. Since the matched pooled raw grows more than that control, could the authors report both methods' AUROC at the first and final cutoffs on the same cohorts, with confidence intervals? This would show whether DigitTrace ends ahead of matched raw or the growth is showing that Digit Trace is catching up to it.

  2. What enables within-family transfer. The Qwen3-8B - Qwen3-14B transfer (63.87 AUROC) is a useful demonstration of fixed coordinate meaning. This finding is interesting to me. I believe it could show some utility of using digits (which upon first inspection seems obscure) in comparison to other commonly used features. To see where this benefit comes from, could the authors report, on the same 16 cells:

    • (a) transfer using the moment statistics alone, since these also have fixed meaning across models;
    • (b) perplexity, which requires no labels from either model, as a reference point; and
    • (c) if feasible, a raw-state baseline transferred through a linear map between the two models' pooled hidden states, fitted on shared unlabelled prompts. If moments alone transfer comparably, the benefit would come from fixed-meaning summaries generally rather than from leading digits specifically.

I primarily would like more evidence that that the digit usage is a sound feature, and I am mainly trying to propose experiments 1 and 2 that might help support digit usage.

I will reconsider my score in light of these answers.

Reviewer_uYCxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This [paper summarises 6 activations streams every fourth layer. For each stream and layer it computes 16 metrics. These are accumulated online during generation into a trace of a few 100 coordinates. The trace is scored by an ensemble of penalized logistic probes. The validation is done as follows. 3 models, 8 benchmarks = 24 cells, with 50 splits and 7 baselines rerun under OpenHalDet. Roc is 76.68 and slightly below below MIND but with one tenth of its coordinates. It leads on GPQA and is strong on math. ROC rises during generation (+10.4 on seven matched cohorts, against +0.9 for a raw control). Incorrect answers shift leading-digit mass from 1 toward 5 and 9. Fusing its score with MIND’s adds approx 3 points

Critical strengths

The representation is new and the protocol careful. Permutation-invariant and model-agnostic summary of activations is appreciated. The evaluation is well controlled.. Resyults are believable.

Critical weaknesses

Ablation studies show leading digits contribute little. moments in Tab 2 alone reach 76.16 compared to 76.68 for the full trace, so digits and Benford statistics add about 0.5. A random projection of raw states to the same size reaches 75.73. The tuned raw probe reaches 78.75, or about 81.2 with the expanded regularisation grid: roughly 4.5 points above proposed and above every others. So claims do not really holds up. The utility claims are also weakly supported. Compactness is not a binding constraint as 10k floats per answer is trivial to store. Fixed meaning across models is contradicted by the finding that score directions do not transfer across model families. Results differ by small margins. The accumulation comparison is mismatched. The raw control averages the last 5 tokens, while DigitTrace pools all generated tokens.Fusion gains were expected with decorrelated models.

Questions for authors

How does the proposed method compare in the accumulation experiment against a running mean pooled raw state over all generated tokens, projected to the same size.

4 comments
AuthorsOpenReview ↗

Answers to W1

Thank you for recognizing that our representation is new, that our permutation-invariant, model-agnostic summary of activations is valuable, and that our evaluation is well controlled and our results believable. This is what we aimed for: not just a performance number, but findings that readers can learn from and trust, each backed by extensive, well-controlled experiments. We kept to the same standard in this response. We analyzed our existing results in more detail and ran additional experiments on the same 50 splits, including the control you asked for.

W1. Contribution of leading digits, and the comparison with raw states.

W1.1 What leading digits add beyond moments. Table 2 averages over all tasks, so the three families look close there, but each carries different evidence and plays a different role. The full DigitTrace was fixed on principled grounds before we saw any test results. By design it keeps all three families, aiming at robustness and generalization across tasks, domains and models, and the results bear this out.

  • When digits help. Digit frequencies alone reach 75.63 AUROC overall, so the leading-digit distribution itself carries substantial correctness evidence. Their added value shows up in two places that the average hides: they add evidence where moments are weak, and they keep detection working in the hardest cases. Digits help most on open-ended QA and science multiple choice, where moments alone are clearly weaker. On ARC-Challenge, for example, adding digits raises Qwen3-8B from 75.11 to 80.45 and Qwen3-14B from 74.41 to 78.46. On TruthfulQA, it raises Llama-3.1-8B from 64.70 to 68.43.
  • The hardest case. On GPQA-Diamond with Llama-3.1-8B, all seven baselines and both raw-state controls (full raw and projected raw) fall below chance, and moments alone stay at chance (50.40). DigitTrace reaches 53.38, the only detector above chance in this cell, and the difference comes from the digit features.
  • Consistency. All results above are averages over 50 random splits. Over all 24 main cells, the 95% confidence interval for the gain from adding digits lies above zero. This gain is therefore reliable, not the product of a lucky split.

W1.2 How DigitTrace compares with raw states. Both comparisons you mention, the projection of raw states and the tuned raw probe, in fact support the paper's claims.

  • A projection of the raw states. Projected raw (75.73) is a direct compression of the full raw states: it uses the value of every neuron and even keeps slightly more coordinates than DigitTrace. DigitTrace still scores 0.95 points higher on average, so its summary loses no correctness evidence, and it is clearly ahead on the tasks where digits help most. On open-ended QA it is 5.52 points higher on average, up to 8.70. On science it is 3.14 points higher on average, up to 9.03.
  • The gap to the tuned raw probe. The 78.75 already includes the 2.41 point gain from the expanded grid (76.34 at a fixed C of 0.3, Appendix E), so it cannot be added again to reach 81.2. We apologize if our wording in Section 6.4 left this unclear. Against full raw at the fixed C, DigitTrace is 0.34 points ahead overall. It leads further on science and open-ended QA, by 4.38 and 2.37 points on average, for example by 6.62 on GPQA-Diamond and 4.46 on CoQA.
AuthorsOpenReview ↗

Answers to W2, W3 and Q1

W2. Utility: compactness, fixed meaning and transfer, and robustness.

  • Compactness serves readability. The value of a small trace goes beyond storage. In fewer than 1,000 coordinates, it covers all generated tokens, every fourth layer and six activation streams, and each coordinate is a named statistic, defined the same way in every model. The next two points build on these fixed definitions.
  • Fixed meaning and transfer. Fixed meaning refers to the coordinates, not to the score direction a probe learns, and the fixed definition is what makes cross-model transfer possible. We applied a probe trained on Qwen3-8B directly to Qwen3-14B answers, and vice versa, without using any target-model labels. Over 16 cells (two directions × eight datasets), the transferred probe averages 63.87 AUROC. MIND, SAPLMA and raw probes take inputs whose size depends on the model, so they cannot be transferred this way. We therefore compare with HaloScope, which is not trained on correctness labels. Even though HaloScope is fitted directly on the target model's own answers and tuned with its labels, the transferred DigitTrace still outperforms it by 4.13 points on average, and by up to 14.05 points (CoQA). Zero-shot transfer across model families remains a common limitation of activation-based detectors, and raw states are no exception. Still, because DigitTrace's coordinates mean the same in every model, such as "the share of activations whose leading digit is 1", findings like the digit shift of incorrect answers (Section 6.2) can be stated directly and compared across families.
  • Robustness beyond the mean. Averaged over tasks from many domains, the strongest detectors are indeed close, so what sets one apart is whether it holds up across tasks. Here DigitTrace is highly robust. Across the 24 main cells, it is the only one of the eight detectors that stays above chance in every cell, and it has the lowest mean standard deviation across the 50 splits (4.11). It is also the only detector never placed in the bottom three. In addition, its readout is a convex fit, so reruns with the same settings give identical results.

W3. The accumulation comparison and fusion gains.

  • Accumulation. We ran the matched control your question proposes (see Q1 for details). Under it, DigitTrace's evidence still grows about two to three times as fast as that of raw.
  • Fusion. The low correlation is itself informative. DigitTrace's scores correlate only moderately with MIND and the tuned full raw probe (0.52 and 0.60), because it summarizes how activation values are distributed, not which neurons produce them. So DigitTrace supplies evidence that neuron-aligned detectors miss, and it is also strong on its own. Fused with MIND, it reaches 79.48, significantly above both, and fused with the tuned full raw probe, 80.36. Both are above every detector in Table 1.

Q1. Running mean of raw states over all generated tokens, projected to the same size. We ran exactly this control. At each cutoff, we averaged the raw hidden states over all tokens generated so far and projected this running mean to DigitTrace's size.

  • Accumulation. DigitTrace's evidence grows much faster. On the seven matched cohorts of the accumulation experiment (the same answers scored at every cutoff), at constant size, it gains 10.37 AUROC points, while running-mean raw gains only 3.8 to 5.3, whether its regularization is fixed or tuned.
  • Endpoint. DigitTrace keeps the correctness evidence of the raw states. Fitted with the same regularization grid and matched in tokens and size, the two are on par over all 24 cells (76.68 for DigitTrace, 76.81 for raw), also when raw reads the same layers and streams as DigitTrace (76.86). Raw here is a projection of every neuron's value, while DigitTrace reaches the same level with statistics defined in advance, whose coordinates stay readable and comparable across models.

In short, digits add evidence that the other two families lack, and under your matched control DigitTrace is on par at the endpoint while its evidence grows much faster. More broadly, DigitTrace offers a new, unconventional way to build representations while remaining competitive. We hope these results answer your questions and would be grateful if you would reconsider the score. We are happy to continue the discussion and hope to earn your further support!

AuthorsOpenReview ↗

Follow-up and results on more benchmarks

We know your time is valuable, but as the discussion period is coming to a close, we wanted to follow up with you. If you feel that our point-by-point response has answered the questions you raised, we would be grateful if you would consider raising your score accordingly. In addition, we have run experiments on more benchmarks, which we share briefly below.

To further verify the generality of our method across diverse tasks and domains, we added seven mainstream benchmarks widely used to evaluate LLM reasoning, covering science, math, economics and business (GPQA-Ext, MMLU-Pro STEM, MMLU-Pro Econ, MMLU-Pro Biz, SciBench, OlympiadBench-Phys and AIME 2018–2025), and evaluated them under the same protocol as Table 1. These extend the evaluation to more types of tasks and domains, such as economics, business, olympiad-level physics and competition math.

\text{Method} & \text{GPQA-Ext} & \text{MMLU-Pro STEM} & \text{MMLU-Pro Econ} & \text{MMLU-Pro Biz} & \text{SciBench} & \text{OlympiadBench-Phys} & \text{AIME} & \text{Avg.} \\\\ \hline \text{Qwen3-8B} & & & & & & & & \\\\ \text{DigitTrace (ours)} & 66.16 & 81.23 & 71.78 & 79.65 & 75.72 & 65.85 & 91.80 & 76.03 \\\\ \text{MIND} & 59.43 & 81.48 & 75.53 & 82.80 & 75.18 & 59.72 & 89.22 & 74.77 \\\\ \text{SEP} & 64.64 & 77.90 & 70.61 & 78.78 & 72.12 & 57.06 & 79.63 & 71.53 \\\\ \text{SAPLMA} & 57.24 & 77.32 & 67.61 & 79.54 & 72.37 & 61.98 & 86.63 & 71.81 \\\\ \text{PRISM} & 51.00 & 79.01 & 71.34 & 82.65 & 75.49 & 59.04 & 83.23 & 71.68 \\\\ \text{ICR Probe} & 46.88 & 71.03 & 59.35 & 74.80 & 67.15 & 58.50 & 64.46 & 63.17 \\\\ \text{Perplexity} & 58.41 & 70.54 & 66.54 & 77.35 & 66.08 & 64.54 & 67.34 & 67.26 \\\\ \text{HaloScope} & 53.93 & 63.81 & 55.23 & 62.66 & 52.82 & 57.85 & 76.58 & 60.41 \\\\ \hline \text{Qwen3-14B} & & & & & & & & \\\\ \text{DigitTrace (ours)} & 67.92 & 74.62 & 70.71 & 83.34 & 70.09 & 59.12 & 90.32 & 73.73 \\\\ \text{MIND} & 67.31 & 81.48 & 71.14 & 82.07 & 75.28 & 57.15 & 94.19 & 75.52 \\\\ \text{SEP} & 66.30 & 74.22 & 63.45 & 78.17 & 68.75 & 58.07 & 76.76 & 69.39 \\\\ \text{SAPLMA} & 63.20 & 74.18 & 63.79 & 77.94 & 71.48 & 55.34 & 92.66 & 71.23 \\\\ \text{PRISM} & 56.58 & 71.54 & 60.83 & 75.33 & 69.88 & 61.60 & 81.43 & 68.17 \\\\ \text{ICR Probe} & 53.59 & 67.89 & 57.55 & 77.50 & 63.00 & 55.57 & 77.23 & 64.62 \\\\ \text{Perplexity} & 59.94 & 66.33 & 66.00 & 79.83 & 64.56 & 60.49 & 68.38 & 66.50 \\\\ \text{HaloScope} & 57.26 & 63.49 & 56.93 & 63.93 & 52.33 & 63.57 & 69.10 & 60.94 \end{array}$$ On Qwen3-8B, DigitTrace has the **highest average** of all eight detectors and ranks first on four of the seven benchmarks. Science reasoning remains its strength: on Qwen3-8B it ranks first on all three science benchmarks (GPQA-Ext, SciBench and OlympiadBench-Phys), and on GPQA-Ext it ranks first for both models. It also ranks first on competition math (AIME) for Qwen3-8B. On MMLU-Pro, it ranks second on STEM and Econ for both models and first on Biz for Qwen3-14B. Averaged over the two models, DigitTrace **ranks first or second on all seven benchmarks**, and it is in the top three in 13 of the 14 results. These results further support the paper's central point: DigitTrace builds representations from a new perspective and is competitive with the strongest detectors. They also show that correctness evidence need not be tied to individual neurons, since it can be read from how activation values are distributed. This opens a new direction for representations whose coordinates mean the same in every model and can be read directly, and we believe it will inspire further work.
AuthorsOpenReview ↗

Revised PDF

We have uploaded a revised PDF that incorporates the results and clarifications from the discussion. New or rewritten text is shown in blue. The main changes are:

  • Introduction and Section 5.2. Robustness across tasks, results on three further model families and on seven further benchmarks, and transfer within a model family.
  • Section 6. The matched raw-state control for accumulation (6.1), why all three feature families are kept and where each helps (6.2), and the length-matched comparison with a clearer comparison to raw hidden states (6.4).
  • Appendix. New subsections on robustness (D.1) and further benchmarks (D.2), and additions on why leading digits (B.1), judge agreement (C), feature families by task (F.1), the matched raw control (F.3), length (F.4), fusion over all models and datasets (F.6), the digit shift (F.8), further model families (F.14), within-family transfer (F.15) and the layer stride (H). We also added four references that support this new content.

Thank you for your constructive feedback, which has helped make the paper clearer and stronger!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.