acceptodds
Under review as a conference paper at ICLR 2027

LatticeSpike: Real-Time Token-Level Hallucination Detection via Box Embeddings and Spiking Dynamics

Abstract

Language models hallucinate mid-generation, yet most detectors only judge text after generation is complete. To address this gap, we propose LatticeSpike, a biologically inspired lightweight detector that runs alongside a frozen language model and scores every token as it is produced, using only the tokens generated so far. At each step, signals read from the language model are embedded as a box whose volume reflects the model's semantic spread. A recurrent spiking neural network, with dynamics modulated by that volume, continuously predicts its own next state. Hallucinated spans break this rhythm and the resulting prediction error, trained against token labels with a ranking objective, is the detection score. Furthermore, we release two human-validated and statistically tested benchmark datasets, spanning two distinct language model families, comprising 10,915 and 10,919 examples with token-level labels from a six-judge LM panel; we call these datasets THRIVE-Llama and THRIVE-Gemma, respectively. We evaluate LatticeSpike against twelve baselines, on four corpora: THRIVE-Llama, THRIVE-Gemma, RAGTruth, and TriviaQA, under a strictly causal protocol. Notably, across the four corpora and multiple generator families, LatticeSpike outperforms all tested baselines in 96/101 individual evaluation cells (95% win rate) spanning average precision (AP) and AUROC, as well as recall, precision, and F1 score at variable false-positive rates (0.1%-5%), under full-corpus and cross-task evaluation protocols. We also provide extensive ablation experiments showing which components are most important to LatticeSpike's success, and statistical testing to show where LatticeSpike's advantage is significant and where it should not be overinterpreted. The code for the project and the released benchmarks can be found at: https://anonymous.4open.science/r/LatticeSpike-8EB2.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Hallucination detection, Language models, Spiking neural networks, Box embeddings, Datasets and benchmarks

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wj6kOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LatticeSpike runs alongside a frozen language model and detects hallucinations token by token as the response is being generated. It first maps hidden states and prediction signals into box embeddings, where the box volume represents the spread of the model’s semantic state. A volume-modulated recurrent spiking network then predicts its next state, and the prediction error is used as the token-level hallucination score. The authors also introduce the THRIVE-Llama and THRIVE-Gemma datasets and compare the method with 12 baselines on four corpora under a strictly causal protocol. The method achieves strong average precision and AUROC results overall.

Critical strengths

It supports real-time, token-level hallucination detection without using future tokens or modifying the generator.
The paper releases two relatively large datasets and validates them through multiple-model judging, human annotation, and statistical testing.

Critical weaknesses

LatticeSpike does not outperform all alternatives at every low-false-positive operating point. For some fixed FPR settings, Transformer-based replacement models perform better.
The THRIVE datasets mainly use a small number of open-source generators, greedy decoding, and a fixed response length. It is still unclear whether the method will generalize to larger models, stochastic decoding, multilingual generation, or interactive systems.

Questions for authors

LatticeSpike does not outperform all alternatives at every low-false-positive operating point. For some fixed FPR settings, Transformer-based replacement models perform better.
The THRIVE datasets mainly use a small number of open-source generators, greedy decoding, and a fixed response length. It is still unclear whether the method will generalize to larger models, stochastic decoding, multilingual generation, or interactive systems.

4 comments
AuthorsOpenReview ↗

Initial Comment

We thank reviewer wj6k for their time and review of our paper. Below we provide experimental results, answers, and clarifications regarding their review points.

AuthorsOpenReview ↗

Response 1

Reviewer wj6k - Response 1

Reviewer Point

"LatticeSpike does not outperform all alternatives at every low-false-positive operating point. For some fixed FPR settings, Transformer-based replacement models perform better."

Author Response

We agree that LatticeSpike does not uniformly dominate the parameter-matched Transformer replacement at every low-FPR operating point. However, it is important to note that this result comes from the controlled architectural ablation, where only the SNN dynamics layer is replaced for an attention mechanism, while the box encoder, six input channels, training objective, data split, checkpoint selection, and evaluation protocol are held fixed. Thus, the transformer variant similarly leverages several of our novel architectural contributions — particularly the box encoder and the concretization channel (our largest signal driver), alongside the rest of the six-channel input, the reconstruction-anomaly scoring, and the training objective — a likely explanation as to what drove its performance beyond attention alone.

Nevertheless, we believe future work could explore a hybrid architecture, as noted in the future work section of the paper. Such a framework would ideally leverage and combine the relative strengths from each design, such as the SNN supplying ranking quality (AP and AUROC) with the transformer supplying detection quality at tighter FPR budgets.

Reviewer Point

"The THRIVE datasets mainly use a small number of open-source generators, greedy decoding, and a fixed response length."

Author Response

Generators. Rather than focusing on breadth of generators, with THRIVE we instead wanted to focus on depth, while still expanding our reach to two distinct generator families and models (Llama-3.2-1B-Instruct and Gemma-2-2b-it); thus, we opted to release only two datasets but of of high quality, each having been validated by humans and statistically tested for quality control purposes.

Moreover, the dataset construction pipeline and code exists such that future work can easily build on these two datasets by expanding THRIVE's reach to other generator families.

As for LatticeSpike, our detection framework as tested extends well past the two generator mark. Beyond THRIVE, the paper evaluates LatticeSpike on RAGTruth (Llama-2-7b-chat) and TriviaQA (Qwen3-8B), and in this rebuttal we add three additional generators, one of which is a larger model (Qwen3-32b, shown below) and the other two which are very recent generators (Gemma 4 E4B and Qwen3.5-9B, shown in our response to Reviewer Cukc). LatticeSpike successfully generalizes across all 7 generators despite these varying in size, family, and recency — demonstrating its robustness and versatility.

Decoding. THRIVE uses greedy decoding by design, so that hallucinations reflect the generator's most likely output rather than sampling noise. Stochastic decoding is nonetheless already covered by our evaluation: every RAGTruth response (all 2,965) was sampled at temperatures between 0.7 and 1.0, so all RAGTruth results in the paper, and those included in the comparison with CHARM (Reviewer Cukc rebuttal), are on stochastically decoded text.

Response length. The 512-token limit is a generation cap, not a fixed length; actual responses vary widely and the vast majority of them fall within this limit. On THRIVE-L, responses have a median of 29 tokens (10th–90th percentile: 14–278); on THRIVE-G, a median of 9 tokens (10th–90th percentile: 4–49). Only 1.8% of THRIVE-L responses and 0.02% of THRIVE-G responses ever reach the cap. The cap, therefore, has a negligible effect on either dataset.

AuthorsOpenReview ↗

Response 2

Reviewer wj6k - Response 2

Reviewer Point

"It is still unclear whether the method will generalize to larger models, stochastic decoding, multilingual generation, or interactive systems."

Author Response

Larger models. To answer the question of whether LatticeSpike generalizes to larger models, we evaluated LatticeSpike on Qwen3-32B, the 4× larger sibling of the paper's TriviaQA generator (Qwen3-8B), using the identical data pipeline and model architecture as the one reported in the paper. Only one training hyperparameter was modified, concretization channel scaling (1 → 12), which we found allowed our model to converge quicker — resulting in a more energy efficient model — while maintaining or even improving performance.

Table 1 reports Qwen3-32B and Qwen3-8B TriviaQA test results for LatticeSpike. Values are mean ± sd over five seeds (42-46). Each generator produces its own answers and therefore its own labelled dataset, so the share of hallucinated tokens differs across rows. Importantly, AP, precision and F1 depend on that share, so they are not directly comparable across generators, AUROC and recall at a fixed false-positive rate are. The Qwen3-8B row represents the paper's reported numbers for LatticeSpike on TriviaQA. The Qwen3-32B run was carried out on a larger GPU (NVIDIA H200) compared to the Qwen3-8B run (NVIDIA RTX 5090) due to the increased GPU capacity requirements of the 32B model.

GeneratorHallucinated tokensAPAP / randomAUROCR@1% FPRP@1% FPRP / randomF1@1% FPRF1 / random
Qwen3-8B (paper)4.7%0.399 ± 0.0088.5×0.888 ± 0.0050.250 ± 0.0020.577 ± 0.00212.3×0.349 ± 0.00321.2×
Qwen3-32B3.6%0.301 ± 0.0148.4×0.866 ± 0.0060.226 ± 0.0180.456 ± 0.02012.8×0.302 ± 0.02119.3×

Table 1: LatticeSpike results on a 4x larger model, as evaluated on TriviaQA. The table shows hallucinated token percentage, AP, AUROC, and recall, precision, and F1 at FPR ≤ 1% across generators; where applicable, "metric / random" columns divide each metric by a random detector's value.

As shown in the results, LatticeSpike successfully transfers to larger generators without architectural changes. Concretely, our model's improvement over a random detector is nearly identical across both generators on AP, precision, and F1: AP 8.4× (32B) vs. 8.5× (8B), precision at 1% FPR 12.8× (32B) vs. 12.3× (8B), F1 at 1% FPR 19.3× (32B) vs. 21.2× (8B). Moreover, on AUROC and recall LatticeSpike performs similarly on both language models — reaffirming its generalizability across generators of varying sizes.

Stochastic decoding. As mentioned in response 1, stochastic decoding is already covered by our evaluation: every RAGTruth response (all 2,965) was sampled at temperatures between 0.7 and 1.0. All RAGTruth results in the paper and in this rebuttal for Reviewer Cukc's response, including the comparison with CHARM, are therefore on stochastically decoded text.

Interactive systems. LatticeSpike is designed for interactive use. It scores each token causally while the response is being generated, using only the generator's internal state up to that step; each score is computed before the token is emitted, so a deployed system can stop, flag, or regenerate as soon as a hallucination is anticipated, rather than after the full response is shown. Our deployment benchmark noted in the paper measures this setting (Section 5.2), additionally we run a subsequent deployment benchmark as part of Reviewer Cukc's rebuttal response (Table 2), where we show the detector running token by token alongside the generator (23.3 ms per token including generation).

Furthermore, because historical user-to-language-model conversation enters the generator as context, which LatticeSpike reads through the generator's internal states, we expect the method to carry over to multi-turn settings; in fact, RAGTruth already hints at this with long conditioning contexts of up to ~2,600 tokens. However, a dedicated multi-turn evaluation, where hallucinations may depend on earlier turns, remains future work.

Multilingual generation. We agree that this remains untested, as it was out-of-scope for the current iteration of this project. All of our benchmarks are in English, and token-level hallucination labels in other languages would require a multilingual annotation pipeline, along with a multilingual training protocol for LatticeSpike. We note this explicitly as a limitation and a direction for future work.

Dear Reviewer wj6k,

Thank you again for your positive review. With the discussion period ending on November 18, 2026, we wanted to check whether our responses have addressed your two points. In brief:

  • Low-FPR operating points. We agree the Transformer variant leads at some tight FPR budgets. That variant is our own ablation: it keeps LatticeSpike's box encoder, input channels, scoring, and loss. None of the thresholded differences between the two is statistically significant in either direction, while the spiking layer's AUROC advantage is significant against all four replacements.
  • Generalization. We added three generators using the paper's TriviaQA pipeline unchanged. Two are recent models (Gemma 4 E4B and Qwen3.5-9B) and one is larger (Qwen3-32B, 4× the paper's Qwen3-8B). AUROC stays between 0.86 and 0.91, and the gain over a random detector is comparable to the paper's results.
  • Stochastic decoding. All RAGTruth responses were sampled at temperatures of 0.7–1.0, so every RAGTruth result, including the new CHARM comparison, is on stochastically decoded text.
  • Response length. THRIVE's 512-token limit is a cap, not a fixed length: only 1.8% (THRIVE-L) and 0.02% (THRIVE-G) of responses reach it.
  • Interactive and multilingual use. LatticeSpike scores each token causally during generation. Multi-turn and multilingual evaluation remain future work, and we will state both explicitly as limitations.

If anything remains unclear, we would be happy to clarify. If our responses have resolved your concerns, we would appreciate your consideration in your final assessment.

Best regards, The Authors

Reviewer_x5W3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LatticeSpike, a biologically inspired, lightweight framework for real-time, token-level hallucination detection in language models. Unlike conventional post-generation detectors, LatticeSpike operates causally alongside a frozen language model, leveraging box embeddings to capture semantic uncertainty and a recurrent spiking neural network (SNN) to identify hallucinations through prediction errors. The authors also introduce two human-validated, token-level benchmark datasets, THRIVE-Llama and THRIVE-Gemma, to support hallucination detection research.

Critical strengths
  1. This paper is in good written and easy to follow.

  2. This paper focuses on SNN, a bio-inspired method, which is an interesting topic to save energy.

  3. In the experimental results, this paper improves results in THRIVE-Llama, THRIVE-Gemma, RAGTruth, and TriviaQA compared with the baselines.

Critical weaknesses
  1. The motivation for using SNNs for real-time, token-level hallucination detection needs to be further clarified. In particular, the authors should explain the advantages of the proposed SNN-based approach compared to alternative solutions, such as model quantization or directly training a lightweight model for the same task.

  2. The authors could further clarify the specific contributions and methodological novelty of this work to the field of SNN itself.

Questions for authors

N/A

6 comments
AuthorsOpenReview ↗

Initial Comment

We would like to thank Reviewer x5W3 for taking the time to read our paper and provide a review. We additionally would like to thank the reviewer for their constructive feedback on our work.

AuthorsOpenReview ↗

Response 1

Reviewer x5W3 - Response 1

Reviewer Comment

"The motivation for using SNNs for real-time, token-level hallucination detection needs to be further clarified. In particular, the authors should explain the advantages of the proposed SNN-based approach compared to alternative solutions, such as model quantization or directly training a lightweight model for the same task."

Author Response

Our motivation for using an SNN comes primarily from the nature of the problem we are trying to solve. Unlike approaches that evaluate hallucinations after an entire response has been generated, LatticeSpike aims to identify hallucinations in real time as they emerge, at the individual token level. This requires the detector to continuously process information from the language model, retain information from preceding tokens, and recognize changes in its internal representations. We view this as a temporal modeling problem, which motivated us to explore recurrent spiking neural networks (RSNNs).

SNNs offer a different way of modeling sequential information through membrane potentials, leakage, and threshold-based neuronal firing. Rather than evaluating each token independently, the RSNN maintains a recurrent state that evolves throughout generations. In LatticeSpike, box embeddings capture changes in the language model's internal representations, and their geometric properties are used to drive and modulate the spiking network. The RSNN learns to predict its own evolving state, and the difference between the predicted and observed states provides a token-level anomaly score. Our hypothesis is that hallucination-related changes in the generator's representations can disrupt these learned temporal patterns.

Another motivation comes from Edge AI and neuromorphic computing. As language models become increasingly deployed in resource-constrained environments, we believe it is important to investigate lightweight monitoring mechanisms that can operate alongside an existing model without requiring another large language model for verification. The event-driven nature of SNNs also makes them interesting candidates for future neuromorphic implementations. However, we do not claim neuromorphic energy savings in this work, since our experiments were conducted on conventional hardware.

Regarding model quantization, we consider it complementary to our approach rather than a direct alternative. Quantization primarily reduces the numerical precision and computational requirements of an existing model, whereas LatticeSpike addresses the separate problem of identifying hallucinations during generation. Quantization alone does not provide a hallucination detection mechanism. Rather, according to M. N. Haque and colleagues [1], “...quantization substantially increases the package hallucination rate (PHR), with 4-bit models exhibiting the most severe degradation”. In principle, LatticeSpike could also monitor a quantized language model, provided the required internal signals remain accessible, although this combination has not yet been evaluated experimentally and remains future work.

We also agree that directly training a lightweight conventional neural network is a reasonable alternative. However, several of the hallucination detectors we compare in the paper (such as HaMI and SAPLMA tok.) are already very light models. Across our original evaluation, which includes these lightweight models, LatticeSpike outperforms the tested baselines in 96 out of 101 evaluation cells. Thus, lighter models can suffer from severe performance degradation.

Regarding the advantage of the SNN over other variants, Appendix B.4.1 in the paper compares precisely this: the RSNN against parameter-matched LSTM and causal Transformer alternatives. For this experiment, only the SNN dynamics layer is replaced, while the box encoder, six input channels, training objective, data split, checkpoint selection, and evaluation protocol are held fixed. This experiment helped isolate the contribution from the RSNN's spiking dynamics. The results are summarized in Table 1.

Dynamics layer (THRIVE-Llama)APAUROC
RSNN (volume-modulated)0.406 ± 0.0120.763 ± 0.007
Transformer without volume0.399 ± 0.0080.742 ± 0.005
LSTM with volume0.395 ± 0.0050.749 ± 0.005
LSTM without volume0.390 ± 0.0070.747 ± 0.004
Transformer with volume0.385 ± 0.0070.734 ± 0.006

Table 1: Controlled dynamics-layer ablation on THRIVE-Llama (Appendix B.4.1, Table 21 of the manuscript). All variants share the box encoder, input features, loss, and evaluation protocol; dynamics-layer parameter counts are matched within 1.5%. Values are mean ± standard deviation over five seeds. Bold indicates the highest mean.

AuthorsOpenReview ↗

Response 1 (continued)

Reviewer x5W3 - Response 1 (continued)

As Table 1 shows, the RSNN achieves the highest AP (0.406 ± 0.012) and AUROC (0.763 ± 0.007). The strongest alternatives reach 0.399 ± 0.008 AP (Transformer without volume) and 0.749 ± 0.005 AUROC (LSTM with volume). The RSNN's AUROC improvement is statistically significant against all four alternatives (adjusted p < 0.001), while its AP improvement is significant against three of them. Interestingly, providing box volume as an additional input to the conventional architectures does not recover the RSNN's ranking performance. This suggests that conditioning neuronal dynamics on box volume provides benefits beyond simply making the volume information available as an input feature.

Beyond model performance, the SNN architecture provides energy and timing efficiency advantages as well. Table 2 reports LatticeSpike's computational efficiency and energy consumption against the SOTA method CHARM (a new baseline introduced as part of Reviewer Cukc's rebuttal), as compared under equal training and evaluation protocols on RAGTruth. These experiments provide further insight regarding LatticeSpike's architectural advantage.

MethodTraining timeTraining energyPeak detector memoryPer-token time (generator + detector)
LatticeSpike (efficient)52.3 ± 4.6 min164 Wh3.08 MiB23.3 ms (RTX 5090 GPU); 23.6 ms (H200 GPU)
CHARM73.3 ± 11.4 min221 Wh3.51 MiB19.9 ms (RTX 5090 GPU); 21.1 ms (H200 GPU)

Table 2: Computational efficiency comparison between LatticeSpike and CHARM. Per-token times include the Llama-2-7B-chat generator and the compared detectors. The generator and detector contributions are 19.1 + 4.20 ms for LatticeSpike and 19.1 + 0.84 ms for CHARM on the consumer card (RTX 5090) GPU, and 20.1 + 3.44 ms versus 20.1 + 1.01 ms on the datacenter card (H200 GPU). Training energy is computed from mean GPU and CPU power consumption multiplied by mean training time.

As shown in Table 2, LatticeSpike requires approximately 29% less training time and 26% less training energy than CHARM under equal training pipelines. It also uses approximately 12% less peak detector memory. These results indicate that the proposed approach can be trained with relatively modest computational and memory requirements.

At the same time, CHARM has lower inference latency. On the RTX 5090, the additional per-token detector latency is 4.20 ms for LatticeSpike compared with 0.84 ms for CHARM, while on the H200, the corresponding values are 3.44 ms and 1.01 ms. At the deployment level, however, these differences are practically imperceptible: both configurations stream text at over 40 tokens per second, several times faster than people read — roughly 5–8 tokens per second [7].

Much of LatticeSpike's inference overhead comes from constructing its input signals rather than executing the recurrent spiking component. As reported in our original deployment experiments, the RSNN core itself requires only 0.14 -- 0.19 ms per token on CPU, suggesting that simplifying signal extraction is a promising direction for reducing latency.

We also distinguish the energy measurements in Table 2 from neuromorphic efficiency claims. The reported energy values measure training energy on conventional hardware, not energy consumed during token-level inference on dedicated neuromorphic processors.

Overall, our motivation for using a spiking neural network lies in whether recurrent spiking dynamics, when conditioned on geometric representations of a language model's internal state, can provide an effective mechanism for identifying hallucination-related changes during generation. Our results show that they can.

We will revise the manuscript accordingly to better explain this motivation, and explicitly identify the observed computational benefits of utilizing an RSNN-based architecture in real-time, token-level hallucination detection tasks.

Reviewer_CukcOpenReview ↗
Rating
3
Confidence
2(+1)
Informativeness
3(+1)
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes LatticeSpike, a supervised token-level hallucination detector that operates alongside a frozen language model using only causally available information. It combines learned box embeddings with volume-modulated recurrent spiking dynamics and uses state-prediction errors, trained with a ranking objective, as hallucination scores. The paper also introduces THRIVE-Llama and THRIVE-Gemma, two matched-prompt datasets with automatically generated token labels and independent human audits. Experiments compare LatticeSpike against 12 baselines on four corpora and include cross-task evaluation, ablation studies, and measurements of computational overhead.

Critical strengths
  1. Consistent empirical improvements. LatticeSpike shows gains across four corpora, supported by results on human-annotated RAGTruth and cross-task evaluations.

  2. Useful benchmark contribution. The two THRIVE datasets provide matched prompts across generator families, automatic token-level labels, and independent human audits that characterize annotation reliability.

  3. Causal token-level design. The detector uses only information available during generation while keeping the language model frozen, making it relevant to online hallucination monitoring.

Critical weaknesses
  1. Lack of more recent generators. Although the paper includes an evaluation on Qwen3-8B, the experiments do not cover small models from more recent families such as Gemma 4 or Qwen3.5. This limits the evidence for the method’s applicability to newer deployment-sized generators.

  2. Comparison with recent related detection methods is incomplete. The main experimental comparisons cover methods published up to 2025, but do not include any methods from 2026 as baselines. For example, CHARM (Neural Message-Passing on Attention Graphs for Hallucination Detection, ICLR 2026) also combines attention and activation signals for detection. Including CHARM as a baseline would make the comparison more comprehensive and help strengthen the empirical support for the paper’s conclusions.

Questions for authors
  1. In practical use, how likely is a fully correct response to trigger a false alarm? Specifically, using the threshold selected on the validation set, what proportion of fully correct test responses contain at least one token with a score above that threshold?

  2. Could the discrepancy between automatic annotations and human judgments affect the comparison between methods?

8 comments
AuthorsOpenReview ↗

Initial Comment

We would like to thank reviewer Cukc for their time and thoughtful review of our paper. In the responses that follow we provide requested experiments, answers, and clarifications.

AuthorsOpenReview ↗

Response 1

Reviewer Cukc - Response 1

Reviewer Comment

"Comparison with recent related detection methods is incomplete."

Author Response
Experiment 1 - CHARM Baseline Addition (Pt.1)

As per reviewer Cukc's request, we added CHARM [1] as a new SOTA baseline to compare our method against. We ran this full-corpus experiment on RAGTruth, as this dataset is well-known, human-labeled, and introduces no author bias.

CHARM was run under the same training and evaluation pipeline established in the paper, with the hyperparameters its authors tuned for their closest setting to RAGTruth (retrieval-augmented QA over Natural Questions), including its full training schedule of up to 75 epochs. Under this author-tuned configuration, CHARM obtained an AP of 0.444 ± 0.013 and an AUROC of 0.852 ± 0.005. It can be seen from Table 1 in the paper (page 7) that LatticeSpike surpasses CHARM in terms of both AP (0.447 ± 0.007) and AUROC (0.866 ± 0.003). As for recall, precision, and F1 at 1% FPR, CHARM achieves 0.280 ± 0.014, 0.651 ± 0.011, and 0.391 ± 0.015, respectively; LatticeSpike comes close at 0.272 ± 0.006, 0.645 ± 0.005, and 0.382 ± 0.007, respectively.

Additionally, we found that simply by tuning two hyperparameters — both selected on validation — (batch size, 10 → 14, and concretization channel scaling, 1 → 12), utilizing the same architecture as the paper's and following the same training and evaluation pipelines, we obtained a version of our model which allows it to train for 5 epochs instead of 15, while improving overall model performance as well. Concretely, this finding cuts training time by ~2.5× (~132 min to ~52 min) on RAGTruth while improving model performance by 1.2%, on average. This version, which we call "LatticeSpike (efficient)", will be included in the final version of the paper and is shown in the tables that follow.

All tables report RAGTruth (Llama-2-7b-chat) test results. Values are mean ± sd over five seeds (same seeds as the paper's) for seeded methods; Lookback Lens and the Hidden probe are deterministic. Best in bold, second best underlined; lower is better in Table 2, higher everywhere else. Everything shown was run on the same hardware (NVIDIA RTX 5090 GPU), with the exception of one additional per-token generation time variant that was included (details in Table 2).

MethodSupervisionAPAUROC
LatticeSpike (efficient)token0.453 ± 0.0060.868 ± 0.005
CHARMtoken0.444±0.013‾\underline{0.444 \pm 0.013}0.852±0.005‾\underline{0.852 \pm 0.005}
Lookback Lenstoken0.4040.831
Hidden probetoken0.4070.844
SAPLMA (tok.)token0.360 ± 0.0160.797 ± 0.006

Table 1: AP and AUROC.

MethodTraining timeEnergy per training runPeak memoryPer-token generation time (generator + detector)
LatticeSpike (efficient)52.3 ± 4.6 min164 Wh3.08 MiB19.1 + 4.20 = 23.3 ms$^{a};20.1+3.44=23.6ms; 20.1 + 3.44 = 23.6 ms^{b}$
CHARM73.3 ± 11.4 min221 Wh3.51 MiB19.1 + 0.84 = 19.9 msa^{a}; 20.1 + 1.01 = 21.1 msb^{b}

a^{a} Consumer GPU (NVIDIA RTX 5090). b^{b} Datacenter GPU (NVIDIA H200).

Table 2: Training time, energy per training run, detector peak memory, and per-token generation time with the detector running. Energy is measured as mean GPU + CPU power during a training period, multiplied by mean training time.

MethodRPF1
LatticeSpike (efficient)0.544 ± 0.0030.421 ± 0.0020.474 ± 0.002
CHARM0.522±0.014‾\underline{0.522 \pm 0.014}0.410±0.007‾\underline{0.410 \pm 0.007}0.459±0.010‾\underline{0.459 \pm 0.010}
Lookback Lens0.4750.3880.427
Hidden probe0.4810.3910.431
SAPLMA (tok.)0.448 ± 0.0170.374 ± 0.0090.408 ± 0.013

Table 3: FPR ≤ 5%.

MethodRPF1
LatticeSpike (efficient)0.271±0.009‾\underline{0.271 \pm 0.009}0.644±0.007‾\underline{0.644 \pm 0.007}0.381±0.010‾\underline{0.381 \pm 0.010}
CHARM0.280 ± 0.0140.651 ± 0.0110.391 ± 0.015
Lookback Lens0.2510.6260.358
Hidden probe0.2400.6160.345
SAPLMA (tok.)0.225 ± 0.0190.599 ± 0.0200.327 ± 0.023

Table 4: FPR ≤ 1%.

MethodRPF1
LatticeSpike (efficient)0.194 ± 0.0040.722 ± 0.0040.306 ± 0.005
CHARM0.193±0.012‾\underline{0.193 \pm 0.012}0.721±0.012‾\underline{0.721 \pm 0.012}0.305±0.016‾\underline{0.305 \pm 0.016}
Lookback Lens0.1720.6970.275
Hidden probe0.1630.6850.263
SAPLMA (tok.)0.145 ± 0.0150.658 ± 0.0240.237 ± 0.022

Table 5: FPR ≤ 0.5%.

MethodRPF1
LatticeSpike (efficient)0.065±0.005‾\underline{0.065 \pm 0.005}0.811±0.011‾\underline{0.811 \pm 0.011}0.120±0.008‾\underline{0.120 \pm 0.008}
CHARM0.068 ± 0.0130.815 ± 0.0370.126 ± 0.023
Lookback Lens0.0630.8090.118
Hidden probe0.0600.8000.112
SAPLMA (tok.)0.046 ± 0.0050.755 ± 0.0190.088 ± 0.009

Table 6: FPR ≤ 0.1%.

AuthorsOpenReview ↗

Response 2

Reviewer Cukc - Response 2

Experiment 1 - CHARM Baseline Addition (Pt.2)
Results Interpretation

Across Tables 1-6, LatticeSpike and CHARM are consistently the two best detectors, with LatticeSpike leading CHARM in total number of cells won (10 vs. 7).

Ranking quality and overall performance. LatticeSpike achieves the best AP and AUROC, which do not depend on any operating point (OP) and can be interpreted as holistic evaluators of model performance and ranking quality.

Training efficiency, memory usage, and predictability. LatticeSpike is the more efficient detector in training cost and memory footprint: CHARM takes 40.2% longer to train (73.3 vs 52.3 min per run), uses 34.8% more energy per training run (221 vs 164 Wh), and occupies 14.0% more peak memory (3.51 vs 3.08 MiB) compared to our method. LatticeSpike is also the more predictable model of the two: across training seeds, CHARM's standard deviations on performance metrics, and hence its 95% confidence intervals, are on average 2.7× wider than our method's. This can be problematic in a real-time deployment setting where retraining events are randomly seeded, and a bad seed draw can result in a noticeably worse model due to its cross-seed variance. Thus, in a real-time deployment setting, where retraining can occur relatively frequently and on random seeds, LatticeSpike can provide savings in energy and memory usage, while also providing cross-seed performance reliability.

Operating point metrics. Operating point metrics are split between the two methods in absolute terms (6 wins vs. 6 wins); however, LatticeSpike wins decisively at 5% FPR, while the three lower FPR thresholds appear to be within noise of each other with no method clearly pulling ahead.

Inference latency. On a consumer GPU (NVIDIA RTX 5090), the end-to-end inference time is 23.3 ms per token with LatticeSpike versus 19.9 ms per token with CHARM, a difference of 3.4 ms per token (17%). The generator accounts for 19.1 ms for either method, and each detector accounts for the rest (detector overheads of 4.20 ms for LatticeSpike and 0.84 ms for CHARM). On a datacenter GPU (NVIDIA H200), the gap narrows: 23.6 vs 21.1 ms per token end-to-end (detector overheads of 3.44 ms for LatticeSpike vs. 1.01 ms for CHARM), a 12% difference, as LatticeSpike's signal computation benefits from faster hardware while CHARM's many small operations do not. At the deployment level, these differences are practically imperceptible: both configurations stream text at over 40 tokens per second, several times faster than people read — roughly 5–8 tokens per second [2]. In practice, a user watching either system generate text would not notice a token-by-token difference. Even over a long response (such as a 200-token response), where the per-token gap adds up, the total difference between LatticeSpike and CHARM is under one second.

Strengths of LatticeSpike and practical guidance. For practitioners choosing between LatticeSpike and CHARM, the results point to LatticeSpike as the favorable detector in the majority of deployment scenarios.

When to choose LatticeSpike:

  • Overall model performance matters most. It is better on AP and AUROC, which measure ranking quality across all thresholds.
  • Retraining must be predictable and reliable. Its seed-to-seed spread is about 2.7× smaller than CHARM's, on average, across metrics. A randomly seeded CHARM retrain can land noticeably below its average; a LatticeSpike retrain stays close to its mean.
  • Energy or training-time constraints exist. It trains in 52 vs. 73 minutes (CHARM takes ~40% longer), consuming 164 Wh of energy vs. CHARM's 221 Wh (CHARM uses ~35% more energy on compute per training run).
  • Memory constraints exist. It occupies less peak memory (3.08 vs 3.51 MiB) alongside a deployed language model.
  • Detection runs at 5% FPR. It wins all three OP metrics (precision, recall, F1), decisively, at the 5% FPR setting.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Rebuttal - General Remarks

We thank all three reviewers and the AC for their time, careful reading of our paper, and insightful comments.

Reviewers agreed that LatticeSpike is a causal, token-level detector that runs alongside a frozen language model, and is well-suited for online monitoring (Cukc, wj6k). Reviewers also found the empirical gains consistent across four corpora (Cukc, x5W3, wj6k), saw the SNN approach as an interesting, energy-motivated direction (x5W3), and called the human-validated THRIVE benchmarks a useful contribution (Cukc, wj6k).

During the rebuttal we added the items shown in the comment above: "Summary of Rebuttal".

AuthorsOpenReview ↗

Summary of Rebuttal

  1. SOTA baseline addition (Cukc). We added CHARM (ICLR 2026), trained for 5 seeds on RAGTruth, with its authors' tuned configuration for retrieval-augmented QA. LatticeSpike has the best AP and AUROC; fixed-FPR results are split, though LatticeSpike wins decisively at 5% FPR while lower budgets are within noise of each other. CHARM trains 40% longer, uses 35% more energy and 14% more memory, and is less reliable as its seed spread is 2.7× wider. CHARM has lower detector latency, but the end-to-end difference is only ~3 ms per token, on average, between CHARM and LatticeSpike — imperceptible at the deployment level since both stream text several times faster than people can read.
  2. Efficient configuration finding (Cukc, wj6k). We found that a validation-selected hyperparameter change for LatticeSpike (concretization channel tuning) cut training time from ~132 to ~52 minutes per run and training energy from ~414 to ~164 Wh on RAGTruth — while improving performance by 1.2% on average. On TriviaQA, the same change yielded comparable results. Such a finding resulted in an even stronger and more energy efficient model.
  3. Newer and larger generators (Cukc, wj6k). Using the paper's TriviaQA pipeline unchanged, we added two recent models (Gemma 4 E4B and Qwen3.5-9B) and one larger model (Qwen3-32B, 4× the paper's Qwen3-8B), each with 5 seeds. LatticeSpike successfully transfers to all three without architectural changes: AUROC stays between 0.86 and 0.91, and its relative improvement over a random detector is comparable to the paper's Qwen3-8B.
  4. Response-level false alarms (Cukc Q1). We measured how often a response containing no hallucinations is wrongly flagged, i.e., has at least one token scored above the detection threshold selected on the validation set. This happens for only 9.3% of fully correct test responses on average (4.1–13.7% across the four corpora), showing that false alarms are rare at the response level.
  5. Robustness to labelling protocol (Cukc Q2). Our four corpora are labelled via three different protocols: a six-judge language-model panel, a three-judge panel checked against gold answers, and human annotators. As shown in the paper, the overall method ranking largely holds across all of them — with LatticeSpike consistently sitting at the top. On RAGTruth, which is labelled entirely by humans, LatticeSpike leads the paper's twelve baselines on 15 of 17 metrics and ties on the other two. This demonstrates that the comparison between methods is fair and consistent, and does not depend on how the labels were produced.
  6. Decoding, response length, interactive use, and multilingual generation (wj6k). The reviewer asked whether results extend beyond greedy decoding and fixed-length responses, and to interactive systems and multilingual generation. Extending beyond greedy decoding: We already do this, every RAGTruth response was sampled at temperatures of 0.7–1.0, so all RAGTruth results, including the CHARM comparison, are on stochastically decoded text. Fixed-length responses: THRIVE's 512-token limit is a generation cap, not a fixed length. Importantly, only 1.8% of THRIVE-L and 0.02% of THRIVE-G responses ever reach it — the cap has a negligible effect on either dataset. Interactive systems: LatticeSpike is designed for interactive use, scoring each token causally as it is generated so a deployed system can intervene mid-response; we expect it to carry over to multi-turn settings, which remains future work. Multilingual generation: This remains untested, as all our benchmarks are in English; we identify it as a limitation and future work.
  7. SNN motivation and contribution (x5W3). We clarified the motivation for using SNNs and our contributions to the field of SNNs, specifically. Motivation: In short, detecting hallucinations as they emerge is a temporal modeling problem, which recurrent spiking dynamics suit naturally. Contribution to the SNN literature: Conditioning spiking neurons on the language model's internal geometry and using their prediction error as the hallucination score is a novel and useful application of SNNs. One of our ablation studies (Table 21 in the paper) shows this: replacing the spiking layer with a parameter-matched LSTM or a Transformer results in lower AP and AUROC relative to the SNN.
  8. Transformer variant at low FPR settings (wj6k). The reviewer noted that, in our architecture ablation, a Transformer replacement for the spiking layer is ahead at some tight false-positive budgets. We acknowledged this and explained that this variant still uses several of LatticeSpike's architectural pipeline contributions — the internal model geometry, box encoder, input signals, and training objective — which seem to be major drivers of performance, not necessarily the attention mechanism itself. We also propose combining SNN and Transformer dynamics into a hybrid architecture to leverage the strengths of both methods, as future work.
AuthorsOpenReview ↗

Summary Comment for AC

Dear Area Chair,

Thank you for overseeing the review of our submission. To help with your assessment, we provide a brief summary of the main concerns raised and how we addressed them during the discussion period.

Reviewer Cukc (Weak Accept): The reviewer acknowledged our rebuttal and indicated positive sentiment towards our work, increasing their confidence score in the process.

We addressed and responded to their concerns — CHARM baseline addition, newer generators, response-level false alarms, labelling protocol — as follows:

  • 2026 baseline: We added CHARM (ICLR 2026) on RAGTruth over five seeds. LatticeSpike achieves the best AP and AUROC; fixed-FPR results are split between the two methods. LatticeSpike trains ~29% faster with ~26% less energy; CHARM has lower per-token latency (imperceptible at the deployment level since both LatticeSpike and CHARM stream text several times faster than people can read).
  • Newer generators: We evaluated Gemma 4 E4B and Qwen3.5-9B with the paper's TriviaQA pipeline; AUROC remains between 0.86 and 0.91 and all other metrics similarly demonstrate strong generalization to newer generators.
  • Response-level false alarms: At the validation-selected threshold, 9.3% of fully correct test responses (4.1–13.7% across corpora) contain at least one flagged token, showing that false alarms are rare at the response level.
  • Labelling protocol: Method rankings hold across three labelling protocols, including the fully human-annotated RAGTruth — demonstrating that the comparison between methods is fair and consistent, and does not depend on how the labels were produced.

Reviewer x5W3 (Weak Reject): As of November 18th, the reviewer has not yet responded to our rebuttal.

We addressed and responded to their concerns — motivation for SNNs versus alternatives, and contribution to the SNN field — as follows:

  • Motivation: Detecting hallucinations as they emerge is a temporal modeling problem, which recurrent spiking dynamics suit naturally.
  • Alternatives to SNNs (including lightweight ones): Quantization is complementary to detection and subsequently to our approach, not an alternative to it. Several baselines are small trained heads, however, this hinders their performance (LatticeSpike leads in 96/101 cells in the paper, consistently outperforming lightweight models). In a controlled, parameter-matched ablation, replacing the spiking layer with an LSTM or Transformer lowers AUROC significantly in all four cases and AP significantly in three of four.
  • Contribution to the field of SNNs itself: We clarified that our contribution to the field of SNNs itself lies in providing and demonstrating a task-specific way to couple structured geometric information with recurrent spiking dynamics to efficiently identify hallucinations in real-time. Conditioning spiking neurons on the language model's internal geometry and using their prediction error as the hallucination score is a novel and empirically useful application of SNNs. Removing this framework severely degrades the performance of our model, as shown in Table 24 of the paper.

Reviewer wj6k (Weak Accept): As of November 18th, the reviewer has not yet responded to our rebuttal.

We addressed and responded to their concerns — low-FPR settings in the architecture ablation, generalization to larger generators, interactive and multilingual use, and THRIVE greedy decoding and response length — as follows:

  • Low-FPR settings: The Transformer variant is our own ablation and shares LatticeSpike's encoder, inputs, scoring, and loss; thus, much of its strong low-FPR setting performance can be attributed to LatticeSpike's own architectural pipeline as opposed to its dynamics layer. Regardless, none of the thresholded metrics in Table 21 of the paper (dynamics-layer ablation) are statistically significant in either direction (Transformer and SNN), while the SNN's AUROC advantage is significant against all replacements.
  • Generalization to larger generators, THRIVE greedy decoding and response length, and interactive/multilingual use: We added a larger generator (Qwen3-32B) alongside the two newer ones, our model demonstrably generalizes to all three generators, including the larger one. We already show LatticeSpike's performance beyond greedy decoding, all RAGTruth results are on stochastically decoded text (temperature 0.7–1.0). THRIVE's 512-token limit is a generation cap, not a fixed response length, and is reached by only 1.8% and 0.02% of responses in THRIVE-L and THRIVE-G, respectively — a negligible effect on either dataset. LatticeSpike is designed for interactive use, though multi-turn and multilingual evaluation are stated as limitations and remain future work.

All new experiments, clarifications, and the stated limitations will be incorporated into the revised manuscript.

We sincerely appreciate your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.