- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper analyzes an existing hallucination detector that compares prediction residuals from dynamical models fitted to correct and hallucinated token-embedding sequences. It proposes guarantees for marginal indistinguishability, sequence-length-dependent classification error, and transfer between embedding spaces, with supporting empirical diagnostics.
Critical strengths
-
The paper asks useful questions about what temporal information adds beyond one-token marginals, how response length affects detection, and when embedding transfer is possible.
-
The explicit residual statistic and moment-based discriminability make the intended mechanism inspectable. Empirical diagnostics attempt to connect the abstract model to token embeddings.
Critical weaknesses
-
The central quantitative guarantees do not hold as stated. In Eq. (16), set P=1, A_c=A_h=0, both noise means to zero, and noise variances to 1 and 0.01. The stationary Gaussian TV distance is approximately 0.7982, exceeding the claimed upper bound 0.495. Eq. (19) also has an incorrect length scaling: its unnormalized score has mean of order sqrt(L), producing an exp(-cL^2) bound for fixed parameters. For A_c=0, A_h=1/2 and unit Gaussian noise, the event that all L states lie in [1,1.1] causes a classification error and has probability p^L>0. This contradicts that bound for any length-independent C_1. These affect the paper’s principal theoretical contribution, rather than only proof presentation.
-
The transfer guarantee omits operator-coordinate compatibility. With no lifting and P=I, take A_c=diag(1/2,0), A_h=0, stationary G_c=diag(4/3,0.01), and let T swap the coordinates exactly. Then chi=0 and sigma_min(T)=1, but Eq. (24) gives target discriminability 0.0025 versus source 1/3, contradicting Eq. (25). Conjugating the operators would avoid this example, but that differs from applying the source operators directly. Empirical residual histograms, length trends and one embedding-pair comparison cannot repair these contradictions or establish the full stochastic assumptions. A weaker transfer lower bound also does not establish that a larger defect causes near-chance accuracy.
Questions for authors
-
How should Eq. (16) be corrected to accommodate the scalar Gaussian counterexample above?
-
Is the score in Result 2 intended to be normalized by sqrt(L-1)? Please reconcile Eqs. (6), (18) and (19), and provide a dependence-valid concentration proof with constants independent of L.
-
Does transfer use coordinate-conjugated operators or unchanged source operators? Please address the exact coordinate-swap counterexample, specify compatible lifted dimensions, and distinguish a non-vacuous guarantee from qualitative empirical agreement.