acceptodds
Withdrawn from ICLR 2027

Guarantees on Dynamical System Distinguishability for LLM Token Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent work has shown that classifying large language models (LLMs)' responses can be distinguished by modeling token embeddings as trajectories of a black-box dynamical system (DS) and comparing prediction residuals of two DSs. Despite the empirical success of this dynamical approach, a theoretical understanding of why it works, how well it scales as a function of the token sequence, and when it transfers across embedding models remains lacking. We address these questions by formalizing the classification task as a binary hypothesis test between two stochastic linear DSs. We show that the total variation distance between the stationary marginal distributions of the two DSs can be arbitrarily small even when the dynamics differ substantially, which provides a fundamental accuracy floor for any classifier that ignores token dynamics. We then show that the misclassification probability of DS-based classification decays exponentially in the sequence length , with the decay governed by a dynamical discriminability quantity that captures the spectral distance between the two DSs. We also characterize cross-embedding generalization by introducing an approximate intertwining condition between embedding models and establishing a lower bound on the transferable discriminability in terms of the intertwining map's smallest singular value. These results explain the empirical performance of DS-based classification and motivate further investigation into using DS theory to analyze AI systems, in contrast to the more common approach of using AI to model dynamical systems.

Primary area:
learning on time series and dynamical systems
Keywords:
dynamical systems, LLM token generation, hypothesis testing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vXzYOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper analyzes an existing hallucination detector that compares prediction residuals from dynamical models fitted to correct and hallucinated token-embedding sequences. It proposes guarantees for marginal indistinguishability, sequence-length-dependent classification error, and transfer between embedding spaces, with supporting empirical diagnostics.

Critical strengths
  1. The paper asks useful questions about what temporal information adds beyond one-token marginals, how response length affects detection, and when embedding transfer is possible.

  2. The explicit residual statistic and moment-based discriminability make the intended mechanism inspectable. Empirical diagnostics attempt to connect the abstract model to token embeddings.

Critical weaknesses
  1. The central quantitative guarantees do not hold as stated. In Eq. (16), set P=1, A_c=A_h=0, both noise means to zero, and noise variances to 1 and 0.01. The stationary Gaussian TV distance is approximately 0.7982, exceeding the claimed upper bound 0.495. Eq. (19) also has an incorrect length scaling: its unnormalized score has mean of order sqrt(L), producing an exp(-cL^2) bound for fixed parameters. For A_c=0, A_h=1/2 and unit Gaussian noise, the event that all L states lie in [1,1.1] causes a classification error and has probability p^L>0. This contradicts that bound for any length-independent C_1. These affect the paper’s principal theoretical contribution, rather than only proof presentation.

  2. The transfer guarantee omits operator-coordinate compatibility. With no lifting and P=I, take A_c=diag(1/2,0), A_h=0, stationary G_c=diag(4/3,0.01), and let T swap the coordinates exactly. Then chi=0 and sigma_min(T)=1, but Eq. (24) gives target discriminability 0.0025 versus source 1/3, contradicting Eq. (25). Conjugating the operators would avoid this example, but that differs from applying the source operators directly. Empirical residual histograms, length trends and one embedding-pair comparison cannot repair these contradictions or establish the full stochastic assumptions. A weaker transfer lower bound also does not establish that a larger defect causes near-chance accuracy.

Questions for authors
  1. How should Eq. (16) be corrected to accommodate the scalar Gaussian counterexample above?

  2. Is the score in Result 2 intended to be normalized by sqrt(L-1)? Please reconcile Eqs. (6), (18) and (19), and provide a dependence-valid concentration proof with constants independent of L.

  3. Does transfer use coordinate-conjugated operators or unchanged source operators? Please address the exact coordinate-swap counterexample, specify compatible lifted dimensions, and distinguish a non-vacuous guarantee from qualitative empirical agreement.

Reviewer_drgFOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper aims to explain why a recent hallucination detection method (Wilson & Akrout, 2026) works. The method fits two linear Koopman models to token embedding trajectories, one for correct and one for hallucinated responses, and classifies a new response by comparing the prediction residuals under the two models. The authors cast this as a hypothesis test between two stable linear dynamical systems with Gaussian noise, and show three things: the stationary distributions of the two systems can be nearly identical even when the dynamics differ; the error of the residual-based classifier decays exponentially with sequence length; and the discriminability transfers across embedding models if the two embedding spaces are approximately related by a linear map. Experiments on HaluEval check the sequence length scaling, and a single embedding pair (Qwen3 and F2LLM) is used to explain the asymmetric cross-embedding transfer.

Critical strengths

The question itself is reasonable: why should a dynamical approach to classifying LLM responses work, and when should it transfer across embedding models? The three results are clearly explained, and the link between sequence length and reliability gives some practical guidance.

I actually like this paper overall, but I cannot recommend a higher score due to the glaring absence of the appendix.

Critical weaknesses

The proofs are missing. All three results point to Appendices A–D, but the submitted PDF ends with the references. Since the theory is the main contribution, in my opinion the paper cannot be properly evaluated in its current form.

The results are largely classical. Result 1 follows from a trivial example: two first-order autoregressive processes with opposite coefficients and the same noise have identical stationary distributions, despite having different dynamics. Result 2 is standard concentration for hypothesis testing between Gauss–Markov processes, and the minimum sequence length follows directly from it. As such, it is unclear what is new beyond applying known results from system identification and signal detection to this setting.

Questions for authors

In Result 2, the exponent scales with the sequence length times the squared gap between the mean score and the threshold, but the mean score itself grows with sequence length. Corollary 1, on the other hand, implies an exponent linear in sequence length. Can the authors clarify?

Reviewer_BBj5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the theoretical foundations of classifying LLM responses by modeling token embedding trajectories as dynamical systems. It provides a theoretical framework of a binary hypothesis test between stochastic linear dynamics systems for understanding the dynamical approach for classifying LLM responses. In particular, they develop the framework to answer theoretically why dynamics matter, how well the dynamical systems method works, and whether the embedding dynamics are generalizable. They also perform experiments on HaluEval to validate their framework.

Critical strengths
  1. The paper studies an interesting problem of understanding why temporal information in token embedding trajectories can be used to classify LLM responses. This could provide useful insights into the theoretical foundations of using such methods for analyzing LLM behavior.
Critical weaknesses
  1. There seems to be some mathematical errors. I think it would be worth going over the results and their corresponding proofs more carefully.
  • The Gaussian TV bound in Result 1 (Equation 16) appears to require additional assumptions. In particular, the covariance KL inequality used in Appendix A (Equation 28) holds if we additionally assume Γh⪰Γc\Gamma_h \succeq \Gamma_c, but fails for general positive-definite covariance matrices. The stated assumptions do not guarantee this, so the result appears incorrect as currently formulated.

  • The exponential classification-error guarantee in Result 2 also appears mathematically incorrect, and its proof contains unjustified concentration arguments. Equation (19) claims

\leq 2\exp\left( -\frac{(L-1)(\mu_L-\eta)^2}{C} \right),$$ where $\Delta E = \sqrt{S_h}-\sqrt{S_c}$, $\mu_L = \mathbb{E}[\Delta E \mid H_c]$, and $C$ denotes the remaining system-dependent terms in the denominator. Since Equation (43) gives $\mu_L = \Omega(\sqrt{L})$, this implies an $e^{-\Omega(L^2)}$ error bound for $\eta=0$, which I believe is generally false. The main issue seems to appear in the transition from Equations (43) to (44) in Appendix B. The authors derive concentration for $S_h-S_c$, but do not properly account for the normalization $$\Delta E = \frac{S_h-S_c}{\sqrt{S_h}+\sqrt{S_c}}.$$ Since the denominator is typically of order $\sqrt{L}$, the fluctuations of $\Delta E$ are generally of constant order rather than $O(L^{-1/2})$. Thus, I think the additional factor of $L-1$ in the exponent of Equation (44) is unjustified, and the resulting bound is false even for simple stable Gaussian systems. 2. The empirical validation also seems to be insufficient to establish that the theoretical assumptions and guarantees apply to actual LLM token embeddings. - The Gaussian residual fits do not verify the full stochastic dynamical-system assumptions, eg. joint Gaussianity, temporal independence of the noise terms. - The experiments in Figure 4 examine the approximate $\sqrt{L}$ scaling of the separability factor, but do not directly validate the claimed exponential decrease in classification error with sequence length. - The cross-embedding validation is limited to one model pair, without demonstrating that the bound is quantitatively informative.
Questions for authors
  1. Could the authors clarify whether additional assumptions are required for the Gaussian TV bound in Result 1? In particular, the covariance KL inequality in Equation (28) appears to potentially require an additional condition such as Γh⪰Γc\Gamma_h \succeq \Gamma_c, which is not guaranteed by the stated assumptions.

  2. Could the authors clarify the concentration argument in Result 2, particularly the transition from Equations (43) to (44) in Appendix B? Since μL=Ω(L)\mu_L=\Omega(\sqrt L), Equation (44) appears to imply an e−Ω(L2)e^{-\Omega(L^2)} bound for a fixed threshold. How is the additional factor of L−1L-1 in the exponent justified after accounting for the normalization in ΔE\Delta E?

  3. Could the authors provide further empirical validation of the theoretical assumptions and guarantees? In particular, it would be helpful to verify the stability of the fitted operators and the temporal independence of the Gaussian noise terms, as well as directly evaluate how classification error scales with sequence length. Additionally, testing more embedding-model pairs and evaluating whether the transfer bounds are quantitatively informative would strengthen the empirical support for the theory.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.