acceptodds
Under review as a conference paper at ICLR 2027

Unveiling Spectral Mechanisms in Training-Free LLM Text Detection

Abstract

The rapid advancement of Large Language Models (LLMs) makes it increasingly difficult to distinguish human writing from machine-generated text. Training-free detection offers a scalable solution, yet common confidence-based metrics mainly measure average token probabilities and often miss the signal fluctuations that characterize human writing, which we call “generative vitality”. Spectral analysis offers a way to capture this vitality, but its mechanism and practical boundaries remain underexplored. In this paper, we analyze spectral detection from both theoretical and empirical perspectives. We connect spectral energy to variance in proxy log-probability trajectories and explain how broader human token choices create the fluctuations used by frequency-domain indicators. We further show that the strength of this signal depends on text length and sampling range: spectral evidence is clearest for long, continuous, constrained generation, while short, fragmented, mixed, and edited settings require complementary confidence and fluctuation views. These findings clarify when frequency-domain detection works and provide guidance for future multi-dimensional detector design.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM-Generated Text Detection, Training-Free Detection, Frequency-Domain Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6BVaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the underlying mechanisms and practical limitations of spectral-based methods for training-free LLM-generated text detection. The authors provide a theoretical analysis connecting differences in low-probability token selection between human and LLM-generated text to log-probability variance and spectral energy. Through experiments, they examine how sequence length, decoding strategies, and human–AI collaborative editing affect the effectiveness of spectral-based detection. The results suggest that spectral-based methods perform better on longer, continuously generated text, while confidence-based methods are more effective for short or locally edited text. The paper further explores combining confidence and fluctuation signals to improve detection robustness across different scenarios.

Critical strengths
  1. The paper systematically evaluates spectral-based detection methods across various settings, including sequence length, decoding strategies, mixed-source text, and human–AI collaborative editing.
Critical weaknesses
  1. Limited novelty in motivation. The motivation of studying fluctuations in token probability sequences has already been explored by prior methods such as Lastde and SurpMark [1]. Moreover, Fourier-based approaches such as FACE [2] and FourierGPT [3] have already investigated spectral differences between human and LLM-generated text, with explicit theoretical motivations. The paper does not sufficiently distinguish its motivation and contributions from these existing works.

  2. Limited theoretical contribution. The theoretical result does not provide a nontrivial explanation for the observed human–LLM differences, and the analysis ultimately reduces the proposed spectral mechanism to a simple variance difference without revealing any genuinely frequency-specific insight.

[1] Black-box Detection of LLM-generated Text Using Generalized Jensen-Shannon Divergence ICML 2026 [2] FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropy NeurIPS 2023 [3] Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihood EMNLP 2024

Questions for authors
  1. The theorem assumes shared head/tail conditional statistics between human and LLM-generated text, which may not hold for real-world distributions. Could the authors empirically quantify how much of the observed variance gap is attributable specifically to differences in tail-token frequency, rather than differences in the conditional statistics themselves?
Reviewer_dSfMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Some approaches to detecting machine generated text rely on fluctuations in the likelihood signal of next tokens in the text. More precisely, a text is fed through a proxy model (not necessarily the same model that we are trying to detect text from) and a sequence of log-likelihoods is produced. A very common method of detecting machine text is to average this sequence of log-likelihoods, the present work is concerned with the question of fluctuations in the sequence of log-likelihoods. A handful of papers have been published introducing methods to measure this fluctuation, the present work is concerned with three questions, why are these methods effective, what is the boundary of their effectiveness, and what can be done to address the case that they fail.

Both theoretical and experimental results related to these questions are given.

Critical strengths

The questions that the paper seeks to address are well motivated and interesting. Fundamentally, the problem which the paper seeks to solve is worth solving.

The writing is generally good and easy to follow. It is very easy to skim read this paper, understand what they do, what problem it solves, and how they justify their arguments.

Critical weaknesses

Theorem 1 gives a strong conclusion, but this strength really comes entirely from the strong assumptions. The actual derivation is two lines. Similarly for Corollary 2. This is not a problem, but I don't think the theoretical section really advances things. When one reads the paper one gets the impression that difficult and deep mathematics is being done, but this is not really the case. (Not really a critical weakness, but I discount the theoretical section as a major strength of the paper)

The choices of decoding strategy (top-k=50, ablation with top-k=20-60) are mentioned casually, but I suspect they have a really profound effect on the result. Claims that human text visit the tail of the probability distribution much more frequently, which are assumptions in Theorem 1, can be entirely explained by the fact that the decoding strategy is steering machine text away from the tail of the probability distribution. See for example the work DMAP: A Distribution Map for Text (Kempton et al, ICLR 2026), which showed that many of the reported effects of likelihood on the detection of machine generated text are entirely an artefact of accidentally applying top-k=50 and are reversed when one samples by pure sampling.

Questions for authors

Page 3, what is the benefit of setting y=f(x)? Have already restricted to x= log probability of next token. Some of the metrics you describe are not purely functions of log-prob of next token (e.g. DetectGPT cannot be written as f(x).

Page 3, paragraph 3, 'Human and LLM text differ in this tail-token rate'. I am not sure that this claim holds universally, as mentioned in the weaknesses section the DMAP work seems to contradict it. I would soften the language here.

I think you miss a third important paper on using fluctuation to detect machine generated text, Black Box Detection of LLM-Generated Text using Generalized Jensen-Shannon Divergence (ICML 2026). This should be included.

Generally, I think the work rests on painting as universal properties of language model generated text which actually depend on specific decoding choices. I would advise to be more careful around these claims.

Reviewer_yhh1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the mechanisms underlying spectral approaches to training-free detection of language-model-generated text.

The authors analyze token log-probability trajectories obtained from a proxy language model and study how differences in their statistical fluctuations can distinguish human-written and machine-generated content.

The paper relates spectral energy to variability in token log-probabilities and examines the effectiveness of spectral detection under different conditions, including short texts, mixed-source documents, and human–AI collaborative editing.

The empirical investigation identifies settings in which spectral information is useful and explores combinations of spectral and confidence-based detection signals.

Critical strengths
  1. Relevant investigation of the limitations of training-free detection.

Understanding why existing detection statistics work, and under which conditions they fail, is an important problem.

The analysis of short, mixed-source, and collaboratively edited documents addresses realistic scenarios that are often insufficiently represented in conventional detection benchmarks.

  1. Useful empirical characterization of complementary detection signals.

The experiments investigate the relationship between confidence-based and fluctuation-based statistics and provide evidence that their relative effectiveness depends on text length and editing conditions.

These observations may help clarify the limitations of existing spectral detection approaches.

Critical weaknesses
  1. Limited theoretical novelty and explanatory depth.

The principal theoretical connection between spectral energy and variance follows from standard signal-processing identities, including Parseval’s theorem.

While this provides a mathematical interpretation of spectral detection statistics, it does not establish a new mechanism explaining why human and machine-generated texts should exhibit systematically different spectral properties.

The central explanation relies on differences in token-probability fluctuations, which are already closely related to the motivation of existing spectral detection approaches.

Consequently, the theoretical contribution appears primarily interpretative rather than a substantive advance in detection methodology or statistical understanding.

  1. Limited methodological advance and insufficient evidence of robust generalization.

Much of the empirical investigation characterizes the behavior of existing detection statistics rather than introducing a substantially new detector.

The effectiveness of the proposed combinations depends on the generation conditions, proxy model, and editing scenario. It remains unclear whether the reported observations generalize reliably across different model families, decoding strategies, and realistic human–AI writing workflows.

Furthermore, the connection between the proposed spectral interpretation and the practical detection improvements is not sufficiently established.

Questions for authors
  1. Beyond the variance–spectral-energy identity, what theoretical result explains the separation between human and machine-generated text distributions? Can the authors formulate assumptions under which the proposed statistics are provably discriminative?
  2. How robust are the reported conclusions to changes in proxy language models, decoding temperature, and generation strategies designed to produce more variable token-probability trajectories?
  3. Do the proposed combinations provide consistent improvements over strong existing training-free detectors under matched evaluation conditions, particularly for mixed and edited documents?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.