- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the underlying mechanisms and practical limitations of spectral-based methods for training-free LLM-generated text detection. The authors provide a theoretical analysis connecting differences in low-probability token selection between human and LLM-generated text to log-probability variance and spectral energy. Through experiments, they examine how sequence length, decoding strategies, and human–AI collaborative editing affect the effectiveness of spectral-based detection. The results suggest that spectral-based methods perform better on longer, continuously generated text, while confidence-based methods are more effective for short or locally edited text. The paper further explores combining confidence and fluctuation signals to improve detection robustness across different scenarios.
Critical strengths
- The paper systematically evaluates spectral-based detection methods across various settings, including sequence length, decoding strategies, mixed-source text, and human–AI collaborative editing.
Critical weaknesses
-
Limited novelty in motivation. The motivation of studying fluctuations in token probability sequences has already been explored by prior methods such as Lastde and SurpMark [1]. Moreover, Fourier-based approaches such as FACE [2] and FourierGPT [3] have already investigated spectral differences between human and LLM-generated text, with explicit theoretical motivations. The paper does not sufficiently distinguish its motivation and contributions from these existing works.
-
Limited theoretical contribution. The theoretical result does not provide a nontrivial explanation for the observed human–LLM differences, and the analysis ultimately reduces the proposed spectral mechanism to a simple variance difference without revealing any genuinely frequency-specific insight.
[1] Black-box Detection of LLM-generated Text Using Generalized Jensen-Shannon Divergence ICML 2026 [2] FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropy NeurIPS 2023 [3] Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihood EMNLP 2024
Questions for authors
- The theorem assumes shared head/tail conditional statistics between human and LLM-generated text, which may not hold for real-world distributions. Could the authors empirically quantify how much of the observed variance gap is attributable specifically to differences in tail-token frequency, rather than differences in the conditional statistics themselves?