acceptodds
Under review as a conference paper at ICLR 2027

AI-generated number sequences contain detectable, non-human biases

Abstract

State-of-the-art AI detection models are classifiers trained on millions of human and AI documents. To tell the difference between human and AI text, trained experts rely primarily on features like characteristic vocabulary, sentence structure, or other visible heuristics. Whether trained classifiers actually depend on these cues, however, is unclear. We test if AI detectors can operate in a domain where none of these heuristics apply: number sequences. Pangram, a state-of-the-art AI detector, distinguishes LLM-generated sequences from identically formatted human-written and true-random baselines with an AUROC of 0.995. Transformation effects are heterogeneous: reversal and some structured recodings preserve detectability, while shuffling and some bijective substitutions eliminate it, implicating an interaction between sequence arrangement and representation. Examining sequence structure across eight open models, we find that LLM-generated sequences differ from human-generated sequences in their statistical structure, including average jump size and complete 10-digit windows. A fitted, interpretable simulation model captures 84% of this structure, and the simulated sequences begin to reproduce AI scores. Sweeping checkpoints, decoding, and chat formatting, we find that post-training, templates, and truncated decoding generally amplify this structure. Thus, Pangram can exploit subtle statistical fingerprints in number sequences when lexical and syntactic cues are absent.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.