Learning Sequential–Structural Discrepancy: Label-Free Pretraining for Domain-Robust LLM-Generated Text Detection
Abstract
The rapid progress of large language models (LLMs) has made it increasingly difficult to distinguish human-written from LLM-generated text, particularly under domain shift. Existing training-based detectors typically learn sequential or syntactic cues under human/LLM provenance supervision, while approaches combining the two often rely on direct feature concatenation, leaving their relationship underexplored and potentially capturing domain-specific artifacts. We empirically find a systematic difference between human-written and LLM-generated text in the discrepancy between sequential and structural representations: human-written text tends to show a larger cross-view discrepancy, whereas LLM-generated text exhibits stronger cross-view consistency. Importantly, this distributional gap persists across domains even when the two views are learned without human/LLM provenance labels. Based on this finding, we propose the **Sequential–Structural Discrepancy (SSD)** framework. SSD first pretrains sequential and structural encoders on unlabeled text through masked representation prediction and subsequently freezes both encoders for detection. The resulting token-level discrepancy plays two complementary roles: it controls the adaptive fusion of the sequential and structural views and is retained as an explicit feature for downstream classification. Supervised training with human/LLM provenance labels updates the downstream fusion and classification modules, while both pretrained view encoders remain frozen. Experiments on in-domain and out-of-domain benchmarks demonstrate that SSD consistently outperforms strong baselines and maintains robust performance under adversarial perturbations.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.