WebMix-Collapse: A Rewrite Benchmark for Sizing the Length Confound in Short-Form Synthetic-Text Detection

On rewrite-based benchmarks for synthetic-text detection, a detector can score well by counting words. We build such a benchmark, WebMix-Collapse, with 10,668 English web documents and model rewrites spanning six contamination rates, ten generator families (1B to 671B parameters) and five rewrite depths, and we measure how much of a statistical detector's accuracy on it is document length. Word count alone reaches AUROC 0.942. Our 19-feature ensemble reaches 0.987, so a single feature recovers 91% of the ensemble's signal above chance. The strongest test uses a split in the release where human documents are the shorter class. There the ensemble does not drift toward chance. It inverts, to 0.289, and matching on length inside the split brings it back to 0.853. Fast-DetectGPT and Binoculars at GPT-2 scale, scored on the same split, do not invert (0.969 and 0.977). The same artifact explains two results that would otherwise pass as findings: accuracy that rises with rewrite depth, where the ensemble beats word count by at most 0.0026, and an apparent DeepSeek-R1 evasion (0.562) that disappears once reasoning traces are stripped (0.994). Out of domain the ensemble keeps most of its ranking (0.855), yet the threshold that gives 1% false positives in domain flags 85% of held-out human documents. A fine-tuned RoBERTa (0.947) and Binoculars at Falcon-7B scale (0.849) hold up out of domain, so the problem is the way rewrite benchmarks are evaluated, not statistical detection itself. We package the controls as the Confound Attribution Protocol (CAP), run it on fifteen detector configurations, and report the two steps that do not work. Our recommendation is simple: report every rewrite-based detection result next to a length-only baseline on the same rows.