- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a model-authorship attribution method that replaces direct source classification with source-conditioned language modeling. Each source family is represented using a learned prefix plus source-specific LoRA adaptation, and attribution is based on relative token likelihoods with clipping to prevent a few extreme tokens from dominating the decision. The paper also introduces a benchmark.
Critical strengths
- MAABench targets a weakness of existing attribution work: robustness to both domain shift and model-version evolution. The benchmark is reasonably broad. This makes the evaluation considerably more relevant to deployment than standard ID-only MAA experiments.
- The proposed ATLAS improves over the strongest baseline. It is also data-efficient.
- The token-level clipping mechanism has a clear robustness motivation rather than being an arbitrary architectural addition. The ablations support that motivation. The additional teacher-attribution experiment also suggests that the learned signal captures persistent family-level characteristics rather than only a benchmark-specific decision boundary.
Critical weaknesses
- ATLAS assumes that every test text comes from one of the predefined candidate families. This is a significant limitation for real deployment, where entirely unseen generators will continually appear. Attribution to previously unseen source families, particularly without retraining, remains unresolved. An open-set evaluation with an explicit unknown class or calibrated rejection criterion would strengthen the work.
- In the reported Domain shift, both the domain changes and the system prompting changes: held-out Patent/Stack Exchange domains use persona-based system prompts rather than the default training prompt. Thus, the experiment demonstrates robustness to a useful compound shift, but it is difficult to determine how much comes from domain versus prompt/style variation. Moreover, generations are relatively controlled, shared prompts, which leaves robustness to paraphrasing, human editing, arbitrary decoding configurations, very short texts, and adversarial obfuscation less established.
- Reported number comes from a single training run with a fixed seed. Given that ATLAS and several learned baselines involve stochastic optimization, reporting mean ± standard deviation across multiple seeds would make especially the smaller performance differences much more convincing.
- Code and data are not realased yet and thus reproducibility is limited.
Questions for authors
Please refer to the weaknesses
5 comments
Response to Reviewer tMGY (2/2)
W2-2. Robustness beyond controlled generations: how robust is ATLAS to paraphrasing, human editing, arbitrary decoding configurations, very short texts and adversarial obfuscation?
ATLAS outperforms the strongest baseline in all five settings (Table 3 and new Appendix C.2). In each table, "Best baseline" is the strongest baseline in each row or column, and all models are trained on clean, full texts.
Paraphrasing, human editing and decoding (OpenTuringBench [2], Table 3, and RAID [1] reconstructed as 5-family attribution (GPT, Llama, Mistral, MPT, Cohere), new, Table 28). E3 lets each model rewrite its own text, E4 lets a human and a model write parts of the same text, and E1 changes the sampling temperature. ATLAS is best in every setting, with the largest gains under human editing and repetition penalty:
Table R3. Accuracy (%) under paraphrasing, human editing and decoding changes.
Setting ATLAS Best baseline Paraphrasing (OpenTuringBench E3) 98.79 95.99 Human editing (OpenTuringBench E4) 41.92 32.36 Sampling temperature (OpenTuringBench E1) 89.71 84.43 Greedy decoding (RAID) 72.89 72.39 Repetition penalty (RAID) 85.88 66.37
Very short texts (new, Tables 26 and 27). Models are evaluated on only 16 or 32 words of each test text. To check that ATLAS does not rely on the opening of a text, we also took 32 consecutive words from the middle of each text. ATLAS stays far more accurate than the baselines at every length, including the middle words:
Table R4. Accuracy (%) on very short texts.
Words ATLAS (ID) Best baseline (ID) ATLAS (Version) Best baseline (Version) First 16 75.82 26.36 52.94 22.81 First 32 90.26 36.36 64.74 28.77 Middle 32 74.67 36.49 52.71 29.35
Adversarial obfuscation (new, Table 25). We applied ten perturbations to the MAABench ID test set, following the existing attacks (i.e., the attack code and default attack rates of RAID [1]). ATLAS is best under nine of the ten attacks, with the largest gains under homoglyphs and zero-width spaces:
Table R5. Accuracy (%) under perturbation attacks on MAABench ID.
Attack ATLAS Best baseline Homoglyph 69.36 58.86 Zero-width space 45.04 25.50 Whitespace 99.19 99.20 Upper/lower case 99.63 99.19 Insert paragraphs 99.66 99.42 Article deletion 99.68 99.02 Number 99.73 99.48 Alternative spelling 99.78 99.51 Misspelling 99.78 99.40 Synonym 90.66 87.40 Avg. 10 attacks 90.25 86.30
[1] Dugan, Liam, et al. "Raid: A shared benchmark for robust evaluation of machine-generated text detectors." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.
[2] La Cava, Lucio, and Andrea Tagarelli. "Openturingbench: An open-model-based benchmark and framework for machine-generated text detection and attribution." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
W3. Single seed: do the results, especially the smaller differences, hold as mean and standard deviation over multiple seeds?
The results hold over three random seeds: ATLAS still outperforms every baseline on all four splits, with large margins on Version shift (12.3pp) and Joint shift (9.0pp) that far exceed the seed variation. It also has the lowest standard deviation on every OOD split. We trained ATLAS and all four baselines with two additional random seeds. Table 2 now reports the three-seed means, with standard deviations in Appendix B.5 (Table 19):
Table R6. Accuracy (%) over three seeds (mean ± std).
Method ID Domain Version Joint ATLAS 99.72 ± 0.05 97.97 ± 0.17 84.30 ± 0.08 74.27 ± 0.25 DeTeCtive 99.51 ± 0.05 94.69 ± 0.60 70.59 ± 0.58 63.34 ± 0.50 OTBDetector 99.49 ± 0.07 94.88 ± 0.63 70.19 ± 1.30 63.47 ± 2.29 DA-MTL 99.58 ± 0.02 95.31 ± 0.27 71.08 ± 0.56 64.99 ± 1.16 Idiosyncrasies 99.57 ± 0.10 95.03 ± 1.18 71.97 ± 0.83 65.27 ± 0.38
W4. Reproducibility: the code and data are not released yet.
We have uploaded the code and data as anonymized supplementary material. The code includes the training, scoring and evaluation scripts for ATLAS. The data includes the MAABench generated texts, seed identifiers and corpus revisions. Human-written texts are not redistributed, as described in the Ethics Statement, but can be recovered with the provided content-addressed identifiers.
We thank the reviewer again for the constructive comments, which helped us strengthen the paper.
Follow-up
Dear Reviewer tMGY,
We hope this message finds you well. We are writing to kindly follow up on our response to your comments, which includes three-seed results with standard deviations, open-set rejection, a domain-only control for Domain Shift, additional robustness results, and the released code and data. We want to make sure that our explanations adequately address the concerns you raised.
If any part of our response remains unclear, we would be very happy to provide further details. As the discussion period is coming to a close, we would greatly appreciate it if you could take a look at your convenience.
Thank you again for your time and thoughtful review.
Best regards, Authors
Response to Reviewer tMGY (1/2)
Thank you for the careful reading of our paper and the constructive comments. Our responses to each weakness follow.
Thank you for this suggestion. ATLAS can reject texts from unseen families: it rejects a held-out family with an average AUROC of 90.3 on ID data, using a score that its likelihood formulation already provides (new, Appendix C.3, Table 29).
The rejection score is the gain of the best source condition over the frozen backbone, maxks~k(y), the quantity behind the rejection rate in Section 6. In a leave-one-family-out evaluation, ATLAS is trained on four families (10k texts per source), the fifth is the unknown class, and one threshold is calibrated on ID validation to accept 95% of known texts. Unknown families are separated well on average, best for Mistral and Llama and hardest for GPT, while known texts are falsely rejected at about the calibrated 5% rate:
Table R1. Leave-one-family-out open-set rejection.
The same score already rejects models outside the candidate families: in Section 6, Falcon3-7B-Instruct and Granite-3.0-8B-Instruct are rejected for 68.2% and 74.6% of their texts. As stated in our conclusion, extending attribution to unseen families, particularly without full retraining, and rejection under domain and version shift are directions we leave to future work.
For ATLAS, the whole drop comes from the domain change. For the baselines, the persona prompt causes a further drop on top of it.
The persona prompt reflects how each domain is written in practice, not an artificial perturbation: patent abstracts are generated as "a patent specialist" and Stack Exchange questions as "a member of an online Q&A community" (Appendix A.3). OpenTuringBench [2] likewise generates its out-of-domain essays with a different system prompt from its training domain (news). To separate the two factors, we added a domain-only control (new, Appendix C.1): texts generated from the same metadata by the same models, but with the default training prompt (means over three random seeds).
Table R2. Accuracy (%) with the domain and the prompt changed separately (mean over three random seeds).
[2] La Cava, Lucio, and Andrea Tagarelli. "Openturingbench: An open-model-based benchmark and framework for machine-generated text detection and attribution." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.