acceptodds
Under review as a conference paper at ICLR 2027

MuST: Multilingual Span-level Machine-Generated Text Detection

Abstract

Detecting machine-generated text (MGT) is critical, as the misuse of large language models (LLMs) has raised concerns about the integrity, authenticity, and trustworthiness of information. In real-world scenarios, the task is challenging because human–AI collaboration produces hybrid texts, motivating the development of fine-grained detection of MGT. This line of research performs well on the English corpus, but its generalization to other languages is under-exploited. To address this challenge, we introduce MuST, a framework for multilingual fine-grained MGT detection. Our framework features unified modeling for two fine-grained MGT tasks, i.e., span-level detection and boundary detection. Besides, to accommodate different languages, we design augmented word embeddings by combining the information of neighboring words, N-grams, the sentence, and the normalized position. To assess our approach under multilingual settings, we further propose MuST-Mix, a fine-grained dataset for multilingual MGT detection. Evaluation shows that MuST outperforms 11 SOTA detectors in eight languages and is robust to unseen generators and unseen domains.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine-generated text detection, Multilingual, Fine-grained text detection.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8UVuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper makes two main contributions: The first is MuST-Mix, a multilingual dataset for span-level detection of AI-generated text. The second is MuST, a detection approach that has been evaluated across eight languages.

MuST-Mix is created by combining existing corpora of human-written and machine-generated texts. The authors use semantic matching to align semantically similar sentences, and controlled replacement to generate hybrid human–AI paragraphs containing different numbers of transitions in authorship. This procedure enables the creation of word-level annotations and sentence-level boundary labels.

The proposed detector is evaluated against several existing approaches using complementary metrics, including mean absolute error, the ratio of AI-generated content, word-level F1, boundary-type F1 and boundary-position F1. The experimental results indicate that MuST outperforms the evaluated baselines across the reported settings, although performance varies across languages and evaluation metrics.

Critical strengths

1. Multilingual coverage. The dataset covers eight languages, providing a useful resource for evaluating AI-generated text detection beyond English and investigating cross-linguistic differences in detection performance.

2. Comprehensive evaluation metrics. The evaluation considers complementary aspects of span-level detection, including word-level classification, AI-generated content proportion, and boundary type and localisation. This provides a more detailed assessment of detection performance than relying solely on document-level classification metrics.

Critical weaknesses

1. Dataset construction and representativeness The dataset uses semantic matching and the controlled replacement of complete sentences to generate hybrid human–AI documents. While this procedure does provide precise annotations, it is unclear to what extent these artificially generated examples accurately reflect realistic human–AI writing scenarios. Human–AI texts may involve partial rewriting, editing within sentences, paraphrasing and iterative revisions, all of which are not necessarily captured by the proposed construction procedure.

The dataset's description in the paper could be enhanced by including representative annotated examples and a clearer analysis of its composition, including differences across languages, source datasets and span lengths. Tables 1 and 8 provide general statistics but they do not report the distribution of samples across individual LLM generators and domains, nor the number of original documents used to construct the hybrid paragraphs. Moreover, the dataset is highly imbalanced across languages, with English, Russian, and Chinese accounting for approximately 76% of the samples. There are also substantial differences in average paragraph length and domain coverage across languages. These differences deserve further discussion, as they may influence the reported detection performance. Given the span-level nature of the task, additional statistics on sentence counts, span lengths, and the proportion of AI-generated content would also help characterize the dataset. A more detailed breakdown, potentially included in the supplementary material, would improve transparency and facilitate the interpretation of the experimental results.

Therefore, I am not fully convinced that the paper successfully demonstrates the broader significance and practical applicability of the proposed approach. While identifying AI-generated spans is a legitimate research problem, the distinction between human-written and AI-generated content is not always straightforward in realistic collaborative writing scenarios. It would therefore be useful for the motivation section to clarify which practical applications the proposed detector is intended to support, and whether its performance on synthetically constructed hybrid texts can be expected to generalise to these scenarios.

2. Limited experimental analysis and interpretation I find the experimental analysis insufficient to understand the proposed framework. The results would improve with a detailed interpretation of performance differences across languages and source datasets, as well as an examination of how span length and the number of human–AI transitions affect detection performance.

Besides, the lack of a qualitative error analysis makes it difficult to understand the types of mistakes made by the detector. Representative examples of false positives, false negatives, and incorrectly located boundaries could help identify whether errors are associated with linguistic characteristics, discourse coherence, or artifacts introduced during dataset construction.

Overall, while the quantitative results are promising, the limited analysis makes it difficult to draw broader conclusions about the detector's behaviour and generalisation capabilities beyond the reported benchmark scores.

Questions for authors
  1. How do the authors ensure that the construction of MuST-Mix does not introduce artifacts that the detector could exploit instead of learning generalizable characteristics of AI-generated text?
  2. Have the authors investigated performance differences according to the source dataset, span length, or number of human–AI transitions? If not, what conclusions can be drawn about the generalization of MuST beyond the specific construction procedure?
9 comments
AuthorsOpenReview ↗

Response to Reviewer 8UVu (1/3)

Thank you for the detailed assessment of the practical setting, dataset composition, representative errors, and generalization across span structures and construction protocols. We have expanded the dataset statistics, added English error examples with plausible linguistic and discourse explanations, and supplied analyses by MGT span length and boundary count together with external-dataset comparisons. Added and substantively revised passages and table entries are highlighted in blue in the revised PDF, including all content from Appendix B onward. The responses below identify the reporting and analysis gaps in the original manuscript and present the corresponding evidence, including the qualitative interpretations supported by the examples.

A1: Practical use and dataset composition.

We thank the reviewer for requesting more detail on practical use, annotation, and dataset composition. MuST is intended to assist human review by identifying candidate AI-origin passages and estimating the AI proportion, with the human reviewer interpreting the evidence. Section 4.1 defines the provenance and boundary labels, and Section 6 discusses finer-grained within-sentence mixing as future work. Appendix B.1 now gives two English construction examples, including the five conditions of an HC3 paragraph and a DetectRL-X example with three human sentences aligned to one rewritten sentence. The following tables report the requested dataset information; the sample counts and average lengths retain the original submission values. MuST-Mix contains 1,398,668 reconstructed samples. English, Russian, and Chinese contribute 1,166,661 (83.4%). The Overall detection results give equal weight to the eight languages, and Section 6 identifies broader and more balanced language coverage as future work. Appendix B supplies the source-document, generator, domain, sentence-count, AI-ratio, span-length, and split-audit details.

Table D1. Language-level sample counts and average paragraph length.

Language#SamplesAvg. length
Arabic44,117529.8
English452,615252.6
French47,365530.1
German46,867440.1
Portuguese46,997530.7
Russian449,406285.9
Spanish46,661479.1
Chinese264,640537.3
Total1,398,668–

Table D2. Source documents and reconstructed rows per source document.

CorpusLanguage#Source documentsRows/doc
DetectRL-XEnglish9,5994.86
DetectRL-XChinese9,6004.88
DetectRL-XRussian9,5934.84
DetectRL-XGerman9,5964.88
DetectRL-XFrench9,5974.94
DetectRL-XSpanish9,6004.86
DetectRL-XPortuguese9,5964.90
DetectRL-XArabic9,4364.68
LLMTraceEnglish89,9233.61
LLMTraceRussian94,2774.27
HC3English8,7165.00
HC3Chinese13,5854.03
C-ReDChinese10,99714.82
M4GT Task 3English7,4995.00

Table D3. Sentence-count, AI-ratio, AI-span-length, and transition statistics of reconstructed paragraphs.

CorpusLang.SentencesAI ratioAI ratioAI spanTransitions
(mean)(mean)(median)(mean)(mean)
DetectRL-XEnglish20.60.2800.105151.30.96
DetectRL-XChinese26.40.2650.075339.31.07
DetectRL-XGerman22.40.2610.083145.01.00
DetectRL-XFrench23.80.2700.097185.91.04
DetectRL-XPortuguese22.60.2750.101185.31.02
DetectRL-XArabic22.00.3050.121156.70.94
LLMTraceEnglish13.00.4000.16498.20.69
LLMTraceRussian24.80.3260.04971.10.58
HC3English8.60.4690.41393.10.99
HC3Chinese5.20.5380.571121.20.37
C-ReDChinese16.00.3930.225201.11.52
M4GT Task 3English8.80.4760.42592.21.06
AuthorsOpenReview ↗

Response to Reviewer 8UVu (2/3)

Table D4. Number of reconstructed samples contributed by each generator, aggregated per corpus and language.

CorpusLang.Generator counts
DetectRL-XEnglishQwen: 10,557; GPT: 10,511; Gemini: 10,506; DeepSeek: 10,403; Unassigned: 4,681
DetectRL-XChineseQwen: 11,818; Gemini: 11,801; GPT: 11,741; DeepSeek: 11,470; Unassigned: 34
DetectRL-XGermanGemini: 10,591; Qwen: 10,562; GPT: 10,544; DeepSeek: 10,488; Unassigned: 4,682
DetectRL-XFrenchGemini: 10,684; Qwen: 10,672; GPT: 10,647; DeepSeek: 10,619; Unassigned: 4,743
DetectRL-XPortugueseQwen: 10,615; Gemini: 10,606; GPT: 10,567; DeepSeek: 10,526; Unassigned: 4,683
DetectRL-XArabicQwen: 10,004; Gemini: 9,943; GPT: 9,927; DeepSeek: 9,839; Unassigned: 4,404
HC3EnglishGPT: 43,580
HC3ChineseGPT: 54,775
C-ReDChinesegemini-2.5-flash: 17,640; gpt-4o: 17,232; doubao-1.5-pro: 16,556; claude-3.5-haiku: 16,324; gpt-3.5-turbo: 15,844; deepseek-v3: 15,728; qwen-3: 15,016; deepseek-r1: 14,248; qwen-2.5: 14,076; Unassigned: 20,337
M4GT Task 3EnglishLLaMA: 26,180; GPT: 11,315

Table D5. Number of reconstructed samples per domain, aggregated per corpus and language.

CorpusLang.Domain counts
DetectRL-XEnglishNovel: 7,171; Wiki: 7,132; News: 7,130; Academic: 6,933; SEO: 6,814; Webtext: 6,797; Unassigned: 4,681
DetectRL-XChineseAcademic: 7,922; Webtext: 7,921; News: 7,892; Wiki: 7,880; Novel: 7,796; SEO: 7,419; Unassigned: 34
DetectRL-XGermanWiki: 7,163; Novel: 7,118; Webtext: 7,111; News: 7,021; Academic: 7,017; SEO: 6,755; Unassigned: 4,682
DetectRL-XFrenchNovel: 7,163; Academic: 7,136; Wiki: 7,129; Webtext: 7,115; News: 7,073; SEO: 7,006; Unassigned: 4,743
DetectRL-XPortugueseNovel: 7,101; Wiki: 7,098; Academic: 7,066; Webtext: 7,060; SEO: 6,995; News: 6,994; Unassigned: 4,683
DetectRL-XArabicWiki: 7,066; Webtext: 6,982; News: 6,814; Novel: 6,768; SEO: 6,341; Academic: 5,742; Unassigned: 4,404
HC3Englishhc3: 43,580
HC3Chinese54,775 total
C-ReDChineseQuestion-answer: 56,688; News: 45,384; Composition: 37,757; Film review: 12,832; Paper: 10,340
M4GT Task 3EnglishPeerRead: 26,730; Outfox: 10,765

Table D6. Source-document split audit; cells report reconstructed rows / source groups.

LanguageTrainValidationTestSource overlapText duplicates
English32,645 / 6,7194,681 / 9599,332 / 1,92100
German32,790 / 6,7164,682 / 9589,395 / 1,92200
Chinese32,807 / 6,7204,668 / 9589,355 / 1,91500

A2: Error examples and plausible linguistic or discourse causes.

We appreciate the reviewer's suggestion to connect representative prediction errors with possible linguistic and discourse causes. Appendix B.2 adds three English examples with source metadata, gold and predicted labels, and relevant excerpts. The table below summarizes the proposed qualitative explanations. These are plausible associations in the examples.

Error typeObserved predictionPlausible linguistic or discourse contribution
False positiveA human opening sentence is labeled AIFormulaic promotional wording and a register change
False negativeA generated closing slogan is labeled humanA short span, repeated product vocabulary, and topical continuity
Displaced boundaryThe human-to-AI boundary is predicted one sentence earlyShared explanatory style and a backward reference across the true junction
AuthorsOpenReview ↗

Response to Reviewer 8UVu (3/3)

A3: Performance differences across languages and source datasets.

We thank the reviewer for requesting a detailed interpretation of performance differences across languages and source datasets. The existing language-wise comparisons in Section 5.2 and the component analyses in Section 5.3 address complementary aspects of this question. Section 5.3 explains the different contributions of contextual cues: removing sentence embeddings lowers Chinese Pos. from 0.8803 to 0.6735, while removing the contextual encoder lowers Russian and Spanish Pos. from 0.7284/0.7625 to 0.6524/0.7182. These correspond to relative decreases of 23.49%, 10.43%, and 5.81%, respectively (Tables 3 and 12). Section 5.4 also compares language-specific generalization under unseen generators and domains. MuST's Overall Pos. is 0.7686 under cross-generator evaluation and 0.6990 under cross-domain evaluation, supporting greater difficulty under domain shift in these protocols.

We have expanded Section 5.2 to connect these ablations to the cross-language results. Word F1 ranges from 0.9800 to 0.9928 across languages, and Pos. ranges from 0.7284 to 0.8803. This larger variation in Pos. identifies exact transition localization as a particularly variable part of the task. The text also explains that the language subsets differ in tokenization, generators, and domains, which jointly shape their evaluation conditions.

Section 5.2 now also compares the source subsets used in Figure 4. Relative to HC3, MuST records a 17.32% decrease in Pos. (0.9614 → 0.7949) and a 324.53% increase in MAE (0.0053 → 0.0225) on C-ReD. C-ReD has more transitions per paragraph on average, 1.52 versus 0.99 (Table 29), alongside differences in language, generators, and domains. Figure 4 supports the structural interpretation: MuST Pos. increases from 0.7296 in the shortest-span bin to 0.8627 in the longest, and MAE rises overall from 0.0174 at one gold boundary to 0.0426 at six. MuST leads Qwen-FT and Qwen-CRF in every bin where the corresponding metric is reported. These observations support span structure as a contributor to dataset-specific difficulty.

We have expanded Section 5.6 with an interpretation of the external-source results in Table 7. On LLMTrace, Word F1 decreases by 0.38% (0.9587 → 0.9551) from English to Russian, and Pos. decreases by 26.36% (0.6897 → 0.5079). This contrast highlights sensitivity of exact transition recovery to the target language and source distribution. Against the strongest Qwen baseline, MuST improves Pos. by 93.52% (0.3564 → 0.6897) in English and 156.00% (0.1984 → 0.5079) in Russian. The external comparisons are summarized below; M4GT Task 3 uses training-split adaptation, and LLMTrace uses its original English and Russian test splits.

BenchmarkMuST Word F1MuST Pos.Qwen-FT Pos.Qwen-CRF Pos.
M4GT Task 3 (EN)0.94150.64330.35110.3576
LLMTrace-EN0.95870.68970.20800.3564
LLMTrace-RU0.95510.50790.12560.1984

Together, Sections 5.2, 5.3, 5.4, and 5.6 connect the observed differences to contextual-cue ablations, span structure, and changes in the target data distribution. Appendix C additionally reports the observed variation over three seeds and one to three epochs.

Reviewer_LmJJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MuST, a model for fine-grained machine-generated text detection in multilingual hybrid texts. It treats the problem as two tasks, word-level provenance labelling and sentence-boundary transition detection. These are trained jointly with a word head, a boundary head and a boundary-count regularizer. The authors also build MuST-Mix, about 1.35M paragraphs in eight languages. It is made by aligning human paragraphs with their LLM rewrites from five existing corpora and swapping in one to three matched machine sentences. MuST beats 11 baselines on in-domain results in all eight languages, with the largest gains on boundary position F1.

Critical strengths
  • S1. The experiments are broad. They cover eight languages, 11 baselines, ablations per language, unseen generators and domains, five attack types, and runtime. Since there are very few multilingual comparisons like this, the tables are useful on their own.

  • S2. Treating word labelling and boundary detection as two joint tasks is a sensible idea, and Figure 1 shows well why either one alone can mislead. The ablations back this up. Removing either head hurts, and even the word-only version beats all baselines on boundary placement.

Critical weaknesses

W1. The model is only tested on the authors’ own way of making mixed text.

  • Every mixed example is made by swapping whole sentences. So AI and human text always switch at the end of a sentence end, and the task becomes mostly “is this sentence AI or not?” which isnt very realistic in human-AI co-writing
  • Pasting sentences from another text can leave signals in flow or style. The model may learn to spot these seams instead of AI writing itself.
  • The authors do not test MuST on other datasets which were created differently, such as the original M4GT Task 3 boundary data, LLMTrace’s own annotations, or AI text edited by humans. So we don’t know if the results carry over to other datasets and real co-writing.

W2. The novelty is overstated. Related work misses a prior paper. Kadiyala et al. (2025) already did word-level detection of mixed human-AI text in 23 languages, with a dataset of 2.4M texts. With that work out there, the main new parts are the joint two-head training and the sentence-swap dataset, which is less new than the paper suggests.

W3. Some results are not explained.

  • From my understanding, Type F1 requires the direction (human-to-AI or AI-to-human), and position F1 does not. So on the boundaries themselves, position F1 is the easier score and should be at least as high. Type F1 only looks higher because it also averages in the “no change” class, which is more common and easy. For every model in Table 2 the two scores agree once you account for this. MuST, for example, gets 0.86 type F1 and 0.80 position F1. Qwen-FT does not. Its type F1 of 0.73 means it places and directs boundaries correctly fairly often, yet its position F1 is only 0.17 (0.12 in English). In Chinese its two scores agree (0.81 and 0.67). What is the explanation for this? It matters because Qwen-FT is the strongest baseline on three of the four metrics and the fairest comparison, since it uses the same backbone as MuST..
  • The parts added to help with multiple languages (word sequences, neighbouring words, position) each change the scores by less than one point. No error bars or multiple seeds are reported, and training is a single epoch. The per-language ablations (Tables 3 and 11) also don’t show these parts helping non-English languages more than English, which is the paper’s reason for adding them.
Questions for authors
  • Q1. Is the train/test split done by source document or by sample? Each source paragraph produces five versions (human only, one to three sentences swapped, fully AI). If these are split separately, the same sentences end up in both training and test data, which would inflate all in-domain results.

  • Q2. Why does Qwen-FT get high word F1 but very low position F1, except in Chinese? Is there a word-alignment or boundary-decoding issue for languages that split words by spaces?

Suggestions

  • Train on MuST-Mix and test on a mixed dataset built by someone else, for example M4GT Task 3, LLMTrace or the Kadiyala et al. data.
  • Compare against the Kadiyala et al. multilingual models.
5 comments
AuthorsOpenReview ↗

Response to Reviewer LmJJ (1/3)

Thank you for the detailed assessment of construction validity, prior multilingual detection work, experimental reliability, metric definitions, and data splitting. Your comments prompted us to correct the contribution statement and Qwen-FT boundary evaluation, expand the baseline and external comparisons, and report results across seeds and training durations. Added and substantively revised passages and table entries are highlighted in blue in the revised PDF, including all content from Appendix B onward. The responses below distinguish the corrections to our positioning and evaluation from clarifications of the metric relationship and splitting protocol, with supporting tables, derivations, and source-level audits.

A1: Construction artifacts and external evaluation.

We appreciate the reviewer's concern about whether cues learned from reconstructed data transfer to other construction protocols. Our evidence addresses this through the original test splits and native annotations of datasets constructed by others. The models are initially trained on MuST-Mix, and the M4GT Task 3 results include training-split adaptation. Table 7 contains the complete comparisons below. MuST leads both Qwen baselines on all four metrics for all three benchmarks. Its Pos. advantage over the strongest Qwen baseline is 28.57, 33.33, and 30.95 percentage points on M4GT Task 3 and the English and Russian LLMTrace subsets, respectively. These results support transfer of learned MGT cues across construction protocols. Table 6 additionally shows the strongest aggregate MAE and Word F1 under expansion, polishing, and condensation.

Table 7. External-construction evaluation.

BenchmarkMethodMAE ↓Word F1 ↑Type F1 ↑Pos. F1 ↑
M4GT Task 3 (EN)MuST0.04490.94150.56150.6433
M4GT Task 3 (EN)Qwen-FT0.07120.92010.43710.3511
M4GT Task 3 (EN)Qwen-CRF0.08640.89210.44050.3576
LLMTrace-ENMuST0.04810.95870.78670.6897
LLMTrace-ENQwen-FT0.19970.75660.40960.2080
LLMTrace-ENQwen-CRF0.12850.85930.54640.3564
LLMTrace-RUMuST0.05120.95510.66440.5079
LLMTrace-RUQwen-FT0.21380.72100.38190.1256
LLMTrace-RUQwen-CRF0.14990.82080.44010.1984

The semantic-matching protocol controls reconstruction-stage prompt and generating-model choices by reusing fixed source pairs. Its five conditions simulate different extents of revision in human–AI collaboration. Section 6 identifies sentence-level construction as the current setting and finer-grained mixing within sentences as future work.

A2: Prior multilingual word-level detection and the contribution of MuST.

We thank the reviewer for bringing this relevant multilingual word-level detection work to our attention. We have corrected our original contribution statement to acknowledge this prior work and specify the joint-task contribution of MuST. The Introduction and Related Work now identify Kadiyala et al. (2025) as a multilingual word-level authorship-classification method. Their CRF-based model assigns binary token labels and maps them to words. Our novelty claim is scoped to explicitly coupling word-level binary authorship classification and sentence-junction directional boundary detection through shared contextual representations and joint supervision. MuST has a dedicated boundary head and supervises provenance, transition direction, and transition count. The contextual N-gram encoder provides shared information for the two heads. Tables 3 and 12 give the head and representation ablations, and the in-domain comparison below shows the performance difference under the same benchmark and metrics.

Table R1. MuST versus Qwen-CRF on MuST-Mix. The better value in each language and metric is bold. Overall assigns equal weight to the eight languages.

AuthorsOpenReview ↗

Response to Reviewer LmJJ (2/3)

LanguageMuST MAE ↓MuST Word F1 ↑MuST Type F1 ↑MuST Pos. F1 ↑Qwen-CRF MAE ↓Qwen-CRF Word F1 ↑Qwen-CRF Type F1 ↑Qwen-CRF Pos. F1 ↑
English0.01030.98720.88950.83860.05500.91460.62810.4699
Chinese0.00580.99280.91830.88030.05230.91620.58600.4054
French0.01020.98690.87270.81360.08320.86430.50510.2943
German0.01410.98000.81480.72940.09730.83160.48320.2602
Portuguese0.01130.98450.85020.78100.09410.84360.47300.2447
Russian0.01460.98080.81490.72840.12520.79220.43860.1924
Spanish0.01520.98100.83720.76250.10100.83950.47230.2510
Arabic0.00730.98900.89360.84420.10360.83250.53680.3310
Overall0.01110.98530.86140.79730.08900.85430.51540.3061

MuST improves Overall Pos. from 0.3061 to 0.7973, a difference of 49.12 percentage points. The partially generated samples described by Kadiyala et al. use continuation of a human prefix and a single human-to-AI boundary. MuST-Mix supports multiple replacement locations and up to six provenance transitions, providing more complex boundary configurations. Their corpus covers 23 languages, and ours covers eight. Section 6 identifies broader language coverage as future work. The requested Kadiyala dataset was unavailable to us at evaluation time under the authors' gated Hugging Face access settings. Direct evaluation on that dataset remains contingent on obtaining access. The completed comparisons are the CRF-based model comparison on MuST-Mix and the external evaluations in Table 7.

A3: Small component effects, multiple seeds, and training duration.

We thank the reviewer for suggesting repeated experiments to assess reliability across random seeds and training durations. We now report eight-language results for seeds 42, 43, and 44 and training durations of one to three epochs in Appendix C. The table below gives the Overall results from Tables 33 and 34. Across seeds, Pos. ranges from 0.7881 to 0.7973. Across epochs, it ranges from 0.7910 to 0.8091. These observations characterize the stability of the complete model.

ExperimentSettingMAE ↓Word F1 ↑Type F1 ↑Pos. F1 ↑
Seed420.01110.98530.86140.7973
Seed430.01110.98470.85540.7881
Seed440.01130.98470.85550.7883
Epoch10.01110.98530.86140.7973
Epoch20.01200.98450.85650.7910
Epoch30.01040.98680.86910.8091

The component ablations address a separate question, the contribution of each part in the reported configuration. All four input-cue ablations lower Overall Pos., with a 6.83-point drop for sentence embeddings and drops of 0.72, 0.77, and 0.81 points for N-gram, adjacent-word, and position embeddings. Table 12 of the revised manuscript also shows how these effects vary by language. For example, removing sentence embeddings lowers Chinese Pos. from 0.8803 to 0.6735, and removing N-gram embeddings lowers German Pos. from 0.7294 to 0.7132. The complete Overall ablations from Table 3 are provided below.

ConfigurationMAE ↓Word F1 ↑Type F1 ↑Pos. F1 ↑
Full MuST0.01110.98530.86140.7973
w/o contextual encoder0.02020.96620.83370.7538
w/o N-gram embedding0.01180.98450.85650.7901
w/o sentence embedding0.02670.95930.81550.7290
w/o adj. word embedding0.01180.98450.85620.7896
w/o position embedding0.01190.98440.85580.7892
w/o word head0.24440.56020.83330.7536
w/o boundary head0.01180.98450.84110.7693

Paired multi-seed repetitions of each ablation configuration would be needed to quantify training-seed uncertainty for the small component differences. The present repetitions support full-model stability, and the component table reports the observed ablation effects.

AuthorsOpenReview ↗

Response to Reviewer LmJJ (3/3)

A4: Type versus Pos. and the Qwen-FT boundary-evaluation correction.

We thank the reviewer for drawing attention to the unusually low Qwen-FT Pos. scores and for asking about the relationship between Type and Pos. These points led us to audit the evaluation implementations and provide a more detailed metric derivation. Pos. measures exact boundary locations, and Type averages the no-change and two directional F1 scores. Appendix A.1 derives their relation, including Type≤(2+Pos)/3\mathrm{Type} \le (2+\mathrm{Pos})/3. The ordering depends on class supports, the no-change score, and direction errors. For MuST, Type/Pos. is 0.8614/0.7973 on MuST-Mix and 0.5615/0.6433 on M4GT Task 3, illustrating both possible orderings.

We also examined the evaluation code for MuST and the other baselines; the audit identified this specific sentence-junction evaluation inconsistency only in Qwen-FT. The anomalously low original Qwen-FT Pos. arose from deriving boundaries directly from raw word-label changes. We corrected this evaluation to obtain sentence labels through majority voting over word predictions and then derive sentence-junction transitions. We have re-evaluated Qwen-FT using the corrected code and updated all affected Qwen-FT results throughout the main paper, appendix, and response tables. The correction affects Qwen-FT Pos. F1; its MAE, Word F1, and Type F1 retain their reported values. The results of MuST and the other baselines also retain their reported values. The table below illustrates the corrected Qwen-FT scores. MuST predicts its boundaries through its dedicated head, and both methods are evaluated at sentence junctions.

LanguageOriginal Qwen-FT Pos.Corrected Qwen-FT Pos.Qwen-FT TypeMuST Pos.
Overall0.16630.59830.72690.7973
English0.12470.68300.78090.8386
Chinese0.67360.67360.80850.8803

A5: Source-level splitting and repeated paragraphs.

We appreciate the reviewer highlighting the importance of preventing source overlap and repeated text across splits. Our split groups all conditions of a source document together by source ID. Keeping the five conditions together prevents versions of the same source from being assigned to different splits. The subsequent text audit identified seven repeated Chinese human paragraphs associated with different source groups. For the dataset release, we removed the affected lower-priority groups and their 34 reconstructed samples. Table 30 reports zero source overlap and zero repeated Chinese human paragraphs in the release, with train/validation/test sizes of 32,807/4,668/9,355 samples from 6,720/958/1,915 source groups. The release cleanup follows the reported evaluations. We reproduce the split-audit results from Table 30 of the revised manuscript here.

Source-document split audit. Each split cell reports reconstructed rows / source groups.

LanguageTrainValidationTestSource overlapText duplicates
English32,645 / 6,7194,681 / 9599,332 / 1,92100
German32,790 / 6,7164,682 / 9589,395 / 1,92200
Chinese32,807 / 6,7204,668 / 9589,355 / 1,91500
Reviewer_gmjJOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes MuST, a framework for multilingual fine-grained machine-generated text detection. It augments word representations with contextual information and jointly optimizes span-level provenance detection and boundary detection. The paper also introduces MuST-Mix, a dataset for multilingual human–AI hybrid text detection.

Critical strengths
  1. MuST-Mix is a valuable dataset with broad coverage, extending multilingual span-level detection to eight languages.

  2. The representation augmentation approach is an interesting aspect of the method.

Critical weaknesses
  1. The Introduction does not highlight the core thread of the method. The method enhances word embeddings to simultaneously improve boundary detection and span-level detection, but the Introduction is not organized around this thread. Contribution also emphasizes only the joint optimization of the two tasks and does not reflect the design of representation enhancement.

  2. The connection between the research motivation and the method design is unclear. The third paragraph points out that existing fine-grained detection lacks multilingual evaluation and that multilingual methods are mostly limited to document-level detection. However, the fourth paragraph directly introduces joint task optimization and representation enhancement without explaining how these designs address the above gap. “Individual words provide limited information” is insufficient to justify the need for “language-specific contextual information,” nor does it explain why fusing adjacent words, N-grams, sentences, and normalized positions helps multilingual detection.

  3. The data construction procedure in the main text is explained too briefly, and the connection between the two stages is unclear, which easily leads to ambiguity.

Questions for authors
  1. Correct “Appendix 8” at Line 132 to “Table 8”.

  2. Explicitly identify the results in Table 2 as in-domain evaluations in its caption.

  3. Explain the meaning of bold values in the caption of Table 2.

1 comment
AuthorsOpenReview ↗

Response to Reviewer gmjJ

Thank you for the detailed assessment of our motivation, representation design, and dataset construction, and for identifying issues in the references and captions. We have clarified the complementary roles of the two detection tasks, explained source-level pairing and dynamic sentence-group replacement, and corrected the reporting details. Added and substantively revised passages and table entries are highlighted in blue in the revised PDF, including all content from Appendix B onward. The responses below identify the presentation issues in the original manuscript and support the clarifications with the construction procedure and ablation results.

A1: Motivation and representation design.

We appreciate the reviewer's suggestion to clarify the motivation and representation design. We have rewritten the motivation around the complementary roles of word-level authorship classification and sentence-boundary detection, and explained how shared contextual representations support both. Figure 2 illustrates the complementarity, Section 4.2 describes the N-gram, adjacent-word, sentence, and positional cues, and the contribution statement now explicitly includes the shared encoder and joint supervision. Tables 3 and 12 substantiate the component contributions. For example, removing sentence embeddings lowers Chinese Pos. from 0.8803 to 0.6735, and the German cue ablations produce the following reductions.

LanguageRemoved cueFull Pos.Ablated Pos.Drop (percentage points)
Chinesesentence embedding0.88030.673520.68
GermanN-gram embedding0.72940.71321.62
Germanadj. word embedding0.72940.71251.69
Germanposition embedding0.72940.70981.96

A2: Paragraph pairing, sentence matching, and controlled replacement.

We thank the reviewer for requesting a clearer distinction between paragraph pairing and sentence-group matching. Section 3 now states that source-provided identifiers and corresponding paragraph or answer indices establish each human–AI pair before sentence-group alignment. Prompt-based generation varies with prompt design and the generating LLM. Reusing fixed source pairs controls additional reconstruction-stage variation from these choices. AI rewriting can expand or condense the sentences within a paragraph, so a monotonic dynamic program matches groups containing one to three sentences on each side. This gives nine size combinations and supports one-to-one, one-to-many, many-to-one, and many-to-many correspondence. Replacement counts one, two, or three human-side sentences and inserts the corresponding AI groups in the original order. The resulting conditions represent different extents of rewriting encountered across rounds of human–AI collaboration. Figure 1 illustrates the pipeline, Algorithm 2 in Appendix A.3 specifies the procedure, and Appendix B.1 supplies worked examples.

A3: References and captions.

We thank the reviewer for the helpful suggestions on references and captions. Following the reviewer's guidance, we have corrected the Section 3 statistics reference to Table 9 in Appendix A.4. The Table 2 caption identifies in-domain evaluation on MuST-Mix and explains that bold marks the lowest MAE or highest F1 in each column. The evaluation statement in the abstract and at the end of the Introduction now specifies 12 SOTA detectors in eight languages and robustness to unseen generators and domains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.