acceptodds
Under review as a conference paper at ICLR 2027

Token-Level Model Blending and AI-Generated Text Detection: Separating Model Switching from the Generation Loop

Abstract

Mixing several language models within one passage is a proposed way to evade detectors of AI-generated text. Token-level blending, which draws a model at random for each short chunk, lowers zero-shot detection scores, yet it also changes the generation procedure, leaving it unclear whether detectors are vulnerable to model mixing at all. We separate the two by running every pool member alone through the identical procedure. Encouragingly for detection, switching itself adds no measurable evasion: four older open models run alone are as hard to detect as their blends (differences of at most 0.0071). When blends and single models are generated together, switching even raises detectability, by 0.0411 to 0.1049 for four newer models and 0.0271 to 0.0655 for four instruction-tuned models. At the same time, the chunked procedure itself creates a large gap: on news articles, Fast-DetectGPT AUROC is 0.7088 for the older models' blends and 0.9283 for their standard text. For the newer models, switching lowers detectability only relative to single models regenerated separately with the original procedure. We examine these patterns across three English domains and several detector families. The results suggest that model switching alone is not a route around current detectors and that evasion claims need same-pipeline single-model baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI-generated text detection, multi-model generation, token-level blending, generation loop, detector robustness, Fast-DetectGPT, Binoculars, evaluation methodology

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tPBdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether the apparent detectability reduction of token-level multi-model generation arises from switching between models or from the accompanying generation procedure. The authors introduce pool-of-one controls, in which each model independently generates through the same chunked loop, and compare an original NLTK-re-spaced loop with a native-string loop. Experiments cover three English domains, classic and advanced model pools, an additional instruction-tuned pool, and multiple detector configurations. Under the primary GPT-Neo-scored Fast-DetectGPT setting, the classic pool shows no measurable switching advantage relative to same-loop controls. In the matched native loop, switching increases detectability for the advanced and instruction-tuned pools. Additional analyses examine score inversion, formatting, paraphrasing, quality, and computational cost. The principal contribution is a controlled evaluation showing that apparent multi-model evasion cannot automatically be attributed to model switching.

Critical strengths
  1. The pool-of-one control is consequential and directly addresses the principal confound. Running each constituent model through the same generation loop is the appropriate control for asking whether switching itself changes detectability. This design separates switching from effects introduced by chunking, repeated tokenization, and string handling. Under the primary Fast-DetectGPT configuration, the control changes the interpretation from an apparent blending advantage to a near-zero effect in the classic pool and an increase in detectability for the advanced native-loop pool. This is a substantive result that supports acceptance as a measurement study.

  2. The evaluation explicitly investigates score inversion and sensitivity to analysis choices. The paper recognizes that AUROC below 0.5 can indicate reversed separation rather than disappearance of information. It considers held-out direction selection, reports realized false-positive rates, and evaluates multiple scorers and formatting treatments. These diagnostics help prevent score inversion or formatting artifacts from being mischaracterized as general detector failure. Although the notation needs correction, the underlying attention to these evaluation issues is valuable.

Critical weaknesses
  1. The original-loop attribution is not fully identified, and the broad concluding claim exceeds the cross-configuration evidence. The historical original-loop blends were generated in fp32, while the corresponding pool-of-one controls were regenerated later using half precision and batched sampling. The authors acknowledge this difference, but disclosure does not make the conditions equivalent. Consequently, the reversal between the original and native loops cannot be attributed exclusively to string handling without separating loop effects from execution-configuration effects. The broader suggestion that switching alone is not a route around current detectors also needs qualification. For example, Table 7 reports a native-loop advanced-SQuAD difference of −0.0312, with a reported 95% interval of [−0.0615, −0.0014], under the OLMo-2-7B Fast-DetectGPT scorer. This does not overturn the primary GPT-Neo and Binoculars findings, but it demonstrates that not all evaluated configurations follow the headline direction. The supported conclusion is that switching provides no consistent, configuration-independent evasion advantage, rather than that switching never reduces detectability in the evaluated setting.

  2. The detectability statistic and the effective comparator need clearer definitions. The manuscript repeatedly combines the definition D(A)=max⁡(A,1−A)D(A)=\max(A,1-A) with a procedure that fits score direction on 20% of prompts and evaluates the remaining 80%. These are different statistics. The former is necessarily at least 0.5; the latter can fall below 0.5 because of direction-selection error or sampling variation. Table 7 includes a value of 0.4975 labeled as detectability for native classic XSum under OLMo-2-7B. This value need not be erroneous, but it cannot literally equal max⁡(A,1−A)\max(A,1-A) evaluated on that test set. The implementation, estimand, and notation should be reconciled.

Questions for authors
  1. Clarification of experimental matching. Are there results in which original-loop blends and pool-of-one controls were generated together using the same precision, software environment, sampling configuration, and matched seed protocol? If not, which conclusions are supported solely by the fully matched native-loop experiments, and which depend on the historical-versus-regenerated comparison?
  2. Clarification of the primary statistic. Do Tables 4, 5, and 7 report D(A)D(A) or held-out oriented AUROC? Please explain the values below 0.5 and provide a pooled pool-of-one comparison with a common score direction. Does this change the estimated switching effects or the advanced-versus-classic contrast?
  3. Clarification of null effects and conclusion scope. For the classic pool, what practically meaningful reduction can the confidence intervals exclude? Please distinguish failure to detect a difference from evidence of approximate equivalence. How should the overall conclusion be revised to incorporate the native-loop reductions observed under alternative scorers?
2 comments
AuthorsOpenReview ↗

Response to Reviewer tPBd (1/2)

Thank you for the careful reading and for regarding the evidence from the pool-of-one control as a substantive result that supports acceptance as a measurement study. This part answers Weakness 1 and Question 1; Part 2 answers Weakness 2 and Questions 2 and 3.

Weakness 1: The supported conclusion is that switching provides no consistent, configuration-independent evasion advantage, rather than that switching never reduces detectability in the evaluated setting.

We adopt this formulation for the abstract and conclusion, restricted to ToBlend's uniform per-chunk scheme. On the original loop, the historical blends and the regenerated pool-of-one arms differ in execution configuration beyond precision (software, seeding, batching, and possibly the Llama-2 sampling configuration, unrecorded in 2024 and set to temperature 0.6 and top-p 0.9 in the regenerated arms), so the revision will associate the original-loop reversal with the loop and its execution configuration rather than with string handling alone. We will also run the Llama-2 pool-of-one arms at temperature 1.0 in both loops, which separates its sampling configuration from the loop effect, and we will include those experiment results in the camera-ready version. In the matched native loop, switching lowers detectability with the interval excluding zero in 6 of the 48 native-loop cells of Table 7, all under scorers or detectors other than GPT-Neo (largest −0.0397), in Table 16 (−0.0310), and under the Hello-SimpleAI formatting diagnostic in all three instruction-tuned cells (Table 11; −0.0532 to −0.1846), while it raises detectability in 17 of these 48 cells. With a pooled pool-of-one reference under the held-out estimand, two of the six Table 7 exceptions no longer hold (OLMo-2-7B advanced SQuAD, 0.0864 [0.0600, 0.1125]; Hello-SimpleAI advanced XSum, 0.0030 [−0.0297, 0.0336]), and the Table 16 SQuAD cell becomes a reduction under the full-sample estimands (−0.0230 [−0.0446, −0.0030]); the revision will list the exceptions under each estimand next to the conclusion.

Question 1: Are there results in which original-loop blends and pool-of-one controls were generated together using the same precision, software environment, sampling configuration, and matched seed protocol? If not, which conclusions are supported solely by the fully matched native-loop experiments, and which depend on the historical-versus-regenerated comparison?

Not in the submission. We will generate original-loop blends and pool-of-one arms of both pools together under one configuration, and we will include those experiment results in the camera-ready version. The Limitations section will carry the following map. Supported by the matched native loop: the classic null under GPT-Neo (0.0016 to 0.0050); the increases from switching in the advanced (0.0411 to 0.1049) and instruction-tuned (0.0271 to 0.0655) pools; RADAR scoring native blends and same-loop single-model text alike (differences of at most 0.0098); seed stability (Table 10); the redraw sweep (Table 22); and the classic pool without GPT-Neo (Table 16). Dependent on the historical-versus-regenerated comparison: the original-loop advanced reductions, the primary RQ3 contrast (0.0965), and the original-loop half of the classic null.

AuthorsOpenReview ↗

Response to Reviewer tPBd (2/2)

Weakness 2: The detectability statistic and the effective comparator need clearer definitions.

Question 2: Do Tables 4, 5, and 7 report D(A)D(A) or held-out oriented AUROC? Please explain the values below 0.5 and provide a pooled pool-of-one comparison with a common score direction.

Question 2: Does this change the estimated switching effects or the advanced-versus-classic contrast?

They report the held-out oriented AUROC: the direction is fitted on 20% of prompts and the AUROC computed on the other 80%, so it can fall below 0.5. The 0.4975 cell is such a case: its full-sample AUROC is 0.5042 (Table 23) with the declared direction, a sampling effect near chance rather than a direction error. The only other value below 0.5 in Table 7, 0.4591 for the GPT-2 detector on the original-loop advanced SQuAD blend, also reflects direction selection near chance: its full-sample AUROC in the calibrated direction is 0.5221 (Table 3), and the direction fitted on the 20% split is the reverse one. The revision will give this statistic its own name and reserve D(A) = max(A, 1 − A) for the full-sample value, reporting both. The pool-of-one comparator in Tables 4, 5, and 7 is the mean of the four arms' held-out values, each arm with its own fitted direction (the P1 columns of Table 7), and the revision will define it next to the statistic. Under the GPT-Neo scorer, the blend and all four pool-of-one arms share one orientation in every native-loop cell, and the p = 0 rung of the redraw sweep is a pooled single-model reference (Table 22); orientations differ within 6 of the 48 zero-shot cells (four scorers, two loops, two pools, three domains). We computed pooled common-direction comparisons with intervals for every cell and for the RQ3 contrast. Under the GPT-Neo scorer, the classic null and the advanced native-loop increases keep their sign and significance under every estimand, and the RQ3 primary contrast stays positive (0.0548 [0.0306, 0.0788] with a pooled reference; 0.1050 [0.0859, 0.1246] with max(A, 1 − A) per arm). The original-loop advanced XSum reduction becomes 0.0283 [−0.0218, 0.0740] with a pooled reference, and two of the six native-loop exceptions do not hold with a pooled reference, as listed in Part 1. The revision will report every estimand beside the printed one.

Question 3: For the classic pool, what practically meaningful reduction can the confidence intervals exclude?

Under the GPT-Neo scorer, the classic-pool intervals exclude switching reductions larger than 0.0276 in the original loop and 0.0168 in the native loop (Table 4), whereas the 2024 classic blends have 0.0624 to 0.2190 lower Fast-DetectGPT AUROC than data-builder single-model text (Table 13). Relative to that drop, the intervals indicate approximate equivalence rather than only a failure to detect a difference. A two-one-sided equivalence test at a margin of 0.03, below the smallest of these reductions, supports equivalence in all six GPT-Neo classic cells (bootstrap TOST p ≤ 0.047), also with a pooled reference; the 90% intervals exclude effects larger than 0.0177 to 0.0297 in the original loop and 0.0123 to 0.0158 in the native loop. We set the margin after seeing the intervals, so these excluded margins are the primary readout. Under the outside-pool scorers and the supervised detectors, the excluded margins are wider in several classic cells (up to 0.0691 for Fast-DetectGPT with the Falcon-7B scorer), consistent with the listed exceptions. The revised conclusion will read: under the primary scorer, ToBlend's uniform switching adds no measurable evasion in the classic pool and raises detectability in the matched native loop for the advanced and instruction-tuned pools; across scorers and detectors, it provides no consistent, configuration-independent evasion advantage, with the listed exceptions.

Reviewer_pHtQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The work reviews an old paper, ToBlend, a training-free attack on AI-text detectors. It builds a passage by repeatedly sampling one model from a pool of four and appending a few tokens of that model's output.

Critical strengths
  • This paper was easy to read and understand. The motivations stated by the authors were pretty clear and commendable. The pool-of-one design was also pretty interesting.
Critical weaknesses
  • Do the RQ2 results hold against a pooled single-model reference with one shared orientation, as in p=0p = 0?
  • It does seem strange that GPT-Neo is used as the scorer while also being one of the blended models.
  • Most robust detectors have emerged, I suggest the authors include 2-3 more detectors as baselines for comparisons.
  • Does the advanced-pool sign reversal survive in RQ3 using the original runs too?
Questions for authors

None, most have been listed in Weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer pHtQ

Thank you for finding the paper easy to read and its motivation clear, and for your interest in the pool-of-one design. Re-examining ToBlend matters beyond that method: in the classic pool, the reduction relative to standard single-model text comes from the generation loop, not from switching (Section 5), so evasion claims for multi-source text need single-model baselines run through the same pipeline.

Weaknesses, bullet 1: Do the RQ2 results hold against a pooled single-model reference with one shared orientation, as in p=0p = 0?

Yes, for the matched native loop. The p = 0 rung of the redraw sweep is such a reference: one model per passage, drawn from the pool, with one shared orientation (Table 22). Against it, under the primary scorer, any switching raises advanced-pool AUROC (XSum 0.7649 [0.7038, 0.8257] at p = 0 against 0.8907 [0.8729, 0.9083] at p = 1; Figure 4), while the classic-pool intervals overlap at every p. Under this scorer, the blend and all four pool-of-one arms also share one orientation in every native-loop cell. In the original loop, they share one orientation in every classic cell and in advanced WritingPrompts, but point in different directions in advanced XSum and SQuAD. We computed the pooled common-direction comparison for every scorer in both loops. Under the primary scorer, it leaves the classic null and the advanced native-loop increases unchanged and keeps the original-loop advanced SQuAD and WritingPrompts reductions (−0.1477 [−0.1870, −0.1111] and −0.0418 [−0.0642, −0.0209]). Advanced XSum becomes 0.0283 [−0.0218, 0.0740] with a pooled reference in its own fitted direction and −0.2054 [−0.2352, −0.1749] with the direction shared with the blend, so that one cell depends on the choice of reference; the revision will report every estimand.

Weaknesses, bullet 2: It does seem strange that GPT-Neo is used as the scorer while also being one of the blended models.

We keep GPT-Neo-2.7B as the fixed scorer for all arms to remain comparable with the audited raw-text tables of Appendix B (Tables 26 and 27); it belongs to the classic pool only. Three controls address the overlap: a single-model mean that excludes GPT-Neo (Section 3), a classic pool without GPT-Neo (Table 16), and two Fast-DetectGPT scorers from outside both pools plus Binoculars with the Falcon-7B pair (Tables 5 and 7). Removing GPT-Neo from the pool leaves a small native-loop reduction on XSum (−0.0310 [−0.0518, −0.0121]) and, under the full-sample estimands (signed AUROC and max(A, 1 − A)), on SQuAD (−0.0230 [−0.0446, −0.0030]); our response to Reviewer tPBd lists these among the exceptions.

Weaknesses, bullet 3: Most robust detectors have emerged, I suggest the authors include 2-3 more detectors as baselines for comparisons.

The study evaluates likelihood, rank, log-rank, and entropy statistics, DetectGPT, DetectLLM, Fast-DetectGPT with three scorers, Binoculars, RADAR, and two RoBERTa-based detectors (Section 3). We will add recent detectors that the paper discusses but does not run, such as DNA-GPT [1] and Raidar [2], on the matched native-loop corpora, and we will include those experiment results in the camera-ready version.

Weaknesses, bullet 4: Does the advanced-pool sign reversal survive in RQ3 using the original runs too?

Within the original runs, the advanced pool's larger reduction holds for GPT-Neo (0.0965 [0.0751, 0.1186]) and Falcon-7B (0.0905 [0.0676, 0.1141]) (Table 5) and stays positive for both under every reference estimand we computed (0.0548 to 0.1328 for GPT-Neo), whereas under OLMo-2-7B its sign depends on the estimand (−0.0169 [−0.0336, −0.0001] in Table 5, −0.0917 [−0.1078, −0.0749] with a pooled reference, and 0.1123 [0.0980, 0.1263] in the declared direction). Read instead as the reversal of the advanced switching effect between loops, it holds for GPT-Neo, Falcon-7B, and Binoculars in all three domains under the paper's estimand (Table 7); for GPT-Neo with a pooled reference, it holds in SQuAD and WritingPrompts but not in XSum. Either way, the size and sign of the advanced-pool effect depend on the scorer and the loop, which is the RQ3 conclusion.

References

[1] Yang, X., Cheng, W., Wu, Y., Petzold, L. R., Wang, W. Y., & Chen, H. (2024). DNA-GPT: Divergent N-gram analysis for training-free detection of GPT-generated text. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=Xlayxj2fWp

[2] Mao, C., Vondrick, C., Wang, H., & Yang, J. (2024). Raidar: Generative AI detection via rewriting. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=bQWE2UqXmf

Reviewer_GfFfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies ToBlend, a blended multi-model generation attack against detectors. The paper conducts numerous experiments to answer several research questions: the effectiveness of attack, the effect of the loop itself with retokenization versus model switching, the difference in effect between older and newer models, and the quality and time costs.

Critical strengths

There are many experiments.

Critical weaknesses

The paper is very hard to read. The positioning within the literature is unclear.

Questions for authors

Questions

My current score is 1 (reject). Addressing all major concerns (A1–A3) is required for me to consider raising it to 2 (borderline reject).

A. Major concerns

A1. Readability and structure

The paper reads as a sequence of experimental results with scattered discussion points, and it is very difficult for a human reader to follow. Technical results are rarely accompanied by paragraphs that give intuition or summarize what was learned, and the sectioning does not guide the reader to the findings.

(a) Section 3 mixes the method with run-level detail. Section 3 ("Method and Evaluation") places the procedure (Algorithm 1) alongside notebook seeding, precision and sampling defaults, an unrecorded 2024 Llama-2 generation configuration, and a scorer substitution made because a model "was unavailable on the server". It also reports results: the AUROC of the regenerated TL4 corpus and the supervised detectors' calibration AUROCs. The "Metrics and uncertainty" paragraph specifies roughly six interval procedures and two split protocols in a single paragraph.

(b) Terminology is overloaded and introduced piecemeal.

  • "Single-model" refers to at least: data-builder text (§3); pool-of-one text in two loops ("single models regenerated separately" in the abstract, "single-model text from the same loop" in §5); the p = 0 rung of the redraw sweep (§5, Table 22); self-scored white-box references (§3, Table 25, and the "Legacy ref." column of Tables 52–54); a 100-prompt pilot control (Tables 33, 50); and pilot arms in App. K.1 and K.6. Captions re-define the term locally (Tables 1 and 3), and in Table 9 the "Single mean" column switches between pool-of-one and data-builder text across row blocks.
  • Corpus labels encode undeclared axes and switch between them. "2024" first appears in "the 2024 battery tables" (p. 3), before the closest thing to a definition ("The 2024 blends come from the original loop"). The paper never states that "2024" denotes the generation year, or why provenance is the distinguishing axis. The same corpus is also called "historical" (App. B) and "battery", and the "Blends" column of Table 5 contrasts "2024" (a date) with "Native" (a procedure).
  • Several labels are undefined or point back to themselves. "FDG" heads the columns of Table 4, and I could not find it expanded anywhere. Table 24 reports decision outcomes as undefined codes ("a", "b", "c", "seed-variation-small", "measurement-title") and refers to an undefined "round-2 rule G3".

(c) Sections titled as research questions do not answer them. Section 1 poses RQ1–RQ4 explicitly and Sections 4–7 carry those titles, but the sections do not tie the results together into centralized conclusions. Section 8 partly synthesizes RQ1–RQ3, but a reader working through Sections 4–7 gets no answer at the point where each question is examined.

(d) The purpose of the text views is not stated where they are introduced. Section 3 defines V0, V1, V1-S, V1-T, and V2 without saying why they are needed. The reason (original-loop outputs are stored NLTK-spaced with leaked markers, and some detectors respond to that formatting) emerges only from §4 and Appendices C–D.

Criterion for score raising. In a revised version:

  1. Separate the method definition (ToBlend, loop variants, reference arms) from run-level implementation details, and move the latter, together with any results, out of Section 3.
  2. Define every reference arm and corpus label once, in a single place (e.g., a terminology table), with descriptive names, and use them consistently in text, tables, and captions. Define or remove undefined codes, and fix self-referential pointers.
  3. End each RQ section with an explicit takeaway paragraph that answers its question.
  4. Add one or two sentences where the text views are introduced, explaining why they are needed.

A2. Contribution and positioning

As I read it, the paper's main finding is that the detection gap of the original blends comes from the generation loop rather than from model switching: "The classic-pool reduction in Table 1 therefore comes from the chunked, re-tokenized loop, not from switching between models" (§5), and switching "adds no measurable evasion" (abstract). I do not question the soundness of this finding. I question its utility within the literature as the paper is currently framed.

(a) Provenance and target of the result. ToBlend is introduced without a citation (§§1–3) and is not listed among contributions (i)–(iv). The motivating claim, that mixing several language models within one passage "is a proposed way to evade detectors" (abstract), is not attributed to any prior work. Related Work cites human–machine mixing (Zhang et al., 2024) and collaborative decoding (Shen et al., 2024), neither of which makes that claim. The negative result therefore has no cited target in the literature.

(b) Which question is primary, and why does it matter? The paper points to three different main results. The title and abstract lead with the switching-versus-loop question (RQ2). The first results table (Table 1) answers RQ1.

(c) Generality of "blending". The general conclusions ("model switching alone is not a route around current detectors", abstract; §8) rest on a single blending scheme: uniform random selection of one model per chunk, with a decoded string handed to the next model. The paper itself names alternatives it does not test: a learned router, a quality objective, and mixing of next-token distributions (§2, "Multi-model decoding"). Varying the chunk size (TL1–TL5, random, sentence-level) and the redraw probability changes the granularity of switching, not its mechanism.

(d) The effective factor is not studied. By the paper's own arms, string handling, rather than switching, carries the effect. The original loop "re-encodes the whole NLTK-re-spaced text, decodes the new span with special tokens kept, … so every model conditions on re-spaced text" (§3), while the native loop "changes only the string handling". Single models run alone through the original loop reach Fast-DetectGPT AUROCs as low as 0.6085 (OPT-2.7B, XSum; §5, Table 8). Yet the original-versus-native contrast is never reported or interpreted as such, and the paper is framed around switching.

Questions for the authors.

  1. Is ToBlend proposed in this submission? If not, please cite its source; if so, why is it absent from the contributions?
  2. Which prior claim does the negative result on model switching correct? Please cite it.
  3. What is the single most important research question this paper answers, and why does its answer matter for detection research?
  4. Why are conclusions stated for model blending or switching in general when only one blending scheme is evaluated?
  5. If string handling is the effective factor, why is it not studied as an attack in its own right?

Additional Requests.

  1. For (c), either evaluate at least one additional blending scheme with a different mechanism (e.g., one of the alternatives named in §2), or restrict all claims about "blending" and "model switching" to ToBlend's uniform per-chunk scheme throughout the title, abstract, and conclusions.
  2. For (d), report the original-versus-native pool-of-one contrast explicitly, and either study string handling as an attack (for example, a single model run through the original and native loops, evaluated on cleaned output) or give a substantive argument for framing the paper around switching instead.

A3. Generation cost and KV caching

As written, the generation loop re-tokenizes and re-encodes the full text at every step: Algorithm 1 tokenizes x with the drawn model's tokenizer on each iteration, and §3 states that "each step re-encodes the whole NLTK-re-spaced text". At TL1 this amounts to about 140–150 steps per passage, each generating k + 3 = 4 model tokens to retain one NLTK token. The paper does not say whether KV caches are reused across steps or maintained per model across switches. Without reuse, every step re-processes the full context, and the attention cost of each step grows quadratically with the context length, whereas standard single-model generation decodes incrementally from a cache. KV caching is not mentioned anywhere in the paper.

The paper claims generation-time costs as a contribution (iv) and poses RQ4, but it reports only logged absolute per-passage times, which it describes as "not controlled hardware benchmarks" (App. I; Table 51). It gives no comparison with single-model generation.

Are KV caches reused across steps and across model switches? If so, how are they maintained for models with different tokenizers?

Additionally, please report generation time for ToBlend against standard single-model generation with KV caching on the same hardware and passage lengths, and report the KV-cache memory footprint if caches are maintained per model. (Optionally, also compare against a pool of one run through the same loop, to separate the cost of the loop from the cost of switching.)

B. Minor concerns

B1. Formal statement of ToBlend and its variants

ToBlend is described across §1 (one sentence), the caption of Figure 1, §3, and Algorithm 1. Algorithm 1 is the only formal statement, and it covers only the buffered original loop with fixed or random k. Variants reported under the same name are described only in prose: the native loop (a single sentence in §3), sentence-level generation, the redraw-probability sweep, and the unbuffered protocol (Figure 1 caption; App. J). For the native loop in particular, the handoff between models is unclear. The paper states that it "changes only the string handling" and "appends the prefix of the model's own decoded string that ends at the k-th NLTK token", which implies, without stating it, that each model still re-tokenizes the full text at every step.

How does the native loop operate across models with different tokenizers, and is the full text re-tokenized by each model at every step?

Can the authors extend Algorithm 1, or add a second algorithm, so that the native-loop, sentence-level, unbuffered, and redraw-probability variants are each stated formally, including the re-tokenization step?

3 comments
AuthorsOpenReview ↗

Response to Reviewer GfFf (1/2)

Thank you for the detailed review and for noting that the soundness of the main finding is not in question. This part answers major concern A1 and questions 1 and 2 of A2; Part 2 answers questions 3 to 5 of A2, major concern A3, and minor concern B1.

Major concern A1: The paper reads as a sequence of experimental results with scattered discussion points, and it is very difficult for a human reader to follow.

We adopt all four score-raising criteria: (1) a method-only Section 3 with one primary interval procedure, and run details moved to an appendix that also states that OLMo-2-7B was the pre-declared fallback scorer; (2) a terminology table with descriptive names used consistently, such as "original-loop pool-of-one", "fixed-model rung (p = 0)", and "Fast-DetectGPT" for "FDG"; (3) a takeaway paragraph ending each RQ section; and (4) a stated purpose for the text views, which separate formatting effects from effects of the generation process (formatting human text alone yields RADAR AUROC of 0.846, 0.724, and 0.603; Table 29).

Major concern A1(b): Several labels are undefined or point back to themselves.

The terminology table reserves "single-model text" for data-builder text, gives the other senses their own names (pool-of-one in each loop, fixed-model rung, self-scored reference, pilot control), drops caption-level redefinitions, and splits Table 9's "Single mean" column by reference type. Corpus labels will name the procedure, "original-loop blends" and "native-loop blends", in place of "2024", "historical", and "battery", including Table 5's "Blends" column; the year remains only as provenance, because it marks the fp32 generation that the later half-precision pool-of-one arms do not share (Section 3). Table 24 will state each rule fixed before the runs and its outcome in words, "round-2 rule G3" will be named as the cleaned-text rule, and self-referential pointers, such as "Appendix I gives medians and maxima by setting" inside Appendix I, will name their target (here Table 51).

Major concern A2, question 1: Is ToBlend proposed in this submission? If not, please cite its source; if so, why is it absent from the contributions?

Major concern A2, question 2: Which prior claim does the negative result on model switching correct? Please cite it.

ToBlend is not proposed in this submission: it was introduced by Huang et al. (2024) [1] as a training-free attack, and this submission re-examines it with matched single-model controls. The revised version will cite [1] where ToBlend is first mentioned and in the abstract. The negative result targets two of that work's central claims: that ToBlend degrades detection by exploiting detectors' assumption of one generator per passage, and that more advanced candidate models disrupt detection more [1]. RQ2 tests the first with the generation loop held fixed, and RQ3 the second. DetectRL similarly reports that mixing sentences generated by different LLMs does not confuse detectors [2].

References

[1] Huang, F., Kwak, H., & An, J. (2024). ToBlend: Token-level blending with an ensemble of LLMs to attack AI-generated text detection (arXiv:2402.11167). arXiv. https://doi.org/10.48550/arXiv.2402.11167

[2] Wu, J., Zhan, R., Wong, D. F., Yang, S., Yang, X., Yuan, Y., & Chao, L. S. (2024). DetectRL: Benchmarking LLM-generated text detection in real-world scenarios. In Advances in Neural Information Processing Systems 37 (Datasets and Benchmarks Track, pp. 100369–100401). Curran Associates. https://doi.org/10.52202/079017-3186

AuthorsOpenReview ↗

Response to Reviewer GfFf (2/2)

Major concern A2, question 3: What is the single most important research question this paper answers, and why does its answer matter for detection research?

RQ2: with the generation loop held fixed, does model switching itself lower detectability? A positive answer would mean that detectors' single-generator assumption can be exploited by mixing alone. For the tested scheme and the primary scorer, the answer is no: switching adds no measurable evasion in the classic pool and raises detectability in the matched native loop for the advanced and instruction-tuned pools, while the original loop lowers detectability even for single models. Under the primary scorer, the advanced blends are less detectable only against original-loop pool-of-one text regenerated separately (−0.0418 to −0.1721; Table 4), a comparison matched in code but not in time (Section 3). RQ1 comes first because it establishes the detection gap that RQ2 then attributes to the loop or to switching. The revision will name RQ2 as the primary question in the introduction and close with its consequence for detection research: evasion claims for multi-source text need single-model baselines run through the same generation pipeline.

Major concern A2, question 4: Why are conclusions stated for model blending or switching in general when only one blending scheme is evaluated?

We will restrict every claim to ToBlend's uniform per-chunk scheme in the title, abstract, and conclusions, and evaluate one scheme with a different mechanism, mixing next-token distributions among the classic members that share a tokenizer; we will include those experiment results in the camera-ready version.

Major concern A2, question 5: If string handling is the effective factor, why is it not studied as an attack in its own right?

Table 8 reports every pool-of-one arm in both loops and Table 7 their means (for example, OPT-2.7B on XSum: 0.6085 in the original loop, 0.8857 in the native loop). With intervals, the native-minus-original contrast for single models under the primary scorer is 0.2068 [0.1894, 0.2249], 0.1320 [0.1119, 0.1517], and 0.0487 [0.0384, 0.0584] in the classic pool and 0.1036 [0.0800, 0.1274], 0.1697 [0.1490, 0.1906], and 0.0898 [0.0774, 0.1027] in the advanced pool, so the original loop's string handling lowers detectability in every cell. Under the held-out estimand, the direction holds under Falcon-7B, Binoculars, and the GPT-2 detector, is mixed under OLMo-2-7B, and reverses under RADAR and the Hello-SimpleAI formatting diagnostic; in the declared direction, it holds for every scorer and detector except RADAR. As an attack, it is therefore detector-dependent. The revision will present this loop effect as a finding in its own right, and we will rescore the single-model original-loop texts after cleaning (V2) and include those experiment results in the camera-ready version.

Major concern A3: Are KV caches reused across steps and across model switches? If so, how are they maintained for models with different tokenizers?

No. As in the original ToBlend procedure, every step calls the drawn model's generation function afresh on the whole current string, re-tokenized by that model's tokenizer; a cache exists only within that call, and models exchange only the decoded string. Logged batched runs on the same A100 80GB cards, though not a controlled benchmark, already separate loop cost from switching cost: cached single-model generation took 0.0729–0.3529 s per passage, single models run through the loop 0.8310–3.7610 s, and blends 1.2307–2.9756 s, so the cost comes from the loop, not from switching. We will state the caching behavior in Section 3 and generalize Algorithm 1 so that the native-loop, sentence-level, unbuffered, and redraw-probability variants are stated formally, including re-tokenization. Because no cache persists across steps or switches, there is no per-model cache footprint to report. We will also add a controlled benchmark on one GPU comparing cached single-model generation, a pool of one in the loop, and blends at matched lengths, with peak GPU memory per arm, and we will include those experiment results in the camera-ready version.

Minor concern B1: How does the native loop operate across models with different tokenizers, and is the full text re-tokenized by each model at every step?

Yes, in both loops each drawn model re-tokenizes the whole current string at every step. In the native loop, the model generates k + 3 tokens, decodes them as its own text, and appends the prefix of this decoded string that ends at the k-th NLTK token (Section 3; Figure 1), so the decoded string is the only handoff between tokenizers. The generalized Algorithm 1 will state this step.

Reviewer_GfFfOpenReview ↗

Response to Rebuttal

I thank the authors for their clarifications. However, I now find the paper's contribution to be insubstantial. To my understanding, the paper is a re-examination of the prior attack method ToBlend, showing that its reported evasion against zero-shot detectors is an artifact of its own string handling rather than of model switching. Honestly, I think this finding is a good starting point for a research project, but presenting the finding on its own hardly suffices for a conference paper. Therefore I maintain my score and urge the authors to keep exploring in this direction and present constructive attack methods in a hypothetical future submission.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to the AC and All Reviewers

We thank the reviewers for their detailed and constructive reviews. Reviewer tPBd regards the evidence from the pool-of-one control as "a substantive result that supports acceptance as a measurement study"; Reviewer pHtQ finds the paper easy to read with a clear motivation; and Reviewer GfFf states that the soundness of the main finding is not in question.

ToBlend was introduced by Huang et al. (2024) [1] as a training-free attack; this submission re-examines it, and the revised version will cite [1] where ToBlend is first mentioned and in the abstract. The paper tests two of that work's central claims; its primary question, RQ2, is whether switching itself lowers detectability once the generation loop is fixed (GfFf). The revision adopts the formulation that switching provides no consistent, configuration-independent evasion advantage, restricts all claims to ToBlend's uniform per-chunk scheme, and lists every matched native-loop configuration in which switching lowers detectability (GfFf, tPBd). Under the GPT-Neo scorer, the classic-pool intervals exclude switching reductions larger than 0.0276 (original loop) and 0.0168 (native loop), against AUROC reductions of 0.0624 to 0.2190 for the 2024 classic blends relative to data-builder text (tPBd). No KV cache is reused across steps, and logged timings show that the loop, not switching, accounts for the cost (GfFf).

We have computed the pooled common-direction comparisons, the equivalence test, and the original-versus-native contrast with intervals (pHtQ, tPBd, GfFf). We will add a matched original-loop run, Llama-2 arms at temperature 1.0, cleaned-output rescoring, a next-token mixing scheme, a controlled timing benchmark, and recent detectors such as DNA-GPT and Raidar. We will include those experiment results in the camera-ready version, together with the readability revisions requested by Reviewer GfFf.

References

[1] Huang, F., Kwak, H., & An, J. (2024). ToBlend: Token-level blending with an ensemble of LLMs to attack AI-generated text detection (arXiv:2402.11167). arXiv. https://doi.org/10.48550/arXiv.2402.11167

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.