acceptodds
Under review as a conference paper at ICLR 2027

Agents Can Use Base Models to Evade AI Detection

Abstract

We show that coding agents equipped with a base language model can successfully assemble responses from its samples to evade detection. Prior "humanization" techniques rely on paraphrasing AI outputs over several iterations, which invariably results in semantic drift. In contrast, equipping coding agents to directly orchestrate the writing process by stitching text samples from a base model allows it to produce outputs that are coherent, task-specific and generally high quality. We find that Claude Opus 5 in a Claude Code harness effectively orchestrates a local 32B parameter OLMo-2 base LM and sacrifices little task accuracy across benchmarks spanning creative writing, factual grounding, health QA and instruction following, while using up to 90% base LM tokens. Responses constructed in this manner reduce the effectiveness of both post-hoc detectors (Pangram v4 detection rate drops from 77% to 24%) and soft watermarking applied a priori to the agent's generations (down to a simulated 10% detection at low FPR). While effective, this evasion requires a significantly larger number of input and output tokens from the agent, increasing the dollar cost per query by up to 27×. Overall, this work demonstrates the effectiveness of a new class of adversarial attacks against AI detection, and urges post-hoc detection providers to include base models in training.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Text Detection, Agents, Large Language Models, Long-form Generation, Watermarking

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_X8afOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents HALO, which uses a coding agent to assemble responses from unwatermarked base-model samples. Across four tasks, it lowers Pangram v4’s aggregate detection rate from 77% to 24% while largely preserving task performance, and estimates reduced soft-watermark detection.

Critical strengths
  • The paper studies an important and interesting problem.
  • The paper demonstrates a concrete way for a capable agent to retain control over a writing task while sourcing much of the final text from a base model. This is a relevant challenge for AI-text detection.
Critical weaknesses
  • The incremental contribution over prior work is unclear. Xu et al. (2026) established that base-model outputs often evade commercial detectors, while Frankentext (Pham et al., 2026) demonstrated that assembling text from external snippets can evade detection. HALO combines these ideas by having a capable agent solicit, select, and splice base-model continuations. This is a useful system demonstration, but the paper does not clearly establish which new mechanism accounts for its gains beyond the known properties of base-model text and snippet assembly. The comparison with HIP also changes the generation workflow and resource budget, making it difficult to isolate the benefit of HALO’s orchestration.
  • The attack has a substantial cost. Table 2 reports roughly 8–27 times the orchestrator API cost of the Agent baseline, excluding the compute needed to run the base model. This matters to the paper’s case for HALO as a broadly usable attack, even if a motivated attacker might accept the expense for selected targets.
Questions for authors
  • Please clarify the specific technical contribution beyond Xu et al.’s base-model observation and Frankentext’s assembly strategy. A controlled comparison with a capable agent that checks and repairs paraphrased outputs, under a comparable inference budget, would help determine whether HALO’s advantage comes from its generation strategy rather than additional agent effort and sampling.
  • Please also present the detection–quality trade-off alongside total cost, including base-model inference.
7 comments
AuthorsOpenReview ↗

Response to reviewer X8af

We thank the reviewer for their detailed feedback. We are glad they found our paper important and interesting, and that our approach presents a relevant challenge for AI-text detection.

Response to Critical weaknesses

  1. We respectfully disagree that our approach of combining the ideas presented by Xu et al and Pham et al is incremental, due to the following reasons: (i) our comparisons to either of their approaches show clear, large gains on the pareto-frontier of the quality vs evasion tradeoff; (ii) we present the novel finding that coding agents can creatively prompt base models for collecting texts to complete a varied set of writing tasks; and (iii) our paper goes beyond post-hoc detection, which is the focus of those prior works, to also devise a technique (via pointer tags) which evades watermarking; moreover, we present a novel analysis technique for determining the strength of watermarking even when the orchestrating agent provides no detection API (as is the case for most frontier models today). This analysis technique may be useful by itself for future work which attempts to study the robustness of watermarking.

    To further isolate the benefits of orchestration we further present a comparison to two additional baselines, both of which use the same orchestrator as HALO, but instead of the base LM sampling tool using: (i) a static pool of ~100K tokens of texts sampled from the base LM given just the task prompt, formatted in a zero-shot manner for the base LM; and (ii) a static pool of ~100K tokens of human text consisting of retrieved passages (again given the task prompt) from reddit WritingPrompts (for Mythos), C4 (for IFEval), and MedQuad (for HealthBench). For FACTS we use the provided context document itself as a human pool of text.

    The results below are computed on n=50 prompts for each benchmark at a copy target of p=0.90, using Opus 5.5 as the underlying model (since we found that Opus 5 now tends to refuse some orchestration requests).

MethodMythos Quality ↑Mythos Pangram ↓Mythos Cost ↓FACTS Quality ↑FACTS Pangram ↓FACTS Cost ↓IFEval Quality ↑IFEval Pangram ↓IFEval Cost ↓HealthBench Quality ↑HealthBench Pangram ↓HealthBench Cost ↓
Agent1.0000.997$0.280.7500.307$0.221.0000.971$0.230.6651.000$0.23
HALO0.8400.011$1.330.9600.005$0.550.9800.086$0.880.6070.301$2.30
HALO (w/ static base LM texts)0.7140.083$1.17———0.9800.316$0.930.5730.725$1.65
HALO (w/ static human pool)0.8200.000$1.450.9400.005$0.501.0000.139$1.020.6340.726$0.96

Observations:

  • Replacing the base LM sampling tool with a static pool of texts from the same base LM generally reduces the effectiveness of orchestration – the pangram AI scores are higher in each case, and quality is lower for Mythos and HealthBench
  • Replacing the base LM sampling tool with a static pool of human texts makes the orchestration harder for IFEval and HealthBench, where we find that the agent gives up before reaching the target copy rate – resulting in high fraction AI scores from Pangram.
  • Re: the reviewer’s comment “agent that checks and repairs paraphrased outputs”. The paper additionally reports a Agent+detector baseline where the agent is given access to Pangram API and allowed to edit its outputs until it gets a low AI score. We show that this method is not effective.
  • While orchestrating human texts is effective on Mythos and FACTS, we note that in general anything published after 2022 on the web has no guarantee of being human generated – and in fact, today, a large fraction of the text found in the wild is AI-generated itself [1]. In the experiments we resolve this by selecting human pools from pre-2022, but we note that in practice relying on genuine human writing online is becoming harder and harder. Base LM orchestration provides a more effective alternative.

[1] Russell, Jenna, et al. "How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text." arXiv preprint arXiv:2609.40295 (2026).

AuthorsOpenReview ↗

(cont) Response to reviewer X8af

  1. We completely agree that the cost of the attack is a deterrent for its adoption – but we would like to respectfully point out that a weakness of the attack is not necessarily a weakness of the paper itself. Our goal is to highlight a novel vulnerability, previously unreported in the literature – along with both its efficacy as well as the key limitation.

    To further check the dependence on cost, we construct 3 cost settings for HALO using Opus 5.5: (i) HALO (high) which uses the exact same setup as the paper; (ii) HALO (med) which reduces the total number of base LM calls available to the agent down to 8 from 30; and (iii) HALO (low) which additionally sets the thinking budget to medium (from xhigh). We compare these 3 variants at a copy target of p=0.90 to the baseline Agent using the same Opus 5.5 model, on a subset of n=50 prompts to restrict costs.

BenchmarkMetricAgentHALO (low)HALO (med)HALO (high)
MythosQuality (coherence) ↑1.0000.7800.7200.840
Pangram ↓0.9970.0030.0330.011
Cost / query ↓$0.28$0.54$0.94$1.33
FACTSQuality (factuality) ↑0.7500.9200.9400.960
Pangram ↓0.3070.0030.0080.005
Cost / query ↓$0.22$0.28$0.62$0.55
IFEvalQuality (strict acc.) ↑1.0001.0001.0001.000
Pangram ↓0.9710.2060.1180.086
Cost / query ↓$0.23$0.30$0.62$0.88
HealthBenchQuality (score) ↑0.6650.5820.6070.607
Pangram ↓1.0000.5560.3510.301
Cost / query ↓$0.23$0.73$1.73$2.30

We make a few observations here:

  • With Opus 5.5, the gap between the Agent approach and HALO approach is smaller, even at the highest attack cost setting, down to 2x-10x as compared to 8x-27x with Opus 5. This reduction is caused due to 2 reasons: (i) Opus 5.5 has a lower cost for cache reads, which form the bulk of the input tokens for HALO (since the base LM samples are read repeatedly from disk); and (ii) the baseline Agent consumes a higher number of tokens with the newer model.
  • Lower cost HALO attacks are less effective since they generally either reduce the quality score, or increase the Pangram fraction-AI score.
  • However, even at the lowest cost setting, which is 1x-3x of the Agent baseline, we still see meaningful evasion while preserving quality on 2 out of the 4 benchmarks.

Overall, we still believe that motivated attackers on subscription plans can easily consume these costs, especially when producing individual texts rather than analyzing data at scale.

Response to Questions

Please see the responses to the weaknesses. We note that the base LM sampling accounts for a small fraction of the orchestration cost. We generally observe the agent sample 1K-28K tokens from the base LM across the four benchmarks – at an estimated cost of \0.15/Mtokensusingserverlessinferenceonaplatformsuchastogether.ai(basedonsimilarsizedmodels),thistotalstoabout0.15/M tokens using serverless inference on a platform such as together.ai (based on similar sized models), this totals to about \\0.004 per query on the higher end. The orchestrator costs, on the other hand range from \0.88−0.88 - \\2.30.

Dear reviewer, we are approaching the end of the author-reviewer discussion period. We have attempted to address all the concerns pointed out in your review -- if there is any additional information we can provide before the deadline, please let us know!

Reviewer_ejgKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The authors bring attention to critical vulnerability in model AI detection methods. The paper provides a creative framework that splices responses from base LLMs while using frontier models/agentic framework for its cognitive (reasoning, instruction following, etc.) capabilities. The authors explain detailed benchmarks, evaluation setup and results which are easy to understand. An oversimplified analogy would be magazine clipping based ransom note creation from old movies.

Critical strengths
  • The paper is well structured and easy to follow.

  • The authors effectively separate a model's cognitive capabilities from its token generation process, highlighting that existing detection methods primarily concentrate on token generation patterns, such as statistical signals derived from post-hoc fine-tuning and watermarking.

  • HALO drastically reduces detection rate by commercial softwares (Pangram, 77% to 24%) and preserving output quality. It provides incremental improvements on anchor paper - Base Models Look Human To AI Detectors

Critical weaknesses
  • The authors successfully pointed out gaps in post-tuning and watermarking methods. However, HALO’s splicing is also expected to leave some statistical patterns (eg, transition and structural anomalies). A counter method (eg, custom classifier) to identify HALO authored text would have been a critical contribution of the paper.
  • The human-as-a-judge evaluation metrics would further establish the efficacy of the proposed method. HALO based text could be easy to identify for an expert human judge.
  • The output text primarily comes from open source base models which leads to two subproblems - 1. Post hoc tuning exercises are required for sensitive/strict domains to enable safety and factuality alignment. Pre-trained only base models lack such alignments. Therefore, use of HALO’s base model based text in such domains (eg, healthcare) would be subpar and easy to identify. 2. Modern AI systems use agentic tools to bring the latest facts in their responses. HALO’s efficacy to successfully integrate outputs from such tools instead of base model’s intrinsic memory will make it more practical. The paper lacks details on this evaluation aspect.
  • Paper uses Claude Code harness as orchestrator. Ablations with other frontier models, framework could have been helpful.
Questions for authors

Have authors tried fine tuning open source models (Llama, Qwen family) as orchestrators? If such an orchestrator can be built locally then cost concerns (27x) can be hugely mitigated making HALO more practical.

6 comments
AuthorsOpenReview ↗

Response to reviewer ejgK

We thank the reviewer for their detailed and informative feedback! We are glad that the reviewer found our paper well structured and easy to understand, our approach creative, and our results convincing.

Response to Critical Weaknesses

  1. We agree with the reviewer that HALO orchestrated text may itself carry statistical signals which may be easy for detectors to classify. In fact, our abstract and discussion both urge post-hoc detector providers to include base model texts and orchestrated texts in their training – one of our main goals for writing this paper was to point out this important vulnerability.

    While training a full-fledged detector to study its robustness and potential for adversarial training is beyond the scope of this paper (partly due to the expense of collecting a large set of HALO responses), below we report preliminary results on training classifiers to distinguish human vs Agent texts as well as human vs HALO texts (at p=0.90).

    For both classifiers we use a collection of 575 LitBench [1] stories which are sourced from the same Reddit collection as Mythos, since they come with accompanying human written outputs. We split these into 425 training, 75 validation and 75 test stories and train the two classifiers above. We train one set of classifiers using Roberta-Large and one using logistic regression over TF-IDF features computed from word unigrams, bigrams and character 3-5 grams. We report accuracy of both classifiers when test either on Agent texts or when tested on HALO texts.

Roberta-large

Train \ TestAgentHALO
Agent1.0000.767
HALO0.8800.880

LR + TF-IDF

Train \ TestAgentHALO
Agent0.9930.593
HALO0.9330.867

The diagonal entries above tell us how the classifier performs on the same AI texts that it was trained on, and the off-diagonal ones tell us how it generalizes to texts outside its training.

We find that accuracy on HALO texts is substantially lower – suggesting these are harder to detect. Moreover, the HALO trained classifier does at least as well on Agent texts as well – suggesting that it is picking up the same features for detecting Agent texts, but only weakly through only the 10% of tokens still generated by the Agent in HALO responses.

We note that these are small scale experiments and need further validation before we can conclude that HALO texts are fundamentally harder to detect – but the initial evidence does suggest so.

[1] Fein, Daniel, et al. "Litbench: A benchmark and dataset for reliable evaluation of creative writing." Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

AuthorsOpenReview ↗

(cont) Response to reviewer ejgK

  1. We conducted a human study by taking 18 prompts from the LitBench test set constructed above showing either the human, Agent or HALO responses to participants in a blind manner. The participants were recruited from the authors’ institutions and were not briefed about this project before performing the task. The generated / written stories are truncated to ~400 words and preprocessed to remove markdown formatting. The participants are asked to label the responses as one of “Definitely human” (1), “Probably human” (2), “Probably AI” (3) and “Definitely AI” (4). We report the results across the 54 annotations below:
SourceRatingsJudged AI (rating ≥ 3)Mean rating (1–4)
Human (LitBench, 2023)184/18 (22%)2.00
Agent (Opus 5.5)1813/18 (72%)2.72
HALO (Opus 5.5)185/18 (28%)2.11

We found that the annotators generally struggle at distinguishing human from AI stories, however, the detection is much higher for the Agent written ones than for the HALO written ones. The annotators also left comments with their predictions; for the HALO stories predicted as human, the reasons included: (i) informal conversational wording that an LLM wouldn’t use, such as “blah blah blah”; (ii) punctuation habits that they associate with humans, such as “--” and parentheses; and (iii) imperfect or clumsy grammar, or unfocused narratives. Overall, these remarks suggest that it is the presence of base LM texts in the HALO responses that causes readers to assume that they are human.

  1. We completely agree that base LM text is not useful for sensitive tasks by itself – this can be seen in our main results where the base LM baseline gets very low quality scores across the benchmarks.

    The main finding of our paper is that this does not pose a problem for capable agents – even though the base LM text in itself lacks the safety / alignment tuning required for practical usage, the agent is still able to prompt, select and stitch together snippets which meet the task objectives. Moreover, the resulting texts are not easy to identify – which can be additionally seen from our post-hoc trained classifier results above, and the human study above.

    We also agree with the reviewer that agents can use other tools for evasion as well. A detailed exploration of such alternative strategies is orthogonal to our contribution which analyzes the base LM tool as one effective choice. However, we further report two additional baselines below, both of which use the same orchestrator as HALO, but instead of the base LM sampling tool using: (i) a static pool of ~100K tokens of texts sampled from the base LM given just the task prompt, formatted in a zero-shot manner for the base LM; and (ii) a static pool of ~100K tokens of human text consisting of retrieved passages (again given the task prompt) from reddit WritingPrompts (for Mythos), C4 (for IFEval), and MedQuad (for HealthBench). For FACTS we use the provided context document itself as a human pool of text.

MethodMythos Quality ↑Mythos Pangram ↓Mythos Cost ↓FACTS Quality ↑FACTS Pangram ↓FACTS Cost ↓IFEval Quality ↑IFEval Pangram ↓IFEval Cost ↓HealthBench Quality ↑HealthBench Pangram ↓HealthBench Cost ↓
Agent1.0000.997$0.280.7500.307$0.221.0000.971$0.230.6651.000$0.23
HALO0.8400.011$1.330.9600.005$0.550.9800.086$0.880.6070.301$2.30
HALO (w/ static base LM texts)0.7140.083$1.17———0.9800.316$0.930.5730.725$1.65
HALO (w/ static human pool)0.8200.000$1.450.9400.005$0.501.0000.139$1.020.6340.726$0.96

We can see these alternatives, while effective on some benchmarks, generally lead to either higher AI score from Pangram, or lower quality scores. This highlights the effectiveness of specifically using the base LM as a sampling tool for orchestration.

AuthorsOpenReview ↗

(cont) Response to reviewer ejgK

  1. We conduct an additional analysis using Opus 5.5 as the orchestrating agent, instead of Opus 5 as in the current version of the paper. Our choice of Claude-based orchestrators is motivated partly by an AI for scientists grant from Anthropic, which enables performing the detailed experiment runs at a much lower cost than what API based pricing or other orchestrators will allow.
MethodOrchestratorMythos Quality ↑Mythos Pangram ↓Mythos Cost ↓FACTS Quality ↑FACTS Pangram ↓FACTS Cost ↓IFEval Quality ↑IFEval Pangram ↓IFEval Cost ↓HealthBench Quality ↑HealthBench Pangram ↓HealthBench Cost ↓
AgentOpus 51.0001.000$0.060.8100.189$0.100.9600.979$0.080.6751.000$0.14
AgentOpus 5.51.0000.997$0.280.7500.307$0.221.0000.971$0.230.6651.000$0.23
HALOOpus 50.8980.039$1.700.9000.023$1.090.9800.242$0.980.6020.655$3.79
HALOOpus 5.50.8400.011$1.330.9600.005$0.551.0000.086$0.880.6070.301$2.30

We find that Opus 5.5 generally leads to significantly higher evasion at a significantly lower cost, and with roughly the same quality. This agrees with our intuition that as even more capable agents are released, the attack becomes more effective.

Response to Questions

Fine-tuning open orchestrators is a compelling direction for future work. We suspect that effective orchestration will rely on the underlying agents being highly capable at tool use. Hence, fine-tuning is likely to require reasonable large-scale models (>30B) – for which we need substantial GPU resources that go beyond an academic budget.

Reviewer_z3UyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper introduces HALO, a framework in which a coding agent constructs responses by selecting and stitching text sampled from a base language model, with additional copying from the user prompt or context where permitted. The motivation is to retain the task competence of a strong instruction-following agent while reducing the statistical signatures associated with its directly generated text. The paper evaluates this approach on creative writing, factual grounding, instruction following, and health question answering. It reports substantial reductions in Pangram scores, with task-dependent quality degradation and considerably higher inference costs. It also estimates watermark detection performance using a proxy language model and analytical or simulation-based calculations.

Critical strengths

1.HALO provides a concrete implementation of this idea through agent-controlled sampling and explicit span pointers. The results extend beyond creative writing to tasks with factuality and mechanically verifiable instruction constraints. For example, Table 1 reports a reduction in the IFEval Pangram score from 0.965 to 0.232 while prompt-level strict accuracy changes from 0.960 to 0.940. This is a substantial empirical effect under the evaluated setting. 2.The copy-rate sweep, alternative base models, regeneration ablation, token-source analysis, and cost breakdown help characterize when the method works and where it struggles. The authors also report bootstrap confidence intervals in the appendix and explicitly acknowledge that their watermark results are estimates. The weaker results on HealthBench and substantial inference overhead are valuable qualifications to the main findings.

Critical weaknesses

1.The abstract describes a reduction in detection rate from 77% to 24%, but these figures correspond to the average Pangram fraction_AI scores in Table 1. This quantity is not the document-level detection rate: Table 11 separately reports AI-label rates of 76% and 19%, alongside Mixed-label rates. The interpretation therefore depends on the document-level decision rule. There are also unresolved reporting discrepancies. The Pangram scores for Agent and HALO differ between Tables 1 and 11, and Table 11’s caption describes 400 examples across all benchmarks even though the reported Frankentext comparison is restricted to Mythos. Table 8 additionally reports DIPPER instruction-level loose accuracy below strict accuracy, which is unexpected under the standard IFEval implementation [5]. The watermark analysis is informative, but remains a proxy-based estimate rather than an end-to-end experiment with a watermarked orchestrator. The sensitivity and WinMax analyses strengthen the discussion, but do not establish that the numerical detection rates transfer to an actual watermarked generation process. 2.Frankentext [1] already uses an LLM to assemble responses from externally sourced text, while HIP [2] establishes the detector blind spot for base-model outputs. HALO makes a useful combination and extension of these ideas. However, the comparison with Frankentext uses previously released Gemini-2.5-Pro outputs, whereas HALO uses Opus-5. Consequently, it does not isolate the effect of replacing human snippets with dynamically generated base-model samples. The FACTS results also have an important attribution limitation: Appendix B states that approximately 60% of sessions never invoke the base model. A context-copying baseline with the same pointer mechanism and copy objective is needed to identify the base model’s contribution on this benchmark. More generally, comparisons with matched orchestration and inference budgets would make the claimed quality–evasion advantage more convincing.

Questions for authors

1.Can you distinguish average fraction_AI, document-level AI detection, and AI-or-Mixed detection throughout the paper? Please explain the differences between Tables 1 and 11 and provide the actual sample counts and benchmark composition for each Table 11 row.

2.Why is DIPPER’s instruction-level loose score 0.333 while its strict score is 0.340 in Table 8? Were identical examples, denominators, and the standard verifier used?

5 comments
AuthorsOpenReview ↗

Response to reviewer z3Uy

We thank the reviewer for the detailed feedback. We are glad they found our implementation “concrete”, our empirical findings “substantial”, and our analysis and ablations useful.

We have uploaded a new draft of the paper which incorporates the fixes mentioned below (highlighted in blue).

Response to Critical Weaknesses

  1. We thank the reviewer for pointing out small discrepancies in our reporting of the Pangram predictions.

    • We agree that the abstract should say “fraction AI scores” rather than “detection rate”. We used “fraction AI score” as the main metric for reporting throughout the paper since: (i) it provides an single aggregate measure as opposed to the array of labels returned by the detection API; and (ii) it provides a more fine-grained comparison across different methods. As the reviewer notes, the Appendix complements this with the actual detection rate as returned by the API, which generally tends to agree with the pattern observed in fraction AI scores.

    • Re: the difference between Table 1 and 11, this actually stems from a small difference in how we average the per-document scores across the two tables. For Table 1, we excluded responses under 50 words (mostly IFEval) following Pangram’s unreliability on very short length texts. For Table 11, we mistakenly included these in the overall average – the per-document scores are identical across the two tables. We have fixed this in the updated draft.

    • Re: DIPPER IFEval loose vs strict scores – another great catch! We investigated this and found the issue to be due to a single example which asks for at least 6 “!” characters in the response. The evaluation script from google that we use for IFEVal, uses a letter-frequency check to evaluate the responses, and ends up replacing “!” with a random a-z character which leads to slight variation in the final aggregate results. Since the loose and strict checks are performed separately, this variation ended up putting the former above the latter. We have now made the fix and re-computed the IFEval scores throughout the paper (the differences are very minor).

    • Re: the watermark analysis, we agree that is a proxy-based estimate. Unfortunately, until one of the frontier labs releases a detection API for checking the generations from their models, it is impossible for us to compute the evasion rate against a watermarked orchestrator. We note that HALO is effective only given a highly capable agentic orchestrator – while some of the largest open-weight model releases may satisfy this constraint, they also require substantial GPU resources to run locally with a watermarking scheme in place. Hence, in this work we limit ourselves to commercial agents, which we believe are also the ones likely to be used by attackers. Having said that, if the reviewer has specific concerns about the validity of our proxy-based analysis, we’d be happy to address those.

AuthorsOpenReview ↗

(cont) Response to reviewer z3Uy

  1. The reviewer raises an important point about isolating the importance of base LM sampling in HALO from the orchestration and increased inference budget. Below we report a comparison to 2 new baselines, both of which use the same orchestrator as HALO, but instead of the base LM sampling tool using: (i) a static pool of ~100K tokens of texts sampled from the base LM given just the task prompt, formatted in a zero-shot manner for the base LM; and (ii) a static pool of ~100K tokens of human text consisting of retrieved passages (again given the task prompt) from reddit WritingPrompts (for Mythos), C4 (for IFEval), and MedQuad (for HealthBench). For FACTS we use the provided context document itself as a human pool of text.

    The results below are computed on n=50 prompts for each benchmark at a copy target of p=0.90, using Opus 5.5 as the underlying model (since we found that Opus 5 now tends to refuse some orchestration requests).

MethodMythos Quality ↑Mythos Pangram ↓Mythos Cost ↓FACTS Quality ↑FACTS Pangram ↓FACTS Cost ↓IFEval Quality ↑IFEval Pangram ↓IFEval Cost ↓HealthBench Quality ↑HealthBench Pangram ↓HealthBench Cost ↓
Agent1.0000.997$0.280.7500.307$0.221.0000.971$0.230.6651.000$0.23
HALO0.8400.011$1.330.9600.005$0.550.9800.086$0.880.6070.301$2.30
HALO (w/ static base LM texts)0.7140.083$1.17———0.9800.316$0.930.5730.725$1.65
HALO (w/ static human pool)0.8200.000$1.450.9400.005$0.501.0000.139$1.020.6340.726$0.96

Observations:

  • Replacing the base LM sampling tool with a static pool of texts from the same base LM generally reduces the effectiveness of orchestration – the pangram AI scores are higher in each case, and quality is lower for Mythos and HealthBench
  • Replacing the base LM sampling tool with a static pool of human texts makes the orchestration harder for IFEval and HealthBench, where we find that the agent gives up before reaching the target copy rate – resulting in high fraction AI scores from Pangram.
  • The reviewer is correct that on FACTS there is no benefit from the sampling tool – copying from the human context alone is sufficient.
  • While orchestrating human texts is effective on Mythos and FACTS, we note that in general anything published after 2022 on the web has no guarantee of being human generated – and in fact, today, a large fraction of the text found in the wild is AI-generated itself [1]. In the experiments we resolve this by selecting human pools from pre-2022, but we note that in practice relying on genuine human writing online is becoming harder and harder. Base LM orchestration provides a more effective alternative.

[1] Russell, Jenna, et al. "How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text." arXiv preprint arXiv:2609.40295 (2026).

Response to Questions

  1. We have uploaded a revised draft which clearly indicates where we are talking about fraction-AI scores versus detection rates, and also resolves the discrepancy between Tables 1 and 11.
  2. Please see the response above.

Dear reviewer, we are approaching the end of the author-reviewer discussion period. We have attempted to address all the concerns pointed out in your review -- if there is any additional information we can provide before the deadline, please let us know!

Reviewer_cR5BOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces an attack, called HALO, that uses a capable agent to prompt a base language model and assemble selected generated passages into a response that evades AI-text detectors. Based on the recent findings saying that base-model outputs are more similar to human text, the method aims to retain the agent’s task performance while borrowing most of the final wording. Experiments across four benchmarks show substantial reductions in detection with varying effects on output quality, at a considerably higher generation cost. The paper also uses a proxy-based analysis to estimate how copying unwatermarked text weakens watermark detection.

Critical strengths

I enjoyed reading the paper. I liked how the authors build on a recent finding in the literature and turn it into an evasion attack against AI-text detectors. They evaluate their attack against Pangram, a strong commercial detector. The experiments and ablations support the main claim that the attack can substantially reduce detection while largely preserving output quality, although the quality trade-off varies across tasks.

Critical weaknesses

The main weakness I see, which the authors also acknowledge, is the cost of the attack. The reported agent API cost is roughly 8-27 times higher than the baseline. Although the cost per response may be acceptable for motivated attackers, this overhead limits the practicality of using the attack at scale. An experiment showing how quality and evasion change under smaller attack budgets would strengthen the paper.

Questions for authors

Have you tried extending HALO to more complex, long-horizon agentic tasks? If so, how do task performance, detector evasion, and cost compare with the tasks evaluated here?

3 comments
AuthorsOpenReview ↗

Response to reviewer cR5B

We thank the reviewer for their detailed feedback. We are glad that they enjoyed reading the paper, and that they found our methods and evaluations convincing.

Response to Critical Weakness

We completely agree that the cost of the attack is a deterrent for its adoption – but we would like to respectfully point out that a weakness of the attack is not necessarily a weakness of the paper itself. Our goal is to highlight a novel vulnerability, previously unreported in the literature – along with both its efficacy as well as the key limitation.

Having said that, we took the reviewer’s suggestion and ran a comparison across attack budgets. This analysis was performed using Claude Opus 5.5 as the orchestrator, since we found that the original Opus 5 model used in the paper’s experiments now refuses some orchestration requests, presumably due to safety updates applied by Anthropic. Opus 5.5 does not refuse any such requests.

We construct 3 cost settings for HALO using Opus 5.5: (i) HALO (high) which uses the exact same setup as the paper; (ii) HALO (med) which reduces the total number of base LM calls available to the agent down to 8 from 30; and (iii) HALO (low) which additionally sets the thinking budget to medium (from xhigh). We compare these 3 variants at a copy target of p=0.90 to the baseline Agent using the same Opus 5.5 model, on a subset of n=50 prompts to restrict costs.

BenchmarkMetricAgentHALO (low)HALO (med)HALO (high)
MythosQuality (coherence) ↑1.0000.7800.7200.840
Pangram ↓0.9970.0030.0330.011
Cost / query ↓$0.28$0.54$0.94$1.33
FACTSQuality (factuality) ↑0.7500.9200.9400.960
Pangram ↓0.3070.0030.0080.005
Cost / query ↓$0.22$0.28$0.62$0.55
IFEvalQuality (strict acc.) ↑1.0001.0001.0001.000
Pangram ↓0.9710.2060.1180.086
Cost / query ↓$0.23$0.30$0.62$0.88
HealthBenchQuality (score) ↑0.6650.5820.6070.607
Pangram ↓1.0000.5560.3510.301
Cost / query ↓$0.23$0.73$1.73$2.30

We make a few observations here:

  • With Opus 5.5, the gap between the Agent approach and HALO approach is smaller, even at the highest attack cost setting, down to 2x-10x as compared to 8x-27x with Opus 5. This reduction is caused due to 2 reasons: (i) Opus 5.5 has a lower cost for cache reads, which form the bulk of the input tokens for HALO (since the base LM samples are read repeatedly from disk); and (ii) the baseline Agent consumes a higher number of tokens with the newer model.
  • Lower cost HALO attacks are less effective since they generally either reduce the quality score, or increase the Pangram fraction-AI score.
  • However, even at the lowest cost setting, which is 1x-3x of the Agent baseline, we still see meaningful evasion while preserving quality on 2 out of the 4 benchmarks.

Overall, we still believe that motivated attackers on subscription plans can easily consume these costs, especially when producing individual texts rather than analyzing data at scale.

AuthorsOpenReview ↗

(cont) Response to reviewer cR5B

Response to Question

This is an excellent suggestion – however, we would like to point out that HALO is applicable only to tasks where the final output is long-form text. Such tasks are rather rare among LLM benchmarks – possibly because reliably evaluating long-form writing outputs remains an open, unsolved problem.

Writing tasks which do have objective evaluation criteria are generally limited to variants of question answering – and we included two representative ones in our evaluations (FACTS and HealthBench). If the reviewer has recommendations on long-form generation tasks which require long-horizon agentic reasoning, we would be happy to incorporate them in our revisions.

Dear reviewer, we are approaching the end of the author-reviewer discussion period. We have attempted to address all the concerns pointed out in your review -- if there is any additional information we can provide before the deadline, please let us know!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.