acceptodds
Under review as a conference paper at ICLR 2027

Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration

Abstract

Large language models (LLMs) can generate fluent and convincing text at scale, creating growing risks for misinformation dissemination, educational misuse, and platform governance. These concerns make robust detection of machine-generated text increasingly necessary. Recent zero-shot detectors mainly exploit probability-based statistical discrepancies, but they do not explicitly account for the training process of LLMs, which leaves a distinct generation mechanism insufficiently modeled and limits detection robustness. To address this issue, we propose EchoPrompt, a training-free detector based on latent prompt restoration. Our key intuition is that machine-generated text is typically produced conditioned on an upstream prompt, and this hidden dependency can be partially reactivated by prepending a unified generic prefix. Specifically, EchoPrompt restores a generic assistant-response context, measures the induced likelihood gain with an instruction-tuned model, calibrates it against the corresponding base model, and aggregates the resulting differences into a score that quantifies latent prompt dependency. Extensive experiments show that EchoPrompt achieves state-of-the-art performance among zero-shot detectors while maintaining strong robustness across challenging evaluation settings.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM-generated text detection, training-free detection, zero-shot detection, large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UiKbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes EchoPrompt, a training-free zero-shot detector for LLM-generated text. Based on the hypothesis that machine text implicitly retains dependency on upstream instructions, EchoPrompt prepends a generic assistant prefix (cgc_g) and scores text by comparing the likelihood gap between an instruction-tuned model and a base model: log⁡Pinst(X∣cg)−log⁡Pbase(X)\log P_{\text{inst}}(X \mid c_g) - \log P_{\text{base}}(X). The method demonstrates performance gains across multiple benchmarks, particularly in the low-FPR regime (TPR@1%FPR).

Critical strengths
  1. Clear Writing and Good Presentation: The paper is well-written, well-organized, and easy to follow. The overall methodology and workflow are clearly illustrated with high-quality figures (e.g., Figure 1).
  2. Comprehensive Experimental Evaluation: The authors conduct extensive experiments spanning multiple public benchmarks, diverse proxy model architectures/scales, and challenging adversarial attack settings, providing thorough empirical results.
  3. Practical Low-FPR Performance: The improvement in macro-average TPR@1%FPR from 73.11% (IRM) to 81.86% is practically meaningful for real-world scenarios where false accusations must be strictly bounded.
Critical weaknesses
  1. Incremental Technical Novelty over IRM: Mathematically, the proposed score is essentially the existing IRM baseline plus a prefix-induced conditional probability shift: . Framing the addition of a hardcoded generic prompt as a new paradigm of "Latent Prompt Restoration" leaves a noticeable gap between the conceptual pitch and the actual technical delta.
  2. Generality of Prefix Selection and the "Restoration" Hypothesis: The core claim that performance gains stem from "latent prompt restoration" requires scrutiny. In Appendix Table 4, candidate prefixes exhibit considerable AUROC variance across prompt families, while random noise yields 0.7992. If the mechanism truly reactivated an intrinsic prompt-response dependency, semantically related assistant prompts should yield stable improvements. Since many candidate variants fail to outperform IRM, it is difficult to confirm a universal "restoration" phenomenon rather than empirical distribution fitting via a dev-set tuned template.
  3. Unexplained Positive Gain on Human Text for Qwen Models: In Section 3.3 (Figure 2 and Table 7), human text exhibits negative gains under Llama proxies as expected by the authors' hypothesis. However, on Qwen2.5 (1.5B and 3B Multi-Domain), human text also obtains positive likelihood gains after prefix injection. If the prefix selectively reactivates machine-specific prompt dependencies, why does human text benefit similarly under Qwen? This contradiction suggests the gain may arise from general prefix priming or attention smoothing rather than true prompt restoration.
  4. Domain and Task Boundaries: AUROC drops markedly to 82.50% on RAID non-chat/completion data. This indicates the method heavily relies on text conforming to conversational assistant conventions and struggles to generalize to open continuation, code, or creative writing.
Questions for authors
  1. In the candidate prefix evaluation (Appendix Table 4), what are the exact performance figures for the unprompted baseline (IRM)? How many prefix candidates actually beat the unprompted baseline?
  2. Under the "latent prompt restoration" hypothesis, how do the authors explain the positive context gains observed on human-written text for Qwen base models?
  3. What is the actual classification F1 performance across benchmarks if evaluated with a fixed decision threshold pre-calibrated on validation data rather than an oracle post-hoc Best-F1?
  4. Experimental Suggestion for Mechanistic Verification: If the authors consider "latent prompt restoration" to be the primary mechanistic contribution of this work, I suggest conducting a more direct mechanistic verification experiment. An intuitive idea would be: on benchmarks where upstream generation prompts are available, evaluate the detector using the original ground-truth prompt for each text. In theory, the true prompt should serve as the oracle upper bound for prompt restoration. If using the ground-truth prompt does not yield a noticeably larger likelihood gain or better class separability than the generic prefix (or even underperforms it), it would be difficult to attribute the empirical improvements to the "restoration of latent prompt dependency," as opposed to generic stylistic steering from an assistant template. Could the authors provide experimental results or discussion on this intuitive check?
1 comment

W1: The technical novelty over IRM is limited.

EchoPrompt's contribution is to expose a detection signal not explicitly measured by unconditioned scoring: machine text retains a measurable dependency on an assistant–response context even when its original prompt is unavailable. We restore this shared context in the instruction-tuned proxy and use the base model to calibrate the likelihood. Sec. 3.3 establishes this contextual response; Fig. 3 identifies the prompt–response clause as the strongest individual component; and Table 3 validates the calibrated score (0.9864 AUROC / 0.9519 Best-F1). The full prefix improves AUROC by 5.33–14.73 points. Across five main settings, EchoPrompt improves Best-F1 by 2.64 points and TPR@1%FPR from 0.7311 to 0.8186. These results demonstrate the value of our training-free detector.

W2 & Q1: Prefix generality and its performance relative to IRM are unclear.

On the independent HC3 development set, IRM achieves 0.6737 AUROC; all 17 prefix candidates exceed it (0.7019–0.9003). We tested the same candidates and one fixed EchoPrompt prefix using three proxy pairs across DetectRL, RealDet, and RAID, without dataset-specific tuning. EchoPrompt exceeds IRM in all nine settings. The same prefix is used throughout, rather than selected separately for each proxy or dataset. Six comparisons against an unprompted score are shown below:

ProxyDatasetNo PrefixEchoPrompt
Qwen2.5-3BDetectRL0.59750.7631
Qwen2.5-3BRealDet0.49200.6331
Qwen2.5-3BRAID0.56280.6588
Llama-3.1-8BDetectRL0.90940.9490
Llama-3.1-8BRealDet0.82610.8830
Llama-3.1-8BRAID0.79490.8512

W3 & Q2: Positive human-text gains under Qwen challenge the hypothesis.

Positive human gain does not contradict our hypothesis: machine text should gain more. Table 7 reports Qwen2.5-1.5B human/machine gains of 0.0209/0.0486 and 0.0040/0.0461; for Qwen2.5-3B, 0.0056/0.0377 and −0.0062/0.0362 (Multi-Domain/Multi-LLM). The machine-minus-human differences are +0.0277, +0.0421, +0.0321, and +0.0424, consistently supporting the predicted relative effect.

W4: Performance on RAID non-chat raises generalization concerns.

DetectRL, RealDet, and RAID contain diverse non-dialogue texts; EchoPrompt does not require conversational formatting. It achieves 0.9998 AUROC on Yelp continuation, 0.9696 on few-shot human-style writing, 0.9960 on ICO/SICO, and 0.9846 on back-translation. On RAID Non-chat, EchoPrompt leads at 0.8249, versus IRM (0.7948), Binoculars (0.7966), Fast-DetectGPT (0.7972), and DNA-DetectLLM (0.7562). This subset challenges other detectors too, while EchoPrompt remains strongest by AUROC.

Q3: Fixed-threshold F1 performance is not reported.

We calibrated method-specific thresholds using 795 examples per class from DetectRL Multi-Domain and transferred them unchanged to five evaluations. We removed exact text overlaps with calibration data and used no test labels to select thresholds.

SettingEchoPrompt F1IRM F1
Multi-Domain95.18%93.97%
Multi-LLM95.31%93.75%
Multi-Attack95.18%93.55%
RealDet87.82%80.60%
RAID84.49%81.07%
Macro Average91.60%88.59%

EchoPrompt leads in every setting, with a 3.01-point macro-average F1 gain.

Q4: The original generation prompt is proposed as an oracle upper bound.

The original prompt is not an upper bound on detection AUROC. The detection proxy can also differ from the actual generator, so the source prompt need not maximize class separation under that proxy. In RAID, machine passages are generated from prompts linked to human reference texts. We apply each original prompt to both classes. Machine-minus-human mean likelihood gains are +0.0958, +0.0687, +0.0257, and +0.0418 across four settings, confirming stronger machine-side contextual responses. However, increased likelihood need not improve class separation:

RAID / ProxyEchoPromptOriginal Prompt
Chat / Llama-30.95580.9588
Chat / Qwen2.50.79080.6945
Non-chat / Llama-30.81550.7869
Non-chat / Qwen2.50.58220.5215

The original prompt is worse in three settings and comparable in one. Because it is also relevant to the human source, it can raise both classes' likelihoods without increasing AUROC. EchoPrompt instead restores a shared response context that improves detection.

Reviewer_mLqtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EchoPrompt, an AI-generated text detection method that does not require training. By adding a general assistant prompt before the text to be detected, and comparing the likelihood score of the text of the instruction fine-tuning model and the base model, it uses the dependence of the machine-generated text on the implicit prompt context to distinguish between AI text and human text.

Critical strengths
  1. A simple and effective new detection signal is proposed. The method does not require training a specialized classifier, nor does it need to know the original prompt used to generate the text. The paper also provides empirical evidence supporting this assumption through prompt ablation experiments and contextual likelihood gain experiments with the same model.
Critical weaknesses
  1. Prior work, such as DPIC (DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection, NeurIPS 2024), has emphasized the coupling between generated prompts and intrinsic model features in AI-generated text detection. Authors should better articulate how their concept of potential prompt dependency differs from this established view.
  2. The paper does not adequately explain the specific construction and selection process of the final prefix. Experimental support indicates that the assistant-style prefix can improve detection performance, but it is not yet sufficient to prove the uniqueness or optimality of the specific prefix chosen.
Questions for authors
  1. The author should design experiments to compare the effects of the original prompt, a semantically similar prompt, a general assistant prefix, style-matching prefix without a prompt-response relationship, and an irrelevant prefix on likelihood gain and detection performance, in order to verify the core mechanism of the explanation.
  2. How was the wording of this final prompt determined? Did more unreported candidate prompts get tried? Does the final prompt have a significant advantage over other semantically similar prompts?
2 comments

W1: The relationship between latent prompt dependency and DPIC is unclear.

The two methods address different questions. DPIC investigates the coupling between generation prompts and intrinsic model characteristics. It reconstructs a content-specific prompt for each passage, generates reference text, and uses a supervised classifier for detection. EchoPrompt instead asks whether text retains a detectable response-context dependency when its original prompt is unavailable. It uses one fixed assistant–response prefix for every passage, scores the conditioned text likelihood, and calibrates it with a base model—without prompt reconstruction, reference-text generation, or classifier training.

The difference is also reflected in the results. EchoPrompt achieves 0.9556 mean AUROC / 0.9198 mean Best-F1 across the five main settings. We additionally evaluated DPIC on matched test samples:

DatasetEchoPrompt AUROCDPIC AUROC
DetectRL Multi-Domain0.98640.7959
RAID0.88900.8152

Thus, EchoPrompt uses a different detection signal and pipeline while outperforming DPIC by 19.05 and 7.38 AUROC percentage points in these two matched evaluations.

W2 & Q1 & Q2: The prefix selection and its supporting evidence need clarification.

Our goal is not to find one optimal sentence, but to establish whether a generic response context provides a transferable detection signal. Sec. 3.2.1 explains the prefix design: the role sentence specifies an assistant, Clause A establishes that the text responds to a user request, and Clause B covers continuations, articles, and question answering. Fig. 3 ablates these components: Clause A is the strongest individual component in all four settings, while the full prefix improves AUROC over the empty prefix by 5.33–14.73 percentage points. The final prefix is kept fixed across datasets and proxies.

To test sensitivity to wording, we evaluated the default prefix and three predefined semantic paraphrases across three proxies and three datasets. Their macro-average AUROCs over the nine settings are 0.8387 (default) and 0.8490/0.8072/0.8428 (paraphrases), versus 0.7769 without a prefix. All three alternative wordings outperform the no-prefix setting; the effect does not depend on one exact sentence.

We also compared the original generation prompt, the generic response prefix, length-matched style-only prefixes, a role-only prefix, and irrelevant prefixes. On Qwen2.5/RAID Chat, the respective AUROCs are 0.7908 (EchoPrompt), 0.6945 (original prompt), 0.7856/0.7978 (style-only), 0.6758 (role-only), and 0.5647/0.5278 (irrelevant travel/cooking). Across both proxies and both RAID subsets, EchoPrompt outperforms the two irrelevant prefixes. These results show that the useful detection signal transfers across response-prefix wordings and does not require recovering each sample's exact original prompt.

Thank the authors for their clarification and the detailed responses. My main concerns are addressed. I will continue to maintain my borderline but positive evaluation, for they achieved better detection performance with a easily understandable and implementable methods.

Reviewer_AVs7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EchoPrompt, a training-free detector for machine-generated text based on latent prompt dependency. It restores the generation context using a fixed assistant-style prefix and computes detection scores by comparing the likelihoods of an instruction-tuned model and its corresponding base model. They evaluate EchoPrompt on three public benchmarks, DetectRL, RealDet, and RAID, and analyze its performance across different proxy models, text lengths, and attack settings.

Critical strengths
  • This paper addresses a practical problem.
  • The proposed method is easy to follow and well-motivated. EchoPrompt uses a fixed assistant-style prefix to restore the generation context and calibrates likelihood scores using a base model, without requiring additional detector training.
  • The low FPR shows a substantial and consistent gain.
Critical weaknesses
  • The latent prompt dependency hypothesis is not adequately validated. Appendix B confirms that machine text gains more from the assistant-style prefix, but the same gap would appear if machine text were merely more typical. For instance, typical sequences benefit more from any added context. And nothing in the design rules this in or out. The scoring function has a related problem: the EchoPrompt score blends a context-induced gain with a base/instruct cross-model difference, and readers are never told how much each contributes.

  • Applicability outside instruction-tuned generation is narrow. As the results show, on RAID's non-chat generators AUROC falls to 82.50, well below the ~95 average in the main table. It is not a minor regression. It suggests the signal is tied to assistant-style post-training behavior rather than to machine generation as such. The authors frame this as consistent with their hypothesis, which it is, but it also bounds the contribution, and this paper does not say how large the affected slice is.

  • The robustness evaluation under non-standard attack settings lacks comparisons. The authors evaluate continuation, human-style generation, back-translation, and non-chat generation, but Table 9 only reports results for EchoPrompt. Although the method performs well in some settings. It is difficult to determine whether EchoPrompt offers better robustness compared with other detectors.

  • Novelty is somewhat limited. EchoPrompt is IRM with a prefix glued on. IRM scores a passage by how much an instruction-tuned model prefers it over its own base model. EchoPrompt does the same thing, except it puts a fixed line about being a helpful assistant in front of the text first.

Questions for authors
  • How can latent prompt dependency be distinguished from the model's preference for certain text distributions?

  • How do the authors distinguish latent prompt dependency from a model's inherent preference for particular text distributions, given that both could produce similar likelihood shifts?

3 comments

W1 & Q1: Text typicality may explain the observed likelihood gains.

Text typicality alone cannot explain EchoPrompt's signal. Sec. 3.3 compares the same text under the same base model with and without response context, rather than raw likelihoods. Fig. 3 isolates the prefix components. Across four Qwen2.5-3B/Llama-3.1-8B settings, prompt–response Clause A is the strongest individual component. Over the empty prefix, it improves AUROC by 12.63, 9.28, 3.84, and 3.41 points, more than the role sentence alone.

On RAID, we apply each original generation prompt to paired machine and human texts. Across Chat/Non-chat × Llama-3/Qwen2.5, the mean human/machine likelihood gains are 0.0304/0.1262, 0.0743/0.1429, 0.1091/0.1348, and 0.0842/0.1260, respectively. Machine text shows a larger gain in all four settings. On Qwen2.5/RAID Chat, EchoPrompt achieves 0.7908 AUROC, compared with 0.5647/0.5278 for irrelevant travel/cooking prefixes. These results show distinct human–machine responses to generation contexts, beyond text typicality alone.

W1 & Q2: The contributions of contextual gain and cross-model differences are unclear.

Sec. 3.2.2 and Table 3 examine both components. The response prefix conditions the instruction-tuned model on a generation context, while the base model provides an unprompted likelihood reference to calibrate fluency and predictability shared by human and machine text. On DetectRL Multi-Domain, EchoPrompt achieves 0.9864 AUROC / 0.9519 Best-F1, compared with 0.9683/0.9162 when both models receive the prefix and 0.9194/0.8416 for the reversed difference of their contextual gains. The instruction model's contextual gain alone yields only 0.5147 AUROC. Thus, response conditioning and base-model calibration form a jointly validated scoring design, rather than two unexplained additions.

W2: Applicability beyond instruction-tuned generation is limited.

No. Our benchmarks do not require explicit assistant personas or dialogue formatting. DetectRL includes machine-generated texts across domains; RealDet covers text from varied sources; and RAID includes summaries, news, books, poetry, recipes, and reviews. These texts are not all conversational assistant responses. EchoPrompt achieves 0.9556 mean AUROC across the five main settings on these three benchmarks. On Yelp continuation, few-shot human-style writing, ICO/SICO, and back-translation, its AUROC is 0.9998, 0.9696, 0.9960, and 0.9846, respectively, covering continuation, human-style generation, and rewritten text.

The drop on RAID Non-chat is also not specific to EchoPrompt. On the same 1,000 human–machine pairs, the AUROCs of EchoPrompt, IRM, Binoculars, Fast-DetectGPT, and DNA-DetectLLM are 0.8249, 0.7948, 0.7966, 0.7972, and 0.7562, respectively. EchoPrompt achieves the highest AUROC. This subset is challenging for multiple detectors and does not establish that EchoPrompt depends on overt conversational style.

W3: Table 9 lacks baseline comparisons.

We add four baselines on the same samples (AUROC):

SettingEchoPromptIRMBinocularsFast-DetectGPTDNA-DetectLLM
Yelp continuation0.99980.98840.99140.99170.9888
Few-shot0.96960.97680.96050.95210.9627
ICO/SICO0.99600.98010.95660.95350.9542
Back-translation0.98460.99060.80700.81340.7163
RAID non-chat0.82490.79480.79660.79720.7562

EchoPrompt leads in three of five settings and outperforms Binoculars, Fast-DetectGPT, and DNA-DetectLLM in all five.

W4: The technical novelty over IRM is limited.

The fixed prefix restores response context; our contribution is identifying, validating, and exploiting machine text's latent dependency on it. When the original generation prompt is unavailable, EchoPrompt evaluates text under a shared assistant–response context and calibrates its conditional likelihood using a base model. This yields a training-free detector without access to the source generator.

In Fig. 3, Clause A is the strongest individual component in all four settings, and the full prefix improves AUROC over the empty prefix by 5.33–14.73 percentage points. In Table 3, the full scoring rule achieves 0.9864 AUROC / 0.9519 Best-F1, outperforming the alternative scoring variants. Across the five main settings, the mean TPR@1%FPR increases from 0.7311 to 0.8186. EchoPrompt therefore contributes a testable detection signal, experimental evidence for its role, and a practical way to use it.

Thank you for the response and additional experiments. I remain concerned about the interpretation of the proposed detection signal. In particular, the style-only prefix achieves performance comparable to, and in one case slightly better than, EchoPrompt on Qwen2.5/RAID Chat. It suggests that stylistic compatibility may explain at least part of the observed improvement, without necessarily requiring latent prompt restoration. Like other reviewers' concerns, the technical novelty remains relatively simple and limited. The analysis does not yet clearly distinguish the proposed mechanism from alternative explanations. Therefore, I will maintain my current rating.

We believe the style-only comparison is being interpreted too narrowly. Both style-only prefixes condition the following passage to be fluent, coherent, and well-structured. They are therefore contextual prompts rather than independent, unconditioned style signals. EchoPrompt also provides a generic description of the following text, but additionally establishes the assistant role and, crucially, the prompt–response relationship. The effectiveness of style-only prefixes is not inconsistent with our approach, nor does comparable AUROC establish that the observed signal is explained by style alone.

Our evidence goes beyond this single comparison. Fig. 3 shows that Clause A, which explicitly establishes the prompt–response relationship, is the strongest individual component across all four settings. The full prefix improves AUROC by 5.33–14.73 percentage points over the empty prefix. Furthermore, when using actual generation prompts rather than our assistant-style prefix, machine text consistently exhibits larger contextual likelihood gains than paired human text across four RAID settings. These results support a broader context-dependent detection signal, not merely the effectiveness of one stylistic template.

Regarding novelty, the contribution is not the complexity of a fixed prefix, but identifying a discriminative contextual signal and exploiting it through base-model-calibrated scoring without additional training. EchoPrompt improves mean Best-F1 over IRM by 2.64 percentage points and TPR@1%FPR from 0.7311 to 0.8186. As another reviewer has also recognized, achieving better detection performance with an easily understandable and implementable method is a strength rather than a limitation. We believe the proposed detection signal, its calibrated scoring design, and the demonstrated improvements constitute a meaningful technical contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.