acceptodds
Under review as a conference paper at ICLR 2027

Direction, Turning, and Dispersion: A Depth-Dynamic Triad for AI-Generated Text Detection

Abstract

AI-generated text detection commonly relies on output statistics or static representation features, whose effectiveness can deteriorate under shifts in generator, domain, language, and editing process. We instead study the transformation path. In a frozen backbone, human- and machine-generated content induce distinct relations among successive residual updates. We organize them into three complementary axes: directional persistence measures whether computation continues in a common direction, turning heterogeneity captures how unevenly token trajectories change course, and state dispersion tracks how token-state magnitudes spread across positions at each depth. Their relative-depth profiles form DEPTHTRIAD, a width-independent description of computation-path geometry. The representation supports DEPTHTRIAD-ZERO, a training-free analytic readout, and DEPTHTRIAD-LINEAR, a lightweight full-profile readout whose coefficients localize evidence by axis and depth. Experiments show strong training-free detection across generation, editing, and languages. The full-profile readout leads the compared cross-benchmark methods in AUROC, and the same representation supports detection in images and audio. These findings recast generative detection as measurement of computation dynamics and point toward a shared representation for multimodal provenance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-generated text detection, depth dynamics, internal representations, training-free detection, cross-benchmark generalization, multimodal provenance

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_D6xvOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DepthTriad, a compact representation for detecting AI-generated text content using a frozen, detector-selected surrogate model. It computes three profiles across network depth: the token-wise mean and standard deviation of the turning between consecutive residual updates, and the standard deviation of token-state RMS magnitudes. DepthTriad-Zero combines three prescribed summaries using pooled source median/IQR normalization, while DepthTriad-Linear fits a linear SVM to the full profiles using 500 matched human/AI-polished ImBD pairs. The scores are transferred to MIRAGE and DetectRL-X. The reported results are strong, including macro AUROCs of 0.9557 and 0.9837.

Critical strengths
  1. Strong transfer from limited source supervision. With 500 source pairs, Linear achieves macro AUROC 0.9557 on MIRAGE and 0.9837 on DetectRL-X (Tables 1, 2, 4). Coverage spans generation and revision, seventeen generators and eight languages, with a source-size-matched DetectAnyLLM rerun in Appendix J. This breadth strengthens the empirical contribution.

  2. Simple features supported by useful controls. The profiles have clear geometric definitions and a small learned head. Exact visible-length matching retains 79.6% of MIRAGE and 95.2% of DetectRL-X with strong performance (Appendix F, Table 12). Table 3 favors the combined Zero score over individual axes and pairs, supporting investigation of the representation.

Critical weaknesses
  1. The representation's contribution is not isolated from the surrogate and probe. Baselines use different backbones and preprocessing: Appendix A lists GPT-Neo-2.7B/2,048 tokens for DetectAnyLLM, while DepthTriad uses its own surrogates/256 tokens. Matching source-pair counts does not remove this confound. The central claim concerns computation-path geometry, so performance gains should be distinguished from those due to the chosen surrogate and probe. Single-layer and layer-wise steering-vector probes should share the same backbone, 500 source pairs, preprocessing, and target records. Table 18 scores terms from a joint fit rather than independently refitting feature subsets, so it does not establish how much angular features add to an optimized S-only detector. Table 3 addresses Zero, not this Linear question. These missing controls are the main reason for my recommendation.

  2. Transfer is better established for ranking than for a fixed operating point. Appendix F reads TPR at 1%/5% FPR from each target ROC; this does not show that a source-calibrated threshold retains those FPRs after transfer. On DetectRL-X, Linear improves AUROC from 0.9584 to 0.9837 but lowers balanced accuracy at threshold zero from 0.8592 to 0.8355 (Table 4). These averages do not determine the achieved FPR. Reporting target FPR and TPR by language/domain at a source-calibrated low-FPR threshold would strengthen the practical claim. The paper acknowledges population-specific calibration. This is a secondary limitation, not an allegation that target labels were used to fit the scoring function.

Questions for authors
  1. Controlled representation comparison. On DiffusionGemma, or another common strong backbone, how do source-selected single-layer and layer-wise steering-vector probes compare under the same 500 source pairs, preprocessing, and target records? Please also independently refit S-only, D-only, and D+T+S full-profile readouts rather than scoring terms from the joint fit. A consistent margin over these controls would directly strengthen the case for the proposed representation and could change my recommendation.

  2. Design and stability of the fixed Zero readout. Appendix C says labeled source trajectories informed the coordinates. What source-only procedure selected the direction anchors, turning band, and signs, and were those choices frozen before any target evaluation? Table 6 shows substantial sensitivity: shifting the direction anchors by -2/28 changes DetectRL-X AUROC from 0.958 to 0.876. Could you report stability over independent source-pair selections, separating design/selection from its validation where appropriate? This would clarify the reproducibility and scope of the training-free readout. I am requesting clarification of the selection protocol, not assuming target leakage.

  3. Threshold transfer. If a low-FPR threshold is calibrated using only held-out source human examples, what FPR and TPR does that fixed threshold achieve on the target languages and MIRAGE operation groups? Please distinguish this experiment from Table 13's target-ROC operating points. Reliable error control under this protocol would strengthen the practical relevance of the claimed frozen transfer.

4 comments

Thank you for the careful review and constructive suggestions. We appreciate your recognition of the strong transfer from limited source supervision and the geometric formulation. Your questions highlight three important aspects of the method: the contribution of the representation, the design and reproducibility of Zero, and the transfer of a source-calibrated threshold. We have conducted additional analyses for each point and present the findings below.

1. Depth integration and layer-wise steering geometry.

Your question concerns the contribution of the representation itself. Our controlled comparisons show that integrating depths improves on every single-depth DTS detector, while D/T steering geometry adds predictive information beyond state magnitudes.

Depth integration. We compare the full profile with three-feature DTS detectors trained separately at each of 29 depths. The comparisons use the same DiffusionGemma backbone, preprocessing, 500 source pairs, and target records, with normalization and the linear SVM fitted independently.

RepresentationMIRAGE AUROCDetectRL-X AUROC
Best individual depth (29 tested)0.86660.9414
Full DTS (88 features)0.95570.9837

Full DTS exceeds the strongest individual depth, depth 5 on both benchmarks, by 0.0891 AUROC on MIRAGE and 0.0424 on DetectRL-X. We will include the complete depth-wise curves in the appendix.

Steering geometry beyond state magnitudes. We isolate the contribution of steering geometry by independently refitting all seven D/T/S feature combinations. D captures residual-update directional persistence, T captures token-wise variation in turning, and S captures the spread of state magnitudes.

FeaturesMIRAGE AUROCDetectRL-X AUROC
S0.84620.9134
D0.92340.9678
T0.88300.9031
D+T0.94570.9795
D+S0.93690.9805
T+S0.91130.9546
D+T+S0.95570.9837

Adding D/T steering geometry to S raises AUROC by 0.1095 on MIRAGE and 0.0703 on DetectRL-X. The corresponding 95% confidence intervals are [0.1066, 0.1123] and [0.0690, 0.0718], from 1,000 paired bootstrap replicates over target human-source clusters.

Each axis also improves AUROC when added to the other two. All six 95% confidence intervals are positive:

Added axisMIRAGE ΔAUROC [95% CI]DetectRL-X ΔAUROC [95% CI]
D+0.0444 [0.0427, 0.0461]+0.0291 [0.0282, 0.0300]
T+0.0187 [0.0176, 0.0199]+0.0032 [0.0028, 0.0036]
S+0.0099 [0.0088, 0.0110]+0.0042 [0.0039, 0.0045]

The advantage over S-only persists across five draws of 500 source pairs. Full DTS achieves 0.9577 ± 0.0038 AUROC on MIRAGE and 0.9795 ± 0.0024 on DetectRL-X, versus 0.8414 ± 0.0054 and 0.9075 ± 0.0041 for S-only (mean ± SD). Together, these results support complementary contributions from D, T, and S. We will add the independent refits alongside Table 18.

2. Design and stability of Zero.

To assess whether the choice of depths depends on the sampled source texts, we repeat the selection using five independently sampled source sets. All five selections choose the original D anchors and T band. Detection performance remains stable, with AUROC standard deviations below 0.001 on both MIRAGE and DetectRL-X.

Original design. The fixed rule was developed from labeled ImBD trajectories, architecture-aware choices, and exploratory evaluation on earlier MIRAGE development sets. It combines D at indices 11 and 22, T at indices 10–14, and S over 25 sliding-attention layers, with signs (−, +, +). Pooled source median/IQR calibration uses no labels. The three normalized summaries are clipped to [−5, 5] and averaged.

Source-only selection and validation. Each draw uses 500 design pairs and 1,000 disjoint validation pairs. Five-fold pair-grouped cross-validation compares 200 configurations: joint D-anchor shifts and T-band-center shifts in {−4, −2, 0, 2, 4}, T widths in {3, 5, 7, 9}, and the original S mask or all layers. Normalization is fitted within each training fold, and the original signs remain fixed. Selections are frozen before source validation and target evaluation.

All five draws select all layers for S, with the original S mask ranking second. AUROC (mean ± SD) is:

ReadoutHeld-out sourceMIRAGEDetectRL-X
Original fixed0.9387 ± 0.00480.9243 ± 0.00060.9580 ± 0.0008
Source-selected0.9394 ± 0.00500.9239 ± 0.00050.9580 ± 0.0008

Source resampling consistently recovers the angular coordinates whose perturbation is tested in Table 6. Both readouts retain stable target performance across source draws. We will clarify the design history and add the selection/validation protocol in Appendix C.

3. Transfer of a source-calibrated threshold.

With thresholds fixed from held-out source humans at nominal 5% FPR, Linear achieves target macro FPR/TPR of 1.07%/72.23% on MIRAGE and 1.73%/67.50% on DetectRL-X. Zero achieves 2.10%/63.73% and 3.45%/62.80%, respectively. Both variants retain macro FPR below 5% and macro TPR above 60% on both benchmarks.

Source calibration. With detector parameters fixed, we calibrate each scoring configuration's threshold using 2,845 held-out ImBD human texts, disjoint from the original 500 fitting pairs. The achieved source FPR is 4.96% for every configuration, and thresholds are applied unchanged to target groups. Table 13 reports target-ROC operating points, and Table 4 uses the default zero-score cutoff; the evaluation here determines the cutoff entirely from source humans.

Transfer across operations and languages. We compare all six methods on the same sampled target records, reporting actual FPR/TPR (%) by MIRAGE setting and operation and by DetectRL-X language.

Training-free methods.

Target groupDepthTriad-ZeroSpecDetectFast-DetectGPT
DIG Generate1.60/90.202.80/9.005.80/43.80
DIG Polish3.40/50.203.60/0.805.80/10.60
DIG Rewrite1.80/48.002.60/1.406.40/11.00
SIG Generate1.60/92.203.60/10.806.00/42.40
SIG Polish2.20/50.601.60/1.404.20/9.00
SIG Rewrite2.00/51.203.60/2.006.60/11.60
MIRAGE macro2.10/63.732.97/4.235.80/21.40
Arabic2.00/57.8098.60/100.0016.40/25.20
Chinese2.20/55.4075.80/98.6011.40/14.20
English7.40/86.400.60/7.8014.80/26.80
French2.80/69.600.40/18.608.40/14.40
German0.80/13.600.40/12.005.60/15.20
Portuguese5.00/84.200.20/13.408.20/15.20
Russian1.00/46.00100.00/100.003.00/3.80
Spanish6.40/89.400.80/12.205.80/18.80
DetectRL-X macro3.45/62.8034.60/45.329.20/16.70

Cross-benchmark learned methods.

Target groupDepthTriad-LinearDetectAnyLLMSV-Detect
DIG Generate1.00/97.203.80/60.401.00/74.20
DIG Polish0.60/59.004.20/75.401.40/48.20
DIG Rewrite1.40/62.804.20/75.800.20/53.20
SIG Generate1.00/95.805.40/60.001.60/68.60
SIG Polish0.80/57.802.80/73.001.00/49.60
SIG Rewrite1.60/60.805.40/76.201.80/52.20
MIRAGE macro1.07/72.234.30/70.131.17/57.67
Arabic0.40/62.400.00/0.000.00/0.20
Chinese1.20/87.400.00/0.000.00/0.20
English5.20/91.003.80/92.200.80/80.80
French1.20/72.600.00/7.400.00/0.00
German0.00/7.600.00/0.000.00/0.00
Portuguese1.80/88.800.20/3.200.00/0.00
Russian0.00/40.000.80/0.200.00/0.00
Spanish4.00/90.201.80/15.200.00/0.20
DetectRL-X macro1.73/67.500.83/14.770.10/10.17

Zero achieves higher macro TPR and lower macro FPR than both training-free baselines on both benchmarks. Linear achieves the highest macro TPR among the learned methods on both benchmarks, together with macro FPRs of 1.07% and 1.73%.

Transfer across domains. We also group the same predictions by domain, using the unchanged thresholds. Entries report actual FPR/TPR (%).

BenchmarkDomainDepthTriad-ZeroDepthTriad-Linear
MIRAGEAcademic0.22/56.390.87/74.43
MIRAGEComment4.91/76.382.61/79.75
MIRAGEEmail2.09/60.280.35/59.58
MIRAGENews1.43/61.070.36/70.18
MIRAGEWebsite2.56/65.360.68/68.60
DetectRL-XAcademic3.83/70.991.62/78.79
DetectRL-XNews1.64/56.650.75/54.11
DetectRL-XNovel3.50/54.340.61/69.10
DetectRL-XSEO3.27/68.053.12/77.86
DetectRL-XWebtext6.84/65.503.04/69.45
DetectRL-XWiki1.66/60.991.20/55.42

Across all reported domains, Linear maintains FPR below 5%. We will add the operation, language, and domain results, together with the source-calibration protocol, alongside Table 13.

I would like to thank the authors for the detailed response and the additional results. The new ablations under matched settings show that the angular features contribute beyond state dispersion. The fixed source-calibrated threshold evaluation also addresses my concern about threshold transfer. To this point, I'm willing to raise my overall recommendation from 2 to 3, given that these new results can be incorporated into the next revision with necessary clarification.

Thank you again for your encouraging follow-up and your willingness to raise your recommendation. We have now incorporated the additional results and requested clarifications into the revised manuscript, with all changes highlighted in blue.

Reviewer_qHWKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DepthTriad, an AI-generated text detection method based on the geometric characteristics of cross-layer residual updates in large language models. By analyzing the computation trajectories of input texts within a frozen surrogate model, DepthTriad extracts three complementary features: directional persistence, turning heterogeneity, and state dispersion, aiming to achieve robust detection across different generators, domains, and languages. The proposed framework includes two variants: DepthTriad-Zero, a training-free approach, and DepthTriad-Linear, which employs a linear SVM classifier. The effectiveness of both variants is evaluated on the MIRAGE and DetectRL-X benchmarks.

Critical strengths
  1. This paper introduces a novel perspective on feature construction for AI-generated text detection by leveraging cross-layer residual updates to derive multiple complementary detection indicators.

  2. The proposed method is simple and straightforward, and the paper is generally well-written and easy to follow.

  3. The experimental comparisons with existing methods are relatively comprehensive in some settings. Moreover, the proposed features demonstrate potential generalizability beyond text to other modalities.

Critical weaknesses
  1. The motivation presented in the Introduction appears somewhat intuitive. In particular, the paper does not sufficiently explain the connection between cross-layer residual updates and detection generalizability. It remains unclear why features derived from cross-layer residual updates should exhibit better generalization than existing detection features under various distribution shifts. After all, computing these residual updates also requires a frozen surrogate model, which is conceptually similar to the design of many existing training-free detection methods. A more convincing explanation of why the proposed geometric features are expected to be more robust to distribution shifts would strengthen the motivation.

  2. The selection of specific depth positions for the D, T, and S features in DepthTriad-Zero appears to rely on manually determined prior assumptions. How can the reliability of these choices be ensured? Additional experiments may be necessary to directly demonstrate that the proposed depth selection strategy is both necessary and generalizable across different settings.

  3. A minor concern regarding the experiments in Table 2 is the selection of baseline methods. Table 1 shows that DNA-DetectLLM is the strongest among the compared existing training-free methods. However, it is excluded from the subset of baselines selected for Table 2. Could the authors clarify the reason for this omission? Including DNA-DetectLLM in Table 2 and the subsequent comparisons would help ensure a more comprehensive and fair evaluation.

  4. Another experimental concern is that the input text length appears to be limited to a maximum of 256 tokens throughout the paper. What is the justification for this setting, and which previous studies support this choice? Many existing studies employ surrogate models that support input lengths of at least 1,024 tokens and commonly adopt a 1,024-token setting. Could truncating longer texts to 256 tokens significantly affect detection performance? Additional experiments with different input lengths would help clarify the impact of this design choice.

  5. Several widely used public benchmarks in this research area, such as MAGE, RAID, and RealDet, are not included in the evaluation. Additional comparisons on these commonly adopted datasets could further strengthen the fairness and persuasiveness of the experimental results.

Questions for authors

See Weakness.

3 comments

Thank you for the constructive comments and for recognizing the perspective of detecting AI-generated text through cross-layer geometry. We clarify the motivation for computation-path geometry and report additional experiments on depth selection, input length, and the requested benchmarks.

1. Why computation-path geometry can support generalization.

A frozen surrogate provides a common reference; the distinction is what we measure. Token likelihoods reflect word probabilities, while raw hidden vectors retain content information. DepthTriad measures how token representations change across layers.

The residual update uℓ,t=hℓ,t−hℓ−1,tu_{\ell,t}=h_{\ell,t}-h_{\ell-1,t} isolates the block’s contribution from the state carried through the skip connection. Cosine comparison of consecutive updates is invariant to positive rescaling of either update and to a common orthogonal change of basis. The angular features therefore capture directional coordination independently of update amplitudes and coordinate orientation.

D summarizes average directional persistence, T measures how turning varies across tokens, and S adds the dispersion of token-state magnitudes. These features summarize geometric relations across positions and depth, rather than individual vocabulary probabilities or all coordinates of hidden vectors.

Our hypothesis is that generation-related regularities recur in this organization across generators, topics, and languages. Angular invariances and tokenwise aggregation provide an inductive bias for detecting shared patterns, while S retains complementary radial information. We will make this reasoning explicit in the Introduction.

2. Reliability and transfer of Zero’s depth choices.

Zero’s summaries are motivated by DiffusionGemma’s source depth profiles: directional anchors near relative depths 0.4 and 0.8, a central turning window, and a dispersion average. We fixed the D/T coordinates and signs across text backbones to test a common analytic readout. Relative-depth interpolation accommodates different layer counts; each backbone uses pooled median/interquartile-range calibration. Appendix C specifies the readout and DiffusionGemma’s structural dispersion mask.

To test these choices, we compared 200 predeclared local variants of the depth positions, turning-window width, and dispersion pool, retaining the original signs. Pair-grouped five-fold cross-validation on each of five DiffusionGemma source draws recovered the original D anchors and T window. All-block S pooling ranked first and the original block-family mask ranked second in every run. The resulting macro AUROCs closely match the fixed configuration; values below are mean ± standard deviation across the five source draws:

ConfigurationMIRAGEDetectRL-X
Fixed coordinates; recalibration across five source draws0.9243 ± 0.00060.9580 ± 0.0008
Source-selected local configuration0.9239 ± 0.00050.9580 ± 0.0008

Table 6 also shows that depth location carries useful information: moving the directional anchors two grid positions earlier changes DetectRL-X AUROC from 0.958 to 0.876, whereas all-block dispersion pooling retains 0.958.

Figure 3 shows that the depth regions and signs of source separation vary across backbones. We therefore tested a common source-selection procedure for LLaDA-1.5, Gemma 4, and Qwen3. Using the original 500 ImBD pairs, pair-grouped five-fold cross-validation selects among 1,728 combinations of coarse depth bands and signs. Each fold estimates calibration from its training portion. The selected configuration and full-source calibration are then frozen for target evaluation, retaining three clipped, standardized scalars and equal-weight fusion. Each cell below reports macro AUROC / TPR@5% FPR, using the predeclared primary fold partition:

BackboneZero readoutMIRAGEDetectRL-X
LLaDA-1.5Original fixed0.6817 / 0.15160.4958 / 0.0311
LLaDA-1.5Source-selected0.7708 / 0.36040.6764 / 0.1559
Gemma 4-26B-A4B-itOriginal fixed0.7821 / 0.34070.8441 / 0.4032
Gemma 4-26B-A4B-itSource-selected0.8350 / 0.42990.9462 / 0.7209
Qwen3-30B-A3B-Instruct-2507Original fixed0.7284 / 0.25140.7731 / 0.2851
Qwen3-30B-A3B-Instruct-2507Source-selected0.8101 / 0.37080.7914 / 0.2762

AUROC improves in all six backbone–benchmark combinations. Four additional source-fold partitions preserve these improvements; Qwen3 selects the same configuration in all five partitions. TPR@5% FPR improves in five combinations, with Qwen3 on DetectRL-X changing from 0.2851 to 0.2762.

These results provide a reproducible source-guided readout with shared geometric definitions. Source labels select bands and signs; numerical calibration pools texts with labels ignored. We will add the rationale and results alongside the original fixed protocol.

3. DNA-DetectLLM on DetectRL-X.

DNA-DetectLLM is a strong training-free detector and achieves the highest macro AUROC among the existing training-free baselines in our MIRAGE comparison (Table 1). On the multilingual DetectRL-X benchmark, however, its performance is substantially weaker. The full language-wise results are:

LanguageAUROCAUPRCTPR@1% FPRTPR@5% FPR
Arabic0.26580.36190.00100.0022
Chinese0.45130.45590.00150.0237
English0.66070.59500.01450.0700
French0.78210.78980.17620.3140
German0.62010.62940.05270.1487
Portuguese0.70220.72350.14670.2457
Russian0.36530.40290.00070.0085
Spanish0.72400.74920.16330.2905
Macro average0.57140.58840.06960.1379

Because of this weak multilingual transfer, we did not include DNA-DetectLLM in Table 2 or the subsequent comparison tables.

4. Choice of maximum input length and sensitivity.

We chose 256 tokens to match DiffusionGemma’s native decoder canvas. The model generates text by denoising a block of tokens at a time. Google specifies a 256-token canvas, and the technical report describes supervised fine-tuning on 256-token denoising blocks with bidirectional attention (Google model card; DiffusionGemma Technical Report, Section 4).

For detection, we place the observed text in this canvas and extract hidden states in one forward pass. Matching the native denoising block provides an architecture-based rationale for the default input budget, which we keep consistent across backbones.

To assess truncation sensitivity, we tested maximum input lengths of 256, 512, and 1,024 tokens on the same sampled texts from MIRAGE and DetectRL-X. The table reports macro AUROC:

Maximum tokensLinear: MIRAGELinear: DetectRL-XZero: MIRAGEZero: DetectRL-X
2560.95630.97960.92280.9552
5120.95210.98910.92490.9622
1,0240.95620.98320.92480.9532

Detection remains stable across the fourfold increase in input budget: the AUROC range is below one percentage point for every method–benchmark combination. We will add this controlled comparison and state the per-method input budgets explicitly.

5. Additional evaluation on MAGE, RAID, and RealDet.

Following your suggestion, we conducted additional experiments on MAGE, RAID, and RealDet. These datasets cover a broad range of generators, including earlier models and open-source models. To focus the evaluation on challenging generation settings within the rebuttal time budget, we selected the strongest generators in each dataset: GPT-4 for MAGE, ChatGPT and GPT-4 for RAID, and Claude 3.7 Sonnet and GPT-4o for RealDet. For MAGE, we use the official Unseen Domains & Unseen Model setting covering CNN/DailyMail, DialogSum, PubMedQA, and IMDb. RAID uses clean examples from its labeled public release exclusively for evaluation. RealDet includes English and Chinese.

All methods use the same fixed samples, selected before scoring. DepthTriad retains its original ImBD-calibrated Zero and ImBD-trained Linear readouts. Each cell reports AUROC / TPR@5% FPR. Bold marks the best value within each method group.

MethodMAGERAIDRealDetMacro average
Training-free methods
SpecDetect0.8874 / 0.65610.9084 / 0.76150.8224 / 0.41400.8727 / 0.6105
Fast-DetectGPT0.9181 / 0.73370.9654 / 0.89730.7906 / 0.44050.8914 / 0.6905
DepthTriad-Zero0.9404 / 0.82340.9802 / 0.90530.9451 / 0.74550.9552 / 0.8247
Cross-benchmark learned methods
DetectAnyLLM0.8956 / 0.57360.9807 / 0.91400.7567 / 0.28950.8777 / 0.5923
SV-Detect0.9432 / 0.77610.9886 / 0.95100.6776 / 0.32000.8698 / 0.6824
DepthTriad-Linear0.9731 / 0.91860.9912 / 0.95980.9367 / 0.75350.9670 / 0.8773

TPR@5% FPR is computed from each evaluation cell’s ROC curve at an observed FPR of at most 5%. Both metrics are averaged equally across MAGE domains, RAID domain–generator cells, and RealDet language–generator cells. The final Macro average gives equal weight to the three datasets.

DepthTriad-Linear achieves the highest overall macro AUROC and TPR@5% FPR among the compared methods: 0.9670 and 0.8773. DepthTriad-Zero leads the training-free group on both metrics for all three datasets, with macro averages of 0.9552 and 0.8247. We will add these comparisons and their sampling and implementation details to the paper.

Thank you for your response, which has addressed most of my concerns. I will raise my score from 2 to 3. Best wishes.

Reviewer_eBsTOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DEPTHTRIAD, a method for AI-generated text detection based on hidden-state dynamics within a frozen surrogate model. The method relies on a detector-side representation backbone that is independent of the model that produced the input text, mapping the text to the hidden states of each layer of the detector backbone.

Critical strengths
  • Introduces a novel paradigm for detecting AI-generated text
  • Extensive experimental evaluation
  • The reported detection performance is strong, outperforming the compared state-of-the-art methods.
Critical weaknesses
  • It is unclear why the measurements extracted from the hidden states of the backbone's layers contain information about the examined task.
  • The claim that the proposed method is multi-modal is a bit overreaching. Minimal experimental evidence was provided.
  • The paper argues about the proposed method's computational efficiency, however the strongest results are achieved when using DiffusionGemma as the backbone, which is a 25B parameters model.
  • The proposed method was calibrated on ImBD and then evaluated on MIRAGE and DetectRL-X. As a result, it is unclear how the method would perform if calibrated on a different dataset. For example, calibrating on MIRAGE and evaluating on DetectRL-X and vice versa would be an interesting experiment that could show whether DEPTHTRIAD captures some universal property of AI generation or properties related to the calibration source.
  • The evaluated backbones are a small number of current frontier models. Given the claim that information about AI generation is reflected in hidden-state geometry, evaluating on a more diverse set of backbones, including older models, could better demonstrate whether this claim holds.
  • I found Section 2 too dense to easily read and follow. A large number of methods is mentioned but minimal comments on their drawbacks are provided.
  • The abstract is also very difficult to understand without having read the paper. Rewriting it and reducing jargon would be beneficial to potential readers.
Questions for authors
  • Can the claim that the measurements extracted from the hidden states of the backbone's layers contain information about the examined task be further supported?
  • How would the proposed method perform when assessing text from generator families not observed during calibration/design? Has a hold-out experiment like this been considered/conducted?
  • How would performance be affected by the length of the text (in terms of number of tokens)? Is performance similar for longer or shorter texts?
  • How robust would the proposed method be to adversarial rewriting?
3 comments

Thank you for the constructive questions. We have completed additional experiments on calibration-source transfer, five further backbones, input length, and adversarial rewriting. We address the four questions first, followed by the remaining concerns.

1. Further evidence that hidden-state geometry contains detection information.

DepthTriad measures the alignment of successive residual updates (D), variation in their turning across tokens (T), and the spread of token-state magnitudes (S). These complementary measurements characterize how a frozen backbone processes the input. Figure 1 shows human–AI differences in the source profiles. We have now tested whether the resulting detection signal transfers when the calibration corpus or backbone changes.

Changing the calibration corpus. We conducted the suggested MIRAGE → DetectRL-X and DetectRL-X → MIRAGE experiments. Each run uses 500 source pairs, with five independently seeded source draws for each new calibration corpus. Linear fits its standardization and classifier on that source; Zero retains its prescribed coordinates, signs, and formula, updating only its label-free median/IQR calibration. Source and target sets have no exact-text or human-source-cluster overlap.

All conditions are evaluated on the same MIRAGE and DetectRL-X datasets. The table reports macro AUROC; ± denotes standard deviation across the five source draws. The ImBD rows are the original fixed-source controls evaluated on these same datasets.

Calibration → evaluationLinearZero
ImBD → DetectRL-X0.97960.9552
MIRAGE → DetectRL-X0.9715 ± 0.00170.9554 ± 0.0004
ImBD → MIRAGE0.95630.9228
DetectRL-X → MIRAGE0.9375 ± 0.00550.9177 ± 0.0009

Both readouts retain strong detection performance after replacing ImBD. This directly supports the presence of transferable information in the geometry beyond the original calibration distribution.

Older and additional backbones. Table 4 evaluates eight backbones: DiffusionGemma, Gemma 4, Qwen3, LLaDA-1.5, and four 0.17B models (Duo, MDLM, Eso-LM-B, and AR). This covers autoregressive, diffusion, and hybrid models across substantially different scales. Following your suggestion, we additionally evaluated five models spanning 2023–2025: Llama-2-7B (2023), ChatGLM2-6B (2023), Llama-3.1-8B-Instruct (2024), Qwen2.5-7B (2024), and Qwen3-4B (2025).

BackboneLinear: MIRAGELinear: DetectRL-XZero: MIRAGEZero: DetectRL-X
Llama-2-7B0.84420.70700.62520.5300
ChatGLM2-6B0.84180.69790.57310.4708
Qwen3-4B0.90780.84770.60760.5046
Llama-3.1-8B-Instruct0.90390.76220.46610.3176
Qwen2.5-7B0.89920.88240.68290.5703

The full-profile readout extracts transferable information from all five added backbones. Comparing Linear and Zero shows that readout choice affects transfer across backbones. Together with the source profiles in Appendix L, these results extend the evidence for detection-relevant depth geometry across model families and generations. We will add the source-transfer and backbone results to the paper.

2. Transfer to generator families absent from calibration.

All 500 AI-polished ImBD calibration examples were generated by GPT-3.5-turbo, as documented in ImBD Appendix A.2. Our calibration texts match its released data exactly. Method development also drew on earlier MIRAGE analyses. The evaluation tests transfer from a GPT-3.5-only calibration source to other generator families, including Claude, DeepSeek, Llama, Qwen, and Gemini.

Appendix E, Tables 8–9, reports results separately for 17 MIRAGE generators and four DetectRL-X generators. For example, on MIRAGE, Linear achieves AUROC 0.995 for Claude-3.5-Haiku, 0.973 for DeepSeek-R1, 0.988 for Llama-3.1-8B-Instruct, 0.939 for Qwen2.5-7B-Instruct, and 0.975 for Gemini-2.0-Flash. On DetectRL-X, its generator-wise AUROCs are 0.976 for DeepSeek-V3, 0.988 for Gemini-2.5-Flash, and 0.986 for Qwen-Max. These results support transfer beyond the generator family supplying the calibration examples.

We will bring the calibration provenance and generator-wise results into the main discussion.

3. Sensitivity to input length.

We have completed a controlled comparison at maximum input budgets of 256, 512, and 1,024 tokens. All three settings use the same evaluation datasets from MIRAGE and DetectRL-X and the same 500 ImBD source pairs. Each length has its own source-fitted calibration/readout, which is frozen before target scoring; the backbone and remaining settings are unchanged.

Maximum tokensLinear: MIRAGELinear: DetectRL-XZero: MIRAGEZero: DetectRL-X
2560.95630.97960.92280.9552
5120.95210.98910.92490.9622
1,0240.95620.98320.92480.9532

Detection remains stable as the maximum input budget increases fourfold: the AUROC range is below one percentage point for each method–benchmark combination. Appendix E separately reports performance by original document word count, complementing this token-budget comparison. We will add the new controlled comparison to the paper.

4. Robustness to adversarial rewriting.

We evaluated the released StealthRL rewriting model on 1,000 MIRAGE AI texts. The sample was frozen before attack generation and drawn from texts initially detected by both Zero and Linear. Each detector uses its own fixed threshold calibrated at 5% FPR on the original ImBD human examples. We generated one candidate per text using the released attack's prompt and fixed generation settings. Every candidate was retained and scored with the original frozen detectors and thresholds.

MethodDetection before → after attackEscape rateAUROC after attack
Linear100% → 95.4%4.6% (46/1,000)0.9950
Zero100% → 71.7%28.3% (283/1,000)0.9525

Escape rate is the fraction of these 1,000 initially detected texts that evades detection after rewriting. AUROC compares the attacked texts with their 1,000 unchanged human controls. The achieved human FPRs at the frozen thresholds are 1.1% for Linear and 3.1% for Zero. DepthTriad-Linear shows strong robustness to the StealthRL rewriting attack, retaining a 95.4% detection rate at the frozen source-calibrated threshold and an AUROC of 0.9950.

5. For the cross-modal detection.

Section 5.5 presents the image and audio experiments as a “cross-modal probe.” We included them to explore whether the geometric measurements developed for text detection also reveal source information in other modalities. The observation we wish to share is that the same D/T/S definitions provide useful detection signals in text, images, and audio, using modality-specific frozen backbones and fitted readouts. This suggests an interesting connection between computation-path geometry and content provenance across modalities.

We will revise this section to make its exploratory role clearer, with claims tied to the evaluated image and audio tasks. This will distinguish the shared geometric representation demonstrated here from a general-purpose multimodal detector.

6. Computational efficiency and backbone size.

DepthTriad processes the observed text in one forward pass through a frozen backbone, reduces the hidden states to an 88-dimensional profile, and applies a lightweight readout. DiffusionGemma is a mixture-of-experts model with 25.2B total and 3.8B active parameters.

Appendix I, Table 16, reports the combined cost of the backbone forward pass and profile extraction. At 256 tokens on one A100 80GB, DiffusionGemma processes 25.25 texts/s at batch size 32, with 54.82 GiB peak memory. The tested 0.17B models process 308–468 texts/s using approximately 6.1 GiB at batch size 128. These measurements quantify the accuracy–resource trade-off across backbone choices. We will bring the throughput and memory results into the main efficiency discussion.

7. Clarity of the abstract and Section 2.

Thank you for the suggestion. We will revise the abstract and Section 2 to improve clarity and readability.

Thank you for addressing my comments. I have updated my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.