acceptodds
Withdrawn from ICLR 2027

ReAgg: Reliability-Aware Aggregation of Token Evidence for Machine-Generated Text Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The widespread use of large language models makes machine-generated text detection crucial for content provenance. Existing detectors typically aggregate token-level statistics into document scores via fixed rules, such as simple means or ratios. However, such uniform aggregation treats every token equally, regardless of its position in the text or how consistently the two reference models represent it. We derive an exact variance excess identity that quantifies the cost of weighting mismatch and support loss, and use this result to guide the construction of reliability weights from two observable cues. Specifically, we propose ReAgg, a reliability-aware aggregation framework that computes a contribution weight for each token using two computable cues: (i) its position in the text, and (ii) the representational stability between a base model and its instruction-tuned counterpart. It further adapts these weights to four scoring interfaces—means, sums, ratios, and spectral statistics—without training any additional model. We evaluate ReAgg across eight carriers spanning five benchmark families (18 subsets), improving macro-average direction-corrected AUROC from to ( pp) across the 13 primary subsets. Ablations support the empirical complementarity of the position and hidden-discrepancy cues.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI-generated text detection, machine-generated text detection, zero-shot detection, reliability-aware aggregation, content provenance

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_frYvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReAgg, a training-free reliability-aware aggregation framework for machine-generated text detection. Instead of uniformly aggregating token-level detection evidence, ReAgg assigns token weights based on two factors: relative token position and hidden-state discrepancy between a base model and its instruction-tuned counterpart. The proposed weighting scheme is adapted to four types of aggregation operators, including means, sums, ratios, and spectral statistics, without modifying the underlying token-level evidence. The paper also provides a theoretical analysis of the variance cost associated with deviations from optimal inverse-variance weighting. Experiments across five benchmark families and eight detection methods show the effectiveness of ReAgg.

Critical strengths
  1. Extensive experiments on multiple public datasets/models.
Critical weaknesses
  1. The use of hidden-state discrepancy between the base and instruction-tuned models as a proxy for token reliability is not well justified.
  2. There is a clear gap between the theoretically optimal weights and the proposed ReAgg weighting scheme. The paper does not establish that the proposed weights are a good surrogate for the theoretically optimal weights.
Questions for authors
  1. The motivation that later tokens have access to longer contexts and may therefore provide more informative detection signals might hold for pure human-written or pure LLM-generated texts. However, in practice, texts contain mixed content, and the locations of these segments are unknown. Does this motivation still hold in such scenarios?
  2. Why can representation stability between the base and instruction-tuned models serve as a proxy for the reliability of human-vs-LLM text detection?
  3. Could the authors provide a more detailed explanation for the substantial performance degradation observed in certain settings, such as Entropy on DetectRL-MultiDomain (−9.21 pp), DNA-style ratio on M4 WikiHow–ChatGPT (−9.8 pp), and Fourier-style on MAGE GPT OOD (−20.5 pp)?
Reviewer_22BiOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

ReAgg reweights existing detectors’ token scores using position and hidden-state differences between base and instruction-tuned models, without additional training. The paper provides a variance analysis and tests eight scoring variants across five benchmark families. Results show modest average gains, though some variants become worse.

Critical strengths

ReAgg offers a simple way to improve several statistical detectors without additional training

Critical weaknesses
  1. The experiments show gains over uniform weighting, but lack a direct comparison with “Exons-Detect: Identifying and Amplifying Exonic Tokens via Hidden-State Discrepancy for Robust AI-Generated Text Detection” (ACL 2026). The Joint-high ablation differs from Exons-Detect in both discrepancy computation and weighting, so it does not substitute for that comparison.
  2. I do not see a clear link between the variance analysis and the weights actually used. Under its assumptions, the theory supports inverse-variance weighting, but the paper does not establish that later tokens or smaller hidden-state differences correspond to lower conditional variance.
  3. The gains are modest and uneven. With only one Falcon base/instruction-tuned pair tested, I am not yet convinced that the weighting rule generalizes across scoring-model pairs.
Questions for authors

1 When does a large hidden-state difference indicate useful evidence, and when does it indicate noise? Can you determine which weighting direction to use without checking target-domain labels? 2 Which datasets informed the choice of position weighting, hidden layer, and low-discrepancy weighting? Were any evaluation datasets left untouched until the design was finalized?

Reviewer_UtjkOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on the question of how to aggregate token-level detection signals into document-level detection protection. The paper starts with the theory of minimizing variance and then introduces two factors to reweight token-level signals (ReAgg). Across tasks and detectors, ReAgg improves the detector AUROC.

Critical strengths
  • Under most detectors and tasks, AUROC are improved.
  • The paper considers various metric-based detection methods.
Critical weaknesses
  • There is a logical gap between the theory and the motivation: minimizing the variance doesn't equal maximizing detection performance.
  • The math proof part doesn't touch on the exact design of weight factors, which I see as the main method contribution of this paper. It is unclear what the motivation is for these designs, e.g., why use a linear function for position, why use base and instruction models as pairs, and why apply multiplication to join two cues. Hence, the method design looks ungrounded.
  • We actually lack knowledge of when reweighting will give improvement and when it will not, which limits its practical use.
  • Generally, I encourage the authors to think more deeply about 1) why the two cues work and don't work based on quantitative or qualitative analysis, and 2) why we only use these two cues but not other possible designs. This might broaden the usage and impact of this research.
Questions for authors
  • Can you conduct an experiment to examine the relationship between the theoretical variance and the cue score?
  • Why should we consider using Falcon-7B as reference pairs? Any reason makes it a good reference?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.