- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ReAgg, a training-free reliability-aware aggregation framework for machine-generated text detection. Instead of uniformly aggregating token-level detection evidence, ReAgg assigns token weights based on two factors: relative token position and hidden-state discrepancy between a base model and its instruction-tuned counterpart. The proposed weighting scheme is adapted to four types of aggregation operators, including means, sums, ratios, and spectral statistics, without modifying the underlying token-level evidence. The paper also provides a theoretical analysis of the variance cost associated with deviations from optimal inverse-variance weighting. Experiments across five benchmark families and eight detection methods show the effectiveness of ReAgg.
Critical strengths
- Extensive experiments on multiple public datasets/models.
Critical weaknesses
- The use of hidden-state discrepancy between the base and instruction-tuned models as a proxy for token reliability is not well justified.
- There is a clear gap between the theoretically optimal weights and the proposed ReAgg weighting scheme. The paper does not establish that the proposed weights are a good surrogate for the theoretically optimal weights.
Questions for authors
- The motivation that later tokens have access to longer contexts and may therefore provide more informative detection signals might hold for pure human-written or pure LLM-generated texts. However, in practice, texts contain mixed content, and the locations of these segments are unknown. Does this motivation still hold in such scenarios?
- Why can representation stability between the base and instruction-tuned models serve as a proxy for the reliability of human-vs-LLM text detection?
- Could the authors provide a more detailed explanation for the substantial performance degradation observed in certain settings, such as Entropy on DetectRL-MultiDomain (−9.21 pp), DNA-style ratio on M4 WikiHow–ChatGPT (−9.8 pp), and Fourier-style on MAGE GPT OOD (−20.5 pp)?