- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes scale-resolved PHDim, an intrinsic dimension estimator based on quantile-trimmed minimum spanning tree (MST) energies. The authors introduce a token cell–skeleton geometric framework to interpret the multi-scale structure of contextual token embeddings and investigate the relationships between intrinsic dimensions at different scales and textual properties such as contextual variability, lexical composition, and text organization. The proposed method is further applied to controlled text transformations, AI-generated text detection, language proficiency analysis, and text error analysis, demonstrating its potential value across several text analysis tasks.
Critical strengths
(1) A novel and mathematically motivated method. The paper extends traditional single-valued PHDim into a scale-resolved intrinsic dimension estimator by selecting different quantile intervals of MST edge lengths. The method has natural connections to persistent homology, graph theory, and asymptotic statistics. Theorem 1 in Section 3.3 provides an asymptotic dimension recovery guarantee for regular distributions, offering a meaningful theoretical foundation for this new perspective on text representation geometry.
(2) An intuitive geometric interpretation and broad experimental exploration. Beyond proposing a new statistic, the authors identify how traditional PHDim may mix statistical signals from different geometric scales and demonstrate the potential of scale separation to distinguish these signals. The cell–skeleton framework, controlled interventions, and experiments across multiple practical tasks provide useful insights. Overall, the paper presents a reasonably comprehensive framework combining geometric intuition, theoretical analysis, and empirical investigation.
Overall, I believe the paper makes a meaningful methodological contribution through its mathematically motivated scale-resolved estimator and broad empirical exploration. While the concerns discussed below limit the strength and generalizability of some interpretations, I consider the paper's novelty and existing contributions sufficient to outweigh these limitations.
Critical weaknesses
(1) A remaining theoretical gap between asymptotic guarantees and the central finite-sample observations. Theorem 1 in Section 3.3 shows that, for sampling distributions satisfying its regularity assumptions, different fixed quantile intervals of positive width recover the same asymptotic intrinsic dimension. However, the main geometric interpretations and experimental conclusions rely on differences in effective dimensions across quantile intervals at finite sample sizes. Although Section 4.2 and Appendix E discuss sampling dependence and cell–skeleton energy mixtures, these analyses do not yet fully establish the connection between finite-sample multi-scale behavior and the asymptotic theory, particularly regarding the stability and geometric meaning of the resulting dimension profiles.
(2) Some linguistic interpretations lack precise definitions and sufficiently independent validation. The paper associates dimensions at different scales with contextual variability, lexical richness, and broader text organization. However, some of these concepts are not clearly operationalized. For example, Section 5.2 suggests that Coarse PHDim may capture diversity organized within coherent text, but the current experiments do not sufficiently distinguish the respective contributions of semantic diversity and discourse coherence.
Furthermore, Appendices V and Y use LLM-generated annotations of textual defects. Although the authors verify whether the quoted text spans actually appear in the original documents, this check does not establish whether the assigned defect categories and severity scores are correct. Without independent human annotation checks or human–LLM agreement analysis, the evidence provided by these annotations for the proposed linguistic interpretations remains limited.
(3) The robustness and representativeness of some experimental results require further investigation. Section 5 applies 26 interventions to 50 source texts and mainly reports median paired changes in the scale-resolved PHDim estimates of their token embeddings for each intervention. The main correlation and regression analyses in Appendix M are then based on aggregated changes from 24 of these interventions. These statistics can reveal meaningful overall trends, but they do not fully characterize the consistency and variability of changes in scale-resolved PHDim across different texts under the same intervention. If substantial cross-text heterogeneity exists, it may limit the strength of interpreting these aggregate trends as general geometric mechanisms.
In addition, the AI-generated text detection experiments in Section 6 compute PHDim using contextual embeddings extracted from only the first 128 tokens of each continuation (Appendix Q.2). This relatively short window, taken from a fixed position, may not adequately represent the embedding geometry of the full generated text.
Questions for authors
(1) Theoretical connection: Could the authors further discuss or propose a preliminary theoretical framework connecting the dependence of finite-sample dimension estimates on quantile interval selection with the interval-independent asymptotic dimension in Theorem 1? For example, would it be possible to establish additional theoretical results under certain structured distribution assumptions, or provide stability guarantees for finite-sample dimension profiles?
(2) Linguistic interpretation: Could the authors use existing computational linguistics metrics, such as sentence-embedding-based semantic diversity measures and discourse coherence measures based on Entity Grid or adjacent-sentence semantic similarity, to quantify diversity and coherence separately? Investigating their relationships with Coarse PHDim and its changes under text interventions could provide more direct evidence for the interpretation proposed in Section 5.2.
Additionally, for the experiments involving automatic LLM annotations, could the authors provide a complete case study showing the original text, the corresponding LLM annotations, and the multi-scale intrinsic dimension estimates, to help readers understand their relationship?
(3) Experimental robustness: For the text interventions in Section 5, could the authors report additional statistics, such as the proportion of source texts exhibiting the same direction of change in their embedding-based scale-resolved PHDim estimates under each intervention, as well as the interquartile range of these changes, to better assess cross-text robustness? For Section 6, could the authors compare results across different token window lengths and positions (beginning, middle, and end) on a representative subset of texts to assess the sensitivity of the main findings to the sampling protocol?