- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents an interesting study of what can be inferred from the geometric structures generated by dimensionality-reduction (DR) methods. The authors investigate four nonlinear DR methods: t-SNE, UMAP, PaCMAP, and TriMap.
In contrast to the often subjective visual interpretation of DR results, this study introduces quantitative measures for analyzing the relationships between the geometric structures of low-dimensional representations and the semantic information they express.
The findings, if presented and contextualized clearly, could provide useful insights for practitioners in data visualization, particularly in interpreting low-dimensional representations of high-dimensional data.
Critical strengths
The paper is generally well written and has the following strengths:
-
The motivation for investigating the relationship between low-dimensional geometric structures and their semantic information is clearly stated.
-
The analytical framework, consisting of controlled experiments addressing Q1–Q3, is systematically developed and clearly explained.
-
The information contained in the high-dimensional representations is quantitatively characterized using mutual information. Furthermore, the semantic expression in the low-dimensional representations is quantified using the measure defined in Eq. (2).
-
The controlled experimental design allows the authors to investigate the relationship between semantic information and geometric structure under well-defined conditions.
Critical weaknesses
The reviewer has several concerns that should be addressed.
- Selection of dimensionality-reduction methods
The authors do not sufficiently explain their rationale for selecting UMAP, t-SNE, PaCMAP, and TriMap. The reviewer understands that these are widely used nonlinear DR methods. However, the extent to which the conclusions can be generalized beyond these particular methods remains unclear.
The authors should provide a clearer justification for their selection and discuss the scope and limitations of generalizing their findings to other DR methods.
- Motivation and interpretation of Q1 (page 4)
The authors state that "practitioners routinely read clear regions in a colored 2D map as evidence that a variable is strongly represented in X."
The reviewer questions the generality of this assumption. An informed practitioner would normally interpret the resulting geometric structure in the context of the objective function and properties of the particular DR method being used.
For example, t-SNE is designed to represent local stochastic neighborhood relationships. Therefore, one should not necessarily expect its resulting low-dimensional representation to preserve arbitrary semantic information that is not explicitly considered in its optimization objective.
In this context, it is unclear whether Q1 reflects realistic expectations of informed practitioners or primarily demonstrates a limitation already implied by the objectives of unsupervised DR methods.
Similarly, the authors state that "information present in high dimensions need not be visually expressed." This observation is reasonable, but also largely expected, since the visual expression of semantic information depends on how that information relates to the original data geometry and the objective of the DR method.
The authors should clarify what new understanding their quantitative results provide beyond these known properties of dimensionality reduction, and under what conditions their conclusions are expected to hold.
- Comparison with supervised dimensionality reduction
The reviewer suggests that the authors consider including supervised DR methods as a complementary analysis.
Such a comparison could help clarify how explicitly incorporating semantic information into the DR objective affects its preservation and visual expression in low-dimensional representations.
In particular, comparing supervised and unsupervised methods within the same original data structure could provide further insight into the relationship between the objectives of DR methods and the semantic information expressed in their resulting representations.
For the authors' reference, some past research has demonstrated that different semantic information given to the same dataset results in substantially different representations. For example:
P. Hartono, Mixing autoencoder with classifier: conceptual data visualization, IEEE Access, Vol. 8, pp.105301 -105310 (2020) DOI: 10.1109/ACCESS.2020.2999155
Although this comparison may be beyond the primary scope of the paper, it could strengthen the interpretation and practical relevance of the findings.
- Practical implications of the findings
The most significant concern is the limited discussion of how practitioners can utilize the insights obtained from this study when analyzing high-dimensional data using DR methods.
The authors demonstrate that low-dimensional geometric structures do not necessarily reflect the amount of semantic information contained in the original representations. However, the paper provides limited guidance on how practitioners should evaluate or interpret the resulting visualizations in light of these findings.
For example, can the proposed quantitative measures be incorporated into a practical evaluation procedure? Under what conditions can a practitioner reasonably trust the semantic interpretation of a low-dimensional representation?
Addressing these questions would substantially strengthen the practical contribution of the paper.
Questions for authors
- Label-blind partitioning (pages 3–4, Eq. 2)
The authors define the semantic expression measure using a label-blind partition of the low-dimensional representation. Appendix B.2 specifies that the partition is obtained using K-means with four clusters.
However, how sensitive are the analytical results to the choice of partitioning method and the number of partitions?
Furthermore, to what extent does this partition-based measure reflect the semantic organization perceived by a human observer? A discussion or sensitivity analysis would help establish the validity and robustness of the proposed measure.
- Supervised encoding of semantic information (page 8)
The authors describe an experiment in which semantic information is incorporated through supervised representation learning. The implementation details are provided in Appendix E.1.
Could the authors briefly explain the supervised encoding procedure in the main text and clarify how the choice of semantic grouping and encoding method affects the analytical results?
In particular, to what extent do the conclusions depend on the specific supervised encoding procedure adopted in this experiment?