- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies semantic structure through changes in local embedding neighbourhoods. It represents neighbourhoods using Semantic Field Subspaces (SFSes), based on singular directions and singular values, and introduces Semantic Shift to measure changes during cluster merging. SAFARI combines agglomerative clustering with a dynamic threshold to select SFSes at notable transitions. Experiments examine hierarchical structure discovery, classification, and an efficient approximation of Semantic Shift across text and image embeddings.
Critical strengths
- The perspective of examining how neighbourhood geometry evolves across scales is interesting. The qualitative examples provide interpretable illustrations of semantic groupings and motivate further investigation of this approach.
- The paper evaluates the framework across text and image embeddings and reports competitive classification results against several standard classifiers.
- The approximation offers a potentially useful computational improvement. The reported speedups and correlation with the exact Semantic Shift suggest that it can capture similar trends at substantially lower computational cost.
Critical weaknesses
The contribution beyond standard hierarchical clustering is not sufficiently isolated. SAFARI constructs its hierarchy by merging clusters according to the cosine distance between their centroids; Semantic Shift subsequently determines which nodes are retained as SFSes. However, neither the main experiments nor the appendix compares against existing hierarchical clustering methods. Table 1 shows classification utility but does not isolate the proposed method's benefits. Likewise, Figure 4 and the qualitative examples do not establish whether comparable semantic structures would already emerge from standard hierarchical clustering using the same embeddings.
Questions for authors
- Could the authors compare SAFARI against hierarchical clustering baselines using the same embeddings, and isolate the effect of Semantic Shift-based node selection from that of the underlying clustering procedure?
- What evidence connects threshold crossings to meaningful semantic boundaries?
We thank the reviewer for the careful reading and for the two precise questions.
Critical Weaknesses.
We ran this comparison. It uses existing hierarchical clustering methods on the same embeddings, and it separates the effect of Semantic Shift from the effect of the clustering procedure. The results are given in Table R4.
Table 1 tests whether SFSes preserve meaningful semantics (L411-412). The benefit of Semantic Shift is isolated in the new experiment.
The new experiment tests this on the same embeddings. Standard hierarchical clustering does not produce comparable structures. The clusters it returns are less pure than the SFSes at every level.
Questions For Authors.
We ran this experiment on the AG-News embeddings. We built four hierarchies on the same embeddings. They are SAFARI's hierarchy and three standard hierarchical clustering methods from the literature, average, complete and Ward linkage. On each hierarchy, clusters are selected in three ways. Semantic Shift (Eq. 1) with the dynamic threshold of Algorithm 1 gives the SFSes of that hierarchy. The largest merge distances give the same number of clusters. A flat cut, the usual way a hierarchical clustering method returns clusters, gives the same number again. Each cell of Table R4 gives the impurity of the selected clusters at Lv1, Lv2 and Lv3. Lower is better.
Table R4
Table R4 isolates the two effects. Along a row the hierarchy is fixed and only the node selection changes. Semantic Shift gives the lowest impurity in every row, at every level. Down the Semantic Shift column the node selection is fixed and only the clustering procedure changes. The impurity stays almost the same, between 0.48 and 0.51 at Lv1. So the gain comes from Semantic Shift-based node selection and not from the underlying clustering procedure. The SFSes of SAFARI are also purer than the flat cut of each standard method. In the revised manuscript, we will add this experiment to Sec. 5.2.
We tested this against reference labels that the method never sees. A meaningful boundary lies where two different classes meet. So we measured the share of merges that join two different reference classes, for the merges at threshold crossings and for all merges (Table R5). Higher is better.
Table R5
At every level on both datasets, a merge at a threshold crossing joins two different classes more often than a merge in general. On MIT-States it does so seven times more often at Lv1.
The clusters formed at threshold crossings are also purer than the same number of clusters taken at the largest merge distances (Table R6). Lower is better.
Table R6
The case study in the paper gives an example. Semantic Shift shows notable evolutions at two iterations of the Sports category (L459-460), and Figure 7 visualises the U.S. university sports groupings. In the revised manuscript, we will add both measurements to Sec. 5.2.