- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LSHC, an LLM-guided bottom-up clustering framework for imbalanced and multi-granularity text data. The method starts from an over-clustered fine-grained partition, selects candidate cluster pairs by centroid similarity, and asks an LLM to choose among MERGE, KEEP, and RECLUSTER using central and boundary representative examples. The authors report strong results with Gemma embeddings on MASSIVE-I, MASSIVE-D, and especially MTOP-I among six benchmarks. The ablation also shows that full LSHC generally outperforms a merge-only variant on MTOP-I and MASSIVE-I.
Critical strengths
S1: The method uses the LLM for local structural decisions over cluster pairs, while leaving large-scale sample assignment to numerical clustering in embedding space. The use of central and boundary examples provides complementary evidence and offers a plausible scalability advantage over per-instance LLM clustering.
S2: This operation provides a local correction mechanism beyond a merge-only bottom-up procedure. In Table 2, adding RECLUSTER yields substantial improvements on MTOP-I; with Gemma, ACC increases from 74.94 to 87.57. This provides empirical support for the module beyond its conceptual motivation.
Critical weaknesses
W1: Under Eq. (12), RECLUSTER permits samples to be reassigned across the two previous clusters. The updated partition therefore need not be a coarsening of the previous one: an old cluster may be split across multiple new clusters. This differs from the nested structure required by conventional agglomerative hierarchical clustering or a dendrogram. Since multi-granularity semantic hierarchy is a central contribution, the paper should clarify how the claimed hierarchy is represented, traced, and used after reclustering, or describe the output more precisely as a sequence of iterative multi-granularity partitions.
W2: Section 4 sets K_target to the ground-truth class count and initializes with twice that number of clusters. Although the paper notes that KEEP decisions and the maximum iteration limit can also terminate the process, the experiments do not show whether the semantic stopping criterion selects a meaningful final number of clusters when the true class count is unknown, or how sensitive results are to the initial cluster count. This limits the evidence for the claim of automatically supporting unknown semantic granularity
Questions for authors
Q1: When RECLUSTER repartitions samples from Ci and Cj into multiple new clusters, how are parent–child relations or another navigable hierarchy representation constructed across iterations? Is the stored H a strictly nested hierarchy, a general graph, or a set of independent intermediate partitions
Q2: Could the authors report results without using the ground-truth K_target? This would directly validate the semantic stopping rule and the multi-granularity claim.
Q3: The paper states that the design substantially reduces token use and inference time, but does not report LLM calls, token counts, or wall-clock runtime. Please report the number of LLM calls, total tokens, and runtime per dataset
Q4: Results on MTOP-I and MASSIVE-I are consistent with the motivation, but overall ACC/NMI does not directly establish that small classes are preserved or minority classes improve. Frequency-stratified results or an analysis tracing small clusters is absent
6 comments
Q2
We appreciate the reviewer's suggestion. To directly investigate the dependence on K_target, we conduct an additional experiment on the complete MTOP-Intent dataset, containing 15,638 samples and 102 annotated categories, without enforcing the ground-truth target cluster number.
Starting from 200 initial micro-clusters, LSHC automatically terminates after 77 iterations when the five most similar candidate cluster pairs receive KEEP decisions. The final result contains 124 clusters, rather than the 102 annotated categories, and achieves 79.45 ACC and 90.52 NMI.
These results demonstrate that the proposed semantic stopping criterion can terminate the clustering process and produce a high-quality partition without requiring the ground-truth final cluster count. The difference between the discovered and annotated cluster counts further confirms that termination is not constrained to reproduce a predefined number of categories.
Regarding initialization, setting (K_0=2*K_target) is an experimental over-clustering configuration rather than a requirement of the proposed operation-selection mechanism. In a fully unsupervised setting, (K_0) can instead be specified independently or estimated from unlabeled embedding-space structure. In the current framework, semantic resolution is influenced primarily by the initial clustering granularity. A larger initial cluster number produces finer micro-clusters, allowing the LLM to examine subtle semantic distinctions during subsequent aggregation. Conversely, a smaller initial cluster count encourages broader initial groups and can support coarser semantic clustering.
| Class frequency | # Classes | # Samples | ACC (%) | NMI (%) |
|---|---|---|---|---|
| Smallest 25% | 26 | 189 | 71.96 | 85.91 |
| 25–50% | 26 | 630 | 83.65 | 93.83 |
| 50–75% | 25 | 1,477 | 81.45 | 91.14 |
| Largest 25% | 25 | 13,342 | 79.79 | 89.85 |
| Overall | 102 | 15,638 | 79.45 | 90.52 |
Q3
We thank the reviewer for highlighting the importance of computational efficiency. To provide quantitative evidence, we add a comparison of LLM calls on the Massive dataset (Table 7).
Specifically, ClusterLLM requires 1,618 LLM calls, LLMEdge requires 892, and k-LLMmeans requires 90, whereas LSHC requires only 69 LLM calls. This corresponds to approximately 95.7%, 92.3%, and 23.3% fewer LLM calls, respectively.
This efficiency in call frequency follows from our lightweight semantic decision mechanism. Rather than asking the LLM to perform sample-level assignments, LSHC uses the LLM only to select MERGE, KEEP, or RECLUSTER based on representative cluster samples. Numerical clustering algorithms subsequently perform local sample reassignment without additional LLM inference.
| ClusterLLM | LLMEdge | k-LLMmeans | LSHC (Ours) | |
|---|---|---|---|---|
| LLM Calls | 1618 | 892 | 90 | 69 |
Q1
We thank the reviewer for this important observation. We clarify that LSHC does not explicitly preserve or reconstruct strictly nested relationships. Instead, the current framework stores an ordered sequence of revised flat partitions.
Unlike conventional agglomerative clustering, where merging decisions are irreversible, LSHC allows RECLUSTER to reorganize samples within the union of two selected clusters. While MERGE preserves nested relationships, RECLUSTER may modify previous cluster boundaries, meaning that consecutive partitions do not necessarily satisfy strict hierarchical nesting. Therefore, (H) is neither a conventional dendrogram nor an explicitly constructed general graph. However, the sequence of the flat partitions may have some hierarchical structure, since the merge operations create higher-level partitions.
The bottom-up design refers to progressively aggregating fine-grained clusters while processing local assignment correction. This design prevents potentially incorrect assignments from being permanently propagated through subsequent merging operations. We have clarified that the current framework produces a sequence of semantically refined partitions at potentially different resolutions, rather than a strictly nested hierarchy.