acceptodds
Under review as a conference paper at ICLR 2027

Clusters Are Proposals: Discovering Hidden Subcategories in Pre-Categorized Text

Abstract

Enterprise call-center analytics often begins with broad topic groups, while the finer distinctions needed for actionable analysis remain hidden. Discovering these distinctions is challenging when interactions within each group are semantically similar, category frequencies are highly uneven, and the number of categories is unknown. The challenge is sharpest when meaningful subcategories are rare: an infrequent but important customer issue may represent only 1% of a broad category’s traffic. Recent LLM pipelines discover categories by proposing candi- date definitions from a sample of documents and then assigning every document against them. We argue that these pipelines lose hidden subcategories before an LLM ever sees them: proposals are drawn from random samples, and small candidates are later pruned, absorbed into coarse labels, or removed by minimum-size rules. We introduce clusters as proposals, which changes where proposals come from. Each category is recursively over-fragmented into size-bounded clusters, so every dense region, however small, reaches the LLM as a candidate subcategory with a written definition. Candidates are merged only when an LLM confirms, from their definitions and supporting texts, that they describe the same subcategory, and every document is then labeled against the final definitions. The design rests on an asymmetry: a redundant proposal costs one merge, but a missing proposal cannot be recovered later. On 10 high-volume categories of a production telecommunications call-center corpus, cluster proposals yield 234 subcategories, compared with 111 from flat clustering and 203 from LLM-first discovery, with higher within-subcategory coherence than both baselines in all 10 categories. In 8 of those categories, at least one discovered subcategory holds less than 2% of that category’s calls—the hidden issues the method is designed to surface.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
fine-grained category discovery, conversation analytics, LLM-guided clustering, overclustering, semantic consolidation, unsupervised text analysis, long-tailed data

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_khMQOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a novel method for large-scale text classification.

By utilizing recursive over-fragmentation, the proposed method extracts every small, dense region as a candidate, thereby overcoming the drawback of prior studies where rare subcategories tend to be buried.

By combining geometric similarity with LLM-based verification, it efficiently consolidates redundant candidates while discovering hidden, important issues.

The effectiveness of the proposed method is demonstrated through evaluation along four axes: topic coherence, topic separation, structural properties of the taxonomy, and operational usefulness via an LLM-as-Judge.

Critical strengths

The method overcomes the critical flaw of conventional approaches—the omission of rare yet important issues (latent subcategories)—by enabling exhaustive extraction through recursive over-fragmentation.

By separating the proposal and assignment processes, a natural language definition is assigned to each subcategory, ensuring high interpretability and auditability.

Furthermore, an independent LLM-as-Judge evaluation verifies that the scores for topic Specificity and Actionability are significantly higher than those of baseline methods.

Critical weaknesses

The evaluation is limited to a single production call-center dataset from a North American telecommunications provider.

Because there is no ground-truth reference taxonomy, evaluations for Recall and semantic completeness are missing.

While the "algorithmic mechanism" of the proposed method itself is dataset-agnostic, its "demonstrated effectiveness" and "preprocessing (data format)" strongly depend on this specific call-center data.

Questions for authors

I understand that the main focus of this paper is on how to discover "actionable hidden subcategories" in a production environment where no reference taxonomy exists. However, what is the reason for not using (or not being able to use) public datasets such as CLINC150 or BANKING77 for further validation?

Reviewer_LTKMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Clusters Are Proposals, a method for finding smaller and more specific topics within existing text categories. The method first divides documents into small clusters using recursive clustering. It then combines similar clusters using vector similarity and LLM-based checks. Finally, an LLM assigns documents to topics based on their text descriptions. The authors test the method on ten categories from a real-world telecommunications dataset. The results show improvements in the number of topics found, topic coherence, separation between topics, and LLM-evaluated actionability.

Critical strengths

[S1]: The paper focuses on an important real-world problem of finding rare or hidden subcategories in text that have similar meanings.

[S2]: The main idea of separating cluster generation from final topic assignment is simple and useful.

[S3]: The method does not need labeled examples or a fixed number of subcategories in advance.

[S4]: The paper compares the proposed method with other methods, uses different evaluation metrics, and discusses failure cases and limitations.

Critical weaknesses

[W1]: Insufficient empirical validation of rare subcategory recovery.

The main goal of this paper is to find hidden and rare subcategories that existing clustering and LLM-based methods may miss. However, the experiments mainly show that the proposed method finds more detailed topics with better embedding-based coherence. They do not directly show whether the method can find rare categories. In Section 7, the authors admit that they do not have a reference list of the correct categories, so they cannot measure how many real categories their method finds or misses. Also, in Section 6.4, they state that they do not know which baseline groups contained the newly found subcategories. Therefore, the results do not clearly prove that the method finds truly meaningful hidden subcategories. It may simply divide existing topics into smaller groups.

[W2]: Substantial coverage degradation and lack of matched-coverage evaluation.

The proposed approach achieves higher topic coherence but leaves a substantially larger fraction of documents unassigned (28.6%) compared to flat clustering (15.3%) and LLM-first discovery (0.2%), as reported in Table 2. This trade-off is especially pronounced in the Service Not Restored category, where the noise rate rises from 1.0% to 37.3% (Appendix C.4, Table 7). Furthermore, the authors explicitly acknowledge in Section 7 that the comparisons do not establish quality at matched coverage. Therefore, it remains unclear whether the coherence improvements reflect genuinely better subcategory discovery or are partly driven by excluding difficult or ambiguous documents.

[W3]: Lack of completely controlled baseline comparison.

The baseline comparison is not fully fair. The flat clustering baseline directly uses cluster membership as its final output, while the proposed method uses an LLM to assign items based on the generated definitions. Therefore, it is unclear whether the improvement comes from recursive clustering or LLM-based assignment. Also, the flat baseline uses the same Optuna settings as the proposed method, which may not be the best settings for flat clustering. The sampling budget and implementation details of the LLM-first baseline are also not clearly explained.

[W4]: Lack of Independent Human Evaluation.

The authors use Claude Sonnet 4.6 to evaluate topic actionability, but the same model family is also used in the discovery process. This raises concerns about the independence of the evaluation. Also, the judge sees only 10 documents per topic, and 43 LLM-first topics are excluded because they contain fewer than 10 documents. The actionability score does not show actual improvement in real-world tasks. Moreover, the improvement over flat clustering is not statistically significant (p = 0.16).

[W5]: Unclear Contribution of Individual Components

The idea of splitting data into smaller clusters and then merging them is already used in previous research. The paper's main contribution may be its specific application and merging method, but the individual contributions of different components are not clearly tested. Important ablation studies are missing, such as recursive clustering without LLM merging, cosine-based merging, final LLM reassignment, and topic-count-matched flat clustering. The effects of the noise penalty and entropy bonus are also not shown with numerical results. Appendix Table 12 only reports "High" and "Low" instead of actual metric values.

[W6]: No Guarantee of Discovering All Rare Categories.

The paper claims that every dense region can be reached during the proposal generation process. However, the minimum cluster size in HDBSCAN ranges from 20 to 200. Appendix A.1 also explains that a subgroup with 30 documents can only be separated when the minimum cluster size is 30 or less. Furthermore, the splitting process stops when no further progress is made, and noise documents are not included in proposal leaves. Therefore, very small, overlapping, or less dense categories may still be missed.

Questions for authors

[Q1]: Check [W1]. Could the authors provide ground-truth-based rare-category precision, recall, or F1 scores on datasets such as CLINC150 or BANKING77, or conduct a human-annotated evaluation to demonstrate that the proposed method genuinely recovers rare subcategories missed by the baselines?

[Q2]: See [W2]. Could the authors provide coherence, separation, and actionability comparisons at matched document coverage levels? In particular, do the improvements persist when the proposed method is constrained to assign a comparable proportion of documents as the baselines?

[Q3]: Check [W3]. Why did the authors not include a flat clustering baseline with the same LLM-based assignment step and optimize it separately for a fair comparison? Also, can the authors provide ablation studies to show the individual contributions of recursive clustering and LLM-based assignment, and explain the sampling budget and implementation details of the LLM-first baseline?

[Q4]: Refer to [W4]. Why did the authors not include human evaluation or use a different LLM model family as a judge? Can the authors report agreement between human evaluators, perform a sensitivity analysis using the same topic selection rules for all methods, and explain how the actionability scores reflect real-world improvements?

[Q5]: Refer to [W5]. Can the authors provide detailed ablation studies to show the contribution of each component, including recursive clustering, LLM merging, final LLM reassignment, noise penalty, and entropy bonus? Also, why does Appendix Table 12 report only "High" and "Low" instead of actual numerical results?

[Q6]: Check [W6]. How can the authors claim that the proposed method discovers every dense region when HDBSCAN uses a minimum cluster size between 20 and 200, and some documents are treated as noise? Can the authors provide experiments on very small, overlapping, and less dense categories, and clarify the limitations of their discovery claim?

[Q7]: The abstract says that the proposed method has higher coherence than both baselines in all ten categories. However, Appendix C.4 shows lower coherence for the Update Account Details category (0.807 vs. 0.819 for flat clustering). Also, the paper says that augmentation reduces noise, but Table 13 shows higher noise for the weighted configuration. Can the authors explain why these results are different from the claims in the paper?

[Q8]: The proposed method uses several steps, including paraphrase generation, clustering, hyperparameter tuning, LLM merging, and final LLM assignment. Can the authors report the runtime, number of LLM calls, token usage, and total cost per 1,000 documents compared to the baselines? Also, how did the authors choose the final configuration from the tested settings, and do the results remain consistent across different random seeds?

Reviewer_wQ85OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Clusters Are Proposals, a framework for discovering hidden fine-grained subcategories within pre-categorized text, particularly rare subcategories that existing clustering and LLM-based methods may overlook. The main idea is to treat clusters as candidate proposals rather than final assignments. The method recursively over-fragments existing categories using UMAP and HDBSCAN, identifies potentially redundant proposals through centroid-centered similarity, and uses an LLM to validate merges and generate category definitions. Documents are then reassigned based on these definitions.

The authors evaluate the approach on 10 categories from a proprietary telecommunications customer-service dataset, comparing it against flat clustering and an LLM-first discovery baseline. The proposed method discovers more subcategories, achieves higher topic coherence and separation, and receives higher LLM-judged actionability scores. However, it also leaves a substantially larger fraction of documents unassigned.

Critical strengths

Clear motivation and identification of limitations in existing methods. The paper identifies a meaningful limitation of existing clustering and LLM-based topic discovery approaches: rare subcategories may be missed during random sampling, absorbed into larger clusters, or eliminated by pruning. This provides a well-motivated problem formulation and highlights an important challenge in fine-grained category discovery, particularly for imbalanced real-world datasets.

Critical weaknesses

1. The central claim of discovering meaningful hidden subcategories is not directly validated. The evaluation relies entirely on a proprietary dataset without fine-grained ground-truth labels. While the proposed method produces more topics with higher coherence and separation, these metrics do not establish whether the additional topics correspond to genuinely distinct and meaningful subcategories rather than over-fragmentation. The LLM-as-judge evaluation provides some supporting evidence but does not directly measure subcategory recovery or semantic correctness. Public benchmarks with known fine-grained labels, such as CLINC150 and BANKING77, would enable a more rigorous evaluation, particularly of rare-category recall. Without such evidence, the paper does not convincingly demonstrate its primary claimed contribution.

2. The experimental comparisons are insufficient to establish the effectiveness of the proposed method. There are two major concerns:

(a) High unassigned rate and potential selection bias. The proposed method leaves 28.6% of documents unassigned, compared with 15.3% for flat clustering and only 0.2% for LLM-first discovery. This raises concerns about selection bias in the reported quality improvements, as excluding difficult samples may inflate coherence and specificity. More importantly, leaving nearly one-third of documents unassigned also raises questions about the method's practical utility in real-world applications.

(b) Weak and limited baselines. The flat clustering baseline is essentially a simplified, single-pass version of the proposed pipeline rather than a strong competing method. Moreover, it directly uses cluster membership for final assignments, whereas the proposed method benefits from LLM-based definition generation and reassignment, confounding the contribution of recursive proposal generation. The only other baseline is a generic LLM-first pipeline, without independent comparisons against established LLM-based topic discovery methods such as TopicGPT, TnT-LLM, GoalEx, or LLooM, several of which are already discussed in the paper. These methods employ different proposal generation, refinement, or coverage strategies and would provide more meaningful comparisons.

Taken together, the substantial coverage differences, insufficiently controlled comparisons, and lack of strong existing baselines make it difficult to attribute the reported gains to the proposed approach or establish its superiority over prior methods.

Questions for authors
  1. Validity of discovered subcategories. Beyond the LLM-as-judge evaluation, is there any human validation or other evidence that the additional subcategories are genuinely meaningful and distinct, rather than redundant or arbitrarily fragmented?

  2. Unassigned document analysis. Could the authors analyze the 28.6% unassigned documents? What types of samples are left unassigned, and do they include coherent, meaningful subcategories that the method failed to discover?

  3. Baseline implementation. Is the LLM-first baseline a faithful implementation of TopicGPT or TnT-LLM, or a simplified pipeline inspired by these methods? What refinement steps are included?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.