acceptodds
Under review as a conference paper at ICLR 2027

Tailoring the Clustering Objective to the Data: Utility-Guided Validity-Index Selection and Candidate Reranking

Abstract

Choosing a clustering configuration without labels requires a criterion, and no single cluster-validity index (CVI) is reliable across geometries. Automated clustering systems adapt the criterion to the dataset, but coarsely: one predicted index, a fixed set, or one global quality surrogate. We present CLUSTOPT, which composes the criterion per dataset for two-dimensional data. Label-free meta-features are mapped to a predicted utility profile: for each of 60 indices, how well it would rank candidate partitions of the data at hand. The search objective is a weighted combination of the indices predicted to be informative, and a learned reranker makes the final choice among the candidates the search visited. To give the profile something to select among, we contribute 46 structural indices. On 1,055 controlled datasets held out from all learned components, weighting the vocabulary by the predicted profile is worth +0.227 ARI over weighting it uniformly. The profile also beats every alternative criterion we tested, including random search, the best single index, a static three-index core and a dataset-independent profile over the same vocabulary. The comparison shows why: all criteria visit candidates of near-identical quality, and differ by up to 0.26 ARI in which candidate they return. Selection is what matters, and CLUSTOPT improves it twice. Reranking the slate adds a further +0.058 ARI and recovers 61% of the distance to the best candidate visited, at under 1% of the search cost. On an independent 50-dataset benchmark CLUSTOPT has the highest mean of any system, a significant paired advantage over ML2DAC, and a lead over AutoClust on non-convex, connectivity-defined structure. Three generalist indices transfer across benchmarks; the remaining new indices are specialists whose value appears only under selection

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
automated clustering, AutoML, cluster validity indices, meta-learning, unsupervised model selection, candidate reranking, dataset meta-features

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pYXvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CLUSTOPT, which predicts the utility of clustering effectiveness metrics based on data features, constructs a weighted search objective, and then reorders the candidate clustering results. The experiments primarily focus on automatic clustering selection in two-dimensional data.

Critical strengths
  1. The motivation for the research is clear; different data structures require different evaluation metrics, and this approach has practical significance.
  2. The paper analyzes the effects of metric weighting and candidate reordering separately and reports that no significant improvements were achieved; the experimental results are sufficiently well-documented.
Critical weaknesses
  1. On external benchmarks, the single-view ARI for ARI is 0.319, and for AutoClust it is 0.321; the relative advantage is primarily concentrated in certain non-convex datasets, and its practical applicability remains limited.
  2. A comparison with a “reordering algorithm using non-conditional criteria combined with independent training” has not yet been conducted; therefore, the additional contribution of data conditioning in a complete system remains unclear.
Questions for authors
  1. If the reordering algorithm is trained separately using non-conditional criteria, does CLUSTOPT still offer an advantage?
  2. Could you provide information on the time required for meta-feature extraction so that the computational cost of the entire process can be evaluated?
3 comments
AuthorsOpenReview ↗

Resposne to Reviewer pYXv

We thank the reviewer for their constructive review. Both questions led to new experiments. We report them here in full and address the weaknesses in the next comment. We have prepared a revised paper that includes them (section and table numbers below refer to it).

Q1. Does CLUSTOPT keep its advantage when a reranker is trained separately for the non-conditioned criterion? We ran this comparison. We trained two rerankers with the same model, hyperparameters, target and training data (48,039 candidate banks from the development splits). One serves the conditioned criterion (the predicted utility profile). The other serves the non-conditioned criterion (the mean profile, with the same 10 indices for every dataset). Each was evaluated on the 1,055 held-out Split-1 datasets, on the candidates visited under its own criterion.

Selection rule (xy-view ARI)ConditionedNon-conditionedPaired difference [95% interval]
Criterion maximizer, no reranker0.49250.4220+0.0705 [+0.0557, +0.0852]
Reranker with predicted-utility features0.58770.5864+0.0014 [-0.0036, +0.0064]
Reranker without those features0.56710.5267+0.0404 [+0.0304, +0.0506]
Best candidate visited (oracle)0.63660.6344+0.0022

The answer is no. When both rerankers see the predicted utilities, the two criteria show no detectable difference: +0.0014, p = 0.70, with 337 wins, 390 ties and 328 losses. This is not a formal equivalence test: the interval rules out neither a small advantage nor a small disadvantage. The revised paper no longer claims that conditioning the criterion adds to the complete system (abstract, Section 5.2 with Table 3, Section 8; details in Appendix V.1).

The experiment also shows where the gain sits.

  • Before reranking, conditioning is worth +0.071, as in the submission.
  • In the main comparison, both rerankers receive the predicted utilities as input features. Without these features, conditioning is again worth +0.040. So the predicted utility profile, the central object of the paper, carries the gain in both systems. It can enter through the criterion or through the final selector; with the second route alone, no further gain from the first was detected.
  • Both criteria visit candidates of nearly equal quality (0.637 against 0.634). Selection, not search, is the bottleneck.
  • Reranking is the larger effect: +0.095 for the conditioned system and +0.164 for the non-conditioned one.

Split 1 informed our design decisions, and this comparison was not run on the external corpus. Two statements are corrected in the revision: the reranker is trained on each candidate's regret relative to the best candidate in its bank, not on ARI directly, and the submission's estimate that this comparison needed 77-107 hours of new search was wrong, because the rerankers are trained on fixed candidate banks that do not depend on the criterion (Appendix V.1).

Q2. Meta-feature extraction time and total cost. We measured the complete pipeline of the three primary systems (the deployed CLUSTOPT and the best AutoClust and ML2DAC arms) on the 50 external datasets (450 runs, one worker, one numeric thread).

Median secondsCLUSTOPTAutoClustML2DAC
xy view13.5214.145.31
All three views33.7242.1617.10

These figures include meta-feature extraction, model loading, search, index evaluation and reranking. They exclude only the refits that our harness performs to log partition hashes, which never affect the selection (with them: 17.29 and 42.14 seconds for CLUSTOPT).

Meta-feature extraction has a median of 1.98 seconds per run, covering all three views. This is 11.5% of CLUSTOPT's execution time. Reranking has a median of 0.019 seconds. CLUSTOPT and AutoClust have similar typical cost (paired median ratio 0.97). ML2DAC is about 2.5 times faster than CLUSTOPT. The timings describe our implementations, including our re-implementations of the two baselines. Appendix A of the submission said that extraction was logged as zero; in fact it lies inside the timed path. The original records also ran with two to four concurrent workers, so they are not comparable with these single-worker figures. The revision corrects this and reports the measurements (Section 7, Appendix S.1); the other arms keep only their original records, labelled as such.

AuthorsOpenReview ↗

Resposne to Reviewer pYXv (part 2)

W1. External results and practical applicability. The reviewer's reading is correct and matches Section 7. On the deployable single view the two systems are level (0.319 against 0.321), and no overall advantage over AutoClust is established. The advantage is concentrated in non-convex, connectivity-defined data, on both benchmarks: arcs and rings (+0.030) and de-interleaving (+0.013) on the controlled benchmark, concentric circles and two moons on the external one. AutoClust leads on isotropic blobs. On the 28 real tabular datasets, 26 of them projected to 2D with PCA, every system stays below 0.16 ARI.

The non-convex regime is the one the method was built for. The work began with a concrete need, separating radio-frequency emitters such as radars in a stream of intercepted pulses. Pulses are clustered in planes of two measured parameters (for example, time of arrival against amplitude or frequency). There, clusters are arcs, parallel bands and interleaved trains, and compactness-based indices can choose the wrong partition. The same holds for other natively two-dimensional data with shape-defined clusters: spatial point data (incidents, disease cases, seismic events), single-molecule localization microscopy, wafer defect maps and 2D laser scans.

High-dimensional data can be brought into scope through 2D embeddings. t-SNE and UMAP maps are widely used to identify groups, for example in single-cell biology [1] and for learned text embeddings at the scale of millions of documents [2]. Clustering in a UMAP embedding instead of the original feature space has also been shown to improve density-based clustering [3]. Our indices and pipeline apply to such a map without change. We have not evaluated this route. We used PCA, which keeps far less cluster structure than these maps do [4], and the embedding test is a natural next experiment.

We state the boundary of the evidence clearly. Our radar-like data are synthetic, and the paper has no result on real data from these domains or on t-SNE or UMAP maps. The reviewer's assessment that practical applicability is not yet demonstrated is fair. The setting is narrow by design, and the paper should have explained why. The revised introduction and limitations now do.

W2. Missing comparison with an independently trained reranker. This is the experiment reported under Q1. It answers the question that the submission left open and narrows our claim about conditioning.

A note on Strength 2. The summary says that the paper reports no significant improvements. For the record, this holds for the overall comparison with AutoClust, but not for the components: profile weighting against uniform weighting (+0.227 ARI) and reranking (+0.058 on the controlled benchmark; +0.083 externally, unadjusted p = 0.0013). Against ML2DAC, the external benchmark shows a positive paired difference of +0.067 [+0.001, +0.157], unadjusted p = 0.039; the best ML2DAC arm was chosen as comparator after unblinding. We withdraw the submission's controlled ML2DAC comparison (+0.042): it was computed against a historical ML2DAC run whose index scorer was defective, and no corrected controlled run exists. On the controlled benchmark the system comparison is therefore with AutoClust only, where the two are at parity (0.616 against 0.614).

We thank the reviewer. The two questions led to the two most informative experiments we have run since submission, whatever the decision.

  • [1] D. Kobak, P. Berens, "The art of using t-SNE for single-cell transcriptomics," Nature Communications, 2019.
  • [2] R. González-Márquez, L. Schmidt, B. M. Schmidt, P. Berens, D. Kobak, "The landscape of biomedical research," Patterns, 2024.
  • [3] M. Herrmann, D. Kazempour, F. Scheipl, P. Kröger, "Enhancing cluster analysis via topological manifold learning," Data Mining and Knowledge Discovery, 2024.
  • [4] J. Lause, P. Berens, D. Kobak, "The art of seeing the elephant in the room: 2D embeddings of single-cell data do make sense," PLOS Computational Biology, 2024. ----- END COMMENT 2 -----

The author's response has addressed part of my concerns.

Reviewer_pZN1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CLUSTOPT, an automated clustering system for two-dimensional data. It learns a per-dataset "utility profile" that predicts how well each of 60 cluster-validity indices would rank candidate partitions in agreement with ARI. The search objective is a weighted combination of the 10 indices with the highest predicted utility, and a gradient-boosted reranker trained to regress ARI picks the final partition. On a held-out split of the synthetic repository, profile weighting improves substantially over uniform weighting. On an external corpus of 50 datasets, CLUSTOPT has the highest mean Best-View ARI, but no significant difference from AutoClust.

Critical strengths

Very thorough evaluation. Many novel components in the method, mostly focusing on prespecified structures of clusters in 2D geometry.

Critical weaknesses

Narrow scope. The method targets clustering in 2D with at most 5 clusters.

It performs comparably to AutoClust which generalises to more clusters.

The paper is interesting for curiosity but I do not see why it would appeal to a wider audience. Therefore, I recommend a rejection.

Questions for authors

Where do you expect to find applications for this?

2 comments
AuthorsOpenReview ↗

Response to Reviewer pZN1

We thank the reviewer for the direct question. We answer it here and address the weaknesses in the next comment. We have prepared a revised paper.

Where do we expect to find applications? In three kinds of data.

  1. Radio-frequency emitter separation. This is where the work began: separating emitters such as radars in a stream of intercepted pulses. Each pulse has a few measured parameters (time of arrival, frequency, pulse width, amplitude), and emitters are separated by clustering the pulses in planes of two parameters. In these planes clusters are arcs, parallel bands and interleaved trains, not compact blobs, so a compactness-based criterion can return the wrong partition. This is why the paper proposes structural indices and a per-dataset choice among them, and why the repository includes a de-interleaving family of 1,733 datasets (Appendix N).

  2. Other natively two-dimensional data with shape-defined clusters. Examples are spatial point data (incidents, disease cases, seismic events), single-molecule localization microscopy, cell positions in tissue, wafer defect maps with their rings and scratches, and 2D laser scans.

  3. 2D embeddings of high-dimensional data. High-dimensional data are routinely mapped to 2D with t-SNE or UMAP, and groups are then identified in the map. This is standard practice in single-cell biology [1] and is used for learned text embeddings at the scale of millions of documents [2]. Clustering in a UMAP embedding instead of the original feature space has also been shown to improve density-based clustering and to make it less sensitive to its parameters [3]. Our indices operate on point coordinates in the plane, so the full pipeline applies to such a map without change. This is a plausible route to data of any original dimension, not one we have tested.

We want to be exact about the evidence. Our radar-like data are synthetic, and the paper has no experiment on real data from these domains. We also did not evaluate t-SNE or UMAP maps. Most of our real datasets were projected with PCA, which keeps little cluster structure, and every system stays below 0.16 ARI there. t-SNE and UMAP maps preserve cluster structure much better [4]. Testing on such maps is a natural next experiment. These are the motivation and the intended uses, not validated applications. The paper should have stated them openly. The revised introduction and limitations now do.

  • [1] D. Kobak, P. Berens, "The art of using t-SNE for single-cell transcriptomics," Nature Communications, 2019.
  • [2] R. González-Márquez, L. Schmidt, B. M. Schmidt, P. Berens, D. Kobak, "The landscape of biomedical research," Patterns, 2024.
  • [3] M. Herrmann, D. Kazempour, F. Scheipl, P. Kröger, "Enhancing cluster analysis via topological manifold learning," Data Mining and Knowledge Discovery, 2024.
  • [4] J. Lause, P. Berens, D. Kobak, "The art of seeing the elephant in the room: 2D embeddings of single-cell data do make sense," PLOS Computational Biology, 2024.
AuthorsOpenReview ↗

Response to Reviewer pZN1 (part 2)

Narrow scope: 2D. The setting is deliberate and follows from the applications in the previous comment. The index vocabulary is 2D by design. The framework itself (predicting index utility per dataset, then reranking) does not depend on the dimension.

At most five clusters. This range is a property of our training data and search domain, not of the design. The indices score any partition. Every system, including AutoClust, searched the same domain, so the comparison is controlled. We do not overstate this: the learned components were trained on two to five clusters and are untested beyond that range. We do not dispute AutoClust's broader applicability. Extending the range requires regenerating the training data and retraining.

Comparable to AutoClust. We agree, and the paper says so: parity on the controlled benchmark (0.616 against 0.614) and on the external single view (0.319 against 0.321). New measurements show similar cost as well: re-measured on the xy view with one worker and meta-feature extraction included, the medians are 13.5 seconds for CLUSTOPT and 14.1 for AutoClust, while ML2DAC is faster at 5.3. These re-measurements cover the three primary systems only. Our claim is not a better general-purpose system. The difference is structured. CLUSTOPT leads where clusters are non-convex or interleaved (arcs and rings, de-interleaving, concentric circles, two moons), and AutoClust leads on isotropic blobs. On the easiest concentric-circles dataset AutoClust returns a partition with 0.15 ARI, although DBSCAN separates the circles perfectly. For the applications above, the first group is the relevant one.

Appeal to a wider audience. Three findings do not depend on the 2D setting.

  • Selection, not search, is the bottleneck. Six criteria, from random search to our predicted profile, visit candidates of nearly identical quality (within 0.006 ARI). They differ by up to 0.26 ARI in which candidate they return.
  • Index usefulness is predictable without labels. Weighting 60 indices by their predicted utility is worth +0.227 ARI over weighting them uniformly.
  • A larger vocabulary helps only under selection. With uniform weights, going from 14 to 60 indices costs 0.058 ARI.

Two analyses run for this discussion sharpen these points. Both narrow our claims.

  • Among the new indices, the 22 pattern-based ones carry the gain (+0.031 ARI on fixed candidates). The image-based group is conditional, and no single index is essential.
  • With a reranker trained for it that sees the predicted utilities, a non-conditioned criterion shows no detectable difference from CLUSTOPT (0.586 against 0.588; +0.001 [-0.004, +0.006]; not a formal equivalence test). The predicted utility profile carries the gain, but it can be used in the search criterion or in the final selector.

Details are in Sections 5.1 and 5.2 of the revised paper (Table 3) and in our responses to Reviewers v4cb and pYXv.

We accept that the validated scope of this paper is narrow. A convincing application paper needs real data from the domains above, more clusters and the embedding experiment. We thank the reviewer for the candid assessment.

Reviewer_v4cbOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The authors introduce CLUSTOPT, an automated way to compose and weight several internal CVIs tailored to a dataset and its properties. The weighting is based on the expected utility of a large range of internal CVIs and learned on a large suite of synthetic 2d benchmark datasets. The authors introduce 46 new structural indices, which are, however, only explained in detail in the appendix.
The main idea is cool and could help to find better hyperparameter settings for given dataset. However, the experiments are only performed on 2d data and it is not clear how this transfers to higher-dimensional data, which is the more typical use case.

Critical strengths

S1) Combining several internal measures is a good idea as they indeed measure very different properties of the data.

S2) Learning which CVIs to use based on the expected utility sounds promising and in the experiments, it lead to better clustering results (measured with external CVIs)

S3) Some of the 46 new indices might be very helpful in assessing clustering results and should be regarded in more detail.

Critical weaknesses

W1) Only two-dimensional data is tested and there is no explanation, why. Nowadays, most datasets have more than two dimensions, so the applicability is limited. Furthermore, several of the indices seem as if they could only work on 2d, so I'm not sure whether the method could be extended.

W2) The main part of the paper is not self-contained and there are so many new measures that it is not clear where the gains in externally measured clustering quality comes from. The novel indices are only briefly mentioned in the main part and not discussed suffiently -they are defined in the appendix, but even there I'd wish for more explanations and, first and foremost, ablation experiments that are required in order to understand the impact of each of them.

W3) Citations are missing. Along the new indices, many existing ones are also used - however, they are not sufficiently well documented - e.g., the noise aware Silhouette coefficient is mentioned (and used) several times in the main part, but never cited nor properly referred to.

Questions for authors

Q1) How does your method perform on higher-dimensional data?

Q2) Is it possible at all to extend it to significantly more than two dimensions?

Q3) What is the influence of each single newly introduced index? Did you perform an ablation to find out whether the main advantages come from a) the sheer amount of indices, b) the utility prediction or c) the combination of specific indices that complement each other?

3 comments
AuthorsOpenReview ↗

Response to Reviewer v4cb

We thank the reviewer for the constructive review. We answer the questions here and address the weaknesses in the next comment. We have also prepared a revised version of the paper (section and table numbers below refer to it).

Q1. Performance on higher-dimensional data. We have no native high-dimensional result. The method takes a two-dimensional dataset as input, and all experiments are two-dimensional.

Q2. Can the method be extended beyond two dimensions? Yes, in two ways.

  • Through a 2D embedding, with no change to the method. High-dimensional data are routinely mapped to 2D with t-SNE or UMAP, and groups are then identified in the map. This is standard practice in single-cell biology [1] and is used for learned text embeddings at the scale of millions of documents [2]. Clustering in a UMAP embedding instead of the original feature space has also been shown to improve density-based clustering and to make it less sensitive to its parameters [3]. Our indices operate on point coordinates in the plane, so the full pipeline applies to such a map as is. We have not evaluated this route yet. Most of our real datasets were projected with PCA, which keeps little cluster structure: every system stays below 0.16 ARI there. t-SNE and UMAP maps preserve neighborhoods and cluster structure much better than PCA [4]. They also distort distances [4], so a proper test must score against labels and repeat over seeds. It is a natural next experiment.
  • Natively, with a different vocabulary. The framework (utility targets, profile prediction, weighted objective, reranking) does not depend on the dimension. The vocabulary does, as the reviewer suspected: the 24 image-based indices use a 2D raster, and the 22 pattern-based ones project to 2D first. A native extension needs dimension-agnostic indices and meta-features, retraining and a new validation. We have not done this.

Q3. Influence of each new index, and the source of the gains. ARI values are means over the 1,055 held-out Split-1 datasets. Brackets are 95% bootstrap intervals.

(a) Not the number of indices. Under uniform weights, going from 14 to 60 indices changes ARI by +0.001 at a fixed candidate budget and by -0.058 under online search (Appendix F).

(b) Mainly the utility prediction. Replacing uniform weights with the predicted profile is worth +0.088 on the 14 pre-existing indices (fixed budget) and +0.239 on all 60 (online search; Appendix F).

(c) New ablation, run for this review (Table 3 and Appendix F.1). We re-scored the same fixed bank of 32 candidates per dataset, with the utility predictor and the top-10 rule frozen. Only the inventory changes.

Inventoryxy-view ARIDifference from the 14 pre-existing
14 pre-existing indices0.5237(reference)
14 + 22 pattern-based0.5550+0.0312 [+0.0233, +0.0393]
14 + 24 image-based0.5279+0.0041 [-0.0037, +0.0120]
All 600.5443+0.0206 [+0.0124, +0.0288]

The pattern-based indices are the clear source of the vocabulary gain. They help in 11 of the 12 structural families, most on bands and stripes (+0.100) and arcs and rings (+0.045). The image-based group is conditional: inconclusive on the xy view, positive on Best-View ARI (+0.0105 [+0.0039, +0.0171]). We state plainly that the full 60-index set is not the best inventory here: its difference from the 14 + pattern set is -0.0107 [-0.0158, -0.0057].

We also removed each of the 46 new indices in turn. Every effect is small, at most 0.0024 ARI. The largest losses come from removing gridness_fft_acf, ribbon_thickness and arc_circle_fit. The value therefore lies in groups of partly redundant structural indices used under selection, not in any single index.

This analysis is criterion-only on fixed candidates, and Split 1 informed our design decisions, so we treat it as supplementary evidence on mechanism. It narrows our claim: the 46 indices are a studied vocabulary with uneven effects, not 46 separately established contributions. The revised abstract, Section 5.1 and Section 8 say so.

  • [1] D. Kobak, P. Berens, "The art of using t-SNE for single-cell transcriptomics," Nature Communications, 2019.
  • [2] R. González-Márquez, L. Schmidt, B. M. Schmidt, P. Berens, D. Kobak, "The landscape of biomedical research," Patterns, 2024.
  • [3] M. Herrmann, D. Kazempour, F. Scheipl, P. Kröger, "Enhancing cluster analysis via topological manifold learning," Data Mining and Knowledge Discovery, 2024.
  • [4] J. Lause, P. Berens, D. Kobak, "The art of seeing the elephant in the room: 2D embeddings of single-cell data do make sense," PLOS Computational Biology, 2024.
AuthorsOpenReview ↗

Response to Reviewer v4cb (part 2)

W1. Only 2D, with no explanation. The reviewer is right: the paper states its scope and does not explain it. The work began with a concrete need, separating radio-frequency emitters such as radars in a stream of intercepted pulses. Each pulse has a few measured parameters (time of arrival, frequency, pulse width, amplitude), and emitters are separated by clustering the pulses in planes of two parameters. In these planes clusters are arcs, parallel bands and interleaved trains, not compact blobs, and no single classical index is reliable. This motivated both the structural indices and the per-dataset choice among them. It is why the repository includes a de-interleaving family (Appendix N).

Other data are natively two-dimensional with shape-defined clusters: spatial point data (incidents, disease cases, seismic events), single-molecule localization microscopy, cell positions in tissue, wafer defect maps and 2D laser scans. A further large source is 2D embeddings of high-dimensional data (Q2).

To be exact about the evidence: our radar-like data are synthetic, and the paper has no experiment on real data from these domains or on t-SNE or UMAP maps. They are the motivation and the intended use, not validated applications. The revised introduction now gives this rationale, and Section 8 states the evidence boundary.

W2. The main text is not self-contained, and ablations are missing. We agree, and the revised paper uses the additional page for this. The main text now has a table of the index groups with representative indices (Table 1), the group ablation (Table 3) and a paragraph on its findings (Section 5.1). Appendix F.1 gives the full results, including the removal of each of the 46 indices. Full definitions remain in Appendix W, now including the 10 established variants (Appendix W.1).

W3. Missing citations. We agree, and the reviewer's example exposed an error on our side. Appendix W defined only the 46 new indices. The revision adds Appendix W.1, which defines each of the 10 "established" variants as implemented and classifies its source. The noise-aware silhouette is our adaptation of the silhouette (Rousseeuw, 1987): the implementation computes the silhouette on the non-noise points, multiplies it by (1 - noise fraction) and maps the result to [0, 1]; we located no published source for the penalty. S_Dbw is our implementation of Halkidi and Vazirgiannis (2001). Two variants are named components of published indices: the single-link separation term of Dunn's index and the within-cluster scatter of Davies-Bouldin. The remaining six (isotropy, two size entropies, two neighborhood purities, minimum centroid distance) are project-defined, with no published source located. We cite only sources we have checked. The sentence in Appendix A saying that the noise-aware silhouette "scores the original labels" was inaccurate and is corrected.

We thank the reviewer for the time invested. The questions made the work more precise, whatever the decision on this submission.

Dear authors, thanks for putting work into the rebuttal.

Unfortunately, the discussion phase is almost over, and with only a day between your rebuttal and the end of the discussion phase, I can only give a brief answer.

Working only on 2d data is not enough for a general paper. I appreciate the effort of referring to single-cell data: however, it is rather concerning that research in this area is performed on the 2d visualizations. This is not an intended use case of t-SNE or UMAP. Recently, [1] showed that t-SNE exaggerates cluster structure. It has been known in the clustering community for almost ten years that non-linear dimensionality reduction should not be used as a preprocessing for clustering [2]. The whole research area of deep clustering has mainly developed because of this problem: Information is lost if projected onto a lower-dimensional space, and this information might be very relevant for clustering as well as the evaluation thereof.

If the work started with a concrete problem that you describe in the other part of the rebuttal, working on radio-frequency emitters such as radars in a stream of intercepted pulses, try to use that as the main motivation and include real-world experiments for it in the main part of the paper to show the relevance to this area.

Regarding Q3 and considering Table 16, it seems that none of the indices has a lot of influence on its own and the tradeoff does not become clear.

Appendix F contains barely any descriptive text about the experiments and does not analyse the many tables in it. One could try to guess from the brief captions and tables what their intended use cases are, but that should be obvious from the paper, even if it is only part of the appendix. Section F.1 contains seven tables and a figure, but only two paragraphs of text, which do not even refer to Tables 10-13 and 15. Please include descriptions of your experiments and analyses of the results when resubmitting.

While I appreciate the work put into the rebuttal, the paper is not ready to be published at ICLR. I recommend to resubmit after having either focused on the 2d use cases in more specialized experiments and including real-world data, or else adapting it to work with higher-dimensional data.

[1] Bergam, N., Snoeck, S., & Verma, N. (2026, April). t-SNE exaggerates clusters, provably. In International Conference on Learning Representations (Vol. 2026, pp. 27823-27848).

[2] Schubert, E., & Gertz, M. (2017, September). Intrinsic T-stochastic neighbor embedding for visualization and outlier detection: A remedy against the curse of dimensionality?. In International Conference on Similarity Search and Applications (pp. 188-203). Cham: Springer International Publishing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.