acceptodds
Under review as a conference paper at ICLR 2027

CLASP: Correlation-Leveraged Across-sample Separability Prioritization for Unsupervised Biomarker Discovery

Abstract

Biomedical data science operates in a regime of high data dimensionality, low sample sizes, and structured noise. As a result, many applications suffer from a reproducibility crisis, in which biomarkers identified in one study fail to replicate in others. To address this challenge, we introduce Correlation-Leveraged Across-sample Separability Prioritization (CLASP), an unsupervised feature selection framework that identifies reproducible feature subsets by maximizing cross-subsample cluster separability. CLASP frames feature selection as a stochastic optimization problem over inclusion probabilities; it uses a Cross-Entropy Method paired with a correlation-augmented Gumbel sampling scheme to find a joint feature set that maximizes cluster distinctiveness. Evaluated both on simulation environments and seven diverse real-world biomedical datasets, CLASP shows state-of-the-art feature selection stability, cluster quality, and out-of-sample cluster stability, thus preserving the reproducibility of selected biomarkers.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Unsupervised Feature Selection, Feature Selection Stability, Clustering, Reproducibility, High-Dimensional Data, Biomarker Discovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_zwVLOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes CLASP, an unsupervised feature-selection method for high-dimensional, small-sample biomedical data.

Critical strengths
  1. The paper studies an important problem.

  2. The proposed method is intuitive and relatively easy to implement.

  3. The paper is well-written, organized, and easy to follow.

  4. To the best of my knowledge, this is a novel approach.

Critical weaknesses

Since I work on theory, I have found some theoretical issues in the paper.

  1. The paper introduces CC data subsamples, but Eq. (1) only computes the silhouette score for a single free index cc. It does not contain an average such as L(m)=1C∑c=1C1∣sc∣∑x∈Xscb(x)−a(x)max⁡{a(x),b(x)}\mathcal{L}(\mathbf{m})=\frac{1}{C}\sum_{c=1}^{C}\frac{1}{|s_c|}\sum_{\mathbf{x}\in X_{s_c}}\frac{b(\mathbf{x})-a(\mathbf{x})}{\max\{a(\mathbf{x}),b(\mathbf{x})\}}. Therefore, the claimed cross-subsample consensus mechanism is not mathematically specified in the current formulation.

  2. The sampling rule uses ∣R∣π\lvert R\rvert\boldsymbol{\pi}, whereas the theorem and its assumption use RπR\boldsymbol{\pi} and the signed row sums of RR. Non-uniform signed row sums do not imply non-uniform row sums for ∣R∣\lvert R\rvert. Thus, Theorem 1 is not valid for the stated algorithm under its current assumptions.

  3. The monotonicity of the elite threshold LtL_t is asserted but not established for the changing sampling distribution. Full support alone does not imply that the upper quantile eventually reaches the global optimum. Moreover, the assumptions state α→1\alpha\rightarrow1, whereas the proof directly substitutes α=1\alpha=1 after a finite iteration. The experiments instead use a fixed α=0.2\alpha=0.2. The proof also treats L(m)\mathcal{L}(\mathbf{m}) as a fixed deterministic function, although it is estimated using random data subsamples in the actual algorithm. Consequently, the theorem does not cover the implemented setting with finite MM, fixed smoothing, stochastic objectives, and early stopping.

Questions for authors
  1. How exactly are the losses from the CC data subsamples aggregated in Eq. (1)? Are subsamples regenerated for every mask and iteration?

  2. Why does Theorem 1 analyze RπR\boldsymbol{\pi} while the algorithm uses ∣R∣π\lvert R\rvert\boldsymbol{\pi}?

  3. Can the authors provide a genuine sample-complexity result relating the mask budget MM to the probability of sampling near-optimal masks?

  4. What guarantees that the elite thresholds LtL_t are monotonic under the changing correlation-augmented sampling distribution?

  5. How does Theorem 2 apply to the practical setting with M=400M=400, α=0.2\alpha=0.2, a stochastic objective, and early stopping?

  6. Can the authors report complete ablations for the correlation prior, data subsampling, and standard CEM?

7 comments
Reviewer_zwVLOpenReview ↗

ICLR Rebuttal: Please Submit Your Response Soon

This is a reminder that the ICLR rebuttal period is ending soon. I would appreciate it if the authors could submit their response as soon as possible so that I can finalize my review.

AuthorsOpenReview ↗

Response to Reviewer zwVL (1/4)

W1/Q1: The paper introduces CC data subsamples, but Eq. (1) only computes the silhouette score for a single free index cc. The claimed cross-subsample consensus mechanism is not mathematically specified in the current formulation.

We thank R4 for bringing up this point, which was also brought up by R1. The noted error is a typo. The implemented objective is L(m)=1C∑c=1C1∣sc∣∑x∈Xscb(x)−a(x)max⁡(a(x),b(x))\mathcal{L}(\mathbf{m})=\frac{1}{C} \sum_{c=1}^C \frac{1}{|s_c|} \sum_{\mathbf{x} \in X_{s_c}} \frac{b(\mathbf{x})-a(\mathbf{x})}{\max(a(\mathbf{x}),b(\mathbf{x}))}, with C=50C=50. We have corrected this typo in the manuscript. The subsamples are redrawn independently for every mask.

W2/Q2: The sampling rule uses ∣R∣π|R|\pi, whereas the theorem and its assumption use RπR\pi and the signed row sums of RR.

We thank R4 for pointing this out. We have updated Theorem 1 to use ∣R∣|\mathbf{R}| and modified assumptions. This theorem was also extended with a condition under which the optimizer moves to a good mask.

W3/Q3: The monotonicity of the elite threshold LtL_{t} is asserted but not established for the changing sampling distribution. Full support alone does not imply that the upper quantile eventually reaches the global optimum. Moreover, the assumptions state α\alpha approaches 1, whereas the proof directly substitutes α=1\alpha = 1 after a finite iteration. The experiments instead use a fixed α=0.2\alpha = 0.2. The proof also treats the loss as a fixed deterministic function, although it is estimated using random data subsamples in the actual algorithm. Consequently, the theorem does not cover the implemented setting.

We thank R4 for highlighting this point, also brought up by R1. Accordingly, we have updated the theory section of the Appendix to reflect more realistic scenarios. Specifically, Theorem 1 has been expanded beyond arguing a search-space reduction to now including the conditions under which we approach a better mask, and the required sampling budget in terms of a failure probability β\beta. Theorem 2 has been updated to account for the finite-sample scenario. We derive almost-sure convergence and a geometric attraction to strict local optima. We recognize that the updated theorems have tempered the claims in our original manuscript, and we have modified the manuscript text to reflect this change.

Regarding the requested sample-complexity result (Q3), Theorem 1(c) addresses this in terms of a failure probability and the probability of sampling a near-optimal mask, and shows that the correlation prior can reduce the required number of masks. This bound applies to a single iteration, not to the whole run.

A.1 Setup

Masks are m∈M={m∈{0,1}p:∥m∥1=z}m \in \mathcal{M} = \lbrace m \in \lbrace 0,1 \rbrace^p : \Vert m \Vert_1 = z \rbrace. The inclusion probabilities π\pi have entries in (0,1](0,1] that sum to zz, starting from π(0)=(z/p)1\pi^{(0)} = (z/p)\mathbf{1}. Each iteration does three things:

  1. It draws MM masks. Each mask is formed by zz draws without replacement (equivalently, Gumbel top-zz sampling), picking feature ii with probability proportional to wi=πiexp⁡(λci)w_i = \pi_i \exp(\lambda c_i). Here c=∣R∣πc = |R|\pi is the correlation reward of Eq. (2) and λ=λcorr\lambda = \lambda_{\text{corr}}. Pπ,λP_{\pi,\lambda} denotes the law of one mask.
  2. It scores each mask by L(m)=(1/C)∑cSil(m,sc)\mathcal{L}(m) = (1/C) \sum_c \text{Sil}(m, s_c), the silhouette averaged over CC random subsamples.
  3. It updates π←(1−α)π+α⋅(mean of the k=γM best-scoring masks)\pi \leftarrow (1-\alpha)\pi + \alpha \cdot (\text{mean of the } k = \gamma M \text{ best-scoring masks}).

For a set BB of features, the margin is ΔB=min⁡i∈Bci−max⁡j∉Bcj\Delta_B = \min_{i\in B} c_i - \max_{j\notin B} c_j. For a mask mm, the outside mass is Um(π)=∑j∉mπjU_m(\pi) = \sum_{j\notin m} \pi_j.

AuthorsOpenReview ↗

Response to Reviewer zwVL (2/4)

A.2 Theorem 1

Let BB be a set of d≤zd \le z features.

  • (a) At initialization, if the rewards cic_i are not all equal, the prior (λ>0\lambda > 0) strictly lowers the entropy of the sampler compared with λ=0\lambda = 0, which is uniform over masks.
  • (b) If ΔB≥0\Delta_B \ge 0, then at every draw, given the features drawn so far, the odds that the next feature lies in BB are at least eλΔBe^{\lambda\Delta_B} times their value at λ=0\lambda = 0. For every rr, the probability that the mask contains at least rr features of BB is at least as large as at λ=0\lambda = 0. If every feature outside BB has a reward at least max⁡i∈Bci\max_{i \in B} c_i instead, both statements hold in reverse. In other words, the prior amplifies whichever group of features has the highest reward, including a nuisance module.
  • (c) M≥log⁡(1/β)/qM \ge \log(1/\beta)/q masks suffice to sample, with probability at least 1−β1-\beta, at least one mask from a set GG of masks of probability qq. For G={B}G = \lbrace B \rbrace with d=zd = z and ΔB≥0\Delta_B \geq 0, the prior multiplies this budget by at most the factor [(1−u)+u⋅e−λΔB]z[(1-u) + u \cdot e^{-\lambda\Delta_B}]^z, where u=UB(π)/zu = U_B(\pi)/z. This is a statement about one iteration, not a budget for the whole run.

Proof sketch.

  • (a) At π(0)\pi^{(0)} and λ=0\lambda = 0 every mask is equally likely, so the entropy equals log⁡(pz)\log \binom{p}{z} minus the KL divergence from the uniform law. An exchange argument on the Gumbel noise shows that a feature with a larger reward is included strictly more often. So the sampler is not uniform, and the divergence is positive.
  • (b) The weights in BB are multiplied by at least eλmin⁡Bce^{\lambda \min_B c} and those outside by at most eλmax⁡Bcce^{\lambda \max_{B^c} c}, so the odds ratio is at least eλΔBe^{\lambda\Delta_B}. Coupling both samplers on the same noise, the prior raises every score in BB relative to every score outside it, which can only increase ∣m∩B∣|m \cap B|.
  • (c) We have (1−q)M≤e−qM(1-q)^M \le e^{-qM}. For G={B}G = \lbrace B \rbrace, m=Bm = B exactly when every Gumbel score in BB beats every score outside BB, so P(m=B)P(m = B) can only drop if we lower the weights in BB to πi\pi_i and raise those outside to e−λΔBπje^{-\lambda\Delta_B}\pi_j (after rescaling). This sampler gives every ordering of BB the same weights as at λ=0\lambda = 0, with less weight outside BB, so we can compare the two order by order and bound each of the zz factors.
Reviewer_e8EhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes CLASP, an unsupervised feature selection algorithm with a particular focus on biomedical applications and high-dimensional data regimes.

The focus of the algorithm is on feature selection stability and cluster separability to circumvent reproducibility issues. The authors provide a simulation study and an evaluation across seven diverse biomedical datasets.

Critical strengths

The paper addresses a highly relevant and important topic.

An evaluation across 7 different data sets in principle highlights the breath and utility of the method. The stochastic formulation of feature selection through cluster separability is interesting, particularly for high-dimensional, low-sample-size datasets.

Critical weaknesses

The objective maximizes silhouette scores across subsamples but does not explicitly enforce agreement between their cluster assignments. Therefore, consistently separable clusters do not necessarily imply reproducible partitions, which challenges one of the main methodological claims.

Another key premise of the paper of not identifying spurious features is not shown or really addressed. Feature stability and cluster separability do not solve this. A common confounding factor in biomedical datasets are cohort or batch effects. These are often the strongest factors in an initial variability or cluster analysis. The paper does not address this issue at all - neither in the simulation nor in the applications and it is not clear how the approach would overcome this. The likely case is that the proposed method would actually target and pick these spurious effects.

While feature stability is desirable, a common goal is to also avoid selecting redundant features. Similarly, one is often interested in the axes of highest variation. The author’s correlation prior may preferentially select redundant features belonging to the same correlated module rather than complementary biological signals. There are obviously certain tradeoffs but the authors do not really investigate/address this here.

The approach shows better performance to comparators but that is also expected as the evaluation criteria used are exactly those that the method optimizes for. This may be fine, but again it is not clear nor do the authors properly make the case that this is really what one wants. Along these lines, the authors do not really show how one can use their algorithm to choose the number of features or cluster size with their algorithm which are critical aspects for any of those methods.

The method fixes the number of selected features to 100, while the synthetic setup in Appendix A.3 specifies 20 informative features. It is unclear whether these settings are used together and how recovery is calculated. Recovering all informative features does not necessarily imply high precision if many irrelevant features are also selected.

Overall this is not an extensive or rigorous simulation study. The chosen setup is trivial and favourable to their approach. At least different correlation structures, spurious clusters, etc. should be evaluated along with the sensitivity of their approach to feature size, cluster size etc. Further,It would be important to also evaluate whether CLASP identifies artificial but stable clusters in homogeneous populations or under technical confounding.

There is no clear analysis of the scalability of their approach. The approach still requires a dense pxp matrix calculation (R).

There is no proper description of the data processing steps.

Little sensitivity to clustering algorithms is claimed but never shown.

The evaluation on the real data sets is rather weak. First, uncertainty/variance estimates should be provided where possible. Given the sample size and 5 fold cross-validation it is likely indiscernible from the other methods in many cases. Second, the claims about reproducibility should have been shown here - how well does the algorithm perform when evaluated across different cohorts, hospitals/centers etc. - these splits are exactly where reproducibility issues typically come up, not random data splits. The authors do not evaluate the selected features. Across those studies does the approach actually identify the relevant known features, i.e. does the approach recover known and relevant biology?

Questions for authors

How does the proposed objective ensure clustering consensus rather than simply maximizing separability?

How does the approach deal with potential confounding sources of variation e.g. batch, cohort effects?

How does your method deal with the fact that selecting overly redundant features is a common aspect avoided by other methods and it seems your method is explicitly encouraging this?

How does your method compare on other commonly used metrics that your method is not explicitly optimizing for e.g. What's the fraction of variance explained by the feature set? How well do the selected features recover the full data cluster structures?

Can you use your approach to select the optimal number of clusters and feature set size ? If not it would be interesting to see how the model performs (the sensitivity analysis in the appendix shows this partially but this would need to be substantially worked out).

How is feature recovery defined, and how does performance change when the selected feature count differs from the true number of informative features if available?

How does the approach fare under more challenging simulation settings? How does CLASP behave when clusters are driven by technical/ batch/cohort effects or when no genuine cluster structure exists?

Does complexity estimate include the clustering algorithm?

How was the data processed ?

How would the approach fare in reproducing findings across real world settings (different studies, cohorts, experiments etc.) ?

Across those studies does the approach actually identify the relevant known features, i.e. does the approach recover known and relevant biology (comparatively to other approaches as well) ?

How would the methods compare when other clustering approaches are used ?

8 comments
AuthorsOpenReview ↗

Response to Reviewer e8Eh (1/8)

W1/Q1: The objective maximizes silhouette scores across subsamples but does not explicitly enforce agreement between cluster assignments. Therefore, consistently separable clusters does not necessarily imply reproducible partitions, which challenges one of the main methodological claims.

We appreciate R3 bringing up this point, and acknowledge that Eq. (1) does not include a term that directly compares the clustering assignments. Our claim in the paper relies on a link between separation and agreement that we had not stated.

Namely, the clustering instability is bounded by the probability mass close to the cluster boundaries, as shown in [1], which is indirectly limited by a high silhouette score. The reason we chose to optimize silhouette score, as opposed to a direct cluster matching, is the continuous nature of the former. In the small sample size regime, a single cluster assignment flip causes a large impact on the loss function, which can lead to instability.

We will add a sentence clarifying the above relationship to the manuscript.

[1] Ben-David, S., & Von Luxburg, U. (2008, July). Relating clustering stability to properties of cluster boundaries. In the 21st annual conference on learning theory (COLT 2008) (pp. 379-390). Omnipress.


W2/Q2: Another key premise of the paper is that identifying spurious features is not shown or really addressed. Feature stability and cluster separability do not solve this. A common confounding factor in biomedical datasets are cohort or batch effects. These are often the strongest factors in an initial variability or cluster analysis. The paper does not address this issue at all - neither in the simulation nor in the applications and it is not clear how the approach would overcome this. The likely case is that the proposed method would actually target and pick these spurious effects.

We thank R3 for bringing up this point of discussion. We recognize that a susceptibility to confounding factors is a weakness of CLASP but note that all the UFS baseline approaches also have this limitation [1]. In fact, batch effects in biomedical data are often addressed using preprocessing techniques, such as ComBat [2] and the removal of principal components associated with technical artifacts [3], as it is difficult for data-driven methods to differentiate between batch effects and true correlational structure [4].

Given this comment, we have conducted two new experiments to evaluate the impact of batch effects on CLASP: (1) feature recovery on simulation data altered to simulate different batches, and (2) stability metrics across ADHD-200 sites when the feature selection method is trained on one site and applied to the others. Summarized results for both of these experiments are available in the tables below. The complete experimental setup and results have been added in Section A.6 and in Tables 7, 8, and 9 of the Appendix.

Setup (1). We extend the simulation of Sec. 4.1 by splitting the samples into two equal batches, assigned independently of the clusters, and shifting a block of 20 correlated noise features between the batches. The batch shift δb\delta_b is expressed as a percentage of the cluster separation.

Batch effect independent of the clusters (a 20-feature block shifted between two batches, δb\delta_b as a percentage of the cluster separation): recall of the 20 informative features, mean of 5 simulated data sets (ρ=0.5\rho=0.5, γ=0.5\gamma=0.5, z=20z=20). CAE selects at chance level (expected recall z/p=0.01z/p=0.01).

Methodnone40%80%160%
Laplacian0.940.620.510.00
SPEC0.940.620.510.00
ICA0.240.220.260.24
MCFS0.090.160.190.00
CAE0.000.000.010.00
GroupFS0.760.330.560.00
SFS-MWK++0.450.420.580.08
CLASP1.001.001.000.00

Setup (2). Each method selects features on the NYU site, as in Sec. 4.2. Without re-selecting, we use these features to cluster the subjects of each of the seven other ADHD-200 sites and compute DBI and ARI per site.

Transfer across ADHD-200 sites. Features selected on NYU (K=2K=2, z=100z=100, 5 folds) are applied without re-selection to the 7 other sites (standardised within site): DBI and ARI consensus, mean over the 7 sites and worst site. Best in bold; within 5% of the best in italics.

MethodDBI ↓ meanDBI ↓ worstARI ↑ meanARI ↑ worst
Laplacian1.4882.4140.6400.307
SPEC1.4962.3260.5950.317
ICA1.8582.5450.5350.238
MCFS2.3032.9500.3490.279
CAE2.6143.8130.1320.066
GroupFS2.6323.9030.1670.038
SFS-MWK++1.5742.5020.6450.371
CLASP1.4612.2170.6390.395

(W2/Q2 continued in the next comment)

AuthorsOpenReview ↗

Response to Reviewer e8Eh (2/8)

W2/Q2 (continued)

With this in mind, we have added a batch-based sampling scheme as an optional setting in CLASP. Given user-provided batch labels, this setting will use batch-aware subsamples of the data in the stochastic optimization to prevent batch effects from dominating. We have evaluated this batch-based sampling scheme on simulated data under different batch shift percentages δb\delta_b. Full details are available in the Appendix, and a summary of the results is available in the table below.

Setup (3). We use the same simulated data as in Setup (1). With batch-aware subsampling, each of CLASP's subsamples is drawn from a single batch, alternating between the two batches.

Batch-aware subsampling (each of the CC subsamples drawn within one batch). Simulated batch shift δb\delta_b as a percentage of the cluster separation, recall of the 20 informative features (mean of 5 data sets).

Methodδb\delta_b = 40%80%160%
Laplacian0.620.510.00
SFS-MWK++0.420.580.08
CLASP (pooled subsamples)1.001.000.00
CLASP (within-batch subsamples)1.001.001.00

[1] Weinberger, E., Covert, I., & Lee, S.-I. (2023). Feature selection in the contrastive analysis setting. NeurIPS.

[2] Johnson, W.E., Li, C., & Rabinovic, A. (2007). Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics, 8(1), 118–127.

[3] Alter, O., Brown, P.O., & Botstein, D. (2000). Singular value decomposition for genome-wide expression data processing and modeling. PNAS, 97(18), 10101–10106.

[4] Nygaard, V., Rødland, E.A., & Hovig, E. (2016). Methods that remove batch effects while retaining group differences may lead to exaggerated confidence in downstream analyses. Biostatistics, 17(1), 29–39.


W3/Q3: While feature stability is desirable, a common goal is to also avoid selecting redundant features. Similarly, one is often interested in the axes of highest variation. The author's correlation prior may preferentially select redundant features belonging to the same correlated module rather than complementary biological signals. There are obviously certain tradeoffs but the authors do not really investigate/address this here.

We thank R3 for bringing up this point of discussion, also brought up by R1. The reviewer is correct that our correlation reward encourages sampling toward correlated groups of features. However, we believe this property is desirable for three reasons: (1) it makes the combinatorial nature of the unsupervised feature selection algorithm feasible, as shown empirically by the experiments and new ablations in the appendix; (2) many real-world biomedical applications are driven by networks of biomarkers [1] [2], and (3) imposing a “winner take all” selection procedure amongst the correlated features runs the risk of overfitting to noise and finite sample effects [3].

While the reviewer is correct that other methods pursue different objectives, such as variance maximization and graph smoothness, we introduce CLASP as an alternative objective for data with a suspected clustering structure based on sparse feature networks. Biomedical tasks often rely on this assumption, so we believe CLASP addresses this gap in the literature.

[1] Ye, J., & Liu, J. (2012). Sparse Methods for Biomedical Data. SIGKDD explorations : newsletter of the Special Interest Group (SIG) on Knowledge Discovery & Data Mining, 14(1), 4–15. https://doi.org/10.1145/2408736.2408739

[2] Chuang, H.-Y., Lee, E., Liu, Y.-T., Lee, D., & Ideker, T. (2007). Network-based classification of breast cancer metastasis. Molecular Systems Biology, 3, 140. https://doi.org/10.1038/msb4100180

[3] Tolosi, L., & Lengauer, T. (2011). Classification with correlated features: unreliability of feature ranking and solutions. Bioinformatics, 27(14), 1986–1994. https://doi.org/10.1093/bioinformatics/btr300

AuthorsOpenReview ↗

Response to Reviewer e8Eh (3/8)

W4/Q4/Q5: The approach shows better performance to comparators but that is also expected as the evaluation criteria used are exactly those that the method optimizes for. This may be fine, but again it is not clear nor do the authors properly make the case that this is really what one wants. Along these lines, the authors do not really show how one can use their algorithm to choose the number of features or cluster size with their algorithm which are critical aspects for any of those methods.

We appreciate the points of concern from R3 regarding the evaluation criteria. While the metrics are aligned with a separability objective, we emphasize that DBI and ARI are computed on held-out folds, and the objective does not directly optimize the Nogueira index. We also note that these metrics are standard in the UFS literature and often used to compare methods [1] [2] [3]. Once again, we recognize that CLASP optimizes a different objective than current variance maximizing methods. Thus, we have introduced CLASP as an alternative objective for data with a suspected clustering structure based on sparse feature networks. Biomedical tasks often rely on this assumption, so we believe CLASP addresses this gap in the literature.

To further address the concern, we have included a new metric: the variance of the full data explained by the selected features, through the held-out R2R^2 of a ridge reconstruction. This analysis is reported in Table 12 of the Appendix. As expected, the baselines outperform CLASP on this metric. However, we emphasize that CLASP is not designed to reconstruct the signal, but rather to find features that make the data consistently separable, regardless of contributed variance.

With regards to the clustering configuration, the user must select the number of clusters, as is standard for most clustering algorithms. We have also added Table 3 to the Appendix that sweeps the percentage of retained features zz = {1%, 2%, 5%, 10%, 20%} as a function of the original data dimensionality. A summary of the results is included in the table below. We observe that CLASP almost uniformly achieves the best Nogueira’s index and ARI consensus. It is also consistently the second-best performer in DBI.

The variability in all three metrics across the feature sparsity zz makes it challenging to provide an a priori heuristic. For example, larger values of zz tend to achieve higher Nogueira’s index but lower ARI, perhaps because the additional features are less informative and dilute the cluster structure, even though larger sets overlap more across folds. Thus, we believe that the user is best suited to determine the desired feature sparsity for their application domain.

[1] Hancer, E., Xue, B., & Zhang, M. (2020). A survey on feature selection approaches for clustering. Artificial Intelligence Review, 53(6), 4519–4545.

[2] Solorio-Fernández, S., Carrasco-Ochoa, J.A., & Martínez-Trinidad, J.F. (2020). A systematic evaluation of filter unsupervised feature selection methods. Expert Systems with Applications, 162, 113745.

[3] Nogueira, S., Sechidis, K., & Brown, G. (2018). On the stability of feature selection algorithms. Journal of Machine Learning Research, 18(174), 1–54.

(W4/Q4/Q5 continued in the next comment)

Reviewer_5eDaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper propose a new unsupervised feature selection method for biomarker detection. More specifically, the paper propose Correlation-Leveraged Across-sample Separability Prioritization (CLASP) that identifies reproducible feature subsets by maximizing cross-subsample cluster separability. Through experiments, the proposed method outperforms existing unsupervised feature selection methods.

Critical strengths

Based on the comparisons, the proposed method outperforms baselines.

Critical weaknesses
  1. For the feature recovery experiments, the proposed method gets 1 for all the cases, while existing methods are low. How this is possible?
  2. The baseline methods are old methods except for GroupFS (Lifshitz et al. (2026)). It should be compared with more recent methods.
Questions for authors
  1. For the feature recovery experiments, the proposed method gets 1 for all the cases, while existing methods are low. How this is possible?
  2. The baseline methods are old methods except for GroupFS (Lifshitz et al. (2026)). It should be compared with more recent methods.
1 comment
AuthorsOpenReview ↗

Response to Reviewer 5eDa

W1/Q1: For the feature recovery experiments, the proposed method gets 1 for all the cases, while existing methods are low. How is this possible?

We appreciate R2's point of concern and clarify the simulation experiments here. The results follow from how simulated data is generated. Only dd signal features carry the mean shift between clusters, the distractor block is strongly correlated noise with no cluster information, and the remaining features are independent noise. Baseline methods that rely on selection criteria such as variance maximization or graph smoothness gravitate towards highly-correlated distractor blocks. In contrast, gradient-based methods operate close to chance in this data regime. CLASP scores a feature set by how well it separates the samples, which is the one property the signal block has by construction. Thus, it achieves the best performance.

W2/Q2: The baseline methods are old methods except for GroupFS. It should be compared with more recent methods.

We thank R2 for this suggestion, also brought up by R1. Accordingly, we have included a new baseline method for all simulation and real-world biomedical experiments. This baseline, called SFS-MWK++ [4], selects features whose Minkowski weighted k-means weights remain consistently high across a range of Minkowski exponents, making it a direct stability-oriented competitor to CLASP in high dimensions.

The table below summarizes the performance of SFS-MWK++ against CLASP across the real-world datasets. The results demonstrate that CLASP achieves a higher average Nogueira’s index (0.778 vs 0.490) and is uniformly better across the 7 biomedical datasets. CLASP also achieves a higher ARI consensus (0.672 vs 0.489) and performs better on 5/7 biomedical datasets. Finally, SFS-MWK++ attains lower mean DBI (0.707 vs 0.854), but the best performing method is split across the datasets. This advantage comes from three datasets where SFS-MWK++ isolates a few outlying samples into their own cluster, which DBI rewards even though the split is not meaningful. Overall, we conclude that CLASP is more robust than SFS-MWK++ across the different real-world settings.

The updated results including SFS-MWK++ can be found in the revised manuscript.

Repeated 5-fold CV (5 repetitions), K=2, z=100: mean ± SD over repetitions of the 7-dataset average, and number of datasets (of 7) on which the method is best against all baselines.

MethodNogueira ↑ (mean ± SD)bestDBI ↓ (mean ± SD)bestARI consensus ↑ (mean ± SD)best
SFS-MWK++0.490 ± 0.00900.707 ± 0.04030.489 ± 0.0141
CLASP0.778 ± 0.01670.854 ± 0.03630.672 ± 0.0285

[4] Zhang, X., & Cordeiro de Amorim, R. (2026). Scalable unsupervised feature selection via weight stability. Information Sciences, 755, 123807. https://doi.org/10.1016/j.ins.2026.123807

Reviewer_VdqDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CLASP, an unsupervised feature selection method tailored to high-dimensional, low-sample-size biomedical data. CLASP casts feature selection as maximizing cross-subsample clustering separability (average silhouette) via a stochastic optimization using the Cross-Entropy Method and a correlation-augmented Gumbel Top‑k sampling scheme over feature inclusion masks. Across controlled simulations and seven biomedical datasets, the authors report notable gains in feature-set stability, clustering quality, and out-of-sample clustering stability compared to several classical and modern baselines.

Critical strengths
  • The central idea of optimizing feature subsets to maximize clustering separability consistently across data subsamples (rather than fitting a single partition) is well-motivated for p ≫ n biomedical regimes.
  • The combination of correlation-augmented Gumbel Top‑k sampling with CEM for a non-differentiable objective is an interesting and pragmatic optimization design that reduces the effective search space.
  • The study covers a broad set of datasets across modalities (fMRI connectivity, microarrays, proteomics, metabolomics, neuroimaging tabular features) and evaluates multiple unsupervised criteria: feature stability (Nogueira), cluster quality (DBI), and clustering stability (ARI consensus).
Critical weaknesses
  • The “convergence” and “search-space contraction” arguments rely on strong/idealized assumptions (e.g., M → ∞, unique global optimum, α → 1) and do not provide actionable finite-sample guarantees or sample-complexity bounds.
  • The correlation reward (|R| π) systematically biases sampling toward correlated groups and “hub” features; while this shrinks search, it may also encourage redundancy in the selected set unless explicitly counteracted. There is limited analysis of redundancy or degree bias in selected features.
  • The objective in Eq. (1) is presented at the subsample level; the paper states maximizing the average silhouette across subsamples but does not explicitly show the averaging operator in the main text, creating ambiguity.
  • Important recent baselines that emphasize stability or unsupervised selection in high dimensions (e.g., SFS‑MWK++, DeepFS, LKFS) are not included, despite relevance and reported scalability strategies; this limits the strength of empirical claims.
  • External validity is not assessed on datasets with known subtypes (e.g., gene-expression cohorts such as ALLAML and GLI 85) using cluster-label concordance (e.g., ARI/NMI to ground truth), nor is biological plausibility probed (e.g., pathway enrichment), which is central for biomarker discovery claims.
  • A fixed selection size z = 100 is used across datasets with widely varying dimensionalities and sample sizes; the impact of z (and a principled way to choose it) is underexplored.
Questions for authors

Please clarify the exact definition of the separability loss: is L(m) averaged over C subsamples (i.e., L(m) = (1/C) Σc Silhouette(m, sc))? What subsample fraction ρ and C were used, and how sensitive are results to these choices? How does λcorr affect redundancy and hub bias? Could you report the mean pairwise correlation among selected features vs λcorr (and λcorr = 0), and provide an ablation showing stability/DBI/ARI as a function of λcorr? Why was z fixed to 100 across all datasets? Can you provide a sensitivity sweep over z and, ideally, an adaptive z-selection heuristic (e.g., stability plateaus or elbow methods)? Spectral clustering details matter (affinity construction, neighbors, scaling). What exact settings were used, and how robust are results to these choices? Have you tried k-means as the backend to test algorithmic dependence? For datasets with known subtypes (e.g., ALLAML, GLI 85), can you report ARI/NMI between clusters (from selected features) and known labels? Any pathway enrichment or overlap with known biomarkers? Could you compare against at least one of DeepFS, LKFS, or SFS‑MWK++ on a subset of datasets, or provide a strong rationale (runtime/memory) for their exclusion? What are the actual wall‑clock runtimes and memory usage for the largest datasets (e.g., p ≈ 22k) and how do they compare to baselines? Is the O(np^2) correlation computation the main bottleneck? Did you explore degree-normalized or sign-aware correlation rewards (e.g., using row-normalized R or partial correlations) to mitigate hub dominance or confounding? Any empirical differences?

4 comments
AuthorsOpenReview ↗

Response to Reviewer VdqD (2/5)

W2/Q3: The correlation reward systematically biases sampling toward correlated groups and "hub" features; while this shrinks search, it may also encourage redundancy in the selected set unless explicitly counteracted. There is limited analysis of redundancy or degree bias in selected features.

We thank R1 for bringing up this point of discussion, also brought up by R3. The reviewer is correct that our correlation reward encourages sampling toward correlated groups of features. However, we believe this property is desirable for three reasons: (1) it makes the combinatorial nature of the unsupervised feature selection algorithm feasible, as shown empirically by the experiments and new ablations in the appendix; (2) many real-world biomedical applications are driven by networks of biomarkers [1] [2], and (3) imposing a “winner take all” selection procedure amongst the correlated features runs the risk of overfitting to noise and finite sample effects [3].

An ablation across different values of λcorr\lambda_{corr} in the biomedical datasets was already provided in the Appendix, and we now add the mean pairwise correlation (absolute value) and effective rank. We have moved the results to a table format (Table 2 in the Appendix) for clarity.

We acknowledge that finding uncorrelated features could be preferential in certain cases. Further analysis on a negative λcorr\lambda_{corr} is left as future work.

Our future work will explore the use of different priors based on domain knowledge or feature properties in order to subselect amongst correlated groups. We will update the manuscript to include the above discussion and proposed future work.

Correlation prior λcorr\lambda_{\rm corr}: redundancy of the 100 selected features (mean pairwise ∣r∣\vert r \vert; effective rank of their correlation matrix) and quality, mean over ADHD-200, ALLAML, CPTAC-BRCA, GLI-85 (K=2K{=}2, paper's split).

mean ∣r∣\vert r \verteff. rankNogueira ↑\uparrowDBI ↓\downarrowARI ↑\uparrow
λcorr=0\lambda_{\rm corr}=00.2922.20.081.570.32
λcorr=0.1\lambda_{\rm corr}=0.10.725.10.750.960.65
λcorr=0.3\lambda_{\rm corr}=0.3 (default)0.725.00.800.950.64
λcorr=0.5\lambda_{\rm corr}=0.50.725.00.710.970.61

[1] Ye, J., & Liu, J. (2012). Sparse Methods for Biomedical Data. SIGKDD explorations : newsletter of the Special Interest Group (SIG) on Knowledge Discovery & Data Mining, 14(1), 4–15. https://doi.org/10.1145/2408736.2408739

[2] Chuang, H.-Y., Lee, E., Liu, Y.-T., Lee, D., & Ideker, T. (2007). Network-based classification of breast cancer metastasis. Molecular Systems Biology, 3, 140. https://doi.org/10.1038/msb4100180

[3] Tolosi, L., & Lengauer, T. (2011). Classification with correlated features: unreliability of feature ranking and solutions. Bioinformatics, 27(14), 1986–1994. https://doi.org/10.1093/bioinformatics/btr300

W3/Q1: The objective in Eq. (1) is presented at the subsample level; the paper states maximizing the average silhouette across subsamples but does not explicitly show the averaging operator in the main text, creating ambiguity.

We thank R1 for bringing up this point, which was also brought up by R4. The noted error is a typo. The implemented objective is L(m)=1CΣc=1C1∣sC∣Σx∈XsCb(x)−a(x)max⁡{a(x),b(x))\mathcal{L}(\mathbf{m})=\frac{1}{C} \Sigma_{c=1}^C \frac{1}{|s_C|} \Sigma_{\mathbf{x} \in X_{s_C}} \frac{b(\mathbf{x})-a(\mathbf{x})}{\max \lbrace a(\mathbf{x}),b(\mathbf{x}))}, with C=50C=50. We have corrected this typo in the manuscript.

W4/Q7: Important recent baselines that emphasize stability or unsupervised selection in high dimensions are not included, despite relevance and reported scalability strategies; this limits the strength of empirical claims.

We thank R1 for this suggestion, also brought up by R2. Accordingly, we have included a new baseline method for all simulation and real-world biomedical experiments. This baseline, called SFS-MWK++ [4], selects features whose Minkowski weighted k-means weights remain consistently high across a range of Minkowski exponents, making it a direct stability-oriented competitor to CLASP in high dimensions.

The table below summarizes the performance of SFS-MWK++ against CLASP across the real-world datasets. The results demonstrate that CLASP achieves a higher average Nogueira’s index (0.778 vs 0.490) and is uniformly better across the 7 biomedical datasets. CLASP also achieves a higher ARI consensus (0.672 vs 0.489) and performs better on 5/7 biomedical datasets. Finally, SFS-MWK++ attains lower mean DBI (0.707 vs 0.854), but the best performing method is split across the datasets. This advantage comes from two datasets where SFS-MWK++ isolates a few outlying samples into their own cluster, which DBI rewards even though the split is not meaningful. Overall, we conclude that CLASP is more robust than SFS-MWK++ across the different real-world settings.

AuthorsOpenReview ↗

Response to Reviewer VdqD (3/5)

The updated results including SFS-MWK++ can be found in the revised manuscript.

Repeated 5-fold CV (5 repetitions), K=2K{=}2, z=100z{=}100: mean ±\pm SD over repetitions of the 7-dataset average, and number of datasets (of 7) on which the method is best.

MethodNogueira ↑\uparrow mean ±\pm SDNogueira ↑\uparrow bestDBI ↓\downarrow mean ±\pm SDDBI ↓\downarrow bestARI consensus ↑\uparrow mean ±\pm SDARI consensus ↑\uparrow best
SFS-MWK++0.490 ±\pm 0.00900.707 ±\pm 0.04030.489 ±\pm 0.0141
CLASP0.778 ±\pm 0.01670.854 ±\pm 0.03630.672 ±\pm 0.0285

[4] Zhang, X., & Cordeiro de Amorim, R. (2026). Scalable unsupervised feature selection via weight stability. Information Sciences, 755, 123807. https://doi.org/10.1016/j.ins.2026.123807

W5/Q6: External validity is not assessed on datasets with known subtypes using cluster-label concordance, nor is biological plausibility probed, which is central for biomarker discovery claims.

We thank R1 for this discussion. The ALLAML and GLI-85 datasets include binary (diagnostic) labels. ALLAML contains bone-marrow and blood microarray profiles (7,129 probes) of 72 acute leukemia patients labeled as acute lymphoblastic (ALL, n=47) or acute myeloid leukemia (AML, n=25), and GLI-85 contains tumor microarray profiles (22,283 probes) of 85 gliomas labeled by histological grade (grade III vs. grade IV). Therefore, we have added the ARI between the clustering results for K=2K=2 and these ground-truth labels to Table 5 of the Appendix. We note that the ARI is uniformly low across all methods (ARI 0.01-0.14, SD 0.12-0.25) with overlapping standard deviations across them. This result suggests that no method can separate the diagnostic classes in an unsupervised fashion.

There are few interpretations for this result. First, there may be a weak relationship between the supervised metric and the raw data in the p>>np>>n scenario due to noise, environmental confounds and finite sampling effects. Second, it is unclear that the ground-truth labels are derived from the same data provided in ALLAML and GLI-85. For example, experts often have access to auxiliary information, like imaging and medical records, which are used to determine diagnostic categories. Therefore, the information in the data that drives the clustering might be incomplete with respect to the supervised labels.

Agreement of the held-out clustering (K=2K{=}2, z=100z{=}100) with the known classes: ARI and NMI, mean ±\pm SD over 25 held-out folds (5 repetitions ×\times 5 folds). Labels are never used for selection.

MethodALLAML (ALL/AML) ARIALLAML (ALL/AML) NMIGLI-85 (two classes) ARIGLI-85 (two classes) NMI
Laplacian0.111$\pm$0.170.142$\pm$0.170.086$\pm$0.140.069$\pm$0.09
SPEC0.129$\pm$0.190.172$\pm$0.170.083$\pm$0.150.077$\pm$0.09
ICA0.073$\pm$0.140.114$\pm$0.130.068$\pm$0.170.129$\pm$0.16
MCFS0.066$\pm$0.140.138$\pm$0.100.089$\pm$0.240.170$\pm$0.20
CAE0.047$\pm$0.170.131$\pm$0.120.141$\pm$0.240.186$\pm$0.19
GroupFS0.012$\pm$0.130.147$\pm$0.120.077$\pm$0.150.123$\pm$0.10
SFS-MWK++0.037$\pm$0.120.111$\pm$0.090.043$\pm$0.140.138$\pm$0.11
CLASP0.130$\pm$0.160.151$\pm$0.150.070$\pm$0.150.077$\pm$0.09

W6/Q4: A fixed selection size z=100z = 100 is used across datasets with widely varying dimensionalities and sample sizes; the impact of z is underexplored.

We appreciate this point brought up by R1. We have added Table 3 to the Appendix that sweeps the percentage of retained features $z=\lbrace 1\%,2\%,5\%,10\%,20\%\rbrace $ as a function of the original data dimensionality. A summary of the results is included in the table below. We observe that CLASP almost uniformly achieves the best Nogueira’s index and ARI consensus. It is also consistently the second-best performer in DBI.

Given the variability in all three metrics across selected feature set size, we do not believe it appropriate to provide a heuristic for the value of zz. For example, larger values of zz tend to achieve higher Nogueira’s index but lower ARI, perhaps because the additional features are less informative and dilute the cluster structure, even though larger sets overlap more across folds. Thus, we believe that the user is best suited to determine the desired feature sparsity for their application domain.

AuthorsOpenReview ↗

Response to Reviewer VdqD (4/5)

Effect of the number of selected features zz (1–20% of pp; K=2K{=}2): mean over the 7 datasets. Best in bold, second best underlined.

Method1%2%5%10%20%
Nogueira stability ↑\uparrow
Laplacian0.621‾\underline{0.621}0.6700.682‾\underline{0.682}0.711‾\underline{0.711}0.731
SPEC0.5810.648‾\underline{0.648}0.6690.7090.740‾\underline{0.740}
ICA0.2680.3090.3870.4580.542
MCFS0.0930.1230.1590.1790.199
CAE0.0500.0810.1600.2190.274
GroupFS0.0490.0600.0610.0720.122
SFS-MWK++0.5260.5340.5600.5820.586
CLASP0.6250.6410.7120.7700.814
DBI ↓\downarrow
Laplacian0.9551.0311.2151.3511.577
SPEC0.9090.9831.1581.2921.545
ICA1.1901.2441.3551.4921.581
MCFS1.0411.1261.3391.4521.454
CAE1.5191.4961.6391.6611.623
GroupFS1.2491.3191.1981.3821.402
SFS-MWK++0.6860.7300.8290.9521.198
CLASP0.871‾\underline{0.871}0.925‾\underline{0.925}1.094‾\underline{1.094}1.252‾\underline{1.252}1.288‾\underline{1.288}
ARI consensus ↑\uparrow
Laplacian0.5860.5870.4750.4120.352
SPEC0.623‾\underline{0.623}0.589‾\underline{0.589}0.494‾\underline{0.494}0.4350.375
ICA0.4800.4550.4040.3520.343
MCFS0.5210.5210.4530.3980.382
CAE0.2890.3170.2900.2430.255
GroupFS0.3210.3120.3550.3560.341
SFS-MWK++0.4930.5090.4910.475‾\underline{0.475}0.410‾\underline{0.410}
CLASP0.6370.6380.5720.5230.460

Q2: What subsample fraction ρ\rho and CC were used, and how sensitive are results to these choices?

We fix the values ρ=⌊0.20ntrain⌋\rho=\lfloor 0.20 n_{train} \rfloor and C=50C=50 throughout the paper. We have included sensitivity results with respect to these parameters on simulated data in the Appendix (Figure 6). The results are not sensitive to ρ\rho with stable performance up until ρ=0.8\rho=0.8. Sensitivity to CC is also low for C>1C>1.

Q5: Spectral clustering details matter. What exact settings were used, and how robust are results to these choices? Have you tried k-means as the backend to test algorithmic dependence?

We thank R1 for pointing out this omission. The settings for spectral clustering (scikit-learn 1.7.2) are as follows: RBF affinity with γrbf=1/(2σ2)\gamma_{rbf}=1/(2\sigma^2), where σ\sigma is the median pairwise Euclidean distance among the training samples in the masked feature space (computed once per mask); kk-means label assignment with a single initialization; features are z-scored on the training fold (no standardization in the simulations).

Based on this comment, we have applied CLASP with three different clustering backends (spectral, kk-means, and Ward linkage) across the seven biomedical datasets. Results show comparable performance across the backends, with spectral clustering arguably providing the most robust performance across the settings. However, we will update our code to allow the user to select a backend clustering algorithm.

Performance comparison across clustering backends inside CLASP (spectral = paper; kk-means; Ward), K=2K{=}2, z=100z{=}100. The evaluation folds are identical for all backends. Best in bold (ties all bolded); values within 0.01 of the best underlined.

DatasetNogueira ↑\uparrow Spec.Nogueira ↑\uparrow kk-m.Nogueira ↑\uparrow WardDBI ↓\downarrow Spec.DBI ↓\downarrow kk-m.DBI ↓\downarrow WardARI consensus ↑\uparrow Spec.ARI consensus ↑\uparrow kk-m.ARI consensus ↑\uparrow Ward
ALLAML0.720.720.720.850.850.850.520.500.50
GLI-850.820.340.340.580.730.730.680.600.60
CPTAC-BRCA0.850.850.851.04‾\underline{1.04}1.031.030.69‾\underline{0.69}0.700.69‾\underline{0.69}
ADHD-2000.730.84‾\underline{0.84}0.851.301.28‾\underline{1.28}1.270.600.61‾\underline{0.61}0.62
ME/CFS0.830.830.830.440.440.440.770.76‾\underline{0.76}0.76‾\underline{0.76}
NCI-600.720.720.71‾\underline{0.71}0.700.68‾\underline{0.68}0.670.680.730.73
TADPOLE0.860.860.861.301.301.300.740.740.74
Mean0.790.740.740.890.90‾\underline{0.90}0.90‾\underline{0.90}0.670.66‾\underline{0.66}0.66‾\underline{0.66}
AuthorsOpenReview ↗

Response to Reviewer VdqD (5/5)

Q8: What are the actual wall-clock runtimes and memory usage for the largest datasets, and how do they compare to baselines? Is the O(np2)O(np^{2}) correlation computation the main bottleneck?

We appreciate R1's concern for runtimes and memory usage, as they can be concerns for sampling-based optimization. Wall-clock runtimes were already available in the Appendix (Table 4), and we have now added memory usage. These metrics are provided in the table below. We note that while not the lowest, CLASP has a manageable runtime for a large pp, as compared to other gradient or sampling-based modern approaches (CAE, GroupFS, SFS-MWK++).

Wall-clock time (s) and peak memory (GB) of one fit on the training fold. CLASP: 28 cores (100 iterations on GLI-85 and NCI-60, 63 on TADPOLE); all others: 1 core.

MethodTADPOLE (p=656p{=}656) time (s)TADPOLE (p=656p{=}656) GBGLI-85 (p=22,283p{=}22{,}283) time (s)GLI-85 (p=22,283p{=}22{,}283) GBNCI-60 (p=25,731p{=}25{,}731) time (s)NCI-60 (p=25,731p{=}25{,}731) GB
Laplacian3.20.22.30.31.90.2
SPEC3.30.23.10.32.80.2
ICA2.50.32.20.22.40.4
MCFS2.10.31.80.21.90.2
CAE7.90.59016.26978.0
GroupFS6.20.5302635.5415447.3
SFS-MWK++1240.35550.4950.4
CLASP3233.05529.151112.1

Q9: Did you explore degree-normalized or sign-aware correlation rewards to mitigate hub dominance or confounding? Any empirical differences?

R1 brings up a great point, and we appreciate the suggestion. The row-normalized reward D−1∣R∣π,Dii=Σk∣Rik∣\mathbf{D}^{-1} |\mathbf{R}| \boldsymbol{\pi}, D_{ii}=\Sigma_k |R_{ik}| is a possible variant of CLASP. Under Theorem 1, its margin is zero for every feature set at uniform initialization, so it has no hub preference at the start. This implementation has been added to the Conclusion section as a future direction.

We keep ∣R∣|\mathbf{R}| instead of signed correlations because features enter the objective through distances, where anti-correlated features carry the same information as correlated ones.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to Reviewer VdqD (1/5)

Reviewer 1

W1: The "convergence" and "search-space contraction" arguments rely on strong/idealized assumptions and do not provide actionable finite-sample guarantees or sample-complexity bounds

We thank R1 for highlighting this point, also brought up by R4. Accordingly, we have updated the theory section of the Appendix to reflect more realistic scenarios. Specifically, Theorem 1 has been expanded beyond arguing a search-space reduction to now including the conditions under which we approach a better mask, and the required sampling budget in terms of a failure probability β\beta. Theorem 2 has been updated to account for the finite-sample scenario. We derive almost-sure convergence and a geometric attraction to strict local optima. We recognize that the updated theorems have tempered the claims in our original manuscript, and we have modified the manuscript text to reflect this change.

A.1 Setup

Masks are m∈M={m∈{0,1}p:∥m∥1=z}m \in \mathcal{M} = \lbrace m \in \lbrace 0,1 \rbrace^p : \Vert m \Vert_1 = z \rbrace. The inclusion probabilities π\pi have entries in (0,1](0,1] that sum to zz, starting from π(0)=(z/p)1\pi^{(0)} = (z/p)\mathbf{1}. Each iteration:

  1. It draws MM masks. Each mask is formed by zz draws without replacement, picking feature ii with probability proportional to wi=πiexp⁡(λci)w_i = \pi_i \exp(\lambda c_i). Here c=∣R∣πc = |R|\pi is the correlation reward of Eq. (2) and λ=λcorr\lambda = \lambda_{\text{corr}}.

  2. It scores each mask by L(m)=1C∑cSil(m,sc)\mathcal{L}(m) = \frac{1}{C} \sum_{c} \text{Sil}(m, s_c), the silhouette averaged over CC random subsamples.

  3. It updates π←(1−α)π+α⋅(mean of the k=γM best-scoring masks)\pi \leftarrow (1-\alpha)\pi + \alpha \cdot (\text{mean of the } k = \gamma M \text{ best-scoring masks}).

For a set BB of features, the margin is ΔB=min⁡i∈Bci−max⁡j∉Bcj\Delta_B = \min_{i\in B} c_i - \max_{j\notin B} c_j. For a mask mm, the outside mass is Um(π)=∑j∉mπjU_m(\pi) = \sum_{j\notin m} \pi_j.

A.2 Theorem 1

Let BB be a set of d≤zd \le z features.

  • (a) At initialization, if the rewards cic_i are not all equal, the prior (λ>0\lambda > 0) strictly lowers the entropy of the sampler compared with λ=0\lambda = 0, which is uniform over masks.

  • (b) If ΔB≥0\Delta_B \ge 0, then at every draw, given the features drawn so far, the odds that the next feature lies in BB are at least eλΔBe^{\lambda\Delta_B} times their value at λ=0\lambda = 0. For every rr, the probability that the mask contains at least rr features of BB is at least as large as at λ=0\lambda = 0. If every feature outside BB has a reward at least max⁡i∈Bci\max_{i \in B} c_i instead, both statements hold in reverse. In other words, the prior amplifies whichever group of features has the highest reward, including a nuisance module.

  • (c) M≥log⁡(1/β)/qM \ge \log(1/\beta)/q masks suffice to sample, with probability at least 1−β1-\beta, at least one mask from a set GG of masks of probability qq. For G={B}G = \lbrace B \rbrace with d=zd = z and ΔB≥0\Delta_B \geq 0, the prior multiplies this budget by at most the factor [(1−u)+u⋅e−λΔB]z[(1-u) + u \cdot e^{-\lambda\Delta_B}]^z, where u=UB(π)/zu = U_B(\pi)/z. This is a statement about one iteration, not a budget for the whole run.

A.3 Theorem 2 (convergence and local attraction)

Let M≥2M \ge 2, k=γM≥1k = \gamma M \ge 1, and let α∈(0,1)\alpha \in (0,1) be constant.

  • (a) For any data and any distribution of the subsample noise, with probability one there is a finite time after which all sampled masks equal a single mask m∞m_\infty. From then on π(t)→m∞\pi^{(t)} \to m_\infty at the geometric rate 1−α1-\alpha, top-z(π(t))=m∞\text{top-}z(\pi^{(t)}) = m_\infty, and the stopping rule is met. Every mask has positive probability of being m∞m_\infty, so (a) guarantees convergence, not quality.

  • (b) Let γ<1/2\gamma < 1/2. Call m♯m^\sharp a θ\theta-strict local optimum if every mask one swap away rarely beats it: it scores above the top-2γ2\gamma quantile of L(m♯)\mathcal{L}(m^\sharp) with probability at most θγ\theta\gamma. Let Ut=Um♯(π(t))U_t = U_{m^\sharp}(\pi^{(t)}) be the mass that π\pi puts outside m♯m^\sharp. Assume:

    • the scores are independent given the masks;

    • the optimum is strict enough, θ=O(1/log⁡z)\theta = O(1/\log z), so that each neighbour rarely enters the elite set;

    • there are enough masks per iteration, M=Ω(1γlog⁡log⁡zγ)M = \Omega\left(\frac{1}{\gamma}\log\frac{\log z}{\gamma}\right): then enough copies of m♯m^\sharp score highly to fill the elite set. With the explicit constants (appendix), the condition on MM holds at the paper's settings (M=400M=400).

    Then m♯m^\sharp attracts nearby runs: there is a neighbourhood Ut≤uˉU_t \le \bar{u}, with uˉ\bar{u} of order γ/log⁡2z\gamma/\log^2 z, such that a run starting inside it converges to m♯m^\sharp with probability at least 1−Ut0/uˉ1 - U_{t_0}/\bar{u}. For λ>0\lambda > 0 this holds as long as the prior does not favor features outside m♯m^\sharp inside this neighborhood; otherwise log⁡z\log z is replaced by eλδlog⁡ze^{\lambda \delta} \log z, where δ\delta is the largest amount by which an outside feature's reward exceeds a selected feature's near m♯m^\sharp.

Full proofs are in the revised Appendix A.

AuthorsOpenReview ↗

General Response: Summary of Revisions

We thank the reviewers for their detailed evaluations. Their comments and suggestions have greatly improved the rigor and validation of our work. Overall, the reviewers found our method novel and intuitive, with R1 and R3 highlighting the breadth of CLASP across the distinct datasets. Below, we summarize the improvements made to our paper in response to the critiques.

1) Adapted the theory to real-world use cases

As noted by R1 and R4, our original theorems relied on assumed idealized conditions that are not satisfied in practice. Accordingly, we have updated the theory section of the Appendix to reflect more realistic scenarios. Theorem 1 has been expanded beyond arguing a search-space reduction to include the conditions under which we approach a better mask and the required sampling budget in terms of a failure probability. Theorem 2 now accounts for the finite-sample scenario. We derive almost-sure convergence and a geometric attraction to strict local optima.

2) Comparison against an additional state-of-the-art baseline method

As suggested by R1 and R2, we compare CLASP with the recently introduced SFS-MWK++ (Zhang & de Amorim, 2026) in all simulated and real-world experiments. In simulation, CLASP recovers all informative features at every distractor correlation level, while SFS-MWK++ degrades as the internal correlation of non-informative features increases. On the real-world datasets, CLASP achieves a 59% improvement on average Nogueira’s index and a 37% improvement on average ARI consensus against the new baseline. SFS-MWK++ attains lower mean DBI, but the best performing method is split across the datasets, with all real-world results now reported over repeated cross-validation.

3) Enhanced the complexity of our simulated experiments and performed a targeted sensitivity analysis

We extended our simulations to three more challenging settings: selecting a different number of features than are truly informative, cluster imbalance, and homogeneous populations with no ground-truth clusters. CLASP is robust when the number of selected features is misspecified, and it is less likely than many of the baselines to identify spurious clusters in homogeneous populations, while it is sensitive to high cluster imbalance. On the real-world datasets, we varied the number of selected features, and CLASP almost uniformly achieves the best Nogueira’s index and ARI consensus. Finally, our new ablations show that each component of CLASP is valuable to the optimization.

4) Explored the susceptibility of CLASP and the baselines to batch effects

Batch effects are a common confound in biomedical data and can drive the feature selection of any unsupervised method. We first simulated a batch shift independent of the clusters. CLASP retains perfect feature recovery up to large shifts, at which point the batch effect dominates CLASP and nearly all baselines. To test this on real data, we selected features using one of the ADHD-200 sites and applied them to the remaining sites. Here, CLASP is the best or within 5% of the best in both DBI and ARI, and remains the strongest method on its most difficult site. We have also introduced an optional batch-aware sampling scheme which, given batch labels, restores perfect recovery even at the largest simulated batch shift.

5) Different clustering backends

As suggested by the reviewers, we evaluated CLASP using spectral, kk-means, and Ward linkage clustering backends across the seven biomedical datasets. Performance is comparable across the backends, with spectral clustering providing the most robust performance across the settings. The software implementation allows the user to select the clustering backend.


Finally, multiple reviewers noted that CLASP selects correlated feature sets, whereas existing methods avoid such redundancy. This is a deliberate design choice. The correlation prior is what makes the combinatorial search feasible, as our ablations show, and many biomedical effects are driven by networks of biomarkers. Imposing a “winner take all” selection amongst correlated features also risks overfitting to noise and finite sample effects. Therefore, we introduce CLASP as an alternative objective for data with a suspected clustering structure based on sparse feature networks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.