acceptodds
Withdrawn from ICLR 2027

Making Generalized Category Discovery Useful for Biodiversity Monitoring

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

By organizing previously unseen categories into coherent groups, Generalized Category Discovery (GCD) can potentially facilitate biodiversity monitoring by organizing observations of new species, enabling models to work with data-deficient species, and accelerating individual re-identification. Yet GCD methods are rarely designed to bear in mind the goals and constraints of biologists or the challenges posed by biodiversity data. This raises a practical question: how can we make GCD useful for biodiversity monitoring? To address this question, we formulate three biodiversity monitoring tasks that can benefit from GCD and propose practical metrics to evaluate their downstream performance under budget constraints. We introduce SciurusSciurus is the genus name of fox squirrels (Sciurus niger), which are known to organize their food caches by nut type delgado2017caching., a GCD-based sample selection framework for identifying informative samples, and show that it substantially improves downstream performance over existing sample selection methods. We further find that clustering accuracy, the most commonly used evaluation metric for GCD, is not always a reliable proxy for downstream utility, suggesting that selecting models based solely on clustering accuracy may be suboptimal for downstream applications. To address this gap, we introduce Homogeneity, a complementary metric that measures within-cluster consistency to better predict downstream performance. Together, our work provides a concrete first step toward making GCD useful for biodiversity monitoring in practice.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
biodiversity, generalized category discovery, species discovery

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tc6pOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses generalized category discovery (GCD) in the framework of biodiversity monitoring, especially under a limited expert annotation budget. To this end, the paper initially defines three application scenarios: discovering new categories, annotating samples from new categories, and training classifiers to recognize these new categories, which could benefit from GCD in biodiversity monitoring. Within this scope, the authors introduce Sciurus, a GCD-based sample selection framework to identify informative samples: Sciurus trains a GCD model for clustering, filters out seen category samples with a confidence threshold, and then repeatedly selects one sample per cluster with a query strategy. A sorting strategy ranks these candidates, and selection continues until the annotation budget is over. In addition, this paper also proposes the metric of homogeneity in place of clustering accuracy due to its limited representativeness for downstream performance. The experimental analysis includes three complementary datasets and backbones together with various GCD methods, showing the effectiveness of the best Sciurus configuration compared to active-learning baselines and random sampling.

Critical strengths

The GCD motivation for biodiversity monitoring, its formulation through well-defined applications, challenges, and the solutions with corresponding claims are all well explained in an easy-to-understand manner. In detail, the evaluation of GCD for downstream use under annotation budget constraints is realistic, which can open up new directions for the biodiversity monitoring community with significant operational impact potential.

The discovery rate results on the first application, which is the most informative test bed for the quality of sample selection with relatively less interference from sample order, features, etc., show the effectiveness of Sciurus across various levels of annotation budgets over three datasets compared to active learning baselines. This also supports one of the main claims of the paper: discovering all new categories does not require every observation to be assigned to the correct cluster.

Critical weaknesses

One of my main concerns about the paper is whether the main results in Figure 2 are fair across Sciurus and baselines. For each dataset and budget, the Sciurus performance is reported as the maximum metric of a quite large number of configurations over GCD methods, backbones, fine-tuning/query/sorting strategies, without using any cross-validation technique or a separate hold-out set, I assume. The reported baseline results are the maximum of five methods, each with three backbones, without considering the possible improvements of adapting query/sorting/filtering strategies, or even clustering used in Sciurus for these methods. To fairly assess the effectiveness of Sciurus, I would expect to see a single configuration chosen, for instance, on a separate set of one dataset, or via a cross-validation technique, and then always comparing this configuration with the baselines. Also, the quite low results of the baseline on Applications 2 and 3, even worse than random selection in some cases, seem to be questionable as well. Especially for Application 3, I can’t fully understand how active learning methods can underperform random selection on 2 out of 3 datasets. Also related to the comparison with other methods, I would expect to see a comparison with Adaptive-Novel due to its high relevance, as also stated by the authors.

My second main concern is that the homogeneity contribution of the paper seems to be overstated, while its novelty is limited. Eq. (4) is already well-known as cluster purity in the machine learning literature. Also, under different names and variants, this metric is widely used in active learning, information retrieval, and general machine learning. However, none of these studies are mentioned in the paper, while it is stated as one of the main contributions of the paper. In addition to the lack of novelty, its motivation over ACC is also questionable, as it can be trivially maximized if samples per class are distributed equally to all the clusters and the number of clusters is high. Thus, homogeneity as an alternative to ACC motivation of the paper requires further justification.

There are also other relatively minor weaknesses.

The related work section mainly focuses on GCD, but lacks discussion about general active learning studies, even though this line of research is highly relevant for budget-constrained applications like biodiversity monitoring and there are already some methods considered as baselines.

There are some typos and English abnormalities (e.g., whil in line 218, the the in line 129, as as in line 141, function function in line 316, etc.), requiring detailed proofreading.

Questions for authors

Can you please clarify whether Sciurus is fairly compared against active learning baselines? Would there be possible improvements by injecting query/sorting/filtering strategies, or even the clustering used in Sciurus, into these methods?

If a single configuration is set on a different dataset/split, etc., what would be the comparison of Sciurus against active learning baselines?

Can you please clarify how the baseline results on Applications 2 and 3 can sometimes be worse than random selection?

Can you please clarify the novelty of the homogeneity metric by stating its difference from class purity?

Can you please further elaborate on the effectiveness of homogeneity over ACC by referring to the trivial high score of homogeneity mentioned above?

Reviewer_bk26OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper defines three GCD (generalized category discovery, i.e. clustering an unlabeled pool containing both known and new classes) downstream tasks under varying annotation budgets: new-category discovery rate, annotation efficiency with label-switch count (number of category changes when an expert verifies the selected images in sequence), and 1-NN (1-nearest-neighbor) test accuracy. It then proposes Sciurus, a method combining OSR (open-set recognition) filtering, clustering, per-cluster query and sorting.

The paper also argues that cluster purity ("homogeneity") beats Hungarian ACC (clustering accuracy under the optimal cluster-to-class matching) for measuring downstream utility. The experiments are conducted on CUB, Herbarium19 and Open-Insect, and the authors claim to beat the baselines (standard active-learning selection methods: Margin, Entropy, Confidence, BADGE, Coreset).

Critical strengths

GCD evaluation driven by real-world constraints is an important and understudied problem The paper provide useful empirical findings.

Critical weaknesses

Methodology

  • The proposed Sciurius picks the best performance over a huge number of configs, where the optimal config is selected per dataset and budget on the ground-truth labels. The “single robust config” somewhat afresses this, but that should be the main paper result, not something that only appears in the appendix. And it is only valid assuming the hparams are tuned on a proper validation set.
  • Multiple of the best configs are essentially agglomerative clustering, the GCD-specific contribution is not properly isolated.
  • The baselines are weak and many are missing, for example LfOSA, MQNet, TypiClust or ProbCover.
  • The introduced homogeneity is just cluster purity, which is a well-known concept. It is maximixed by overclustering, which might explain why over-estimating k helps on Herbarium.
  • There are some unjustified things about the metrics, such as effectiveness being the mean of two values at different scales. Also it is assumer that the cost of switches is linear, but there is no user study.
  • Sciurus and the baselines seem to differ not only in the selection strategy but also in training data - Sciurus uses DlDu while baselines have been trained on Dl only. The improvement thus cannot be attributed to the proposed method
  • In App. 1 the novel-class count (ground truth) is needed to set the budget.
  • Samples are counted at equal cost while App. 2 argues the presentation order is important

Novelty:

  • As mentioned above with some examples, many baselines are missing. related work is extremely short, omitting active learning, open-set AL and rare-category detection literature
  • Many of thei introduced concepts are just renamed concepts from prior work:
  • Homogeneity -> purity, center query -> TypiClust cluster repre., filter -> MSP OSR threshold
  • The main contribution seems to be the task/metric framing + the evaluation, which might better fir a datasets track, or an applied venue.
  • As is, I think the contributions are too limited for an ICLR main paper.

Presentation, framing

  • Abstract redundancy: "clustering accuracy… is not always a reliable proxy" and "selecting models based solely on clustering accuracy may be suboptimal" make the same point.
  • "Concrete first step" is a bit of overclaim: for example, AGCD or rare-categiort discovery tackle the same problems.
  • generally the presented tasks map to standard ML problems and it would be helpful to use these for a ML venue like ICLR. For example Task 1 = category-level recall/coverage …. Currently the paper only uses biology-centric terms
  • .Fig. 1: the bottom-right panel can't be understood before Efficiency is defined.The "selected samples" and the Old/New legend are unclear.
  • Fig 3 seems to show that homogeneity underperforms on ⅔ datasets, including long-tailed, which contradicts the text and caption.
Questions for authors

The novelty and performance of the proposed Sciurus are currently not established.

  1. Provide results with configs selected on a held-out validation set.
  2. How does Sciurus compare to the listed baselines?
  3. Fix the improvement attribution problem (Sciurus and the baselines seem to differ not only in the selection strategy but also in training data.)
Reviewer_JhqXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how generalized category discovery (GCD) can support biodiversity monitoring under limited annotation budgets. Sciurus combines GCD-based clustering with confidence filtering and sample selection, with experiments on category discovery, annotation efficiency, and downstream classification.

Critical strengths

GCD is a practically relevant research problem.

Critical weaknesses
  • Sciurus mainly combines existing GCD models, confidence-based filtering, sampling across clusters, and center-based or uncertainty-based selection. The framework does not introduce a new learning objective, and the novelty beyond combining these established components is limited.
  • The main performance comparisons are concentrated in Figure 2. More detailed comparisons against individual GCD and sample selection methods are needed. Evaluation is restricted to three offline datasets covering tiny birds, plants, and insects, without larger-scale experiments to demonstrate the practical claims.
  • For each application, dataset, and budget, the paper reports the best performance across all configurations. This represents the best observed performance after extensive search. Although a validation set exists, it is used only to calibrate the old-category filtering threshold. An independent protocol for selecting configurations is needed.
  • “Homogeneity” in Equation (4) is mathematically identical to the established cluster purity metric and should not be presented as a new metric. Section 5.2 also states that all sorting strategies use descending order, whereas Section 7.2 explains that Sample Margin prioritizes lower-margin samples. Since the recommended configurations use this strategy, the inconsistency directly affects reproducibility and interpretation.
Questions for authors

see weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.