- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies Multi-Label Generalized Category Discovery (ML-GCD), where labeled training images contain only known categories, while unlabeled images may contain novel categories, or both known and novel categories. The goal is to recognize known categories and discover consistent novel-category identities across images, assuming the number of novel categories is given. The proposed Patch-guided Discovery (PaD) framework uses a DINO-pretrained Vision Transformer with independent sigmoid outputs and sparse cross-view pseudo-labels. A frozen teacher guides student learning through known-category prediction distillation, global relation preservation, and spatially aligned patch distillation. Experiments on VOC2007, MIRFlickr, and COCO2014 include baseline comparisons, component ablations, and analyses of mixed-category images. Anonymous novel-category outputs are globally matched to ground-truth categories for evaluation.
Critical strengths
- The paper introduces a valuable task setting and provides a detailed discussion of relevant prior work. To the best of my knowledge, the most closely related studies are appropriately acknowledged.
- The paper presents a distinctive methodological design and describes its components and implementation in detail.
- Experiments on three datasets evaluate performance using multiple metrics, while component ablations provide evidence for the contributions of individual modules.
- The paper transparently reports negative results and experimental limitations, including reduced joint recognition on mixed-category images, variability across random seeds, and optimization issues in some baseline adaptations.
Critical weaknesses
1. Weak exposition and insufficient motivation for key design choices.
The paper devotes substantial space to equations and implementation details but does not clearly connect the problem’s challenges to the proposed design choices. For example:
- Although the differences between ML-GCD and earlier settings are specified, its motivation remains unclear: which applications require these supervision restrictions, and what distinct challenges do they introduce beyond similar weakly supervised settings? Later sections do not resolve these questions.
- The method description is highly technical but offers limited justification for using DINO, applying Sinkhorn normalization, assigning a second positive pseudo-label to half of the unlabeled images, or choosing the associated hyperparameters.
Conversely, the task definition, global evaluation matching, and baseline adaptations are explained too briefly. For example:
- Section 2 describes DLCL and MCUL through the phrases “latent label columns” and “latent columns with missing observed entries.” These descriptions do not adequately convey their supervision assumptions to readers unfamiliar with the papers.
- The evaluation protocol is difficult to follow even for readers familiar with Average Precision. The equation is explicit, but the meaning of “global” alignment—one matching computed over the entire evaluation set and reused across subsets—must largely be inferred.
These issues hinder both comprehension and assessment of the paper’s contributions.
2. Insufficient justification of the task setting and its distinction from prior work.
ML-GCD requires all training images containing novel categories to enter the unlabeled pool, with all annotations—including known-category annotations—withheld. The paper does not explain why it excludes the natural alternative of retaining known-category annotations in such images, or which applications motivate this restriction.
Although DLCL and MCUL are acknowledged, their supervision assumptions are not explained sufficiently to establish the significance of the differences. Table 1 also risks conflating task specifications with method capabilities. These studies do not stipulate “Labelled subset excludes Novel” or “Fully unlabeled training pool,” but this does not establish that their methods cannot accommodate these conditions through adaptation.
3. Insufficient explanation of the discovery mechanism and the main performance gains.
The method uses anonymous category slots, sparse cross-view pseudo-labels, and self-training, but provides limited explanation of how these components encourage the slots to represent the intended semantic categories. The initialization of category directions and the rationale for pseudo-label selection are also insufficiently specified.
On VOC, Novel mAP rises from 6.80 for ML-SimGCD to 53.95 for the category-learning control, then to 54.93 with PaD. Thus, the largest gain occurs before the distillation components emphasized in the paper are added. The ablations mainly isolate these distillation components rather than explain the much larger gain from the base discovery framework. It remains unclear how much improvement comes from the loss, pseudo-label rules, the samples receiving supervision, or baseline adaptation.
4. Insufficiently controlled external baseline comparisons.
Table 3 mixes adapted external methods, an internal ablation, and a hybrid framework without clearly distinguishing their purposes. The category-learning control is a direct ablation; MulSupCon with an adapter combines an existing representation-learning framework with a discovery module; and ML-SimGCD is an adapted discovery method whose particularly low performance warrants investigation. On MIRFlickr and COCO, the main table compares PaD only with the internal control.
The appendix includes relevant methods such as DLCL, MCUL, and DMNL. However, these methods use frozen features while PaD fine-tunes the backbone, and some adaptations have unresolved convergence or calibration issues. These differences limit the conclusions that can be drawn about PaD’s advantage over prior discovery methods.
5. Insufficient analysis of discovery quality and performance trade-offs.
The component ablations and mixed-image evaluations are useful, but several central questions remain unanswered:
- What do the novel slots learn? Do they duplicate known categories, split individual categories, capture background patterns, or fail to cover some categories?
- Are the proposed relation and patch constraints necessary, or could freezing the backbone or applying a simpler feature-matching loss achieve comparable results?
- Why do aggregate mAP, F1, or precision improve while joint known–novel recognition declines on VOC and COCO? Is this caused by score calibration or by reduced recognition of one category group?
These questions directly concern the stated goal of recognizing known and novel categories when they co-occur. The current analysis does not sufficiently explain how the method achieves this goal or where it fails.
Questions for authors
- What accounts for the large gain from ML-SimGCD to the category-learning control? On VOC, Novel mAP increases from 6.80 to 53.95 before distillation is added. Can the authors identify which changes drive this improvement and provide evidence that the ML-SimGCD adaptation was appropriately configured?
- What evidence establishes PaD’s advantage over the closest prior methods? The appendix reports DLCL, MCUL, and DMNL adaptations, but their frozen representations and, in some cases, unresolved optimization issues complicate comparison. Are there results under comparable representation-training conditions, or evidence that these adaptations were adequately optimized?
- How are the novel classifier directions initialized, and what evidence shows that the resulting slots capture distinct target categories? In particular, do the reported results reveal duplicated semantics, overlap with known categories, or novel categories that remain undiscovered?
2 comments
Thank you for taking the review comments seriously and investing substantial effort in providing additional explanations and experiments. I appreciate these efforts. However, several key concerns remain unresolved, and I therefore maintain my original rating.
W1. Weak exposition and insufficient motivation for key design choices.
I acknowledge the additional explanations and experiments. Nevertheless, the writing issues in the paper continue to substantially affect its readability and the completeness of its arguments. Technical details are organized within a insufficiently clear logical framework, making it difficult for future readers to understand the connections between the underlying challenges, the proposed design choices, and their intended effects. Furthermore, explanations that are essential to understanding the paper's core contributions, should not rely primarily on the rebuttal or supplementary material. Therefore, until the main paper undergoes substantial revision to incorporate these explanations and establish a clearer line of argument, I will maintain this concern unresolved.
W2. Insufficient justification of the task setting and its distinction from prior work.
I still think that the proposed ML-GCD setting is quite similar to existing task formulations (table 1), and that its substantive distinctions from them have not been sufficiently established.
I do not dispute that these constraints define a different supervision protocol. However, the authors have not sufficiently explained why these differences warrant an independently motivated research task. In particular, it remains unclear which practical applications necessitate these restrictions and what fundamentally different research challenges they introduce compared with existing settings. Consequently, I believe that the novelty of the proposed formulation, particularly its claim of being the first to formulate Multi-Label Generalized Category Discovery, is limited.
W4. Insufficiently controlled external baseline comparisons.
Given the substantial similarity between ML-GCD and related settings, I think the effective comparisons for these related methods, including DMNL, DLCL, and MCUL, is particularly important. However, the main paper does not provide convincing direct comparisons, and the adaptation experiments in the supplementary material do not adequately address this concern.
In addition, these experiments suffer from substantial comparability issues. The related methods use frozen features, whereas PaD fine-tunes the backbone. Furthermore, some adaptations exhibit unresolved optimization convergence, kernel degeneracy, or score-calibration issues. These factors make it difficult to assess the relative capabilities of the methods themselves. In particular, when the adaptations of certain baselines have not been sufficiently validated, their extremely poor performance cannot serve as reliable evidence of PaD's superiority. I appreciate the authors' transparency in reporting these experimental limitations, but this does not resolve the underlying concern regarding the validity of the comparisons.
W5 & Q2. Insufficient analysis of discovery quality and performance trade-offs.
I still find the current experimental evidence insufficient to establish confidence in the proposed method.
First, the actual quality of Novel category discovery remains insufficiently analyzed. The additional experiments reported by the authors show that the 10 Novel categories correspond to only 7 distinct raw outputs when selecting the highest-AP output for each category. This raises concerns about whether different Novel slots actually learn semantically distinct categories. Furthermore, the decline in joint Known–Novel recognition on VOC and COCO remains insufficiently explained. Given that recognizing Known and Novel categories simultaneously is a central objective of the proposed task, this issue should not be treated merely as a secondary performance trade-off.
Second, the methodological contribution of PaD remains unclear. The performance gains brought by the proposed global-relation preservation and local-feature preservation constraints are limited, and the additional comparisons with simpler alternatives further weaken the case for their necessity. For example, CLS matching and the EMA patch teacher achieve Novel mAP scores of 56.32 and 55.74 on VOC, respectively, both exceeding PaD's 54.93. The authors also acknowledge that these simpler alternatives are competitive. Therefore, the current experiments have not established a clear advantage of the proposed complex constraints over simpler feature-preservation strategies.
Response to Reviewer ieQJ
Thank you. We appreciate your recognition that we report negative results and limitations transparently; this was deliberate. Section and table numbers refer to the revised manuscript.
1. Setting, motivation and exposition.
ML-GCD deliberately uses a strict supervision boundary to test whether annotations of known-category compositions transfer to unannotated compositions containing novel categories (Sec. 3). Retaining Known annotations in mixed images is a relaxation with additional supervision; it must be compared within its own regime (Appendix A), and the revised text positions it as an extension. Preliminaries now state the visible labels and the supplied Novel-category count.
Table 1 marks conditions not stipulated in the original formulations, not inability to adapt, and we adapted DLCL, MCUL and DMNL to ML-GCD supervision (Appendices G.1, I.8). We will replace the compressed descriptions of DLCL and MCUL in Related Work with explicit statements of which category labels are known and which annotations are missing.
On design choices (Sec. 4): DINO follows GCD's self-supervised pretraining without manual category labels. Sinkhorn supplies a batch-balancing prior for ranking targets, not multi-label probabilities. Selecting a second positive for a prescribed fraction of images is a sparse training prior (about 1.5 positives per image), not an estimate of true label counts; Table 11 shows non-monotonic sensitivity to this fraction and to the Sinkhorn iteration count. We will move component purposes ahead of the equations and consolidate numerical settings in the appendix. Global alignment is one AP-maximising Novel matching over the full test set, reused across metrics and subsets, with Known correspondences fixed (Sec. 5.1, Appendix A).
2. Attribution and the ML-SimGCD adaptation (Q1).
The largest gain (VOC Novel mAP 6.80 to 53.95) comes from multi-label target adaptation, which we regard as a core part of our contribution; the revised text foregrounds it. Table 3 separates the steps cumulatively: cross-view learning restricted to unlabelled images (6.80 to 10.42), ASL on Known labels (8.69), sparse masking on unlabelled images (32.60) and ASL on the selected entries (53.95). PaD then reaches 54.93. The largest steps coincide with replacing dense targets by sparse ones, which admit multiple positives and leave intermediate entries unsupervised instead of treating every unselected category as absent, and with ASL on those entries. These are ordered contrasts, not independent causal estimates.
ML-SimGCD applies rescaled Bernoulli targets and BCE to both labelled and unlabelled images (Appendix A). It shares the backbone, trainable block, initial head, learning rate, batch size and 200-epoch horizon with the other single-stage methods.
3. External comparisons and simpler preservation (Q2).
Table 2 separates external adaptations, the internal learner and the hybrid MulSupCon framework; Table 14 reports the longer 150+200 MulSupCon schedule, which neither establishes convergence nor aligns total budgets. DLCL, MCUL and DMNL use frozen features and therefore do not isolate discovery objectives from representation learning. We state this in the main text, do not claim superiority over the original methods, and leave fine-tuned counterparts for future work. For simpler preservation, Table 10 compares a frozen backbone, CLS matching (Novel mAP 56.32) and an EMA teacher (55.74) with PaD (54.93). These controls are competitive, and we do not claim that every PaD component is necessary.
4. Discovery quality and mixed-image recognition (Q3).
Classifier directions are randomly initialised and normalised, and no Novel labels determine them (Appendix B.1). Shared outputs and cross-view supervision encourage consistent predictions across views but do not guarantee distinct semantic categories. In the additional VOC pair (Appendix E.1), ten Novel categories select only seven distinct highest-AP raw outputs in both models, despite PaD's higher assigned Novel mAP, which shows overlap in preferred outputs. We will add image-grounded inspection of category overlap and missed categories.
On joint recognition, mAP and aggregate F1 do not require both category groups to be correct within each image, whereas joint hit does. The VOC/COCO joint-hit losses accompany lower Known recall even though COCO Novel recall improves (Table 12), and the revised conclusion states this deficit directly. Whether it reflects score calibration or reduced recognition of one category group is not yet resolved. The Known-only recall drop in Appendix E.1 suggests it is not confined to mixed images, and we will add teacher-calibration measurements on Known-only versus mixed images.