acceptodds
Withdrawn from ICLR 2027

TransGCD: Fine-Grained Semantic Transfer for Generalized Category Discovery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

13 views

Abstract

Generalized Category Discovery (GCD) aims to uncover novel categories from unlabeled data while preserving recognition of known categories. Recent GCD methods have achieved substantial performance gains by generating pseudo-text representations for unlabeled samples to provide additional semantic guidance. However, these methods typically derive pseudo-text from global visual representations, which are susceptible to background interference and fail to capture fine-grained object attributes. To address these limitations, we propose TransGCD, a fine-grained semantic transfer framework that transfers object-centric visual semantics to textual representations for enhanced pseudo-text generation. Specifically, we introduce two complementary techniques, termed Object-centric Semantic Decomposition (OSD) and Attribute-guided Semantic Injection (ASI). OSD decomposes global visual representations into complementary object-centric semantics, isolating category-relevant cues from background interference. Building on these decomposed semantics, ASI transfers fine-grained visual attributes into textual cues and injects them into the text encoder to enhance pseudo-text representations. By transferring fine-grained visual semantics into the textual space, TransGCD provides more discriminative semantic guidance for recognizing known categories and discovering novel ones. Extensive experiments across multiple GCD benchmarks demonstrate consistent improvements in known, novel, and overall accuracy, with particularly notable gains on fine-grained datasets. e.g., achieving increases of 8.7% on CUB and 6.4% on Stanford Cars. Code is available at redhttps://anonymous.4open.science/r/TransGCD.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Generalized Category Discovery, Vision-Language Models, Fine-Grained Visual Recognition

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_36MaOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on the Generalized Category Discovery field and aims to enhance pseudo-text representations by transferring fine-grained visual attribute semantics into the textual space. This paper proposes a plug-and-play framework named TransGCD which introduces two complementary techniques, termed Object-centric Semantic Decomposition (OSD) and Attribute-guided Semantic Injection (ASI). The OSD module decomposes visual representations into complementary object-centric attributes, which are subsequently injected into the text encoder by the ASI module to generate fine-grained pseudo-text representations.

Critical strengths

The research motivation of this paper is clear, and the problem is worthy of further investigation. In recent years, GCD research has increasingly shifted its attention from unimodal visual learning (e.g., SimGCD) toward multimodal learning (e.g.,GET). However, existing pseudo-text representations are mainly derived from global visual features and often overlook fine-grained semantic information. Therefore, investigating how to effectively incorporate fine-grained visual semantics into pseudo-text representations is meaningful and worthwhile.

Critical weaknesses
  1. Numerical inconsistencies in Table 3. The reported improvements in Table 3 are inconsistent with the corresponding baseline and full-model results. Specifically, the improvement on CUB should be 2.5 (87.9 − 85.4), rather than the reported 2.4. Similarly, the improvement on FGVC-Aircraft should be 3.5 (69.9 − 66.4), rather than 3.6. The authors should clarify whether these discrepancies result from rounding or calculation errors and ensure the numerical consistency of the reported results.
  2. Computing overhead is not discussed. The authors should report the additional parameters, FLOPs, training time, and inference time compared with the baselines to demonstrate whether the performance improvements justify the extra computational overhead.
  3. The ablation studies are incomplete. The discussion about the loss functions LconL_{con}, LdivL_{div},LitcL_{itc},and$L_{sem}$ is not provided.
  4. Table 3 is not referenced in the main text. The authors should explicitly refer to Table 3 when discussing the effectiveness of different components in Section 4.3.
Questions for authors
  1. The numerical inconsistencies in Table 3 raise concerns about the reliability of the reported experimental results. Could the authors clarify the reasons for these inconsistencies?
  2. Due to the lack of ablation studies on the proposed loss functions, their individual effectiveness and necessity are not sufficiently demonstrated. Could the authors provide additional ablation experiments to further validate the contributions of these loss functions?
Reviewer_hRS2OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TransGCD, a plug-and-play framework combining object-centric semantic decomposition, attribute-guided injection, and LLM-generated descriptions to improve fine-grained pseudo-text representations for generalized category discovery across multiple benchmarks.

Critical strengths
  1. The paper addresses background interference and insufficient fine-grained attribute modeling in existing pseudo-text-based GCD methods.
  2. The proposed OSD and ASI modules provide a coherent pipeline for extracting object-centric visual attributes and transferring them into textual representations.
Critical weaknesses
  1. Tables 1 and 2 explicitly state that bold values indicate the best results, yet the authors also highlight average improvements in bold, creating an exaggerated impression of competitiveness. Across the 18 evaluation metrics, TransGCD achieves only 6 best results, despite the extensive use of bold formatting. This presentation substantially undermines confidence in the experimental reporting.
  2. Despite being presented as a plug-and-play extension, TransGCD remains inferior to several recently developed standalone GCD frameworks, even after integration with existing methods. More importantly, the selected baselines originally rely on DINO, but the authors replace their backbones with CLIP, despite the availability of native CLIP-based GCD methods. The motivation and fairness of this experimental design remain insufficiently justified.
  3. Existing CLIP-based GCD approaches commonly exploit predefined or shared textual information. TransGCD introduces an additional LLM-based attribute-generation pipeline without a controlled comparison against these existing textual resources. Although Table 6 examines different text generators, it does not establish that the additional LLM-generated information is necessary or superior to established textual supervision. This weakens the motivation and practical significance of the proposed approach.
Questions for authors
  1. Why are average improvements highlighted in bold in Tables 1–2 despite only six best-performing metrics out of 18? Could the authors adopt consistent formatting?
  2. Why replace DINO with CLIP in existing GCD frameworks rather than evaluate against native CLIP-based methods under identical experimental conditions?
  3. Can the authors demonstrate that LLM-generated attributes outperform predefined textual descriptions or shared vocabularies under controlled settings with identical backbones and training budgets?
  4. Given the modest incremental improvements in Table 3, can the authors justify the complexity of OSD and ASI through computational-cost analysis and stronger controlled ablations?
Reviewer_Ky2HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TransGCD, a fine-grained semantic transfer framework for Generalized Category Discovery (GCD) that enhances pseudo-text representations using object-centric visual attributes. It introduces Object-centric Semantic Decomposition (OSD) to extract fine-grained attributes while suppressing background noise, and Attribute-guided Semantic Injection (ASI) to incorporate these attributes into CLIP's textual representations. An LLM-guided pipeline further provides attribute-rich semantic supervision. Experiments on six GCD benchmarks demonstrate improvements over multiple CLIP-based baselines, particularly for fine-grained and novel-category recognition, supported by ablation studies.

Critical strengths
  1. Effective fine-grained semantic representation learning. The proposed OSD and ASI modules address an important limitation of existing pseudo-text-based GCD methods, namely their reliance on holistic representations that may overlook discriminative object attributes. The improvements on fine-grained benchmarks, particularly CUB and Stanford Cars, demonstrate the potential value of incorporating part-level semantic information for novel-category discovery.

  2. Compatibility with existing GCD frameworks and extensive experimental evaluation. TransGCD can be integrated into multiple existing GCD methods, including SimGCD, CMS, and SelEx, without modifying their primary training objectives. Evaluation across six datasets, together with component-wise ablations and comparisons of textual supervision strategies, provides useful evidence of the method's practical applicability.

Critical weaknesses
  1. Limited methodological novelty and insufficient isolation of the proposed contributions. The framework combines established techniques, including learnable queries, attention-based saliency filtering, cross-attention feature injection, and LLM-generated attribute descriptions. Although their integration is effective, the methodological distinction from existing fine-grained vision-language alignment and pseudo-text generation approaches is somewhat limited. Furthermore, the large improvement from attribute-rich textual supervision makes it difficult to determine how much of the performance gain is attributable specifically to the proposed OSD and ASI mechanisms.

  2. Inconsistent improvements and incomplete validation against strong baselines. Although TransGCD improves its integrated baselines in most settings, it does not consistently outperform existing state-of-the-art methods. For instance, on Stanford Cars, the proposed SelEx-CLIP+TransGCD achieves 82.2% overall accuracy, compared with 89.2% for SSR2-GCD. The gains are also substantially smaller on generic datasets, and known-class performance occasionally decreases after integration. Additionally, the absence of repeated-run statistics, computational overhead comparisons, and quantitative evaluations of background suppression limits the strength of the empirical conclusions.

Questions for authors
  1. Contribution of individual components: Can the authors provide a more controlled comparison separating the benefits of attribute-rich semantic supervision from those of the proposed OSD and ASI modules? In particular, how does TransGCD compare against a simpler approach that directly injects LLM-generated attribute embeddings without the query-based decomposition module?

  2. Competitiveness against stronger methods: While TransGCD improves its selected baselines, several existing methods achieve higher performance on particular datasets. Can the authors clarify why the proposed framework remains below these methods and whether integrating TransGCD into stronger recent baselines would yield additional improvements?

  3. Robustness and efficiency: Can the authors provide mean and standard deviation over multiple random seeds, together with training/inference overhead? Additionally, can they quantitatively evaluate whether the saliency-guided decomposition genuinely reduces background interference, particularly when foreground objects are small or partially occluded?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.