acceptodds
Under review as a conference paper at ICLR 2027

Anchored Residual Integration with Structure-regularized Adaptation for Generalized Category Discovery

Abstract

Generalized category discovery uses labeled examples from known classes to recognize both known and novel categories in unlabeled data. In fine-grained settings, closely related novel categories often share substantial semantic structure, with distinguishing cues expressed as subtle deviations in the representation. We identify a geometric mechanism in cosine classification that can weaken these cues: relative to a semantic anchor derived from known classes, residual directional similarity is scaled by the product of the sample and prototype residual magnitudes. Consequently, useful discriminative information may contribute weakly to classification even when it remains encoded in the representation. We propose ARISE, a two-stage framework combining structure-regularized adaptation with anchored residual integration. First, ARISE adapts pretrained representations under known-class supervision while constraining changes in local relations through a momentum reference encoder. It then constructs a semantic anchor for each novel prototype and applies square-root reweighting to mitigate residual-magnitude attenuation while retaining sensitivity to residual strength. A novelty-and-ambiguity gate selectively applies the correction during novel-category learning and joint inference. Across six benchmarks, ARISE achieves an average overall accuracy of 71.9% and the highest overall accuracy among the compared methods on all four fine-grained benchmarks.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Generalized Category Discovery;Representation Learning;Fine-Grained Visual Recognition

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CZnGOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ARISE for generalized category discovery, combining geometry-regularized feature adaptation with candidate-specific residual correction. The method uses EMA-based relational regularization and gated square-root residual reweighting to improve fine-grained category discrimination. Experiments on six benchmarks show competitive performance, with the best overall accuracy among the compared methods on four fine-grained datasets.

Critical strengths
  1. Well-motivated residual correction. The anchor-based decomposition provides a concrete mechanism for underweighted directional evidence. The low-magnitude diagnostics (Fig. 5; Table 13) support the motivation beyond overall accuracy.
  2. Strong fine-grained benchmark results. ARISE achieves 76.4% on CUB-200, 70.1% on Stanford Cars, 49.6% on Herbarium19, and 63.1% on FGVC-Aircraft (All accuracy), leading the methods in the main comparison (Table 1). Ablations and three-seed results add useful validation.
Critical weaknesses
  1. Material reporting inconsistencies. The main text describes inference with corrected logits, but Appendix A.6 (including Eq. 37) specifies original logits. Tables 5 and 8 also disagree on apparently matching ablations (e.g., CUB No Sinkhorn: 71.4% vs. 69.8%), while numerical changes stated in Appendix B.3–B.4 do not match Tables 8–9. These conflicts obstruct reproducibility.
  2. Limited isolation of the main correction. Adding the residual component gives only +1.1/+1.3 All points on CUB/Stanford Cars (Table 2); inference-only correction gives +0.7/+0.1 points, with slight New-accuracy declines (Table 3). Comparisons with simple novel-logit calibration or alternative residual exponents are missing.
Questions for authors

Q1. Were Table 1 predictions made with original logits or corrected logits? Please provide the exact inference path and a same-checkpoint comparison.

Q2. Why do Tables 5 and 8 differ, and why do Appendix B.3–B.4 calculations disagree with Tables 8–9? Please provide reconciled settings/results.

Q3. Under matched settings, how does square-root lifting compare with simple logit calibration and other residual exponents? Does it improve low-magnitude top-1 accuracy?

Reviewer_tepdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ARISE, a two-stage framework for Generalized Category Discovery (GCD). Experiments show that the method improves overall performance across multiple datasets.

Critical strengths
  1. The paper analyzes how cosine classification weights residual-direction consistency relative to known semantic anchors and shows that the product of residual magnitudes may suppress candidate-specific evidence.

  2. The experimental evaluation is fairly comprehensive, as the method is assessed on six datasets, including comparisons with multiple GCD methods, ablation studies, and hyperparameter sensitivity analyses.

  3. The implementation details are well documented, covering relation-based batch sampling, prototype initialization, residual decomposition, centering, and gating, which ensures good reproducibility.

Critical weaknesses
  1. The main experiments are based on a DINO-initialized ViT backbone, whereas recent GCD methods have increasingly been built upon stronger or more generalizable pretrained models such as DINOv2 and CLIP. Relying solely on DINO as the backbone fails to fully demonstrate the effectiveness and generalization ability of ARISE under stronger representations.

  2. The paper lacks an analysis of computational efficiency. ARISE introduces multiple components that may significantly increase training and inference overhead. It is recommended that the authors report training/inference time, parameter count, and GPU memory usage, compare efficiency against the main baselines, and discuss the scalability of the method on large-scale datasets.

  3. The paper may have a tendency to weaken the baselines and exaggerate the gains. In Table 7, when comparing with plug-and-play GCD methods, the paper uses only SimGCD as the baseline; however, stronger baselines such as SelEx clearly outperform SimGCD on fine-grained tasks, and the gains of these plug-and-play methods over SelEx tend to be larger. Therefore, pairing the comparison only with SimGCD may exaggerate the relative advantage of ARISE.

Questions for authors

See Weaknesses.

Reviewer_EBpROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a new two-stage GCD framework. They show competitive performance to other SOTA methods on classical datasets and surpass SOTA methods on fine-grained benchmarks. They motivate their methodological contributions partly empirically by showing that similar novel classes can be separated along directions that are orthogonal to the most similar known class prototypes. This leads them to develop GEOMETRY-REGULARIZED KNOWN-CLASS ADAPTATION, a new supervised student-teacher objective for their first learning stage, that acts to preserve local structure from the pretrained backbone along the most significant PCA-directions, while training a classifer for the known classes. The remaining methodological contributions come into play in the second stage. Each sinkhorn-initialized novel protoype gets a KNOWN-SEMANTIC ANCHOR, which is a similarity-weighted average of known prototypes. To get the cosine simialrity score between a sample and a novel prototype, they get decomposed (RESIDUAL DECOMPOSITION AND LIFTING) into the cosine similarity of their projections on the known semantic anchor and a square-root reweighted cosine similarity of the residuals of the projections. They use the square root here to boost weak residuals. As not all weak residuals are helpful, they add SELECTIVE GATING for novelty and ambiguity and CENTERING to REGULATE NOVEL CANDIDATE CORRECTIONS. The authors do some emperical analysis to validate their proposed methods. Furthermore, they show on how their model surpasses SOTA methods on four fine-grained benchmarks. The paper additionally includes many ablation studies to showcase individual contributions of their methods.

Critical strengths

Interesting and novel perspective of failure case of GCD in FGVU. The paper offers a new perspective of fine-grained GCD: Rather than attributing failure to missing discriminative features, it ponders whether cosine scoring simply under-weights certain discriminative features. Their solution of exploiting this phenomenon by decomposing embeddings relative to known-semantic anchors is sensible, and their design choices follow logically from their analysis.

Empirical evidence of consistent performance gains on FGVU benchmarks: ARISE shows consistent gains across multiple fine-grained benchmarks compared to other SOTA-methods: 1.2–1.5 All over the strongest baseline on all four fine-grained benchmarks, with small seed variance, makes the method practically relevant

Critical weaknesses

W1: The authors do not give enough evidence of their claim regarding their first contribution: Analysis of how cosine classification weights residual directional agreement relative to known-semantic anchors, showing that the product of residual magnitudes can suppress candidate-specific evidence). The main part of the paper does not show any empirical evidence; instead, Fig.1 is merely an illustrative example. Supplementary part C gives partial emperical evidence but still does not show and even slightly contradicts that the effect is strongest for small magnitudes as claimed. Fig 5 a) would benefit from showing evidence retained in Q2-Q5 quantiles and some sort of baseline like the anchor-term or full-cosine margin (is the direction is correct where the base score is wrong?). Fig 5b shows effects on the novel decision margin, which can be dominated by a few large changes, showing change in accuracy would show actual effects. For CUB, the biggest margin effect is at Q3 not at Q1. Consider adding more dataset examples (Aircraft/Herbarium) to have a better emperical argument that low magnitude effects are present and generalize. The added correction sqrt(m)-m*q peaks at 0.25, True mean magnitudes of CUB and SCars are at 0.22, 0.25 for Q1, without controls it is hard to say whether low-magnitude residuals carry evidence or whether the operator just acts most strongly at m ≈ 0.25. Consider shuffling residual directions while keeping magnitudes fixed or changing the lifting function to distill effects.

W2: The ablations in the main paper do not isolate the proposed contributions. Big parts of the main text ablations are not part of the contributions (e.g. table 5 & sinkhorn temp analysis in fig 3). Ablations of contributions such as correction gates and anchor construction (table 10) + geometry-batch construction (table 11) are in the appendix. The authors explicitly motivate their square root normalization against full normalization as the latter would discard information about residual strength but do not show this. Furthermore, there are irregularities between appendix ablations experimental numbers and main text: Table 5, no Sinkhorn ALL CUB: 71.4 vs 69.8 for Table 8 ; no class scalling ALL CUB 74.9 (table 5) vs 74.2 (table 8) and fixed reference CUB ALL 64.6 (table 4) vs 65.4 (table 9).

Questions for authors

I ask the authors to clarify discrepancies and to report more evidence for their claims as outlined above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.