- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper deals with zero-shot learning (ZSL): object classes are associated to a set of attributes, a visual/attributes mapping is learned on seen classes such that at test time, the model is able to classify samples from unseen classes that are associated to the same attributes as the seen classes but have not been used at training time.
Previous works proposed to learn a hyperbolic (visual) embedding for ZSL. The first claimed novelty of the current paper is to propose two mappings: the CLS visual token to the the class name on the one hand (global branch) and the patch tokens to the attrbute prototypes on the other hand (local branch). The visual representations are projectd (lifted) to the hyperbolic manifold. Both benefit from an aditional projection, the radius offset head (ROH), which modulated the direct representation. In addition, the local representation is aggregated. This lorentz centroid pooling (LCP) consists to compute the average o gthe local listed points in the ambiant Euclidean space then project back to the hyperbolic manifold. Before the projection (hyperbolic lifting), the author propose to ``spread'' the semantic representations (class name or attribute) in order to improve the ability to distinguish them (process named PBE: Prototype bundle expansion). After projection, the resulting logits then probabilities are combined linearly. Last, on the local branch, the method include a regularization (LGRA: local geodesic-ray alignment) that force the local representation to align to the attribute prototype.
The method is compared to several recent approaches on CUB, SUN and AWA2, which are usual benchmarks in the ZSL literature. A quite significant ablation study is also reported, allowing to compare the relative merits of each module of the full model. The stability of the performance is also reported for two important hyperparameters. Last, the paper reports qualitative results, in particular to illustrate the effect of the ROH.
Critical strengths
-
[S1] the protoype bundle expansion is an original contribution, that seems efficient if one compare (Table 2) the model with all the proposed modules and that without PBE (line f). One also appreciate that it construction is motivated with some details in appendix A.2
-
[S2] another original contribution is the radius-offset head that acts as a sort of attention on the normalized local and global representations. A study of its specific interest would have been interesting (see [Q2])?
The paper reports good performance on three datasets that are usual in the ZSL literature. However, I consider that the protocol is unfair, as explained in [W1] below, thus it can not be considered as a strength at the same time.
Critical weaknesses
[W1] the evaluations are unfair, because most hyperparameters were optimzed directly on the test set. To the authors’ credit, they are quite transparent on this point, in appendix B.3, on line 952 and in Appendix B.4 on line 991 and in section 4.4 (line 422 and 424). It is particularly stricking for the setup of on SUN (Table 3, line 855) that correspond to "peak of performance" on the test set (Figure 6b), but it seems quite general, including and fusion weights. One must nevertheless recognize that it is a common practice in the ZSL literature, SVIP and VSPCN having been published (ICCV and CVPR 2025 respectively) with likely the same problem, and that reporteing better results than them with a fair protocol is difficult. It nevertheless remains a fundamentaly bad practice that should be banned as soon as possible. In 2017, Xian et al. published a famous paper to highlight ``ugly'' practices in ZSL. It seems that there is still work to do on this point, althought it is not specific to the current paper.
- another good practice, not applied in this paper nor in the usual ZSL literature, consists to run several times the experiments and report either a standard deviation or applying a statistical test to assesss the significance of the differences of performance.
- [W1a] regarding the performances reported, it is also surprising that the baseline model of the proposed approach without any of the proposed contribution (but PBE) (line 'a' of Table 2) has performance on-par (CUB, SUN) or even much better (AWA2) than recent state of the art methods (Table 1). It is likely it explains most of the good performance reported in Table 1. However, it is diffcult to understand and not commented in section 4.3: See questions.
[W2] regarding the novelty of the contribution, it looks like it essentially consists to aggregate existing approaches, or their utility is arguable.
- as mentioned in the paper, hyperbolic projection has already been proposed in the ZSL literature (Liu et al, 2020). While not mentioned in the related works, multi-branch mapping (image/class + attributes/token) has also been exhaustivly explored in the prototype network literature (e.g [a] and subsequent works; a paper such as [b,c] have similar dual-branch architecture as the current paper). The first claim, that consist to mix these two ideas, has thus a limited novelty.
- the LCP is a naive average followed by a projection on the manifold (equation 6). This choise seems odd: why the average is not directly computed in the manifold with its natural metric? While efficient in practice (Table 2; lines h versus full) computing a naive average to agregate can not be considered as particularly inovative.
- LGRA seems to very very close, if not identical to the of (Desai et al, 2023). In particular, equation (11) of the current manuscript is essentially similar to equations (11) and (12) of (Desai et al, 2023). Here again, the novelty seems marginal in comparison to this previous work.
[a] Xu et al (2020) Attribute Prototype Network for Zero-Shot Learning, NeurIPS
[b] Xiang et al. (2023) Dual Feature Augmentation Network for Generalized Zero-shot Learning, BMVC
[c] Shen et al. (2025) Zero-shot image classification based on class representation learning and attribute embedding learning, PLoS One
Questions for authors
-
[Q1] The ablation contains several interesting comparison but it misses the simplests, namely that with P and C onlyn without PBE (and possibly "C only" and "P only") wich is is the most basic baseline. On line 'a', with PBE only, the performances are already around the state of the art on CUB and SUN and much better on AWA2. It seems to explain most of the top results reported with the contributions. However the baseline results, withour most of the proposed contributions, are diffcult to understand. Is it due to PBE: what are the results without this module at all (line (a) without PBE)? How the very good baseline results can be explained?
- comparing lines (f) and (full), it seems that the specific contribution of PBE is not enough important to explain the baseline good results
-
[Q2] did the author conducted ablation to evaluate the specific merit of the ROH? In other words, what would be the results with in equation (4) and in equation (5)?
Minor questions:
- [MQ1] in equation (7), Does "clip(.,.,.)" a cosine similarity between the CLIP embeddings? what is ? It seems (from appendix A.2) that it allows to stabilize the normalization: reporting the exact formulation in this appendix, and studying its importance would have been appreciable.
Minor remarks (not to be addressed in the rebuttal)
- DRF and LCP are used on line 203 before they are defined. At this level, it would be better to give an overview without using acronym, even BPE and LGRA which meaning has been given in section 1
- the projections on line 215 (attr and cls) have not the same typography as that on Fig. 2
- in equation 4 and 5, it should be clarified that and are in and e.g with
- on line 246, 247, the text alternates between notation that relate to ( , , ) and other that do not (, ), which makes reading tedious.
- at least it should start with "let be the normalized descriptor for class ..."
- introducing the notation and while they are only the normalized version of and does not help either.
- by the way, the are not explicitly defined, it is only the Fig. 2 that indicates they are and whici are -normalized.
- the motivation for PBE (line 105) should be reminded and detailled in the dedicated section (3.4)
- What means "ROH" on line 258?? It is only explained in the caption of Figure 4, line 405! Possibly, one can guess it from Figure 2. It should be introduced on line 211, or line 221 (after introduction). In any case, it would be nice to explain what it is (in which range it lies...)
- In section 4.5, the experiment leading to Figure 4 should be explained in the text before refeering to its results