acceptodds
Under review as a conference paper at ICLR 2027

Beyond Holistic Alignment: Structured Semantic Transfer for Zero-Shot Hashing

Abstract

Zero-shot image hashing aims to learn a compact binary code that generalizes to unseen classes without real training images of those classes, using semantic side information. Recent generative pipelines synthesize pseudo-images for unseen classes and align image features through contrastive objectives, yet the hash space itself is supervised mainly through holistic features and class centers without explicitly exploiting structured semantic relations. Two complementary forms of semantic structure therefore remain underexplored: attribute-related patterns encoded in visual representations and the relational geometry that connects seen and unseen classes. We propose a novel framework termed Structured Semantic Transfer (SST) for zero-shot hashing, which preserves semantic structures beyond holistic alignment. A lightweight branch captures attribute-aware representations from local patch features, and a pairwise structure distillation loss aligns the similarities among these representations with those of their classes, enabling the encoder to learn transferable semantic structures. Meanwhile, a geometry alignment loss preserves the attribute-induced relations among hash centers, guiding unseen class centers through structured semantic propagation. We further introduce a per-code-length component configuration that adapts the overall scale of the two loss weights at each operating point while keeping the framework unchanged. Experiments on standard zero-shot hashing benchmarks show that SST consistently improves retrieval over strong baselines, and ablations confirm that the two semantic transfer mechanisms are complementary.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
zero-shot hashing, image retrieval, representation learning, knowledge transfer

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9HNwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Structured Semantic Transfer (SST), a zero-shot image hashing framework built upon a PZSH-based generative retrieval pipeline. The main idea is to transfer semantic relationships into both visual representations and learnable hash centers, rather than relying primarily on holistic feature alignment. SST introduces two components. First, patch-level relational distillation employs learnable attribute queries to aggregate patch tokens and aligns pairwise similarities of the resulting embeddings with class-attribute similarities. Second, attribute geometry alignment constrains the cosine similarities between unseen and seen class centers using attribute-presence information. The authors also provide a theoretical bound relating continuous center geometry to binary Hamming similarities under additional assumptions. Experiments on AWA2, CUB, and SUN, across four hash lengths, demonstrate consistent improvements over the authors' PZSH-based implementation. Ablation studies, selected multi-seed experiments, and alternative design controls further support the effectiveness of the proposed objectives.

Critical strengths
  1. A coherent and practically motivated approach to structured semantic transfer. The two proposed objectives address complementary aspects of zero-shot hashing: transferring class-attribute relationships into patch-derived visual representations and imposing semantic structure on learnable hash centers. Their integration into a shared visual encoder and hash-learning pipeline is conceptually coherent. The distinction between batch-level representation relations and cross-class center geometry provides a reasonable motivation for the overall framework.
  2. Consistent empirical improvements supported by meaningful ablations. SST improves over the authors' internal baseline across all twelve dataset–code-length configurations. Both transfer objectives contribute positively, and the full model generally performs better than either single-component variant. The additional multi-seed results in Appendix A.10 and the design controls in Appendix A.11 strengthen the empirical evidence. I also appreciate the authors' explicit discussion of evaluation-protocol differences and the limitations of their theoretical analysis.
Critical weaknesses
  1. The comparison against PZSH is not sufficiently controlled to support the broader improvement claim. The main concern is that the reported PZSH baseline is not a faithful reproduction of the original PZSH optimization objective. The original PZSH method uses center-based cosine classification, quantization, and proxy losses, whereas the current paper adopts learnable continuous centers supervised through MSE and momentum consistency. Consequently, removing the two proposed transfer losses does not recover the original PZSH method. While the ablations convincingly demonstrate improvements over this modified implementation, they do not establish whether the proposed objectives improve the original PZSH framework under matched conditions. Furthermore, the paper reports mAP@5000, while published PZSH results use mAP@all, and some imported comparisons use different class partitions. The authors acknowledge these differences, but they substantially limit conclusions about competitiveness against established methods. This issue is particularly consequential because PZSH is the primary foundation and strongest directly controlled baseline of the paper.
  2. The methodological novelty and the evidence for the intended semantic mechanisms remain somewhat limited. Previous zero-shot hashing studies, particularly COMAE (ICMR 2025), RAZH (2025), and DA2H (2026), have already investigated attribute-related local representations and visual-semantic alignment. SHC (TOIS 2025) also considers semantically structured hash centers, although in a different setting. The specific relational objectives proposed here differ from these methods, but their broader motivation substantially overlaps with prior work. Moreover, the patch-level objective matches image-level similarity matrices without directly supervising individual attribute coordinates or spatial correspondence. Consequently, it does not establish that the learned queries identify meaningful attribute-specific regions. The authors appropriately acknowledge this limitation, but additional mechanistic evidence would be valuable. Similarly, the geometry objective constrains continuous unseen–seen center similarities, whereas the principal retrieval task depends on ranking unseen images using binary codes. Theorem 3.3 provides a conditional relationship between these spaces, but it does not directly characterize unseen–unseen retrieval or mAP. The empirical improvements are encouraging, yet the precise role of semantic geometry in improving retrieval discrimination could be more convincingly demonstrated.
Questions for authors
  1. Could the authors evaluate the two proposed transfer objectives on top of a faithful implementation of the original PZSH framework, retaining its original center construction and hash losses? Reporting the original method, the modified MSE-based implementation, and both variants with the proposed components under identical retrieval protocols would clarify whether the improvement comes from structured semantic transfer rather than other changes to the baseline.
  2. Could the authors further isolate the contribution of the proposed structural relations? In particular, a semantic-target control using shuffled or mismatched class-attribute relations would help determine whether the geometry loss benefits from meaningful semantic structure rather than generic center regularization. Likewise, more direct evidence connecting patch-level aggregation to attribute-specific visual information, beyond pairwise similarity agreement, would help distinguish SST from existing attribute-aware hashing approaches.
  3. Can the authors provide stronger evidence of competitiveness against recent attribute-aware methods such as COMAE and DA2H under matched dataset splits and evaluation metrics? The current literature scores are useful contextual references, but are not sufficient for a controlled state-of-the-art comparison.
3 comments

Dear Reviewer 9HNw,

Thank you for your careful review and constructive feedback. We respond to this concern below.

Q1. Evaluate the transfer objectives on original PZSH and distinguish them from the MSE modification under identical retrieval protocols.

We now report the original-center/original-loss reproduction separately from PZSH-MSE. The original loss implementation uses fixed centers with center-classification, quantization, and proxy terms. PZSH implementation

We first add patch supervision without changing those fixed centers (O0 versus O1). We then isolate the effect of making centers learnable (O2), adding patch (O3), and adding geometry (O5). O2/O3/O5 retain the original hash losses and the original center initialization construction. This separate adaptation is necessary because a loss defined only on fixed centers cannot update those centers. The M0/M1/M2/SST rows independently isolate the objectives within the MSE branch. CUB mAP@5000 at seed 42 is:

Variant24 bits48 bits64 bits128 bits
O0: original losses, fixed centers0.13180.21640.22440.2691
O1: O0 + patch0.14800.21950.23420.2718
O2: original losses, learnable centers0.14780.22100.23590.2802
O3: O2 + patch0.14980.22130.23890.2813
O5: O3 + geometry0.15310.22260.24400.2819
M0: MSE baseline0.13970.21600.22810.2704
M1: M0 + patch0.15710.23120.23510.3035
M2: M0 + geometry0.15560.22260.24050.2881
SST: M0 + patch + geometry0.15810.23620.25470.3060

O1 improves over fixed-center O0 at every length. In the original-loss, learnable-center branch, the increments O3−O2 are 0.196/0.034/0.300/0.106 points, and O5−O3 are 0.334/0.130/0.510/0.064 points, in 24/48/64/128-bit order. Both increments are positive under @all as well. The results provide controlled evidence that the objectives' benefits are not confined to the MSE branch, while identifying the required center adaptation. We will rename the internal baseline, retain the distinction between fixed and learnable centers, and report both evaluation cutoffs.

Critical Weakness 2 — methodological distinction from prior attribute-aware hashing.

We acknowledge prior attribute-aware modeling in COMAE, RAZH, and DA2H, and semantic centers in SHC. SST combines batchwise attribute-relation matching on patch-derived embeddings with unseen–seen attribute-presence matching on learnable centers in generative zero-shot hashing.

Q2. Could a shuffled or mismatched semantic target isolate the structural relations, and could more direct evidence connect patch aggregation to local visual information?

On CUB (training seed 42), we separately test shuffled geometry targets (class–attribute permutations within seen/unseen groups) and class-only patch targets (same/different class = 1/0), keeping other settings fixed. Localization compares visible-part hits in top-attention versus random 20% regions on unseen images, using query–part mappings fixed from seen validation. Masking applies identical 20% attention or random masks to SST and CLS queries; attention masks come from frozen SST, database/checkpoints/fill remain fixed, and random results average ten mask seeds.

Table metrics: Geometry/Masking = mAP@5000; Patch = mAP@5000 / @all; Localization = hit rate (%). Retrieval values use the 0–1 scale.

TestVariant24 bits48 bits64 bits128 bits
GeometryNo geometry0.15710.23120.23510.3035
GeometryShuffled0.14410.22000.23160.2756
GeometrySST0.15810.23620.25470.3060
PatchClass-only0.1532 / 0.15270.2272 / 0.22690.2477 / 0.24810.2407 / 0.2405
PatchAttribute (SST)0.1581 / 0.15750.2362 / 0.23570.2547 / 0.25410.3060 / 0.3059
LocalizationAttention39.96%35.88%45.17%49.92%
LocalizationRandom20.13%20.10%20.03%20.13%
MaskingSST, unmasked0.15810.23620.25470.3060
MaskingSST, random0.14330.21010.21900.2725
MaskingSST, attention0.09690.20700.10870.2633
MaskingCLS, unmasked0.13180.20890.23830.2925
MaskingCLS, random0.11800.18280.20640.2542
MaskingCLS, attention0.07420.16150.10350.2342

Correct geometry targets improve mAP@5000 over shuffled targets by 1.40/1.62/2.31/3.04 percentage points; the same ordering holds at @all. Attribute targets outperform class-only targets at both cutoffs. Attention regions yield higher part hit rates and larger masking-induced retrieval drops than random regions. These results support meaningful semantic correspondence and part-associated, retrieval-relevant information.

The target substitution changes loss scale; localization does not certify each query's named attribute. Masking also affects CLS, so this effect is not exclusive to SST. We will add these controls and clarify the contribution accordingly.

Q3. Can COMAE and DA2H be compared under matched dataset splits and evaluation metrics?

We have added the requested comparison on AWA2, CUB, and SUN, retaining all four code lengths. The reproduced methods share the class split, query/database membership, relevance definition, and AP calculation. AP@K averages precision at relevant ranks among the first K retrieved items; queries with no relevant retrieval contribute zero. For each method, @5000 and @all use the same checkpoint and retrieval lists. The comparison controls data and evaluation; method-specific architectures and pretraining are retained.

Our SUN setting uses 574 seen / 143 unseen classes, 1,430 queries, and 7,170 database images, following the default split in the released COMAE loader. The manuscript's 645/72 split and retrieval counts were description errors and will be corrected; this clarification does not change the reported scores. We will explicitly state the implemented setting instead of transferring the textual setting from another paper.

Each entry below is mAP@5000 / mAP@all, at seed 42. The DA2H entries are our reproduction.

DatasetMethod24 bits48 bits64 bits128 bits
AWA2PZSH-original0.5117 / 0.45020.5846 / 0.54060.6164 / 0.56330.6412 / 0.5997
AWA2PZSH-MSE0.5132 / 0.45960.5877 / 0.53630.6149 / 0.57540.6336 / 0.6104
AWA2COMAE0.3836 / 0.33370.5121 / 0.45200.5086 / 0.46830.5705 / 0.5327
AWA2DA2H (our reproduction)0.4897 / 0.43130.5444 / 0.50320.5613 / 0.52030.6046 / 0.5736
AWA2SST0.5769 / 0.55630.6192 / 0.59390.6288 / 0.60530.6643 / 0.6516
CUBPZSH-original0.1318 / 0.13170.2164 / 0.21640.2244 / 0.22440.2691 / 0.2691
CUBPZSH-MSE0.1397 / 0.13920.2160 / 0.21530.2281 / 0.22760.2704 / 0.2701
CUBCOMAE0.1119 / 0.11150.1831 / 0.18270.2054 / 0.20500.2496 / 0.2490
CUBDA2H (our reproduction)0.1246 / 0.12410.1730 / 0.17260.2024 / 0.20210.2415 / 0.2412
CUBSST0.1581 / 0.15750.2362 / 0.23570.2547 / 0.25410.3060 / 0.3059
SUNPZSH-original0.1057 / 0.10510.1437 / 0.14180.1729 / 0.17170.1930 / 0.1906
SUNPZSH-MSE0.1055 / 0.10310.1458 / 0.14330.1771 / 0.17420.1949 / 0.1921
SUNCOMAE0.0950 / 0.09420.1233 / 0.12220.1482 / 0.14770.1874 / 0.1869
SUNDA2H (our reproduction)0.0855 / 0.08520.1345 / 0.13350.1653 / 0.16430.1873 / 0.1867
SUNSST0.1273 / 0.12600.1691 / 0.16670.2097 / 0.20700.2169 / 0.2154

SST is higher than original PZSH, PZSH-MSE, COMAE, and our DA2H reproduction in all twelve dataset–code-length settings under both cutoffs. These matched comparisons support the empirical advantage without mixing literature protocols or evaluation cutoffs. We will place them separately from contextual literature scores and document the implementations used.

Reviewer_GeNmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies zero-shot image hashing in a generative training setting, where real images are available for seen classes and Stable Diffusion supplies pseudo-images for unseen ones. The patch branch attends to local visual tokens and matches pairwise image-embedding similarities to the similarities between the corresponding class attribute vectors. The geometry loss aligns the continuous hash-center similarities of unseen–seen class pairs with a target built from attribute-presence vectors. The authors conduct experiments on several datasets to evaluate the proposed method.

Critical strengths
  1. The paper introduces two complementary semantic transfer mechanisms, targeting patch-level representations and hash center geometry. Their roles in the framework are reasonably well explained.

  2. The method shows improvements over the baseline across different datasets and code lengths. The ablation studies provide evidence that both proposed components contribute to the performance gains.

Critical weaknesses

1.The empirical comparison is most convincing against PZSH, the only recent baseline evaluated under the same protocol. Results for other published methods are contextual rather than directly matched, which limits the strength of the broader comparison. 2. The two transfer losses impose relational constraints, but the evidence linking them to fine-grained attribute transfer and improved binary retrieval remains indirect. The theoretical analysis also covers a narrower setting than the retrieval task evaluated in the paper.

Questions for authors
  1. COMAE (ICMR 2025) already explores local attribute representations and pairwise semantic relationships. Could the authors further clarify the main methodological novelty of SST compared with COMAE?

  2. The paper lists per-code-length component configuration as a contribution. How does this differ from standard hyperparameter tuning, given that the loss weights are selected separately for different code lengths?

  3. The PZSH baseline used in Table 1 adopts learnable centers and an MSE-based hash loss. How does this implementation differ from the original PZSH, and could these differences affect the comparison?

  4. The patch-level loss mainly matches pairwise similarities between image embeddings and class attributes. What evidence shows that it captures fine-grained local semantics rather than simply providing additional class-level regularization?

  5. Equation (4) aligns continuous class centers, while retrieval relies on binary hash codes. Could the authors provide more direct evidence that the learned geometric relationships are preserved after binarization?

  6. The geometry loss only considers unseen–seen class pairs, whereas retrieval also depends on relationships among unseen classes. Why are unseen–unseen pairs excluded from the geometry constraint?

4 comments

Dear Reviewer GeNm,

Thank you for your careful review and constructive feedback. We respond to this concern below.

Q1. COMAE already explores local attribute representations and pairwise semantic relationships. What is the main methodological novelty of SST compared with COMAE?

COMAE combines local-prototype attribute regression with attribute-level contrastive and class-wise objectives. SST instead jointly matches patch-embedding cosine similarities to class-attribute similarities and unseen–seen learnable-center similarities to attribute-presence similarities. Our contribution is this combination of representation-level and center-level relational supervision in generative zero-shot hashing. Q4 and Q6 provide controls for these choices.

Critical Weakness 1. The empirical comparison is most convincing only against the internal PZSH baseline.

We reproduced original PZSH, COMAE, and DA2H alongside PZSH-MSE using identical class splits, query/database membership, relevance rules, and AP calculation, while retaining method-specific architectures and pretraining. AP@K averages precision over relevant ranks within the top K (zero if none). Both cutoffs use the same checkpoint and ranking for each method.

SUN uses 574 seen / 143 unseen classes, 1,430 queries, and 7,170 database images, following the released COMAE loader. We will correct the manuscript's 645/72 split and retrieval counts; the evaluated data and reported scores remain unchanged.

Each cell is mAP@5000 / mAP@all, at seed 42:

DatasetMethod24 bits48 bits64 bits128 bits
AWA2PZSH-original0.5117 / 0.45020.5846 / 0.54060.6164 / 0.56330.6412 / 0.5997
AWA2PZSH-MSE0.5132 / 0.45960.5877 / 0.53630.6149 / 0.57540.6336 / 0.6104
AWA2COMAE0.3836 / 0.33370.5121 / 0.45200.5086 / 0.46830.5705 / 0.5327
AWA2DA2H (our reproduction)0.4897 / 0.43130.5444 / 0.50320.5613 / 0.52030.6046 / 0.5736
AWA2SST0.5769 / 0.55630.6192 / 0.59390.6288 / 0.60530.6643 / 0.6516
CUBPZSH-original0.1318 / 0.13170.2164 / 0.21640.2244 / 0.22440.2691 / 0.2691
CUBPZSH-MSE0.1397 / 0.13920.2160 / 0.21530.2281 / 0.22760.2704 / 0.2701
CUBCOMAE0.1119 / 0.11150.1831 / 0.18270.2054 / 0.20500.2496 / 0.2490
CUBDA2H (our reproduction)0.1246 / 0.12410.1730 / 0.17260.2024 / 0.20210.2415 / 0.2412
CUBSST0.1581 / 0.15750.2362 / 0.23570.2547 / 0.25410.3060 / 0.3059
SUNPZSH-original0.1057 / 0.10510.1437 / 0.14180.1729 / 0.17170.1930 / 0.1906
SUNPZSH-MSE0.1055 / 0.10310.1458 / 0.14330.1771 / 0.17420.1949 / 0.1921
SUNCOMAE0.0950 / 0.09420.1233 / 0.12220.1482 / 0.14770.1874 / 0.1869
SUNDA2H (our reproduction)0.0855 / 0.08520.1345 / 0.13350.1653 / 0.16430.1873 / 0.1867
SUNSST0.1273 / 0.12600.1691 / 0.16670.2097 / 0.20700.2169 / 0.2154

SST exceeds every reproduced baseline in all twelve dataset–code-length settings under both cutoffs. We will add these matched results and the clarified protocol to the revision.

Q3. How does the learnable-center, MSE-based PZSH baseline differ from original PZSH, and could the differences affect the comparison?

The baseline in the submitted main table is an MSE-based adaptation: removing the two transfer objectives recovers this internal baseline. We will label it PZSH-MSE and separately identify the reproduction retaining original PZSH's fixed-center construction and center-classification, quantization, and proxy losses. Original PZSH loss implementation

We added controls that separate the original-loss branch from the MSE branch. O0/O1 retain fixed centers; O2/O3/O5 retain the original hash losses and allow centers to learn. All rows use the same CUB retrieval protocol. Values are mAP@5000 at seed 42.

Variant24 bits48 bits64 bits128 bits
O0: original losses, fixed centers0.13180.21640.22440.2691
O1: O0 + patch0.14800.21950.23420.2718
O2: original losses, learnable centers0.14780.22100.23590.2802
O3: O2 + patch0.14980.22130.23890.2813
O5: O3 + geometry0.15310.22260.24400.2819
M0: MSE baseline0.13970.21600.22810.2704
M1: M0 + patch0.15710.23120.23510.3035
M2: M0 + geometry0.15560.22260.24050.2881
SST: M0 + patch + geometry0.15810.23620.25470.3060

With fixed original centers, adding patch supervision improves O1 over O0 by 1.62/0.31/0.98/0.27 points. Within the learnable-center original-loss branch, O3 exceeds O2 and O5 exceeds O3 at all four lengths; the geometry increments are 0.334/0.130/0.510/0.064 points. The same ordering holds under @all. Thus the positive effects are observed under the original hash losses as well as the MSE objective. Because geometry acts on trainable centers, O2/O3/O5 are explicitly labeled as center adaptations, not unchanged fixed-center PZSH. We will add these distinctions and controls to the revision.

Q4. What evidence shows that the patch-level loss captures local semantics rather than simply providing additional class-level regularization?

We added three controls on CUB (training seed 42):

Target. Replace attribute-cosine targets with same-class/different-class labels (1/0), keeping the remaining model, initialization protocol, and loss weights fixed.

Localization. Freeze query-to-part mappings selected on seen-validation images, then measure visible-part hit rates within top-attention 20% regions on unseen images, against equal-area random regions.

Masking. Mask 20% of each query image while fixing the database, checkpoint, and fill value. SST and CLS share regions from frozen SST attention; random-mask results average ten mask seeds.

Table metrics: Target = mAP@5000 / @all; Localization = hit rate (%); Masking = mAP@5000. Retrieval values use the 0–1 scale.

TestVariant24 bits48 bits64 bits128 bits
TargetClass-only0.1532 / 0.15270.2272 / 0.22690.2477 / 0.24810.2407 / 0.2405
TargetAttribute (SST)0.1581 / 0.15750.2362 / 0.23570.2547 / 0.25410.3060 / 0.3059
LocalizationAttention39.96%35.88%45.17%49.92%
LocalizationRandom20.13%20.10%20.03%20.13%
MaskingSST, unmasked0.15810.23620.25470.3060
MaskingSST, random0.14330.21010.21900.2725
MaskingSST, attention0.09690.20700.10870.2633
MaskingCLS, unmasked0.13180.20890.23830.2925
MaskingCLS, random0.11800.18280.20640.2542
MaskingCLS, attention0.07420.16150.10350.2342

The attribute target improves mAP at all four lengths and both cutoffs. Attention regions yield higher part hit rates and larger masking-induced retrieval drops than random regions. Together, these controls support part-associated, retrieval-relevant visual information.

The target comparison uses fixed weights despite a changed loss scale; localization does not certify each query's named attribute. Masking effects also occur for CLS, so they are not exclusive to SST. We will add these controls and state their scope explicitly.

Q6. Why are unseen–unseen pairs excluded from the geometry constraint?

Our design connects unseen centers supported by synthetic training examples with seen centers also supported by real images. To test this choice empirically, we added U–U-only and combined-block controls while retaining the remaining configuration. CUB mAP@5000 at seed 42 is:

Variant24 bits48 bits64 bits128 bits
U–S only (SST)0.15810.23620.25470.3060
U–U only0.14200.22120.24070.2834
0.5 U–S + 0.5 U–U0.14400.21950.23740.2769

Each block is averaged separately. The combined variant uses 0.5 times the U–S mean plus 0.5 times the U–U mean, with the same outer geometry weight. U–S gives the highest retrieval score at all four lengths, and this ordering also holds at @all. These results support the selected U–S configuration at the tested weights.

Reviewer_591ROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The proposed approach SST adds two transfer modules to a PZSH-style zero-shot hashing pipeline. (i) patch-level relational distillation. Attribute queries attend over BLIP patch tokens and a Gram-matrix loss aligns batchwise embedding similarities with class-attribute similarities. (ii) attribute geometry alignment, which constrains unseen–seen center cosine similarities using attribute-presence vectors.

Critical strengths
  1. The paper presents a clear motivation: insufficient semantic transfer in generative zero-shot hashing.
  2. The paper is easy to follow.
Critical weaknesses
  1. The "patch module" overlaps substantially with DA2H (cited in the paper). DA2H already introduces an attribute-aware Transformer with a semantic-aware decoder that localizes attribute-relevant regions for zero-shot hashing. SST's attribute-query attention over patch tokens is close to it. The paper cites DA2H but does not compare against it in the experiments.

  2. The "geometry module"'s gains are modest. Removing it causes drops of 0.0196–0.0217 at several settings, smaller than the patch term's effect on AWA2-24 (0.0421). The contribution is incremental relative to center-based methods.

  3. Relational distillation for hashing is not new. Missing related work DRKDH (Deep Relational Knowledge Distillation Hashing via Relaxed Masking Triplet Optimization for Large-scale Image Retrieval). DRKDH distills relational knowledge into hashing networks. SST uses fixed attribute similarities rather than a learned teacher, but the paper neither discusses this related work nor explains why fixed targets are preferable.

  4. The comparison is not fair. Table 1 states that literature scores retain their source protocols as contextual references, but the evaluation protocols and class splits are inconsistent across rows. For example, SUN contains 717 classes in the cited source, while this paper uses 500 seen and 217 unseen classes. Imported scores therefore do not share the same class identities, database membership, or AP normalization as SST's runs, and direct numerical comparison across rows is not meaningful.

Questions for authors

NA

5 comments

Dear Reviewer 591R,

Thank you for your careful review and constructive feedback. We respond to this concern below.

Critical Weakness 1. The patch module overlaps with DA2H, and DA2H is not compared experimentally.

We agree that attribute-aware regional modeling has prior precedents, including DA2H. The contribution we wish to establish is the particular combination of two relation objectives in generative zero-shot hashing: a batchwise attribute-similarity target for patch-derived visual embeddings, and an unseen–seen attribute-presence target for learnable hash centers. The two objectives supervise different objects and are optimized jointly through the shared hashing pipeline.

We added our DA2H reproduction under the shared class split, query/database membership, relevance definition, and AP calculation. Each cell reports mAP@5000 / mAP@all at seed 42, on the 0–1 scale. The comparison controls data and evaluation; method-specific architectures and pretraining are retained.

DatasetMethod24 bits48 bits64 bits128 bits
AWA2DA2H (our reproduction)0.4897 / 0.43130.5444 / 0.50320.5613 / 0.52030.6046 / 0.5736
AWA2SST0.5769 / 0.55630.6192 / 0.59390.6288 / 0.60530.6643 / 0.6516
CUBDA2H (our reproduction)0.1246 / 0.12410.1730 / 0.17260.2024 / 0.20210.2415 / 0.2412
CUBSST0.1581 / 0.15750.2362 / 0.23570.2547 / 0.25410.3060 / 0.3059
SUNDA2H (our reproduction)0.0855 / 0.08520.1345 / 0.13350.1653 / 0.16430.1873 / 0.1867
SUNSST0.1273 / 0.12600.1691 / 0.16670.2097 / 0.20700.2169 / 0.2154

SST obtains higher mAP than this DA2H reproduction on all three datasets and at all four code lengths, under both cutoffs. We will add this comparison and frame the contribution around the specific relation objectives and measured gains, while explicitly acknowledging prior attribute-aware regional modeling.

Critical Weakness 2. The geometry gains are modest and the contribution appears incremental.

To isolate the contribution of geometry, we compare the full model with removal of geometry and with a mismatched semantic target. The patch branch remains enabled in all three variants. CUB mAP@5000 at seed 42 is:

Variant24 bits48 bits64 bits128 bits
Without geometry0.15710.23120.23510.3035
Mismatched geometry target0.14410.22000.23160.2756
SST0.15810.23620.25470.3060

Adding geometry improves mAP by 0.10/0.50/1.96/0.25 percentage points at 24/48/64/128 bits. In the target control, class-to-attribute correspondence is permuted within seen and unseen groups; the loss form, weight, and target-value distribution are preserved. Correct correspondence improves mAP over the mismatched target by 1.40/1.62/2.31/3.04 points. This identifies a contribution of the correctly assigned semantic target beyond merely adding the same loss with mismatched relations.

The original appendix also reports three matched training seeds for six settings (AWA2-64, CUB-24/64/128, and SUN-64/128). The full model exceeds the corresponding no-geometry run in each of these 18 paired comparisons. These existing repetitions complement the new seed-42 controls; they are not repetitions of the newly added target control. We will present geometry as a complementary contribution whose magnitude depends on the setting, without calling the single-run differences statistically significant.

Critical Weakness 3. Missing DRKDH and the motivation for fixed attribute-relation targets.

Thank you for pointing out DRKDH. We will add it to the related-work discussion and acknowledge relational knowledge distillation as established prior work. The released DRKDH implementation provides a two-stage workflow with saved embeddings between stages.

Our use of fixed relations is motivated by the information available in this zero-shot setting: class attributes are provided for both seen and unseen classes, whereas real images of the final unseen classes are unavailable for training. These attributes therefore supply a common relation target for the two class groups without requiring an additional relation-teacher training stage. Our CUB semantic-mismatch control preserves the geometry loss form, weight, and target-value distribution while permuting class–attribute assignments. Correct correspondence improves mAP@5000 by 1.40/1.62/2.31/3.04 percentage points at 24/48/64/128 bits, providing evidence that the attribute correspondence is useful. We will state this setting-specific motivation, rather than claim that fixed targets universally outperform learned teachers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.