Beyond Holistic Alignment: Structured Semantic Transfer for Zero-Shot Hashing

Zero-shot image hashing aims to learn a compact binary code that generalizes to unseen classes without real training images of those classes, using semantic side information. Recent generative pipelines synthesize pseudo-images for unseen classes and align image features through contrastive objectives, yet the hash space itself is supervised mainly through holistic features and class centers without explicitly exploiting structured semantic relations. Two complementary forms of semantic structure therefore remain underexplored: attribute-related patterns encoded in visual representations and the relational geometry that connects seen and unseen classes. We propose a novel framework termed Structured Semantic Transfer (SST) for zero-shot hashing, which preserves semantic structures beyond holistic alignment. A lightweight branch captures attribute-aware representations from local patch features, and a pairwise structure distillation loss aligns the similarities among these representations with those of their classes, enabling the encoder to learn transferable semantic structures. Meanwhile, a geometry alignment loss preserves the attribute-induced relations among hash centers, guiding unseen class centers through structured semantic propagation. We further introduce a per-code-length component configuration that adapts the overall scale of the two loss weights at each operating point while keeping the framework unchanged. Experiments on standard zero-shot hashing benchmarks show that SST consistently improves retrieval over strong baselines, and ablations confirm that the two semantic transfer mechanisms are complementary.