- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The manuscript proposes a novel distillation framework for cross-modal hashing that decouples reliable knowledge from inherent teacher noise. Recognizing that pre-trained teacher models often produce redundant and unstable hash codes, especially when dealing with complex, multi-label semantic relations, the manuscript designs a dual-granularity purification system. At the micro-level, the method employs an elbow-based and label-cardinality-guided clustering approach to compute the entropy and polarity of each bit, which subsequently informs a continuous reliability mask. At the macro-level, the framework adjusts the contrastive learning objectives by filtering out false-positive supervision and shielding latent semantic neighbors from incorrect false-negative repulsion. Comprehensive evaluations illustrate its capability to yield efficient students that consistently surpass their teacher counterparts across multiple code lengths.
Critical strengths
(1) The core mechanism of decoupling reliability from noisy supervision offers an insightful perspective in the domain of multimodal hashing. Rather than treating the output of teacher as an absolute standard, the framework utilizes information-theoretic measures to assess the trustworthiness of each individual hash dimension. This design prevents the student from inheriting the representational biases of teacher and memorizing random noise, which bridges the gap between information theory and discrete optimization in cross-modal retrieval. (2) The manuscript provides thorough ablation studies that isolate and validate the individual contributions of the reliability masks, semantic filtering mechanisms, and internal contrastive objectives. The logical flow from identifying the problem of teacher noise to addressing it via bit-wise and semantic-wise purification is coherent. The consistent improvements, especially the performance leaps observed when distilling from weaker, noisier teachers, corroborate the theoretical claims of the paper.
Critical weaknesses
The problem formulation and technical contributions are well articulated, but there are a few areas regarding algorithmic robustness, parameter sensitivity, and computational transparency that require clarification: (1) The framework heavily depends on an offline dual-clustering step, particularly relying on the elbow method to determine the coarse partitions and pseudo-labels. Heuristic clustering can be quite fragile under extreme noise distributions, the potential failure modes of this initialization phase are underexplored. An incorrect clustering granularity could systematically propagate erroneous ones. (2) Furthermore, although the paper mentions the offline nature of the preprocessing steps, it fails to explicitly quantify the exact computational time cost it adds to the overall pipeline. While bit-wise calculations on centroids might be efficient, performing K-means clustering over the entire dataset could become a bottleneck. (3) The framework introduces several heuristic thresholds for bit masking (e.g., , , and parameters). Although setting defaults promotes cross-dataset generalization, the lack of deep discussion on how these parameters affect the accuracy.
Questions for authors
Q1. How robust is the elbow-based clustering approach when applied to varying noise levels? Are there scenarios where incorrect clustering granularity severely degrades the downstream semantic filtering? Q2. Explicitly quantify the additional time cost introduced by the offline secondary clustering and bit-wise entropy/polarity calculations. How does this preprocessing overhead scale computationally as the size of the dataset increases? Q3. The overall method relies on several threshold values. Is the framework highly sensitive to small perturbations in these heuristic thresholds? Besides, the current methodology focuses on image-text pairs. If this framework were to be extended to more than two modalities (e.g., integrating video, audio, and text) simultaneously, would the dual-granularity purification still hold?
7 comments
Response to Reviewer CB94 [2/5]
Response to Weakness 1 & Question 1: robustness of elbow-based clustering and safeguards against noise.
1. Low-dimensional discrete space avoids the disadvantage of dimensionality.
Different from traditional clustering applied to high-dimensional continuous visual or text features, where heuristic elbow criteria often become unstable. The proposed dual-clustering operates strictly on compact, low-dimensional binary hash codes ( bits). Specifically, the elbow-based clustering employs standard -means with -means++ initialization, searching within a bounded range guided by dataset label cardinality to establish a stable coarse baseline ().
2. Multi-layered error mitigation and graceful degradation.
Even if the initial elbow clustering produces imperfect boundaries under noisy teacher outputs, three built-in safeguards prevent error propagation:
- Complementary second granularity and reliability filtering: any boundary ambiguity in the coarse partition is actively rectified by the secondary label-cardinality clustering () and dual-criterion bit filtering (polarization and entropy).
- In macro-level semantic filtering, cluster assignments are used to check cross-modal consensus (). An imperfect partition simply causes uncertain sample pairs to fail the consensus check and be excluded from distillation candidate sets.
- Compensation via intra-student contrastive learning: any supervision sparsity caused by strict filtering is compensated by our holistic intra-student contrastive objective (), ensuring continuous global manifold utilization.
3. Empirical verification across varying noise levels.
In our setting, noise directly originates from the inherent imperfections of different teacher models. Across all seven teachers in Table 1, RDDH achieves consistent gains, notably delivers the largest improvements on weaker, noisier teachers, e.g., +13.4% average I2T mAP gain on DJSRH vs. +2.8% on DHRL.
Response to Reviewer CB94 [3/5]
Response to Weakness 2 & Question 2: quantitative preprocessing overhead and dataset scaling analysis.
1. Strict offline execution with zero inference overhead.
All dual-clustering, entropy, polarization, and mask construction operations are executed strictly once during an offline initialization phase prior to iterative network training. Consequently, they add 0 ms of computational overhead during query retrieval, where the lightweight student simply performs a single forward pass.
2. Theoretical complexity and empirical runtime measurements.
- -means clustering on compact binary hash codes has a time complexity of , where is the number of iterations, is the cluster count, and is the code length. Because it operates on low-dimensional binary vectors, it scales linearly with the number of training samples .
- On an NVIDIA RTX 4070 GPU, the complete preprocessing pipeline takes only about 4 seconds on MIRFlickr (, 128 bits) and about 2 minutes on the large-scale NUS-WIDE dataset (, 128 bits).
- Crucially, bit-wise polarization and entropy are computed exclusively on the cluster centroids. This yields a constant complexity of , which is completely independent of dataset size .
- Even on large datasets, total offline preprocessing accounts for less than 1.5% of the overall training time, confirming that it does not become a computational bottleneck.
Response to Reviewer CB94 [1/5]
Dear Reviewer CB94,
We appreciate your thorough review and constructive suggestions concerning the noise robustness of elbow-based clustering, offline preprocessing overhead, threshold sensitivity, and extension to more than two modalities. In response, we have quantified the exact computational complexity and runtime of the offline clustering and reliability calculations, and provided detailed analyses on parameter sensitivity and multi-modal generalizability. We have addressed your comments in five sections. We will release the corresponding code and experimental logs to support reproducibility.
We would be profoundly grateful if you would consider re-evaluating our work.
Sincerely,
Co-authors