- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates how to further enhance the retrieval performance of existing cross-modal hashing systems without increasing hash code length or retraining the base models. The authors propose ReCapHash, a plug-in enhancement framework applicable to various base hashing methods. The core idea is to treat existing hash codes as reallocatable coding resources. By evaluating the contribution of individual hash bits, the framework retains important bits and reclaims inefficient ones. It then encodes previously under-represented semantic information into the reclaimed bits by integrating residual relationship mining, binary recovery, and cross-modal distillation. The framework requires no modification to the original hashing models' learning objectives, and the final encoding length remains unchanged. Experimental results demonstrate that ReCapHash achieves strong retrieval performance across multiple public datasets and varying code lengths, and enhances the performance of various existing hashing methods.
Critical strengths
(1) The plug-in design of ReCapHash offers practical value. It improves retrieval performance across different base hashing methods without modifying their original learning objectives, demonstrating good flexibility and applicability.
(2) The capacity reallocation strategy under a fixed bit budget is well conceived. By reusing low-utility hash bits, ReCapHash improves retrieval performance without increasing the original code length or storage overhead, making it practical for large-scale retrieval.
Critical weaknesses
(1) ReCapHash employs a uniform semantic candidate range for different query samples. However, the density of semantic neighborhoods can vary significantly across samples. A fixed number of candidates may result in some queries including weaker semantic relationships while simultaneously limiting the candidate coverage for semantically rich samples.
(2) Samples with fewer connections in the residual relation graph may have insufficient relational information for reliable residual recovery. The paper does not examine how node degree affects recovery quality.
(3) The paper primarily reports overall retrieval performance, offering limited analysis of improvements across queries of varying difficulty. Residual capacity reallocation may yield greater benefits for samples where the baseline hashing model performs poorly, whereas gains for simple queries might be constrained by their already high baseline performance.
(4) ReCapHash constructs the residual relation graph only once during training, which may limit its adaptability to new training data or distribution shifts. The paper does not discuss whether the graph can be updated efficiently in such cases.
Questions for authors
-
When evaluating the contribution of hash bits, the paper selects candidates ranked immediately after the semantic positive set as negative samples. Why were these relatively similar candidates selected instead of samples with lower similarity?
-
Since ReCapHash introduces additional residual prediction heads, how much inference-time overhead does it add compared with the original base hashing models?
-
If ReCapHash is extended to three or more modalities, such as image, text, and audio, how should the residual relationship mining and shared residual graph construction be adjusted?
4 comments
Response to Reviewer bTUx (Weaknesses) [1/2]
W1. Fixed Semantic Candidate Range
Thanks for your suggestion. ReCapHash employs a two-stage selection mechanism. The top- semantic candidates define a candidate pool, while only the top- core-conditioned residual relations are retained, prioritizing semantically supported yet underrepresented relationships. To further evaluate candidate selection, we compare different fixed ranges with density-adaptive selection, whose average candidate budget matches the default .
Table 1: Comparison of semantic candidate selection strategies at 32 bits (mAP@50).
| Candidate Strategy | MIR I→T | MIR T→I | NUS I→T | NUS T→I | IAPR I→T | IAPR T→I |
|---|---|---|---|---|---|---|
| Fixed | 0.8922 | 0.8528 | 0.7904 | 0.7618 | 0.7608 | 0.7425 |
| Fixed | 0.8858 | 0.8493 | 0.7912 | 0.7590 | 0.7621 | 0.7424 |
| Density-Adaptive | 0.8862 | 0.8530 | 0.7921 | 0.7623 | 0.7604 | 0.7425 |
| Fixed (Ours) | 0.8977 | 0.8626 | 0.7953 | 0.7632 | 0.7631 | 0.7497 |
As shown in Table 1, the default consistently outperforms both smaller and larger fixed candidate ranges, as well as density-adaptive selection, across all datasets and retrieval directions. These results suggest that a moderate fixed candidate range provides a favorable balance between semantic coverage and noise suppression, without requiring query-specific adaptation.
W2. Effect of Node Degree on Residual Recovery
Thanks for your comment. Appendix C.4 analyzes the structural properties of the residual graphs, with Table 4 showing that all three graphs remain connected, with average degrees of 88.00, 80.17, and 81.36. To further examine recovery reliability, we evaluate the correlation between node degree and residual graph agreement, and compare recovery quality across low- and high-degree nodes.
Table 2: Relationship between node degree and residual recovery quality at 32 bits.
| Metric | MIRFlickr | NUS-WIDE | IAPR-TC12 |
|---|---|---|---|
| Average Degree | 88.00 | 80.17 | 81.36 |
| Degree–Recovery Spearman's | -0.1399 | -0.0375 | -0.0048 |
| Low-Degree Agreement (Bottom 25%) | 0.8644 | 0.8480 | 0.8376 |
| High-Degree Agreement (Top 25%) | 0.8562 | 0.8390 | 0.8304 |
Degree–recovery correlations are weak across all three datasets, while low-degree nodes achieve comparable or slightly higher recovery agreement than high-degree nodes. These results indicate that lower graph connectivity is not systematically associated with poorer residual recovery.
W3. Query-Level Improvements Across Different Difficulties
Thanks for your suggestion. To examine query-level improvements, we rank test queries by their AP@50 under the original hashing model and divide them into three equal-sized groups: Hard, Medium, and Easy. The corresponding retrieval gains are reported below.
Table 3: Retrieval improvements across query difficulty levels at 128 bits (ΔmAP@50).
| Dataset | Task | Hard | Medium | Easy |
|---|---|---|---|---|
| MIRFlickr | I→T | +0.1975 | +0.0896 | +0.0515 |
| T→I | +0.2030 | +0.0970 | +0.0613 | |
| NUS-WIDE | I→T | +0.1243 | +0.0701 | +0.0357 |
| T→I | +0.1206 | +0.0611 | +0.0370 | |
| IAPR-TC12 | I→T | +0.1108 | +0.0697 | +0.0287 |
| T→I | +0.1069 | +0.0619 | +0.0218 |
Hard queries consistently achieve the largest retrieval gains across all datasets and directions, demonstrating the effectiveness of ReCapHash on challenging queries.
W4. Incremental Updating of Residual Relations
Thanks for your comment. ReCapHash constructs the residual graph during offline training, but its learned modality-specific predictors can directly encode unseen samples without reconstructing the graph or repeating capacity reallocation. For newly added training data, local graph updates are possible in principle if the reclaimed coordinates remain fixed. Efficient incremental residual recovery and adaptation to distribution shifts remain promising directions for future work.
Response to Reviewer bTUx (Questions) [2/2]
Q1. Selection of Hard Negative Samples
Thanks for your feedback. We select candidates immediately following the semantic positive set as informative hard negatives. These candidates are semantically close to the query but lie outside its positive neighborhood, making them more effective for evaluating bit-level ranking discrimination. In contrast, distant negatives are often trivially separable and provide limited information about individual bit contributions. This design enables more discriminative marginal utility estimation.
Q2. Inference-Time Overhead
Thanks for your comment. ReCapHash introduces only lightweight modality-specific residual prediction heads while keeping the original feature encoders and hash learners unchanged. Residual graph construction and spectral recovery are performed offline and introduce no inference-time graph computation. We further measure the encoding latency before and after applying ReCapHash.
Table 4: Inference-time overhead of ReCapHash on MIRFlickr at 32 bits.
| Base Method | Base Latency (ms) ↓ | +ReCapHash (ms) ↓ | Overhead (%) ↓ |
|---|---|---|---|
| HEH | 0.2296 | 0.2315 | 0.83 |
| USFTH | 0.1036 | 0.1050 | 1.35 |
| ENHash | 0.2055 | 0.2071 | 0.78 |
| HINT | 0.0947 | 0.0957 | 1.06 |
| UWMCH | 0.2053 | 0.2063 | 0.49 |
ReCapHash introduces only 0.49%–1.35% encoding overhead across five base methods, demonstrating its lightweight inference cost.
Q3. Extension to Multiple Modalities
Thanks for your suggestion. ReCapHash can be naturally extended to multiple aligned modalities. For image, text, and audio, residual relations can be mined across all modality pairs by comparing pretrained semantic affinities with retained-core affinities. These relations can then be aggregated into a shared residual graph, where each aligned multimodal instance forms a node. Global spectral recovery produces shared residual targets, which are distilled into modality-specific prediction heads. This preserves the fixed-budget capacity reallocation principle across multiple modalities.
Thank You
We hope our additional analyses and experimental results have addressed your concerns and further demonstrated the robustness and practical effectiveness of ReCapHash. We will release the corresponding code and experimental logs to support reproducibility. We sincerely appreciate your consideration and welcome any further feedback.
Sincerely,
The Authors
Thanks for your review
Dear Reviewer bTUx,
Thank you for your thorough and constructive review. We greatly appreciate your recognition of the practical value and key contributions of ReCapHash. Your feedback provides a valuable opportunity to further clarify our methodology and strengthen its experimental validation. Below, we provide detailed responses and additional experimental evidence.