- Rating
- 3
- Confidence
- 1(−2)
- Informativeness
- 2
- Presentation
- 2(−1)
- Soundness
- 2
Read review
Paper summary
This paper proposes HOCH for unsupervised cross-modal hashing. Cascaded interaction blocks combine normalized low-rank multiplicative features with gated residual updates to progressively refine paired image and text representations. Adaptive modality fusion constructs a multimodal teacher, whose cross-modal and intra-modal similarity structures are distilled into independently deployable unimodal students. Teacher and student branches share modality-specific encoders but use separate hash heads. Experiments cover three datasets, two retrieval directions, and four hash lengths using fixed CLIP features, supplemented by component ablations, parameter sensitivity analyses, and efficiency measurements.
Critical strengths
The framework addresses a practical training–deployment distinction: paired modalities can provide richer supervision during training, while each deployed student requires only its own modality. The architecture makes this distinction explicit, and the reported inference paths exclude the interaction and fusion modules. This gives the approach a useful deployment rationale beyond improving a jointly evaluated multimodal representation.
The empirical evaluation is reasonably broad. The paper states that all methods use identical partitions, fixed CLIP features, hash lengths, and evaluation protocols. HOCH achieves the highest reported mAP@50 in 20 of 24 settings, with particularly substantial improvements on MSCOCO at short code lengths. The mAP@K curves provide additional evidence beyond one retrieval depth. Table 3 also reports measured training costs relative to FirstOrder, supporting the practical value of the proposed compact interaction design in that setting.
Critical weaknesses
The paper does not specify the number of independent training runs or report variability for retrieval accuracy; the latency standard deviations do not address this uncertainty. Several important settings are also missing, including the pseudo-similarity quantile, hash sharpness, ranking margins and temperatures, regularization weights, and teacher/student weighting schedules. Encoder and aggregation architectures are not fully specified, and checkpoint-selection and tuning procedures remain unclear. These omissions matter when interpreting small component-level gains and reproducing the method without the planned code release.
Questions for authors
Could you provide the exact FirstOrder formulation and explain the parameter-count difference in Table 3? On one representative dataset at 64 bits, could you compare HOCH with an additive variant retaining the same low-rank projections, normalization, gates, and output dimensions? A single-block multiplicative variant would also help determine whether recursive refinement provides an additional benefit beyond multiplicative interaction itself.
Could you report mean and standard deviation over at least three training seeds for HOCH, FirstOrder, and MeanFusion on one representative 64-bit setting, keeping the data split fixed? Reporting the paired differences between variants would be particularly informative. For example, on NUS-WIDE, the reported gains over FirstOrder and MeanFusion are 0.0037 and 0.0007, respectively; their consistency across runs could materially affect the assessment of the individual modules.
Could you provide a compact configuration table specifying the pseudo-similarity quantile, hash sharpness, DropPath probabilities, ranking margins and temperatures, loss coefficients, and teacher/student weighting schedules, together with the encoder and aggregation architectures? Please also explain how checkpoints and hyperparameters were selected, including the dataset-specific feature-preservation weights, and whether any labeled validation data were used. This would clarify the reproducibility and supervision requirements of the reported results.
5 comments
Response to Reviewer XY4F [2/5]
Response to Weakness 1 & Question 1: FirstOrder formulation, parameter-count differences, and controlled interaction analysis.
Thank you for asking us to separate the effects of the interaction operator and recursive refinement, and to make the parameter comparison reproducible. We audited the models module by module. At 64 bits, HOCH contains 12,549,890 parameters: 4,726,784 in the two modality encoders, 3,150,336 in the shared aggregation network, 4,540,416 in the four interaction blocks, 131,328 in the four teacher/student hash heads, and 1,026 in adaptive fusion.
In the originally submitted FirstOrder variant, the entire multiplicative interaction branch in each block is replaced by a full-width additive projection:
where is a learnable 512-by-512 projection matrix. This additive projection and its LayerNorm contain 263,680 parameters per block, compared with 83,456 in HOCH's multiplicative interaction branch. The difference is 180,224 per block and 720,896 across four blocks, giving FirstOrder 13,270,786 parameters (13.27M). The gates and residual updates are identical in these two variants; the parameter difference is confined to the interaction branch.
We also report the A4* additive control from our controlled experiments. A4* retains HOCH's four-block recursive architecture, normalized low-rank projections, gates, output dimensions, aggregation, fusion, and distillation, replacing only element-wise multiplication with addition. Both variants have 12.55M active parameters.
Table R1(a). Parameter-matched interaction comparisons at 64 bits (three-seed mean ± sample SD of bidirectional average mAP@50).
| Method | Operator | Interaction inputs | MIRFlickr | NUS-WIDE | MSCOCO |
|---|---|---|---|---|---|
| HOCH | Multiplication | Updated | 0.9181 ± 0.0010 | 0.8260 ± 0.0019 | 0.9411 ± 0.0026 |
| A4* | Addition | Updated | 0.9174 ± 0.0005 | 0.8263 ± 0.0019 | 0.9414 ± 0.0016 |
We additionally evaluated M1, a single-block multiplicative variant that retains the first interaction block's gate and residual update and omits the subsequent three blocks. At 64 bits, the three-seed means ± sample standard deviations of bidirectional average mAP@50 are:
Table R1(b). Single-block versus four-block multiplicative interaction at 64 bits. Values are the mean ± sample standard deviation of bidirectional average mAP@50 over three training seeds. The last column reports the difference between the HOCH and M1 means.
| Dataset | M1 (one block) | HOCH (four blocks) | HOCH − M1 |
|---|---|---|---|
| MIRFlickr | 0.9176 ± 0.0012 | 0.9181 ± 0.0010 | +0.0005 |
| NUS-WIDE | 0.8250 ± 0.0021 | 0.8260 ± 0.0019 | +0.0010 |
At 64 bits, four-block HOCH improves mean bidirectional mAP@50 over the single-block M1 variant on MIRFlickr and NUS-WIDE, showing that additional interaction stages contribute beyond a single multiplicative block in these settings. In the revised manuscript, we will clarify the FirstOrder formulation and parameter-count difference in Appendix C.4, and briefly summarize the parameter-matched A4* and single-block M1 comparisons there.
Response to Reviewer XY4F [3/5]
Response to Weakness 2 & Question 2: multi-seed ablation results and statistical significance analysis.
Thank you for pointing out that the small single-run ablation gains need a variability estimate. We repeated HOCH, FirstOrder, and MeanFusion on NUS-WIDE at 64 bits with the same data split and three training seeds (2024, 2025, and 2026).
Table R2(a). Three-seed NUS-WIDE ablation at 64 bits.
| Method | I2T | T2I | Average | HOCH − variant |
|---|---|---|---|---|
| HOCH | 0.8385 ± 0.0015 | 0.8135 ± 0.0025 | 0.8260 ± 0.0019 | — |
| FirstOrder | 0.8321 ± 0.0029 | 0.8111 ± 0.0026 | 0.8216 ± 0.0009 | +0.0044 ± 0.0012 |
| MeanFusion | 0.8399 ± 0.0015 | 0.8128 ± 0.0042 | 0.8263 ± 0.0024 | −0.0003 ± 0.0012 |
Table R2(b). Teacher-relation Precision@50 with original and shuffled AMF weight assignments on NUS-WIDE at 64 bits.
| seed | Original P@50 | Shuffled P@50 | Paired difference |
|---|---|---|---|
| 2024 | 0.826210 | 0.824166 | +0.002044 |
| 2025 | 0.839207 | 0.829679 | +0.009528 |
| 2026 | 0.824007 | 0.824009 | −0.000002 |
HOCH exceeds FirstOrder in all three training seeds; its mean paired gain is +0.0044 ± 0.0012. HOCH and MeanFusion are approximately tied in bidirectional retrieval performance. However, the sample-specific fusion weights still help the teacher model preserve semantic neighbors. To test whether the learned sample-to-weight assignment matters to the multimodal teacher, we held the teacher checkpoint and input features fixed and shuffled the learned image-text fusion weights across samples. The original assignment gave higher teacher-relation Precision@50 in two seeds and a negligible difference in the third; its mean paired advantage was +0.003857 ± 0.005017.
In the revised manuscript, we will add the three-seed results and paired differences to the ablation appendix and update the discussion accordingly.
Response to Reviewer XY4F [1/5]
Dear Reviewer XY4F,
We sincerely thank you for your careful review and constructive questions about the FirstOrder comparison, the reliability of the ablation results, and the reproducibility of our experimental protocol. In response, we clarified the FirstOrder formulation and parameter counts, conducted controlled interaction comparisons and three-seed ablations, and brought the training settings and model-selection procedure together in a concise account.
We hope these additions make the evidence behind our conclusions easier to assess, and we would be grateful if you would reconsider our work.
Sincerely, Co-authors