- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work examines fine-grained semantic degradation arising when multi-label semantic relations get mapped into finite Hamming spaces. It looks at distance collisions, ranking inversion, and inconsistent ranking behavior across different code lengths. The proposed MultiBit framework introduces a fractional relation teacher built from label co-occurrence and specificity, scale-aware bit organization, plus a multi-length prefix alignment strategy. With nested prefix design, a single model can produce multiple code lengths in one forward pass. The authors test image-to-text and text-to-image retrieval over several datasets and four code lengths. They also run ablations to study relevance resolution, quantization error, and ranking consistency.
Critical strengths
- The paper studies a meaningful problem in multi-length cross-modal hashing, and the proposed components fit together reasonably well. It investigates preserving graded semantic relations within binary hash codes of variable lengths. FRT, SBO and MPA each tackle different subproblems in this setup. Overall, I find the method sufficiently novel in the context of cross-modal retrieval.
- Evaluations span multiple datasets, both retrieval directions, and several code-length settings. Ablations and ranking consistency analyses help support the method’s benefits. Released code and supplementary implementation details also aid reproducibility. Overall, the experiments support the main claims, although I still have a few methodological questions.
Critical weaknesses
- In Section 3.4 and Appendix B, 16-bit, 32-bit and 64-bit prefixes only use the first scale subblock; multi-scale diffusion only appears for the 128-bit code. For shorter hash codes, gains from SBO mostly come from supervision within that first block, not cross-scale fusion. The authors should make this explicit in the main text. Comparisons with alternative bit allocation or scale ordering would help quantify how much performance depends on the current prefix split.
- The paper frames collisions, inversions and cross-length ranking inconsistency as a unified motivation, which is intuitive. Still, limited discrete distances and ranking shifts are known intrinsic properties of binary hashing. To me, the more interesting part is the connection to graded semantic relations and the attempt to preserve ranking structure across code lengths. I recommend separating these well-known limitations from the new problem formulation in the Introduction or Section 3.1.
Questions for authors
- Why can mini-batch-based normalization guarantee stable ranking on the full retrieval database? It would be helpful to report sensitivity to batch size or batch composition, or to provide some intuition for why the current approximation works well.
- How is the discrete sequence extended to a continuous function in the integral formulation(Eq.2)? Please supply a more detailed derivation and proof.
3 comments
Authors have addressed most of my concerns.
Thank you for your detailed rebuttal. The authors have answered most of my questions with their clarification. I have decided to maintain this positive review.
Response to Reviewer BhX5
Thank you for your positive feedback and support. This is very encouraging to us. Thank you again.
Response to Reviewer BhX5
Q1. Mini-batch normalization and full-database ranking.
Thank you for your recognition of our work and your constructive suggestions. Your feedback is very encouraging to us. We address your question below.
(1) Scope of the ranking property. Equation (10) states a conditional property for a given candidate set: when the listwise divergence vanishes, the student's continuous Hamming ranking preserves the teacher's strict ordering within that set. The full-database performance of the learned binary codes is evaluated empirically.
(2) Why mini-batch alignment provides a consistent ranking reference. The integrated teacher g∗ is constructed from the training-set label graph and supplies the same relevance scores across batches. For a query q and two candidates a,b present in a batch,
P∗(b∣q)P∗(a∣q)=exp(Ttg∗(q,a)−g∗(q,b)).
The normalization denominator cancels, so adding other candidates leaves this pair's teacher ordering unchanged. For fixed model parameters, the same argument applies to the student's distribution. Random mini-batches therefore expose different candidate comparisons against a shared teacher, and every code length learns from this same reference. Batch composition can still affect the loss, gradients and candidate coverage.
(3) Empirical evaluation. We train with batch size 128 and evaluate mAP@all over the complete retrieval database. The table summarizes the mean Kendall's τb over the six unordered pairs of 16/32/64/128-bit codes from Table F-2 (higher is better). Full denotes MultiBit with MPA.
DatasetMIRFlickr-25KIAPR TC-12MS-COCOI2T: w/o MPA0.8200.8010.833I2T: Full0.8550.8170.839T2I: w/o MPA0.8060.7900.822T2I: Full0.8450.8030.824
MPA improves cross-length ranking agreement in both directions on all three datasets. These results support its effectiveness under the reported training configuration; Table F-2 includes the full comparison.
Q2. Continuous diffusion on the discrete label graph.
Thank you for requesting this clarification. In our retrieval setting, two candidates can share the same number of labels with a query yet differ in relevance because of label specificity and relationships among their labels. FRT uses the training-label graph to construct a graded ranking reference for transferring these distinctions into finite Hamming codes.
(1) In our formulation, t denotes diffusion time on a graph with finitely many label nodes. We write its symmetric positive-semidefinite Laplacian as L=Udiag(λk)UT, with λk≥0. For any real t≥0,
H(t)=e−tL=Udiag(e−tλk)UT.
This matrix-valued function is continuous in t and satisfies H′(t)=−LH(t) and H(0)=I. Applied to a weighted label vector, it describes propagation through label relationships. Small t emphasizes local neighborhoods, while larger t incorporates longer propagation paths. Thus, continuous time provides different relational scales on the same discrete graph.
(2) Equation (2) expresses how FRT combines these scales with the weight tα−1e−ηt/Γ(α). For each eigenvalue, the substitution u=(λk+η)t in the Gamma integral gives
Γ(α)1∫0∞tα−1e−(λk+η)t,dt=(λk+η)−α.
Here α>0 and η>0 ensure convergence. Applying this identity to the finite set of eigenvalues yields
Γ(α)1∫0∞tα−1e−ηte−tL,dt=Udiag((λk+η)−α)UT=(L+ηI)−α.
Equation (2) therefore integrates continuous-time diffusion on a discrete graph.
(3) In MultiBit, the integrated kernel defines the query--candidate relevance score g∗ through the kernel cosine in Eq. (3), combining label specificity with relations across propagation scales. MPA uses this shared teacher to supervise every code-length prefix. To expose a finite scale structure for the available bits, Eq. (4) uses Gauss--Laguerre nodes and weights to guide SBO and the scale-specific objectives. The integrated teacher is computed from (L+ηI)−α, and each scale teacher from e−τℓL. This connects multi-scale label relations to the bit organization and ranking objectives used for binary retrieval.