- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper identifies a previously underexplored problem in lifelong cross-modal hashing: different task partitions of the same ordered data stream lead to inconsistent historical binary codes, even when query heads remain similar. The authors propose PILCH, which achieves partition invariance through frozen features, Dice-coefficient-based permanent semantic target encoding, additive prefix moments, and a shared sample-count clock. Theorem 1 proves invariance of prefix heads, historical codes, and ranking lists under explicit assumptions. Experiments introduce the TRR metric to quantify cross-partition retrieval reproducibility and validate the method on four datasets, where the PILCH core achieves 100% TRR and a calibrated clocked extension attains the highest directional mean mAP.
Critical strengths
- PILCH addresses partition sensitivity at the indexing level, offering clear computational advantages. By freezing features and using additive prefix moments, the method completely avoids the gallery rewriting or feature retraining overhead common in existing lifelong hashing schemes, with each write strictly O(1) complexity. This design forms a clear contrast with methods like LEMON that require periodic gallery rewriting, and demonstrates practical deployment value in large-scale stream data scenarios.
- The theoretical derivation and experimental design are both solid. The proof chain for Theorem 1's invariance claim is complete, and the appendix documents constants, seeds, and baseline recurrence details, which is a level of transparency uncommon in this area. TRR@20 as a new metric has practical utility for audit scenarios, and directional mean mAP avoids bias from single-direction reporting. Ablation experiments cover component contributions and state separation, making the overall evaluation thorough.
Critical weaknesses
- The choice of Dice coefficient for target code construction lacks comparative justification. PILCH uses Dice coefficients based on global label co-occurrence frequencies to generate permanent semantic target codes, but the paper does not discuss why Dice was chosen over other set similarity measures (e.g., Jaccard, overlap coefficient, cosine similarity). Different measures would produce different target code distributions, which could directly affect both the strength of partition invariance and final retrieval performance. While Dice offers algebraic convenience, the absence of comparison against alternatives leaves this core design choice insufficiently motivated and makes it difficult to judge whether this selection is critical to the method's success.
- The causal source of mAP gains is not isolated. PILCH outperforms baselines on both TRR and mAP, but these two improvements may stem from different mechanisms: TRR gains come directly from the partition invariance design, while mAP gains may arise from the quality of the Dice target encoding itself, independent of invariance. Table 8's ablation isolates component contributions but does not compare PILCH against the strongest baseline under a single fixed partition. Therefore it remains unclear whether the mAP advantage depends on the multi-partition setting.
Questions for authors
- Is the choice of Dice coefficient critical to target code construction? Can you provide a comparison with alternative label co-occurrence measures (e.g., Jaccard, overlap coefficient) on TRR and mAP, or theoretically analyze which properties of Dice are essential for partition invariance?
- Table 8's ablation does not include a single-partition baseline comparison. Can you provide mAP numbers for PILCH vs. the strongest baselines under a single fixed partition? I want to know whether the mAP improvement actually comes from the invariance mechanism.
- The analytic core and fixed-clock discrete extensions both achieve partition invariance, but through somewhat different mechanisms. Could the authors clarify which parts of the guarantee rely on additive prefix statistics and the unique ridge solution, and which follow from synchronizing model updates and database writes at fixed sample counts?
- Equation (4) recovers the full-prefix pairwise relation objective without explicitly storing cross-task similarities. Could the authors briefly explain which algebraic properties make this exact recovery possible, and why the additive statistics in Equation (2) are sufficient to preserve cross-task relations?
8 comments
Response to Reviewer mJTn[2/6]
Response to Q1 and W1
Thank you for asking for a direct comparison and a clearer explanation of why we use Dice.
We respectfully clarify that the target is computed from each item's own label vector. Corpus-wide label frequencies play no role in this computation. For nonempty label sets , Dice similarity is . This measures their shared labels directly. The paper realizes this similarity with a fixed feature map and Gaussian projection. Equal label sets receive equal targets throughout the stream.
1. Applicability of Dice
Dice offers a useful mathematical property. For nonempty label sets, its positive-semidefinite, unit-diagonal kernel has an exact finite feature map . The inner product of two mapped labels equals their Dice similarity. A single Gaussian projection is then drawn and fixed to produce the binary targets. For Dice similarity , the probability of a matching bit over that draw is Higher label similarity therefore gives a higher probability of matching bits. This is the link between the semantic measure and the permanent code. For binary labels, Jaccard similarity satisfies . Their continuous ordering agrees, while finite-bit codes can differ.
2. Controlled affinity comparison
Following your advice, we completed a controlled comparison of Dice, Jaccard, overlap, and cosine. We used the analytic encoder-write learner on NUS-WIDE, COCO, MIRFlickr, and IAPR. The original three seeds, frozen CLIP features, streams, solver, hyperparameters, shared clock, and queries were held fixed. Codes have 64 bits.
The four measures use the same encoder: up to 256 distinct bootstrap label vectors serve as fixed anchors, and a shared Gaussian projection maps their affinity vectors to bits. Only the affinity formula changes. The common affinity-vector projection gives a defined code for all four measures. Native Dice retains the paper's exact finite feature map; its original-query results were reproduced separately. The shared-encoder comparison isolates the similarity choice.
The solver retains code and relation weights of 10, ridge , and the original sample-count solve schedule. Anchors are selected once by a fixed rule.
For each variant, we replayed the stream under . The replays compare the state and head at every solve, the historical gallery codes, query codes, and write times. Every variant reaches 100% TRR@20, with identical scoring inputs across these taskifications. The complete experiment includes all four datasets and all three original seeds. The four affinity variants plus the native-Dice reference give 60 completed experiment cells and 180 taskification replays. The native reference checks reproduction; the four shared-encoder columns provide the controlled affinity comparison.
3. Support for the results
This result agrees with the proof. A fixed deterministic target yields the same additive statistics when and the execution conditions are shared. Positive ridge then yields the same unique head. The target's permanence gives this guarantee; the chosen similarity affects which labels receive similar codes and hence retrieval relevance.
Table 3.1 reports full-gallery mAP over all final-prefix queries, averaged across both directions and three seeds. Relevance means any shared label, and Hamming ties use item ID. Within the shared encoder, Dice is within 0.24 mAP points of the best tested measure on every dataset. This shows that its mathematically convenient construction also gives competitive retrieval accuracy.
Section 3.2 now specifies item-label construction. Section 4.4 and new Appendix C.8 report the controlled comparison. Dice provides an exact fixed map and an explicit link to bit agreement; all four tested affinities preserve invariance with permanent targets.
Table 3.1. Shared-encoder mAP (%)
| Dataset | Dice | Jaccard | Overlap | Cosine |
|---|---|---|---|---|
| NUS-WIDE | 71.60 | 71.84 | 70.88 | 71.55 |
| COCO | 75.96 | 76.08 | 75.83 | 76.04 |
| MIRFlickr | 80.42 | 80.01 | 80.05 | 80.52 |
| IAPR | 52.50 | 52.28 | 52.38 | 52.53 |
We will release the corresponding code and experimental logs to support reproducibility.
Response to Reviewer mJTn[3/6]
Response to Q2 and W2
Thank you for asking whether the reported mAP gains remain under one fixed partition and which components produce them.
Table 1 evaluates mAP under a single fixed taskification. The relevance scores are obtained within that partition. The cross-partition comparisons separately evaluate reproducibility. We reproduce the requested baseline comparison in Table 3.2 below.
1. Comparison under fixed P10
The comparison uses the paper's frozen features, ordered streams, 64-bit codes, both retrieval directions, and the three original seeds. Each method is evaluated on the same eligible query cohort against the same gallery population. Hamming ties follow the same item-ID rule. Values average directions within each seed and then average the three seeds. The tested uncalibrated baseline is selected separately for each dataset, so the table includes the strongest corresponding reference.
The calibrated pipeline has higher mean mAP on each dataset. These gains occur within the single fixed partition. They are therefore present in an ordinary retrieval comparison, independently of computing the cross-partition TRR metric.
2. The role of components in retrieval
We agree that this result needs precise component attribution. The high-accuracy row combines the selected discrete-code gallery with fixed-gallery query calibration. The gallery supplies binary codes learned by DOCH or LEMON under a sample-count clock. The query stage combines current, semantic, aligned alternate, and pairwise predictions, followed by the selected residual. It then binarizes the query prediction for Hamming retrieval. These stages fit later query predictions to the codes already stored in the gallery.
The analytic core explains why the historical index remains consistent. Its permanent targets and additive moments give an exact full-prefix objective and a unique regularized solution. Together with the shared write schedule, they preserve historical codes and rankings across taskifications. The highest-accuracy row is the calibrated discrete extension of that indexing framework.
The added target experiment in our response to Q1/W1 isolates similarity choice within the analytic encoder-write learner. It holds the encoder, solver, clock, features, and query population fixed while changing the affinity. All four affinities preserve TRR, while their mAP values differ. That comparison directly separates the target's relevance geometry from the permanence needed for invariance.
The original accumulation ablation further tests the use of historical information. With P10, the permanent Mode-T gallery, features, and solve counts fixed, retaining full-prefix statistics improves mAP by 0.432–7.720 points across the four datasets. This comparison isolates retained training history within the analytic learner.
Applying a shared schedule to LEMON yields perfect task-partition reproducibility while retaining its retrieval utility. Query calibration then improves compatibility with the selected fixed gallery.
Revised Section 4.2 explicitly states the fixed-P10 setting and attributes its gains to the calibrated pipeline. Section 4.4 and Appendix C.8 add the affinity comparison. Together with the shared-clock intervention, these results distinguish relevance from partition reproducibility.
Table 3.2. Fixed- mAP (%)
| Dataset | Uncalibrated baseline | Calibrated pipeline |
|---|---|---|
| NUS-WIDE | DOCH 83.20 | 83.94 |
| COCO | LEMON 82.02 | 84.46 |
| MIRFlickr | DOCH 92.20 | 92.68 |
| IAPR | LEMON 67.59 | 69.84 |
Response to Reviewer mJTn[1/6]
Dear Reviewer mJTn,
Thank you for your detailed review and for recognizing the indexing formulation, theoretical analysis, and reproducibility of our evaluation. Your suggestions helped us strengthen the justification for the permanent targets and explain the mechanisms more clearly.
We have added the controlled comparison of Dice, Jaccard, overlap, and cosine to the manuscript. We also clarified the fixed-partition accuracy comparison and expanded the derivation showing how the moments recover cross-task relations. The four responses below address each question in order and distinguish the analytic and discrete mechanisms behind the retrieval guarantee.
We are grateful for your constructive suggestions and for considering the revised explanations and evidence.
Sincerely,
The Authors