acceptodds
Under review as a conference paper at ICLR 2027

Diagnosing Learner–Codec Mismatch in Low-Bit Retrieval

Abstract

Retrieval comparisons at a fixed code rate can depend on the export codec as well as the learner. On Stanford Online Products, FIT-only range calibration improves frozen twenty-dimensional four-bit triplet descriptors by 14.22 average-precision points at the same ten-byte payload, changing the allocation effect against ten-dimensional eight-bit descriptors from -5.61 to +8.50 points. At common four-bit output, a historical four-bit-training advantage falls from +9.74 to -0.36 points; learned-scale DINOv2 and SigLIP2 contrasts have conditional intervals including zero. Product quantization fitted separately to each learned representation supplies a further comparison. In one additional paired training realization, native contrastive descriptors trail triplet descriptors by 2.081 points, whereas CAL-selected ten-byte codecs favor contrastive descriptors by 1.854 points; both adjusted intervals exclude zero. Range calibration alone also reverses this pair's learner ordering. The finding concerns codec-dependent rankings on an exposed population, not universal contrastive superiority. Training-step-preserving and range-reset exports remain distinct, and CUB retains adverse allocation comparisons. These controls diagnose learner–codec compatibility without establishing a new quantizer, equivalence, or a minimum-bit rule.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
image retrieval, compact embeddings, deep metric learning, quantization, codec calibration, low-bit representations

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_5o9GOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper asks whether fixed-payload low-bit retrieval rankings reflect the learner or the export codec. With DINOv2 (and some SigLIP2) checkpoints held fixed, it compares 80-bit allocations Q8D10 versus Q4D20 under a ladder of codec interventions: native grid (D0), FIT cell-mean decoder with fixed indices (D1), nearest-centroid reassignment (D2), and FIT-only global/coordinate-wise range calibration (C1/C2), plus own-latent PQ/OPQ and common-export training-grid controls. On SOP, C1 alone recovers +14.22 AP on frozen 4-bit triplet descriptors and reverses the allocation effect from −5.61 to +8.50; a historical 4-bit training advantage also collapses under equal calibration. CUB provides adverse transfer cases. The authors are careful not to propose a new quantizer or a universal bit-allocation law.

Critical strengths

The codec ladder does a good job of isolating the mechanism. D1/D2 versus C1 shows fairly cleanly that the SOP triplet failure is largely a scale/grid interface mismatch, not just bad centroid reconstruction. This is the most consequential positive result in the paper.

Critical weaknesses

The presentation cost is real. The clarity problems are not cosmetic: they make it hard to see what is new relative to classical quantization hygiene, such as Lloyd steps and scalar range fitting on a 7-point menu.

The generalizable contribution is limited for ICLR. There is no new lasting method, only 2–3 paired heads, two reused datasets, and conclusions that correctly refuse universality. As written, this reads more like a rigorous evaluation note than a venue-level advance.

Questions for authors
  1. Why does FIT always select R = 0.25, the menu floor, for Q4 triplet U? Is this latent scale collapse, activation saturation, or a menu-boundary artifact? Would a continuous R change the story?
  2. Can you report seed-level variability, say ≥5–10 independent paired fits, for the key allocation reversals, rather than only conditional product/species bootstrap on fixed models?
  3. After C1, what is the actionable recommendation for practitioners? Should they prefer Q4D20 on SOP-like data, or is the message simply “always FIT-calibrate before comparing learners/allocations”?
  4. A prospectively held-out dataset, or larger-scale retrieval, under the same protocol would materially strengthen the contribution.
3 comments
AuthorsOpenReview ↗

Response to Reviewer 5o9G

Why did the original range search select its lower boundary?

We re-examined the original two training seeds with a finer range search. The four-bit triplet models select ranges of 0.3076 and 0.3038, both above the original menu floor of 0.25. Reconstruction error decreases and clipping falls from about 5.5% to 1.4–1.5%. This shows that coarse menu spacing affected the selected value; choosing its floor alone did not identify latent collapse.

We refined the four-bit and eight-bit models separately using reconstruction error on the fitting data, then evaluated the fixed choices. The original allocation difference, four-bit minus eight-bit AP, is −5.61 points under the original fixed-grid export, +8.50 after the original range calibration, and +8.45 after refinement. Both seeds retain the positive calibrated difference. This reuses the four original frozen checkpoints and adds no independent training repetitions. Section 5 and Appendix G.2 give the separate scores and range choices. Refinement lowers reconstruction error without necessarily increasing retrieval AP. The same distinction appears in the new pairs (Appendix G.4); no test-AP tuning is involved.

The thirty-model learner cohort in Appendix G is separate from the original allocation comparison.

What do the independent repetitions establish?

We completed three new allocation pairs under the original training and export protocol. Each seed trains a twenty-dimensional four-bit model and a ten-dimensional eight-bit model, both at ten bytes. The original triplet loss, architecture, data use, and training budget are retained. Calibration selects from the original seven ranges by reconstruction error on FIT, without using test AP.

Training pairNative Q4−Q8 AP pointsOriginal calibration Q4−Q8 AP points
1−5.62+7.99
2−6.35+7.72
3−5.04+9.07

All three pairs reverse: the mean allocation difference changes from −5.67 to +8.26 points. Between-pair standard deviations are 0.66 and 0.71 points. All six model-conditional product-bootstrap intervals exclude zero after Bonferroni correction for 95% family-wise confidence across the three training pairs and two exports. These intervals condition on the fitted models; they do not replace the reported training-seed variation. All three new pairs and the full evaluation set are included.

Appendix G.4 reports the new three pairs separately from the original two. The auxiliary five-pair means are −5.65 and +8.36 points; all five have the same sign reversal. These repetitions start from the same pretrained backbone with new head initializations and training streams; both backbone and head are fine-tuned. They share evaluation products. Continuous-range refinement is a secondary analysis; the replication above uses the original calibration rule.

Appendix G.1 separately reports ten learner pairs at fixed four-bit allocation and the newer-regime allocation pairs. Neither is counted as an original-condition replication.

What should a practitioner do after calibration?

Compare models under the intended deployment codec before choosing one, with separate fitting, validation, and final evaluation data. With equal codec-fitting work, this improves SOP selection by 5.43 AP points with DINO and 5.70 with CLIP. At full budget, OPQ-first selection followed by codec reselection reaches the same pipelines as joint selection; extra complexity has no established benefit. In-shop selection intervals include zero, so this is a procedure for making a fair comparison, not a guarantee of improvement or a universal preference for four-bit descriptors.

A SigLIP2 control compares training without quantization with four-bit training under the same final twenty-dimensional, four-bit calibrated scalar export, with parameters fitted separately to each representation. Across four paired seeds, AP without training-time quantization minus AP with four-bit training is +0.55/+0.61 points for triplet and −7.10/−4.77 for contrastive models at 7,200/14,400 updates. All four seeds retain these directions at both budgets. Under the same final ten-byte export, training without quantization yields higher AP for triplet learning, while four-bit training yields higher AP for contrastive learning at both tested budgets. This motivates evaluating training precision for each learner under the intended deployment codec. It is a separate training-precision comparison from the original allocation replication; the budgets use their own learning-rate schedules.

The initial In-shop protocol was fixed before training and uses the official query/gallery roles with item-separated fitting and validation data. It supplies additional-domain evidence. We distinguish this recorded evaluation procedure from the broader history of benchmark use. The original CUB and teacher controls are retained.

The rebuttal and the new experiments are solid. They effectively address my questions about the variance and boundary choices. I have updated my score to Weak Accept.

Thank you for your careful reassessment and constructive feedback. Your questions helped us clarify the range-boundary choices and strengthen the evidence on variability across training runs. We appreciate your time and thoughtful engagement with our work.

Reviewer_oJRtOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper shows interesting findings: many allocation preferences cannot survive by using FIT-only range calibration or CAL-selected ten bytes codec. The performance can be increased without establishing a new quantizer, equivalence, or a minimum-bit rule and extra training. The problem and findings are attractive. The research is worth of being followed. However limitations are still existing.

Critical strengths
  1. The paper decouples learner quality from export-codec quality at a fixed code rate and shows a single range calibration recovers 14.22 AP points and flips both the allocation preference and the learner ordering.
  2. The four frozen-checkpoint interventions map onto Lloyd's two steps plus a scale repair, cleanly isolating reconstruction, assignment, and range. Freezing checkpoints and fitting only on FIT keeps each step interpretable.
  3. Matched setup, per-head conditional bootstrap intervals, and thorough adverse controls (memory, normalization, teacher compression, partial recoding) lend credibilit. The CUB counterexample prevents over-generalization.
Critical weaknesses
  1. The paper's evidence and all headline reversals come from single-modal image retrieval; the authors do not discuss the transferability of their findings to other retrieval settings (e.g. cross-modal retrieval). I would ask the authors to either bound their claims to the tested single-modal regime or add one adjacent-task direction to show the range-calibration effect is not specific to this setting.
  2. The paper is hard to follow for a reader not already inside this notation. Several invented or composite abbreviations appear without a definition or intuition on first use, including in the abstract, which raises the entry cost considerably. The related-work section is also terse: for each of the three areas it lists works without sufficiently stating how this paper relates to them and which issue it newly separates.
  3. The paper is entirely confined to low-bit rates. No higher-bit (>8-bit) or higher-payload configuration is examined. Since range-calibration benefits would be expected to diminish as bit precision grows, the authors do not clarify whether their reversal is a low-bit phenomenon or a general interface effect.
Questions for authors

Please see the weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer oJRt

Does the effect remain above eight bits?

The ordering can still change when the learned quantizer clips the representation. We exported the same frozen twenty-dimensional models at sixteen bits, using forty bytes per item. With their learned ranges, triplet and contrastive AP are 51.28% and 50.02%. After range refinement, they are 52.60% and 57.92%. The contrastive-minus-triplet difference therefore changes from −1.26 to +5.32 AP points across the ten DINO SOP pairs.

Increasing precision reduces rounding error, but cannot recover values clipped outside the chosen range. We selected the range using four-bit reconstruction error on the fitting split (FIT), then held it fixed across export precisions. These are fixed-model export comparisons, not sixteen-bit retraining or separate retrieval-optimal range searches at each precision. Table 23 in Appendix G.3 reports the trained, reset, and refined quantizers at 4, 8, and 16 bits, together with clipped and unclipped floating-point exports.

The large repair of a reset [−1,1] grid is concentrated at low precision: for triplet models, refined minus reset AP changes from +21.63 points at four bits to −0.08 at sixteen bits. That reset grid differs from the trained-quantizer baseline. The evidence therefore separates a coarse-grid effect from clipping that can persist at higher precision. Calibration is not uniformly beneficial at every precision.

The corresponding payloads are 10, 20, 40, and 80 bytes for four-bit, eight-bit, sixteen-bit, and floating-point export. Increasing export precision is also distinct from training in full precision and subsequently deploying a ten-byte OPQ code; Appendix G.3 reports the latter as a separate training comparison.

What is the scope beyond the tested retrieval task?

We restrict the empirical conclusion to single-modal image retrieval. CLIP supplies image descriptors here; we have not tested text-to-image retrieval. SOP, In-shop, and the adverse CUB results define the tested scope. In-shop shows a substantial codec interaction but no mean learner reversal.

Our recommendation is to fit the intended deployment codec on FIT, choose the learner on separate validation data (CAL), and evaluate the frozen choice on TEST. It does not prescribe a universal bit width or require joint optimization. The revised abstract and Sections 1–3 introduce the problem and data roles before the abbreviations; Section 2 explains how the question differs from established quantization and metric-learning methods. Section 6 and Appendix G separate the precision, training, and model-selection comparisons.

AuthorsOpenReview ↗

Range calibration at higher export precision

Dear Reviewer oJRt,

You asked whether the benefit of range calibration diminishes above eight bits. We extended our earlier 20-dimensional comparison to frozen 40-dimensional models on Stanford Online Products. For each model, we fitted the export range on separate data using four-bit reconstruction error and reused that range at sixteen bits. Compared with keeping the range learned during training, refitting improved average precision (AP) by 1.61 points at four bits and 2.26 at sixteen for triplet models, averaged over three runs. For contrastive models, the corresponding gains were 4.17 and 4.85 points across three runs. For the sixteen-bit export with the learned range (80 bytes per descriptor), clipping accounted for almost all of the AP loss relative to the unclipped floating-point output. Thank you for raising this question.

Reviewer_FZd2OpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper is a diagnostic study of compact image-retrieval descriptors at a fixed budget of 10 bytes. It states that conclusions such as "which bit allocation is better" or "which training loss is better" depends on the export quantizer rather than on the learned representation. With frozen models, simple training-only codec adjustments, mainly recalibrating the range of the uniform quantizer, are enough to change or reverse these rankings. The authors conclude that low-bit comparisons should be made with a properly calibrated codec. The evidence comes from few training runs on reused benchmarks.

Critical strengths

The paper shows a relevant methodological point, which is that low-bit retrieval comparisons can be confounded by a poorly calibrated export quantizer. Its practical message is that one needs to calibrate the codec before attributing differences to the learner or to the bit allocation.

The paper is honest about scope. Negative and adverse results (CUB, normalization, a strong compressed teacher) are reported, the claims are explicitly limited, and costs beyond payload size, such as decoder state and re-encoding, are discussed.

Critical weaknesses

There is a massive presentation problem, which sounds like the paper was mostly written by an AI without a human to make it human-readable.

This starts as soon as in the abstract, which does not set up the context or motivation: it starts directly with specific numbers and uses terms that are only defined later (e.g., FIT/CAL splits, U/XBM, Q4/Q8, T4/T8, E4). A reader cannot tell from the abstract what problem is addressed, why it matters, or what the main takeaway is. The paper would not some major restructuring. For instance for the abstract, it could be (i) the setting (compact descriptors for retrieval under a fixed byte budget); (ii) what is at stake (comparisons between learners or bit allocations are commonly drawn from a single export quantizer); (iii) the question (whether such rankings reflect the learner or the codec); and (iv) the main finding, stated qualitatively (simple, training-only codec recalibration can reverse these rankings), with details deferred to the body.

The same issue affects the introduction and much of the paper, which is hard to follow due to dense, hedged phrasing and many internal acronyms.

In addition to this presentation issue, my main concern is the limited novelty and significance. The central finding is that recalibrating a quantizer's range on training data matters, and that a fixed [−1, 1] grid on tanh outputs can be badly miscalibrated. Calibrating ranges, fitting centroids and using PQ/OPQ are standard practice in quantization and post-training quantization. The "native" baseline therefore looks like a weak, uncalibrated export. Showing that fixing it changes the conclusions is expected. The paper offers no new method and no general guideline beyond "calibrate the codec".

Finally, the experimental validation is underwhelming. First analysis is narrow and therefore provides fragile evidence: The setting is very specific: one 10-byte budget, two allocations, two losses, mostly one backbone, and two datasets. But most importantly, the paper does not report results on the widely adopted datasets from the Approximate nearest neighbor search literature.

Questions for authors

Have you tried evaluation at scale? For approximate nearest-neighbor indexes, datasets of at least 1 billion vectors is what matters in practice.

3 comments
AuthorsOpenReview ↗

Response to Reviewer FZd2

Does the result depend on a weak, uncalibrated baseline?

The reversal remains when we preserve each model's learned quantizer. Across ten independently trained pairs on Stanford Online Products, contrastive models trail triplet models by 1.27 average-precision (AP) points under their trained quantizers, but lead by 5.47 points after standard optimized product quantization (OPQ), at the same ten-byte item payload. All ten pairs reverse in point estimate. The nominal seed-level t-based 95% intervals are [−1.55, −0.99] and [5.12, 5.83]; these describe variation between training pairs. One pair's fixed-model product-bootstrap interval includes zero, so we do not claim a separately significant reversal in every pair.

The baseline retains the learned step size and range. We fit OPQ separately to each representation using the same procedure and fitting opportunity. Table 2 reports the absolute scores:

ExportTriplet AP (%)Contrastive AP (%)
Trained quantizer50.5849.30
Standard OPQ51.3056.78

Thus the deployment codec changes which trained representation is preferred. Appendix G.1 separates the initial five pairs, the additional five, and all ten. The original fixed-grid allocation experiment also repeats in three newly trained pairs: four-bit minus eight-bit AP changes from −5.67 to +8.26 points on average using the original calibration rule, with reversal in every pair (Appendix G.4).

A separate completed control isolates scale updating during training. Across four paired DINO SOP seeds, we match the initial scale, weights, training draws, and 14,400-update budget, then fit the same ten-byte global scalar calibration to each representation. Learned-scale minus frozen-scale AP is +5.84 points for triplet and +0.16 for contrastive models. The corresponding descriptive 97.5% intervals resampling seeds and products are [3.54, 8.15] and [−0.16, 0.50]. The triplet gain remains after both exports are calibrated, providing a training-time mechanism control alongside the OPQ comparison above.

What does this change for model selection?

The codec changes the model-selection decision, not only a fixed model’s score. With equal codec-fitting work, native-first selects triplet in every SOP pair and OPQ-based selection chooses contrastive in every pair. Test AP rises from 51.27% to 56.70% with DINO and from 50.00% to 55.70% with CLIP: gains of 5.43 and 5.70 points. The simultaneous 95% intervals are [4.93, 5.93] and [5.23, 6.16]. We resample training pairs and products separately and adjust across four comparisons within each selection analysis. These are full-procedure comparisons, including fitting-work allocation; total scoring time is not matched. At full budget, choosing under OPQ and then reselecting that learner's codec yields the same pipelines and per-query AP as joint learner–codec selection on DINO SOP and In-shop. Additional screening has no established benefit. Section 6 and Appendix G.1 specify these distinct procedures.

The sixty-fit CLIP study repeats the SOP reversal. It also shows that full-precision rather than four-bit training helps triplet models but hurts contrastive models under the same final ten-byte OPQ export. In-shop has no mean learner reversal and its selection-gain intervals include zero; the adverse CUB and compressed-teacher controls remain. A matched hard-OPQ training intervention also fails to improve the strong validation-selected In-shop reference with either backbone (Appendix G.5). Our contribution is this controlled diagnosis and its model-selection consequence, rather than the invention of range fitting or OPQ.

Have we evaluated billion-vector ANN retrieval?

No. These experiments evaluate exact semantic image retrieval, not approximate-nearest-neighbor index throughput. Item payload and shared codec state are reported separately. We have rewritten the abstract and Sections 1–2 around the deployment decision and bound the claims to the tested setting. We plan to release the implementation and evaluation scripts; a public code link is not included in this rebuttal.

Thank you for the answers.

The answer still does not address what happens when varying the byte budget. The answer to another reviewer and reference to Appendix G3 reports some results, but this is only about varying the precision. I believe that, without such an ablation on the number of component, the study remain incomplete, the paper should provide a more detailled analysis of the trade-off, i.e., the dependance of the budget, otherwise the conclusions remain fragile.

I have still concerned by the contribution and the presentation. I have read the other reviews and answers, and these don't address my concern. I therefore keep a negative rating, yet raising to weak reject to account for the additional feedback by authors.

AuthorsOpenReview ↗

Descriptor dimension, byte budget, and model selection

Dear Reviewer FZd2,

Thank you for pointing out that our earlier precision comparison kept the number of descriptor components fixed. On Stanford Online Products, we compared 20- and 40-dimensional models trained at the same four-bit precision. At each byte budget, we fitted optimized product quantization separately to each representation and compressed both query and gallery descriptors.

The table shows the difference in test average precision (AP points, 40 dimensions minus 20).

Training objective10 bytes per descriptor20 bytes per descriptor
Triplet−0.74+3.22
Contrastive−1.46+0.86

The 20-dimensional model led at 10 bytes and the 40-dimensional model led at 20 bytes in all three paired training runs for each objective. This test-set reversal persisted with the same codec family fitted separately to each model.

We then chose the width twice on the same validation products, once with 10-byte codes and once with 20-byte codes. We evaluated both choices at 20 bytes per descriptor on the test set. Changing the validation budget changed the selected width in one of three triplet runs and two of three contrastive runs. Averaged over all three runs per objective, matching validation to the 20-byte deployment budget improved test AP by 0.938 points [0.788, 1.090] for triplet models and 0.708 points [0.475, 0.945] for contrastive models.

For each test product, we first averaged the difference between the two selection rules across the three fixed training pairs. We then resampled test products using the same draws for both rules and adjusted the intervals across all 24 deployment comparisons.

Does this comparison address your concern about how descriptor width interacts with the byte budget and clarify why that matters when selecting a model for deployment?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We have uploaded a revised manuscript incorporating the additional analyses and clarifications described in our responses. References to newly added sections, tables, and appendices refer to this revision. We will consolidate further changes arising from the discussion into a subsequent update and clearly summarize those changes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.