acceptodds
Under review as a conference paper at ICLR 2027

EVERY BYTE MUST RANK: A RETRIEVAL-AWARE CODEC FOR FROZEN MULTIMODAL EMBEDDINGS

Abstract

Multimodal retrieval systems increasingly store every candidate as a high-dimensional embedding, and the bill is paid per vector: one billion 2,048-dimensional FP32 embeddings occupy 8.2 TB before any index is built. This scale makes embedding storage a critical systems problem. Yet the tools used to compress these vectors were designed to reconstruct them. Product quantizers minimize reconstruction error even though retrieval depends on the resulting rankings, and fixed-width codes charge every symbol equally regardless of predictability. We introduce EmbedCodec, a codec trained for ranking rather than reconstruction. A lightweight adapter shared by queries and candidates is optimized through the hard product-quantizer assignments used at deployment, reshaping the frozen encoder’s geometry for post-quantization retrieval. A task-conditioned Gaussian mixture models the resulting indices through codeword geometry; rANS then entropy-codes those indices, and rate is measured from serialized bytes rather than nominal bit width. Results on five retrieval tasks spanning text, cross-modal, and visual-document search with two frozen encoders show that EmbedCodec uses about 125 B/vector, a roughly 65× storage reduction relative to FP32, while retaining 99.6–99.8% of the corresponding five-task average NDCG@10. At about 500 B/vector, it exceeds the FP32 average on both encoders. Every persisted stream is exactly decodable back to its input index tuples, and each of the ten encoder–task panels contains at least one Pareto-nondominated EmbedCodec operating point. Our findings suggest that retrieval-aware source and entropy coding can substantially reduce embedding storage without requiring faithful coordinate reconstruction.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
multimodal embeddings, representation compression, cross-modal retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_DnTFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EmbedCodec, a retrieval-aware codec for compressing frozen multimodal embeddings. It introduces a shared query–candidate adapter trained through the hard product-quantization assignments used at deployment, aiming to preserve retrieval rankings after quantization. To further reduce storage, it models quantized indices with a task-conditioned Gaussian mixture model and applies rANS entropy coding. The authors evaluate the method on five retrieval tasks with two frozen embedding encoders, measuring NDCG@10 and serialized storage. The results indicate substantial storage reduction over FP32 embeddings with only a small drop in five-task average NDCG@10. Additional experiments examine adapter placement, retrieval scoring functions, entropy modeling, and cross-domain performance.

Critical strengths
  1. The paper addresses the substantial storage cost of frozen multimodal embeddings in retrieval systems and evaluates compression using actual serialized bytes rather than relying solely on theoretical code lengths. This makes the problem formulation practically relevant and the reported storage–quality trade-offs more informative for deployment.

  2. The evaluation covers five retrieval tasks and two embedding encoders, while the supplementary material examines adapter placement, deployment scoring functions, entropy-table alternatives, and exact decoding. These analyses provide useful evidence that the proposed pipeline is operationally viable and that its components affect the resulting compression–retrieval trade-off.

Critical weaknesses

1.The proposed framework mainly integrates retrieval-aware adaptation, OPQ/PQ quantization, task-conditioned entropy modeling, and rANS coding. The current experiments do not sufficiently isolate the contribution of the rate-aware optimization or establish a clear methodological advance over existing retrieval-oriented compression methods.

2.The training objective is not fully aligned with the deployment ranking function, and comparisons retain different methods' native scoring functions and rate definitions. Consequently, the results do not fully establish the method's advantage under consistent retrieval protocols and comparable storage costs.

Questions for authors

1.What is the key methodological advance of EmbedCodec over existing retrieval-aware quantization and entropy coding methods? Specifically, can the authors isolate the contribution of rate-aware optimization from retrieval-aware adaptation and entropy modeling?

2.How sensitive are the results to the mismatch between the training objective and the deployment ranking function? Would optimizing with the same negative squared Euclidean distance used at inference improve retrieval performance?

3.How would the reported results change under consistent storage accounting and retrieval evaluation protocols across competing methods, including the costs required for practical deployment?

2 comments
AuthorsOpenReview ↗

Official Response to Reviewer DnTF (Part 1/2)

We thank the reviewer for the valuable comments and seek to address reviewer's concerns below:

Q1: Beyond combining a retrieval-aware adapter, OPQ/PQ, a task-conditioned entropy model, and rANS, what component is methodologically new, and which ablation isolates the contribution of rate-aware optimization from adaptation and final coding?

A1: The methodological contribution is to train a pre-OPQ analysis transform jointly for post-quantization retrieval and the model code length of the selected indices, while the encoder, OPQ rotation, and codebooks remain fixed. Removing only the rate term increases mean code size for both encoders at all three OPQ widths after matched coder fitting.

This design differs from JPQ, which optimizes its query encoder and PQ centroids for ranking (Zhan et al., 2021, Section 3.1). In EmbedCodec, the forward pass uses hard assignments, and soft-assignment gradients carry both the retrieval loss and the geometric GMM's coding cost back to the analysis transform. See Line 232-284 in the paper (Eqs. 6–9, pp. 5–6).

  1. A matched supervised control on Seed1.6 VDR-en shows better retrieval after the same fixed OPQ.

    The ordinary supervised control matches the architecture, initialization, training batches, optimizer, and 6,808 updates, with quantization and the rate term removed during training. Both models then use the same fixed M128 OPQ.

    TrainingNDCG@10 before OPQNDCG@10 after OPQSize (B/vector)
    Ordinary supervised training0.8263130.810736125.208
    EmbedCodec0.8263560.823626125.066

    Nearly identical pre-quantization scores become different post-quantization scores at comparable code sizes. See Line 869-897 in the paper (Appendix C.1, Table C.2, p. 17).

  2. Removing only the rate term increases actual code size for both encoders at all three OPQ widths.

    This control retains quantization and the matched training conditions, changing the rate weight from 0.001 to 0. After training, both models separately fit the same type of task-conditioned three-component GMM on the same 65,536 training candidates, with 64 fitting updates and identical rANS settings.

    Removing the rate term increases the five-task mean code size by 0.106%–0.226% across all six settings. The sum of marginal index entropies also rises by 0.114%–0.249% across the six settings. These entropies are computed directly from index frequencies, independently of the fitted GMM.

    Two task examples also improve retrieval with the rate term:

    Encoder / M / TaskWith rate term B/vectorλ=0 B/vectorWith rate term NDCG@10λ=0 NDCG@10
    Seed1.6 / 128 / VDR-en125.066125.2470.82360.8110
    Qwen3-VL / 256 / FIQA247.186247.7850.58090.5729

    Code sizes count actual serialized main + hyper bytes per candidate. See Line 996-1012 in the paper (Appendix C.3, Table C.6, p. 19); Line 1015-1068 in the paper (Appendix C.3, Table C.7 and Figure C.3, pp. 19–20); Line 1069-1101 in the paper (Appendix C.3, Figure C.4, pp. 20–21).

  3. Task conditioning reduces code size without changing retrieval scores.

    Holding the transform, OPQ, and indices fixed, we independently fit shared and task-conditioned three-component GMMs under identical conditions. The shared model requires 1.57%–3.05% more bytes on the six five-task means and increases all 30 task streams; every model recovers the same indices and preserves the same retrieval scores. See Line 1106-1127 in the paper (Appendix C.4, Table C.8, p. 21).

AuthorsOpenReview ↗

Official Response to Reviewer DnTF (Part 2/2)

Q2: Training uses cosine-prefix losses, whereas deployment ranks with negative squared Euclidean distance. Does this mismatch affect retrieval, and what happens when the model is trained with the deployment-aligned L2 objective?

A2: On Seed1.6 M128 FIQA, cosine-prefix training with L2 retrieval gives the highest score in the matched comparison: 0.7161. Keeping L2 retrieval and changing the training objective to L2 gives 0.7024. Keeping cosine-prefix training and changing the retrieval scorer to cosine gives 0.7047.

  1. On FIQA, L2 retraining lowers retrieval quality even when evaluation uses L2.

    We change only the score in the retrieval loss on quantized candidates. The analysis-transform initialization, fixed OPQ, training batches, optimizer, other losses, and 6,808 updates are unchanged. The L2 loss retains decoded candidate norms, as at inference.

    Training scoreCosine retrieval NDCG@10L2 retrieval NDCG@10
    Original cosine prefixes0.70470.7161
    L2 prefixes0.69030.7024

    See Line 971-992 in the paper (Appendix C.2, Tables C.4–C.5, pp. 18–19).

  2. The effect is encoder-dependent: Qwen's mean is almost unchanged, whereas all five Seed tasks decrease.

    Encoder / M128Original training, L2 retrievalL2 training, L2 retrievalChange
    Qwen3-VL0.70000.7001+0.000110
    Seed1.60.78170.7741−0.007616

    Scores are five-task mean NDCG@10; changes use unrounded values. See Line 971-992 in the paper (Appendix C.2, Tables C.4–C.5, pp. 18–19).

  3. With the model fixed, L2 remains the better deployment scorer in most comparisons.

    Holding the checkpoint, queries, candidates, and compressed indices fixed, L2 gives higher NDCG@10 in 24 of 30 task–width comparisons and five of six five-task means across both encoders and M128/M256/M512. Only the scorer changes; stored bytes remain identical. See Line 945-969 in the paper (Appendix C.2, Figure C.2, p. 18).

Q3: Under a common retrieval protocol, a common scorer, and complete deployment storage accounting, how do the reported results change relative to competing methods?

A3: With the same retrieval protocol and negative squared L2 scorer, EmbedCodec retains higher retrieval scores than matched JPQ-style. After including all stored model and index objects, its shared five-task package remains about 26× smaller than FP32.

  1. Common scoring preserves the retrieval advantage over matched JPQ-style, with a clear storage trade-off.

    At M128, both methods use the same frozen encoder outputs, evaluation queries and full task candidate pools. The matched training uses the same sampled positives and in-batch negatives, batch size 512, and 6,808 updates. Scores are five-task mean NDCG@10; package sizes include all five indexes and one copy of each method's shared model.

    MethodScorerQwen NDCG@10Seed NDCG@10Qwen package MBSeed package MB
    JPQ-style, matchedNative inner product0.52190.6154119.684119.684
    JPQ-style, matchedCommon negative squared L20.44520.5721119.684119.684
    EmbedCodecNegative squared L20.70000.7817156.224156.746

    Under common L2, EmbedCodec gains 0.2548 NDCG@10 on Qwen and 0.2096 on Seed, with an additional 36.5–37.1 MB of complete five-task storage. See Line 693-750 in the paper (Appendix B.2, Table B.1, pp. 13–14).

  2. Complete package storage remains substantially below FP32 and 16-bit storage. Across the five tasks, it is about 26× smaller than FP32 and 13.2× smaller than 16-bit storage.

    Totals count compressed indices, the analysis transform, OPQ state, probability tables, candidate identifiers, offsets, and framing. The five-task deployment contains 501,964 candidate entries and shares one transform and OPQ state; MB denotes 10610^6 bytes.

    DeploymentFP32 package MB16-bit package MBEmbedCodec Qwen MBEmbedCodec Seed MB
    WebQA, 377,482 candidates3,097.5681,551.401129.303129.646
    Five tasks, shared4,128.2772,072.233156.224156.746

    WebQA alone retains about a 24× storage reduction relative to FP32 after all these costs are included. See Line 1239-1276 in the paper (Appendix D.1, Table D.2, pp. 23–24).

Reviewer_oeQ1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EmbedCodec proposes a retrieval-aware compression framework that optimizes embedding quantization for retrieval quality rather than reconstruction accuracy. It combines a trainable adapter with product quantization and task-conditioned entropy coding. Contributions: The main novelty is jointly optimizing retrieval performance and compression rate while keeping pretrained embedding encoders frozen. Validation: Evaluated on five retrieval tasks using two embedding models, EmbedCodec achieves ~65× storage reduction while retaining 99.6–99.8% of FP32 average NDCG@10. Ablations validate the adapter and entropy coding, although entropy coding contributes only an additional ~1–3% storage reduction.

Critical strengths

The first strength is that it is a practical and relevant problem. The paper addresses the significant storage cost of large-scale embedding retrieval. Optimizing compression directly for retrieval quality is a meaningful approach with practical applications. Second, the paper shows strong empirical results: The method achieves approximately 65× storage reduction with minimal retrieval degradation across five tasks and two embedding models. The comparisons and ablation studies provide useful evidence supporting its effectiveness.

Critical weaknesses

One of the biggest weakness is the limited comparability with baselines. The methods use different retrieval scoring functions and storage accounting conventions, making it difficult to determine whether the reported improvements come from the proposed compression approach or differences in evaluation settings. Second, there is limited incremental novelty. Retrieval-aware quantization has been explored in prior work, and the proposed entropy coding provides only an additional ~1–3% storage reduction. This raises questions about the extent of the technical contribution beyond existing methods.

Questions for authors

My biggest concern is the fairness of baseline comparisons. Could the authors provide a more controlled comparison with existing methods using consistent retrieval scoring and storage accounting? How much of the reported improvement remains under these conditions? Once the controlled comparison is fixed, it leads to my second question: given that entropy coding provides only ~1–3% additional storage savings, how much of the overall improvement can be attributed specifically to retrieval-aware optimization?

2 comments
AuthorsOpenReview ↗

Official Response to Reviewer oeQ1 (Part 1/2)

We thank the reviewer for the valuable comments and seek to address reviewer's concerns below:

Q1: If all methods use a consistent retrieval scorer and the same storage-accounting convention, what numerical retrieval and storage advantage does EmbedCodec still retain?

A1: With a common negative squared L2 scorer and matched M128 training, EmbedCodec retains higher five-task mean NDCG@10 than JPQ-style. The storage comparison depends on the baseline: matched M128 EmbedCodec uses more package storage than matched M128 JPQ-style, but about half the storage of the original M512 JPQ-style baseline. Complete five-task storage is also about 26× smaller than FP32.

  1. Under matched M128 training, the NDCG@10 gains are 0.2548 for Qwen3-VL and 0.2096 for Seed1.6.

    Both M128 models use the same sampled training batches and 6,808 updates, evaluated on the same queries and candidate pools with negative squared L2; scores are five-task mean NDCG@10. JPQ-style preserves initial document assignments and retrieves with learned centroids.

    EncoderJPQ-style NDCG@10EmbedCodec NDCG@10GainJPQ-style total (MB)EmbedCodec total (MB)
    Qwen3-VL0.44520.7000+0.2548119.7156.2
    Seed1.60.57210.7817+0.2096119.7156.7

    JPQ-style's native inner-product scores are 0.5219 for Qwen3-VL and 0.6154 for Seed1.6, also below EmbedCodec. See Line 693-750 in the paper (Appendix B.2, Table B.1, pp. 13–14).

  2. Compared with the original M512 JPQ-style baseline, M128 EmbedCodec delivers higher quality at about half the total storage.

    The original M512 JPQ-style baseline is trained on pooled data for 2,000 updates; EmbedCodec uses five-task training with 6,808 updates. Both use negative squared L2 on the same evaluation pools.

    EncoderJPQ-style M512 NDCG@10EmbedCodec M128 NDCG@10JPQ-style total (MB)EmbedCodec total (MB)Storage saved vs. JPQ-style
    Qwen3-VL0.64930.7000312.4156.250.0%
    Seed1.60.72770.7817312.4156.749.8%

    Total storage counts each method's persistent model state, codes, probability tables, identifiers, offsets, and framing; shared state is counted once. MB = 10⁶ bytes. See Line 693-750 in the paper (Appendix B.2, Table B.1, pp. 13–14).

  3. Relative to FP32, complete storage remains 24× smaller on WebQA and 26× smaller across the five collections.

    Collection (stored entries)FP32 total (MB)Qwen codec total (MB)Seed codec total (MB)
    WebQA (377,482)3,097.6129.3129.6
    Five task collections (501,964)4,128.3156.2156.7

    See Line 1239-1276 in the paper (Appendix D.1, Table D.2, pp. 23–24).

AuthorsOpenReview ↗

Official Response to Reviewer oeQ1 (Part 2/2)

Q2: Given that entropy coding contributes only about 1–3% additional storage reduction, how much of the overall result is specifically due to retrieval-aware optimization rather than ordinary quantization, generic supervised adaptation, or post-hoc coding?

A2: Retrieval-aware training, rate training, and entropy coding affect different parts of the result. Retrieval-aware training improves five-task mean NDCG@10 under the same fixed OPQ; the rate objective changes the learned index distribution and reduces final code size; entropy coding then stores the resulting indices without changing retrieval scores.

  1. The learned transform improves retrieval quality with OPQ fixed.

    The comparison uses the same frozen encoder, OPQ rotation and codebooks, queries, candidate pools, and negative squared L2 scorer. The table reports five-task mean NDCG@10 at M128.

    EncoderFrozen FP32OPQ without learned transformEmbedCodecGain over OPQ
    Qwen3-VL0.70250.68940.7000+0.0106
    Seed1.60.78310.77470.7817+0.0070

    The gains remain positive at all three widths for both encoders; all six paired-bootstrap 95% confidence intervals are above zero. See Line 910-943 in the paper (Appendix C.1, Table C.3, pp. 17–18).

  2. A matched supervised control on Seed1.6 VDR-en shows better preservation through quantization.

    The supervised control removes quantization and rate terms during training, using the same architecture, initialization, sampled batches, optimizer, and 6,808 updates.

    TrainingBefore OPQ NDCG@10After fixed M128 OPQ NDCG@10Quantization lossCode size (B/vector)
    Ordinary supervised training0.8263130.8107360.015578125.208
    EmbedCodec0.8263560.8236260.002730125.066

    Quantization losses use unrounded scores. With nearly identical quality before quantization and comparable code sizes, the difference arises after the same OPQ. Refitting OPQ on the control's training outputs gives 0.8114, still below EmbedCodec's 0.8236. See Line 869-897 in the paper (Appendix C.1, Table C.2, p. 17).

  3. The rate objective reduces bytes beyond the effect of post-hoc coder fitting.

    This control removes only the rate term, retaining quantization and the matched training conditions. Afterward, task-conditioned GMM-K3 coders are separately fitted to each model's indices, using the same 65,536 training candidates, 64 fitting updates, and identical rANS settings. At M128:

    EncoderFixed-width indices (B/vector)λ=0 + coding (B/vector)With rate term + coding (B/vector)Increase when rate term is removed
    Qwen3-VL128123.797123.531+0.215%
    Seed1.6128124.873124.722+0.122%

    Values are five-task means of actual main + hyper bytes; increases use unrounded with-rate values as the reference.

    Removing the rate term increases actual bytes in all 30 task streams, by 0.106%–0.226% on the six five-task means. Separately, lossless rANS recovers the same indices in all 30 streams with zero change in NDCG@10. See Line 996-1012 in the paper (Appendix C.3, Table C.6, p. 19); Line 1197-1224 in the paper (Appendix C.5, Figure C.6, p. 23).

Reviewer_hb6tOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EmbedCodec, a method for compressing stored embeddings from frozen multimodal retrieval models. A adapter shared by queries and candidates is trained in front of a fixed product quantizer so that compressed codes preserve retrieval ranking. The resulting codes are entropy-coded using a task-specific probability model to save additional space. On five text, image and document retrieval tasks with two embedding models, the paper reported roughly 65x smaller storage than full-precision vectors nearly the same retrieval quality and better quality at large code sizes.

Critical strengths
  1. Storage is measured from the actual serialized streams than theoretical code sizes. For example, the lowest-rate setting is reported as 124.9 and 126.1 bytes per vector, against a nominal 128 bytes. The authors also check that all 30 stored streams decode exactly back to their original codes, giving zero change in NDCG@10 after decoding.

  2. The 2,048-dimensional embedding models stay frozen, the OPQ quantizer (K = 256) is fixed and no learned decoder is added. The method therefore fits into existing retrieval systems while reaching roughly 65× smaller storage than FP32 with 99.6 to 99.8% of FP32 retrieval quality. The authors also state openly that their comparisons are not perfectly matched and they show results that do not follow a clean trend such as two tasks peaking at the lowest rate.

Critical weaknesses
  1. With 128 subspaces of one byte each, a standard OPQ already stores 128 bytes per vector which is 64× smaller than FP32. The paper's entropy coder brings this down only to about 125 bytes, a saving of 1.2–2.4%. The adapter improves average NDCG@10 by only 0.004–0.015 over no adapter. The claim that the method "exceeds FP32" is also confounded. The adapter is trained on each task's training data and applied to the queries, which amounts to task-specific fine-tuning that the unsupervised baselines do not receive. On Seed FIQA, for example, the 128-byte code scores roughly 0.715 against about 0.67 for near-lossless baselines (approximate, read from Fig. 4).

This matters because the abstract presents the "65× compression" and "FP32-level quality" as the method's achievement, but the paper never measures FP32 with the same adapter, so it cannot show how much compression actually costs or what its own contribution is.

  1. The following controls and storage information are missing from the paper.
    • There is no run with the rate term turned off and the codes entropy-coded afterward, so even the 1–2% saving may not come from the given proposed training.
    • The FP32 baseline looks inflated. Lossless ZSTD shrinks the FP32 vectors about 2× around (8,192 to 4,034 bytes), which suggests the embeddings carry only about 16 bits per value against a 16-bit baseline, the reported 65× would be closer to 32×.
    • The per-task probability tables which are reported (about 66 KB each at the lowest rate) could outweigh the entropy-coding savings for collections smaller than 20k–35k vectors, and the paper does not say whether they are included.
    • The Latency is not reported. Decoding speed and query latency are missing, although entropy-coded codes cannot be searched directly the way fixed-width PQ codes.

This points matters for a retrieval embedding paper because it claim advantage is a better trade-off between storage, quality and system cost and these missing pieces are what would establish that trade-off.

Questions for authors
  1. What is the per-task NDCG@10 of FP32 embeddings with the same adapter, without quantization?

  2. What happens if the adapter is trained without the rate term (λ = 0) and the resulting codes are entropy-coded afterward? Also, report the per-subspace index entropy in each case.

  3. What is the native output precision of both encoders, and how does 16-bit (BF16/FP16) storage compare?.

  4. What does the "hyper" stream contain, and are the per-task probability tables included in the reported bytes per vector?

  5. Address weakness 2 each sub point.

6 comments
AuthorsOpenReview ↗

Official Response to Reviewer hb6t (Part 1/6)

We thank the reviewer for the valuable comments and seek to address reviewer's concerns below:

Q1: Is the advertised “65× compression with FP32-level quality” mostly standard OPQ, which already gives 64×, while entropy coding saves only 1.2–2.4% and the adapter improves mean NDCG@10 by only 0.004–0.015?

A1: Standard OPQ accounts for most of the raw compression ratio. EmbedCodec's contribution is to improve retrieval at a fixed OPQ rate and then reduce the stored index bytes without changing the indices. For example, on Qwen COCO I→T and Seed FIQA, EmbedCodec at about 125 B/vector achieves higher NDCG@10 than standard OPQ at 256 B/vector.

  1. EmbedCodec can use roughly half the index bytes of the best-scoring OPQ point while achieving higher NDCG@10. On these tasks and the Qwen five-task mean, the byte reduction is 50.8%–51.9%.

    The curves use the same task candidate pools and each method's reported scorer. Across M128, M256, and M512, OPQ's highest scores occur at M256 for these two tasks and M512 for the Qwen five-task mean.

    Encoder / EvaluationMethodMB/vectorNDCG@10
    Qwen3-VL / COCO I→TOPQ256256.0000.5887
    Qwen3-VL / COCO I→TEmbedCodec128123.1190.6470
    Seed1.6 / FIQAOPQ256256.0000.6770
    Seed1.6 / FIQAEmbedCodec128125.8790.7161
    Qwen3-VL / Five-task meanOPQ512512.0000.7010
    Qwen3-VL / Five-task meanEmbedCodec256251.0110.7122

    See Line 379-455 in the paper (Figures 3–4, pp. 8–9); Line 664-727 in the paper (Appendix B.1, Figures B.1–B.2, pp. 13–14).

  2. The learned transform improves retrieval with OPQ fixed, narrowing the gap to frozen FP32.

    At M128, this retains 99.6% (Qwen) and 99.8% (Seed) of frozen FP32 mean retrieval quality. The ablation uses the same frozen encoder, OPQ rotation and codebooks, queries, candidate pools, and negative squared L2 scorer. All scores below are NDCG@10 at M128.

    Encoder / EvaluationFrozen FP32OPQ without learned transformEmbedCodecGain over OPQ
    Qwen3-VL / Five-task mean0.70250.68940.7000+0.0106
    Seed1.6 / Five-task mean0.78310.77470.7817+0.0070
    Qwen3-VL / COCO I→T0.58530.58670.6470+0.0603
    Seed1.6 / FIQA0.66880.67670.7161+0.0394

    The 95% paired-bootstrap confidence intervals for the mean gain are above zero in all six encoder–width configurations. See Line 841-868 in the paper (Appendix C.1, Table C.1, pp. 16–17); Line 910-943 in the paper (Appendix C.1, Table C.3, pp. 17–18); Line 379-455 in the paper (Figures 3–4, pp. 8–9); Line 664-727 in the paper (Appendix B.1, Figures B.1–B.2, pp. 13–14).

  3. Entropy coding further reduces index bytes without changing retrieval.

    rANS decoding recovers the identical PQ indices in all 30 task streams, with zero change in NDCG@10. Rates are five-task means of actual serialized index bytes.

    MFixed-width indices B/vectorQwen3-VL B/vectorSavingSeed1.6 B/vectorSaving
    128128124.9052.42%126.0861.50%
    256256251.0111.95%253.0121.17%
    512512500.9482.16%501.6662.02%

    Savings are relative to fixed-width indices, with one byte per OPQ subspace. See Line 1197-1224 in the paper (Appendix C.5, Figure C.6, p. 23).

AuthorsOpenReview ↗

Official Response to Reviewer hb6t (Part 2/6)

Q2: For the same trained adapter, what is the per-task NDCG@10 before quantization, and how much quality is actually lost after M128 OPQ? Does this change the “exceeds FP32” claim?

A2: For the same trained transform, M128 OPQ retains 97.9% (Qwen3-VL) and 99.1% (Seed1.6) of its pre-quantization five-task mean NDCG@10. The abstract's “exceeds FP32” statement compares the M512 compressed representation with the original frozen FP32 baseline on the five-task mean.

  1. The M128 quantization loss is 0.0154 NDCG@10 for Qwen and 0.0070 for Seed on the five-task mean. Each encoder uses one analysis transform shared across all five tasks. Before and after OPQ, its weights and transformed queries remain fixed; only the candidate vectors are quantized.

    Qwen3-VL

    TaskFrozen FP32Before OPQAfter OPQLoss
    FIQA0.56810.58000.57010.0099
    COCO T→I0.80540.80180.79310.0086
    COCO I→T0.58530.64680.6470−0.0002
    VDR-en0.78140.78120.75060.0306
    WebQA0.77240.76700.73900.0280
    Mean0.70250.71530.70000.0154

    Seed1.6

    TaskFrozen FP32Before OPQAfter OPQLoss
    FIQA0.66880.71320.7161−0.0029
    COCO T→I0.82930.82520.81760.0076
    COCO I→T0.78370.77560.76590.0097
    VDR-en0.82640.82640.82360.0027
    WebQA0.80730.80290.78530.0176
    Mean0.78310.78870.78170.0070

    All scores are NDCG@10. Loss = before OPQ − after OPQ, computed from unrounded scores; negative values indicate a score increase.

    Qwen COCO I→T and Seed FIQA show retrieval gains learned before OPQ and retained after M128 compression. See Line 841-868 in the paper (Appendix C.1, Table C.1, pp. 16–17).

  2. At M512, EmbedCodec achieves higher five-task mean NDCG@10 than the original frozen FP32 baseline for both encoders. Here, FP32 refers to the original frozen encoder outputs. The quantization losses above compare each trained transform before and after OPQ.

    EncoderFrozen FP32 meanM512 mean
    Qwen3-VL0.70250.7150
    Seed1.60.78310.7884

    See Line 379-455 in the paper (Figures 3–4, pp. 8–9); Line 664-727 in the paper (Appendix B.1, Figures B.1–B.2, pp. 13–14).

AuthorsOpenReview ↗

Official Response to Reviewer hb6t (Part 3/6)

Q3: What happens when the adapter is trained with λ=0 and the resulting codes are entropy-coded afterward with the same coder? How do actual bytes and per-subspace index entropy change?

A3: Removing the rate term makes the indices less compressible: actual coded bytes and summed subspace index entropy both increase in all 30 task comparisons, even when both models are entropy-coded after training under the same conditions.

  1. Fitting the final coder separately to both models does not remove the byte increase.

    We compare λ=0.001 with λ=0, keeping quantization, architecture, initialization, sampled batches, optimizer, fixed OPQ, and 6,808 updates unchanged. We fit a task-conditioned three-component GMM separately to each model's indices from the same 65,536 training candidates, with 64 updates and identical rANS settings.

    EncoderMWith rate term B/vectorλ=0 B/vectorIncrease
    Qwen3-VL128123.531123.797+0.215%
    Qwen3-VL256248.883249.378+0.199%
    Qwen3-VL512496.796497.918+0.226%
    Seed1.6128124.722124.873+0.122%
    Seed1.6256250.849251.116+0.106%
    Seed1.6512496.950497.675+0.146%

    The table reports code sizes after the separate coder fits, as five-task means of actual main + hyper bytes. Increases are relative to λ=0.001. See Line 996-1012 in the paper (Appendix C.3, Table C.6, p. 19).

  2. The index frequencies show the same trend: rate training lowers empirical entropy in 91.8% of task–subspace comparisons.

    For each task and subspace, we compute Hm=−∑jpm(j)log⁡2pm(j)H_m=-\sum_j p_m(j)\log_2 p_m(j) directly from candidate index frequencies. Removing the rate term increases entropy in 8,224 of 8,960 task–subspace pairs. Summing HmH_m across subspaces gives:

    EncoderMWith rate term ∑mHm\sum_m H_m (bits/vector)λ=0 ∑mHm\sum_m H_m (bits/vector)Increase
    Qwen3-VL128979.749981.919+0.222%
    Qwen3-VL2561976.0471980.213+0.211%
    Qwen3-VL5123946.9673956.788+0.249%
    Seed1.6128990.441991.649+0.122%
    Seed1.62561993.9141996.187+0.114%
    Seed1.65123954.0883960.334+0.158%

    Values are five-task means; increases are relative to λ=0.001. Figure C.3 shows the complete per-subspace differences. These frequency-based measurements are independent of the fitted coder, showing that the byte benefit is accompanied by more compressible indices. See Line 1015-1068 in the paper (Appendix C.3, Table C.7 and Figure C.3, pp. 19–20).

Reviewer_gk5AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies compression of frozen multimodal embeddings for retrieval. EmbedCodec trains a residual adapter shared by queries and candidates before a fixed OPQ quantizer. Training uses hard candidate assignments in the forward pass and soft-assignment gradients in the backward pass. A task-conditioned Gaussian mixture model assigns probabilities to codewords, and a rate penalty is optimized jointly with retrieval losses. The selected indices are stored using rANS. Retrieval compares continuous adapted queries with decoded candidate vectors using negative squared Euclidean distance.

Experiments cover two embedding models and five retrieval tasks. At approximately 125 B/vector, EmbedCodec retains 99.6–99.8% of the corresponding FP32 average NDCG@10. The reported entropy-coding savings over fixed-width indices are 1.167–2.418%. The paper's main proposed advance is the combination of adaptation to a fixed quantizer with joint optimization of retrieval quality and index compressibility.

Critical strengths
  1. The problem is practically relevant, and the reported storage–quality trade-offs are encouraging. The evaluation spans two encoders and five tasks covering text, image–text, and visual-document retrieval. Retaining nearly all of the FP32 average retrieval quality at approximately 125 B/vector is useful. The adapter improves average NDCG@10 by 0.004–0.015 across all six encoder–width configurations, supporting the usefulness of adapting the representation to quantization.

  2. The paper implements and checks the actual coding pipeline. It reports serialized stream sizes and verifies exact recovery of the stored index tuples across all 30 evaluated streams. This provides concrete evidence that entropy coding introduces no additional retrieval distortion after quantization.

Critical weaknesses
  1. Unclear incremental contribution and insufficient controls. Retrieval-oriented quantization has precedents in JPQ and CQ. The paper compares against original FP32 embeddings, but the main text does not report an unquantized adapter trained with matched retrieval supervision and budget. Consequently, the reported gains over FP32 do not isolate the contribution of the quantization design from generic task adaptation. The JPQ adaptation to frozen embeddings is also insufficiently specified, including its trainable components, supervision, and negatives; differences in scoring and storage accounting further limit the comparison.

  2. Small entropy-coding gains and unclear benefit from joint rate optimization. Entropy coding saves only 1.167–2.418% over fixed-width indices. At the lowest width, quantization already reduces storage from 8,192 to 128 B/vector (64×), before entropy coding brings it to approximately 125 B/vector (65×). The headline compression factor therefore largely reflects quantization. The main-text results do not isolate the benefit of joint rate training over a retrieval-only adapter followed by the same entropy coder, or establish why the geometric GMM is preferable to simpler probability tables.

Questions for authors
  1. What exactly is the implemented JPQ baseline? Please specify whether it is the original method or an adaptation to frozen embeddings, the query-side architecture, trainable codebooks and assignments, negative-sampling procedure, initialization, and training budget. Please reconcile “JPQ-style” in Figure 1 with “JPQ” in Figures 3–4 and provide equivalent configuration details for the other learned baselines.

  2. How are training and evaluation separated and matched across methods? Please clarify the query and candidate splits, which data fit OPQ and the entropy tables, hyperparameter selection, and whether evaluation collections are available during fitting. Uncertainty estimates for the adapter gains would also help determine their reliability.

4 comments
AuthorsOpenReview ↗

Official Response to Reviewer gk5A (Part 1/4)

We thank the reviewer for the valuable comments and seek to address reviewer's concerns below:

Q1: With the same supervised adapter, what are the retrieval scores before quantization and after the same fixed OPQ? How much of the gain comes from generic task adaptation, and how much survives compression?

A1: Ordinary supervised training alone does not explain EmbedCodec's post-quantization advantage. On Seed1.6 VDR-en, the two models reach nearly identical NDCG@10 before quantization, but EmbedCodec loses less retrieval quality after the same fixed OPQ. The control matches the architecture, initialization, training batches, optimizer, and 6,808 updates; it removes both quantization and the rate term during training.

  1. VDR-en: EmbedCodec retains more retrieval quality through the same fixed M128 OPQ.

    TrainingNDCG@10 before OPQNDCG@10 after OPQSize (B/vector)
    Ordinary supervised training0.8263130.810736125.208
    EmbedCodec0.8263560.823626125.066

    OPQ lowers NDCG@10 by 0.015578 for the control and 0.002730 for EmbedCodec. Since the pre-quantization scores and compressed sizes are nearly equal, the meaningful difference is how much retrieval quality survives quantization. See Line 869-897 in the paper (Appendix C.1, Table C.2, p. 17).

  2. VDR-en: refitting OPQ for the control does not close the gap. OPQ is refitted on the supervised model's training outputs.

    MethodNDCG@10Size (B/vector)
    Ordinary supervised training + refitted OPQ0.8114124.971
    EmbedCodec + fixed OPQ0.8236125.066

    See Line 869-897 in the paper (Appendix C.1, Table C.2, p. 17).

  3. Seed1.6 FIQA provides a second example: EmbedCodec exceeds supervised FP32 quality with 66.5× compression.

    RepresentationNDCG@10Size (B/vector)
    Original frozen FP320.66888,192
    Ordinary supervised training, FP320.71078,192
    EmbedCodec before quantization, FP320.71328,192
    Ordinary supervised training + fixed OPQ0.7153123.276
    EmbedCodec + fixed OPQ0.7161123.164

    See Line 869-897 in the paper (Appendix C.1, Table C.2, p. 17).

Q2: If the adapter is trained with the rate weight set to zero and the same task-conditioned coder is fitted afterward, do the resulting indices actually require more bytes?

A2: Yes. Even when each model receives its own task-conditioned coder, fitted under identical conditions, removing the rate term increases the actual serialized size in all 30 task streams.

  1. The increase is 0.106%–0.226% on the six five-task means.

    We compare λ=0.001 with λ=0. Both models retain quantization during training and use the same architecture, initialization, sampled batches, optimizer, fixed OPQ, and 6,808 updates.

    After training, we separately fit a task-conditioned three-component GMM to each model's indices from the same 65,536 training candidates. Both fits use 64 updates and the same fitting and rANS settings.

    EncoderMWith rate term B/vectorλ=0 B/vectorIncrease
    Qwen3-VL128123.531123.797+0.215%
    Qwen3-VL256248.883249.378+0.199%
    Qwen3-VL512496.796497.918+0.226%
    Seed1.6128124.722124.873+0.122%
    Seed1.6256250.849251.116+0.106%
    Seed1.6512496.950497.675+0.146%

    Rates are five-task means of actual main + hyper bytes per candidate; increases are relative to the model trained with the rate term. See Line 996-1012 in the paper (Appendix C.3, Table C.6, p. 19).

  2. The index distribution itself becomes more compressible with rate training. Removing the rate term increases the sum of marginal subspace entropies in all 30 task configurations, by 0.114%–0.249% on the six five-task means. These entropies are computed directly from index frequencies, independently of the fitted coder, so they corroborate the coded-byte results without relying on the GMM fit. See Line 1015-1068 in the paper (Appendix C.3, Table C.7 and Figure C.3, pp. 19–20).

  3. Two task-level examples also show higher retrieval scores with the rate term.

    Encoder / M / TaskWith rate term B/vectorλ=0 B/vectorWith rate term NDCG@10λ=0 NDCG@10
    Seed1.6 / 128 / VDR-en125.066125.2470.82360.8110
    Qwen3-VL / 256 / FIQA247.186247.7850.58090.5729

    See Line 1069-1101 in the paper (Appendix C.3, Figure C.4, pp. 20–21).

AuthorsOpenReview ↗

Official Response to Reviewer gk5A (Part 2/4)

Q3: Why use a task-conditioned geometric GMM during training if a simple empirical table compresses the final fixed indices better? How much do task conditioning and mixture components each save?

A3: The geometric GMM (K_GMM=3) provides a compact, geometry-based probability model for joint training with the analysis transform. After training, an empirical table serves a different role: it can further shorten the final stream without changing retrieval.

  1. The GMM provides a compact, geometry-based training objective. For each task and subspace, GMM (K_GMM=3) uses 27–99 trainable parameters to estimate probabilities for 256 codewords. Shared means, scales, and mixture weights make coding costs depend on codeword locations. The rate objective fits these parameters to the hard indices and sends gradients to the analysis transform through soft assignments. See Line 244-284 in the paper (Eqs. 7–9, pp. 5–6).

  2. Both task conditioning and mixture capacity reduce final code size. Task conditioning saves 1.54%–2.96%, while GMM (K_GMM=3) saves 0.15%–0.46% over GMM (K_GMM=1). We freeze the transform, OPQ, and indices, then independently fit each model on the same 65,536 training candidates for 64 updates with the same optimizer and learning rate. K_GMM denotes the number of Gaussian components. All models use the same rANS settings and recover identical indices and retrieval scores.

    EncoderMShared GMM (K_GMM=3) B/vectorTask-conditioned GMM (K_GMM=1) B/vectorTask-conditioned GMM (K_GMM=3) B/vectorSaving vs shared GMM (K_GMM=3)Saving vs task-conditioned GMM (K_GMM=1)
    Qwen3-VL128127.293124.033123.5312.956%0.405%
    Qwen3-VL256254.270249.409248.8832.119%0.211%
    Qwen3-VL512507.652497.533496.7962.138%0.148%
    Seed1.6128127.628125.301124.7222.278%0.462%
    Seed1.6256254.778251.452250.8491.542%0.240%
    Seed1.6512506.015497.813496.9501.792%0.173%

    Rates are five-task means of actual main + hyper bytes per vector. Each saving is relative to the named baseline. Both changes shorten all 30 task streams. See Line 1106-1127 in the paper (Appendix C.4, Table C.8, p. 21).

  3. An empirical table can further reduce bytes after training. It preserves retrieval and saves a further 0.117%–0.407%. We estimate task-specific frequencies from the same training indices with a fixed pseudocount of 0.5, using the same CDF precision, block size, and raw fallback.

    EncoderMTask-conditioned GMM (K_GMM=3) B/vectorEmpirical table B/vectorSaving vs task-conditioned GMM (K_GMM=3)
    Qwen3-VL128123.531123.0290.407%
    Qwen3-VL256248.883248.0550.332%
    Qwen3-VL512496.796495.4630.268%
    Seed1.6128124.722124.3700.282%
    Seed1.6256250.849250.2970.220%
    Seed1.6512496.950496.3690.117%

    Both coders recover the same indices, so this additional saving leaves retrieval unchanged. See Line 1129-1180 in the paper (Appendix C.4, Table C.9, pp. 21–22).

AuthorsOpenReview ↗

Official Response to Reviewer gk5A (Part 3/4)

Q4: What exactly is the implemented “JPQ-style” baseline, and after matching its assignments, supervision, negative samples, training budget, scorer, and storage accounting, how does it compare with EmbedCodec?

A4: Our baseline is an adaptation of JPQ to frozen Qwen/Seed embeddings, not the original text-to-RoBERTa pipeline. The released JPQ implementation generates RoBERTa embeddings from text, whereas our baseline receives precomputed 2,048-dimensional Qwen/Seed vectors. We therefore replace its text query encoder with a residual MLP on these vectors, retain fixed candidate assignments and joint query–centroid optimization, and label the implementation “JPQ-style.” With matched M128 supervision and updates, EmbedCodec scores higher than this baseline under both its native inner-product scorer and a common negative squared L2 scorer.

  1. The adaptation preserves the frozen embedding source. Running the original text encoder would change the embedding space being compressed. We train a query-only residual ReLU MLP (2048 → 4096 → 2048) directly on the frozen Qwen3-VL and Seed1.6 outputs with a multi-positive inner-product loss. The MLP starts at identity; the initial PQ codebook is fitted for 25 iterations on 65,536 frozen candidates from the historical pooled training source. Candidates are assigned using this initial codebook; retrieval uses the learned centroids at those fixed indices. Figures 3–4 consistently label this adaptation “JPQ-style.” See Line 630-638 in the paper (Appendix A.3, p. 12).

  2. EmbedCodec retains its quality advantage after matching supervision, negatives, and training budget. Original JPQ mines hard negatives using its current PQ index. For the matched M128 control, we hold retrieval supervision fixed by using the same 43,957 training queries and sampled in-batch positives and negatives as EmbedCodec. Both methods use batch size 512, the same AdamW settings, and 6,808 updates, and are evaluated on the same five candidate pools.

    MethodRetrieval scorerQwen3-VL mean NDCG@10Seed1.6 mean NDCG@10
    JPQ-style, matchedNative inner product0.52190.6154
    JPQ-style, matchedCommon negative squared L20.44520.5721
    EmbedCodecNegative squared L20.70000.7817

    See Line 693-750 in the paper (Appendix B.2, Table B.1, pp. 13–14).

  3. The five-task package uses 36.5–37.1 MB more storage than matched JPQ-style.

    EncoderMethodCode size B/vectorFive-task package MB
    Qwen3-VLJPQ-style, matched128.000119.684
    Qwen3-VLEmbedCodec124.905156.224
    Seed1.6JPQ-style, matched128.000119.684
    Seed1.6EmbedCodec126.086156.746

    Code size counts stored index bytes per vector: raw indices for JPQ-style and main + hyper for EmbedCodec. Package totals additionally include each method's stored model, codebooks, identifiers, and framing, with one shared model across the five indexes; MB denotes 10610^6 bytes. See Line 693-750 in the paper (Appendix B.2, Table B.1, pp. 13–14); Line 1239-1276 in the paper (Appendix D.1, Table D.2, pp. 23–24).

AuthorsOpenReview ↗

Official Response to Reviewer gk5A (Part 4/4)

Q5: Which data are used to train the adapter, fit OPQ and the entropy model, select hyperparameters, and report final evaluation? Is any evaluation query or candidate reused during fitting?

A5: The training and test sets share neither queries nor candidates. OPQ and the entropy model are fitted only on training candidates.

  1. Training and test content is disjoint after deduplication. Deduplication covers normalized text and exact or near-duplicate images. COCO images and their captions are treated as one source group. A query is excluded from the test set if filtering removes any required positive. See Line 579-609 in the paper (Appendix A.1, Table A.1, pp. 11–12).

  2. Training and fitting use only training data. The fitting sample contains 65,536 training candidates: 13,108 from FIQA and 13,107 from each other task.

    StageData used
    Joint transform and GMM training43,957 training queries; candidates drawn from a 292,935-item training pool
    OPQ fittingFrozen embeddings of the 65,536 training candidates; OPQ then remains fixed
    GMM prefitOPQ indices of those same candidates, fitted for 64 updates
    Probability models fitted after trainingIndices produced by each model for those same training candidates

    See Line 579-609 in the paper (Appendix A.1, Table A.1, pp. 11–12); Line 613-626 in the paper (Appendix A.2, p. 12).

  3. Validation data are used for configuration selection; final results are reported on the test sets. EmbedCodec reserves 300 queries per task for model development, trains on the remaining 43,957 of 45,457 queries, and uses the final checkpoint after 6,808 updates. These held-out training queries are separate from the validation queries listed below. JPQ/BINCE configurations are chosen by mean validation NDCG across the five tasks, subject to a limit on bytes per vector.

    TaskTraining queriesValidation queries / candidatesTest queries / candidates
    FIQA5,166293 / 33,718366 / 32,768
    COCO T-to-I9,70019,506 / 3,90719,360 / 3,874
    COCO I-to-T9,7003,907 / 19,5193,874 / 19,360
    VDR-en9,691191 / 68,8082,020 / 68,480
    WebQA9,700723 / 379,4171,001 / 377,482

    FIQA, COCO, and WebQA use content-filtered official splits; VDR-en uses a holdout constructed from its English document dataset. Each test query searches its task's full candidate pool. See Line 579-609 in the paper (Appendix A.1, Table A.1, pp. 11–12).

Q6: The adapter improves the five-task mean by only 0.004–0.015 NDCG@10. Are these gains reliable, or could they be explained by query-sampling variability?

A6: The gains exceed the variability captured by paired query resampling: the 95% confidence intervals are above zero in all six encoder–width configurations.

The comparison measures the effect of the analysis transform with OPQ fixed. We compare the learned analysis transform with identity (no learned transform) on the same test queries and candidate pools, keeping the frozen encoder, OPQ, and retrieval scorer unchanged.

EncoderMFive-task NDCG@10 gain after OPQ95% confidence interval
Qwen3-VL128+0.01059[0.00681, 0.01443]
Qwen3-VL256+0.01472[0.01117, 0.01819]
Qwen3-VL512+0.01382[0.01071, 0.01700]
Seed1.6128+0.00702[0.00330, 0.01081]
Seed1.6256+0.00357[0.00040, 0.00674]
Seed1.6512+0.00499[0.00174, 0.00831]

See Line 910-943 in the paper (Appendix C.1, Table C.3, pp. 17–18).

Comments on the paper

3 comments
Area_Chair_HtXzOpenReview ↗

From AC.

Dear Authors,

Given the deadline on 18 November, please submit your responses to the reviews to facilitate a productive discussion with the reviewers.

Thanks, AC

Authors: please submit responses to the reviews. The author-reviewer debate is an important part of the ICLR process.

AuthorsOpenReview ↗

Review and Rebuttal Summary

We thank all reviewers for their careful reading and constructive comments. We are encouraged that Reviewers gk5A, oeQ1, and DnTF recognize the practical importance of embedding storage and the reported storage–quality trade-offs. We also appreciate that Reviewers gk5A and hb6t highlight our measurements of actual serialized bytes and exact recovery of the stored indices.

The reviews raise three central questions: (1) what retrieval-aware and rate-aware optimization contribute beyond standard OPQ, ordinary supervised training, and post-hoc coding; (2) whether the baseline comparisons remain favorable under matched training, common retrieval scorers, and consistent evaluation protocols; and (3) how the method performs when complete storage and deployment costs are included. The additional experiments and revisions address these questions as follows:

  1. Retrieval-aware training improves retrieval after fixed OPQ. We compare against ordinary supervised training with matched architecture and training conditions, evaluate both models before and after the same fixed OPQ, and test OPQ refitting. On Seed1.6 VDR-en, the models have nearly identical pre-quantization quality, but EmbedCodec incurs a smaller quantization loss. Paired query resampling also gives 95% confidence intervals above zero for the mean gain over OPQ without the analysis transform in all six encoder–width configurations. See Line 869-897 in the paper (Appendix C.1, Table C.2, p. 17); Line 910-943 in the paper (Appendix C.1, Table C.3, pp. 17–18).

  2. Rate training and final probability modeling play distinct roles. Removing only the rate term increases actual stored index bytes in all 30 task streams, even after each model receives a separately fitted coder of the same type under identical conditions. The index frequencies show the same trend through their marginal entropies. With the indices fixed, further comparisons identify the effects of task conditioning and mixture components. We also explain how the geometric GMM provides rate gradients during training and show that empirical final tables can further reduce bytes without changing retrieval scores. See Line 996-1012 in the paper (Appendix C.3, Table C.6, p. 19); Line 1106-1127 in the paper (Appendix C.4, Table C.8, p. 21); Line 1129-1180 in the paper (Appendix C.4, Table C.9, pp. 21–22).

  3. The JPQ-style comparison remains favorable under matched training and common scoring. We specify the JPQ-style adaptation and match its supervision, in-batch negatives, and training budget. EmbedCodec achieves higher five-task mean NDCG@10 for both encoders whether JPQ-style uses its native inner-product scorer or the common negative squared L2 scorer. Additional experiments compare retrieval scorers and direct L2 training. We also document the training, validation, and test data, including the absence of shared queries or candidates between training and testing. See Line 579-609 in the paper (Appendix A.1, Table A.1, pp. 11–12); Line 630-638 in the paper (Appendix A.3, p. 12); Line 693-750 in the paper (Appendix B.2, Table B.1, pp. 13–14); Line 971-992 in the paper (Appendix C.2, Tables C.4–C.5, pp. 18–19).

  4. EmbedCodec improves the reported storage–quality trade-offs. On the same test candidate pools with each method's reported scorer, EmbedCodec at about 125 B/vector achieves higher NDCG@10 than OPQ at 256 B/vector on Qwen3-VL COCO I→T and Seed1.6 FIQA. Comparisons with physically stored FP16/BF16 arrays show that M128 uses 32.5–32.8× less vector storage. See Line 379-455 in the paper (Figures 3–4, pp. 8–9); Line 664-727 in the paper (Appendix B.1, Figures B.1–B.2, pp. 13–14); Line 1231-1250 in the paper (Appendix D.1, Table D.1, pp. 23–24).

  5. Complete storage and runtime measurements quantify deployment costs. Including probability tables, the analysis transform, OPQ state, identifiers, and framing, complete M128 storage is about 24× smaller on WebQA and 26× smaller across the five task collections than FP32. At M128 on WebQA, one-time index preparation takes about 6.8 s on one NVIDIA H20 with four CPU threads. Subsequent queries from precomputed embeddings have a median latency of about 10 ms, with reconstructed candidates cached on the GPU. See Line 1239-1276 in the paper (Appendix D.1, Table D.2, pp. 23–24); Line 1288-1328 in the paper (Appendix D.2, Table D.4, pp. 24–25).

We thank the reviewer again and look forward to further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.