acceptodds
Under review as a conference paper at ICLR 2027

Generative Late-Interaction Embeddings For Visual Document Retrieval

Abstract

Late-interaction retrieval is the state-of-the-art for visual document search, but storage costly, where compression techniques retain a subset or local average of the vectors per page. Under aggressive memory budgets, these methods degrade sharply, while alternatives require retraining the encoder. Investigating this degradation across three encoders revealed two geometric properties: vectors lie on the unit sphere and concentrate within a 5-6 dimensional manifold. Consequently, a) standard -means centroids fall inside the sphere, underestimating MaxSim scores and normalizing them to the surface fixes this ( nDCG@5). b) the low-dimensional manifold allows full vector regeneration from a few. To this end, we introduce Generative Late-Interaction Embeddings (GLIE): vectors per page learned from the normalized centroids to serve as both a lightweight index and a basis for regenerating a page's full embedding set. At query time, search runs exclusively on these vectors, and a decoder expands only the top candidates back to all vectors for exact re-scoring. At four vectors per page on ViDoRe v1, GLIE retains nearly 80% of the uncompressed system's nDCG@5, against 70% for the best prior post-hoc method. These results use a 415K-parameter network fitted in within three GPU-minutes on just a few thousands training pages. At a matched training budget, fine-tuning the encoder does not reach even the training-free stage of GLIE, and the full system beats it at every budget. These patterns are true for a second encoder and ViDoRe v2. By reconstructing evidence on demand rather than sampling it, GLIE opens a new axis for storage-efficient retrieval, with the decoder as its main design surface.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
visual document retrieval, late interaction, multi-vector compression, generative embeddings, ColPali, MaxSim

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KcQPOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Generative Late Interaction Embeddings (GLIE), a post-hoc method that compresses the NN ~ 1000 patch embeddings per page of late-interaction visual document retrievers to k<<Nk << N representitive vectors. The key idea is from observation that NN per-page embeddings lie on the unit sphere and have low intrinsic dimension (5-6 when estimated with TwoNN). Therefore, they first use normalized k-means centroids to build compressed representation that preserves unit norm property. Then, a zero initialized cross attention refiner adjusts the kk centroids. Finally, a shared decoder expands the refined vectors back to NN vectors for reranking. At query time, all pages are scored with their kk stored vectors, and only the top-L candidates are decoded and reranked. Due to this decoder, they can discard full NN patch embeddings from the storage and only maintain k vectors per page. They also keep the encoder frozen, and only train cross-attention refiner and MLP decoder, which only needs few GPU minutes for training. On ViDoRe v1 and v2, they outperforms raw k-means, token pooling and cluster merging. They also show they can outperform Light-ColPali on small training budget.

Critical strengths
  • Practical and well-motivated method. The paper propose solution to practical problem from simple and well-explained observation. Table 2 shows that per-page patch embeddings have low intrinsic dimension consistently across three encoders with different dimensions. This well motivates compressing a patch embeddings with fewer vectors. Overall solution is low-cost. It works post-hoc on a frozen, public encoder, and only require few GPU minutes to train their refiner and decoder.
  • Strong results, tested across different backbones. They empirically show their method is efficient with smaller number of vectors compared with other training-free methods. Finally, they also show thorough evaluation across ViDoRe v1 and v2, two different encoders, and different values for kk.
  • Analysis of k-means centroids. The paper proposes k-means centroids as intuitive compressing method. Normalization of these centroids are proposed as free solution to fix underestimation problem of k-means centroids (Proposition 1). Renormalizing cluster means itself is not new (Light-ColPali already does it, as noted in Appendix C), but Proposition 1 gives a useful explanation of why it helps, and the gain shrinks as k grows as predicted.
Critical weaknesses
  1. Critical baseline missing from the main comparison. Main comparison (Tables 3, 4 and Figure 1) does not compare GLIE against normalized k-means baseline (i.e. GLIE without refiner, decoder and rerank). Against it, GLIE gains +0.052 at k=4 and almost nothing at k≥32k \geq 32 (Table 7), which changes the reading of the headline claims. Additionally, the use of k-means centroids is also not justified nor compared with alternatives (e.g. spherical k-means).
  2. Clarify stage 2 trade-off on memory cost vs performance. The benefit of the generative decoder over reranking with the original embeddings is limited to storage on disk and transfer at query time, not memory during search. At stage 2 holding the L shortlisted pages as regenerated vectors or as original patch embeddings requires the same RAM. The decoder therefore lets one discard the original embeddings X entirely, at the cost of lower accuracy than reranking with X (e.g., 0.657 vs. 0.782 for the shortlist oracle at k=4 on ViDoRe v1, Table 3). This is still meaningful when total storage is the bottleneck, but the paper should state this trade-off more clearly, explaining clearly the benefit of the decoder.
  3. The geometric analysis is not tightly connected to the method or the experiments. The paper measures an intrinsic dimension of about 5 or 6, but no design choice depends on it (e.g., the budget k is not chosen from it). It is also not discussed what this intrinsic dimension implies for the number of vectors needed per page.
  4. Presentation can be clearer. Same component appears under many different names (e.g. normalized k-means, normalized clustering, spherical anchoring, etc). Limitation of k-means suddenly appear in Section 3.1 without clearly stating it will be used as compression method. Figure 4 starts y-axis at 0.5 and do not show raw numbers for each components, which distort the relative size of the others. Some statements are stronger than it is. e.g., 'training can only move it where the objective improves' is not guaranteed (Stage 1 in DocVQA or InfoVQA is actually degraded compared to training-free stage).
Questions for authors

See weaknesses.

3 comments

We thank the reviewer for the constructive comments. We answer each point below.

W1 (first part). Normalized k-means in the main comparison, and the gain of the learned components.

The reviewer is right that normalized k-means belongs in the main comparison. It was only in the ablation table (Table 7) because the main tables compare complete systems with prior work. We will add it as a row to Tables 3 and 4 and as a curve to Figure 1. The reviewer's reading of the gains is also right: over normalized k-means the learned components add about +0.05 at k = 4, and the gain shrinks toward k = 32, where normalized k-means alone already keeps 94% of the uncompressed score (0.784 of 0.836). The scope of the paper is the geometry of page vectors, the training-free correction that follows from it, and a minimal generative read-out. We believe decoder design is a large topic of its own, so we kept the read-out minimal and limited the paper to two results that show this direction is promising: (1) the design of the decoder matters (Table 10, re-run with three seeds in our reply to reviewer tiqY), and (2) a headroom of 0.13 nDCG@5 at k = 4 remains between GLIE and the original vectors on the same shortlist (Figure 4). The generative read-out is therefore a first step in this direction, and we will reflect this scope more clearly in the revision.

W1 (second part). Why k-means, and the comparison with spherical k-means.

k-means is the clustering that Proposition 1 points to. For unit-norm tokens, the k-means objective equals the total amount by which the centroids fall inside the sphere, which is the quantity that spherical anchoring corrects. We also compared it with other training-free ways to choose the k vectors.

Training-free choice of the k vectors (nDCG@5, ViDoRe v1)k=2k=4k=8k=16k=32k=64
Farthest-point sampling0.3740.3810.4090.4860.5800.685
Cluster merging0.4470.4710.5310.6130.6900.763
k-means medoids0.5000.5460.6250.6970.7620.792
Normalized k-means (ours)0.5520.6050.6840.7360.7840.809

Averaging and then normalizing beats selecting existing tokens at every budget. Spherical k-means projects the centroids at every iteration, where we project once. Run in the same solver, it stays within about 0.01 of normalized k-means at every budget (0.010 higher at k = 4, equal from k = 32). GLIE runs unchanged on a spherical k-means partition, so we adopt the reviewer's suggestion as a variant and will add both comparisons to the appendix.

W2. The trade-off of stage 2.

The reviewer is right that the read-out saves storage and not working memory. Its benefit is a higher score from the same stored bytes, for the case where the original vectors cannot be kept. The same index supports three deployments.

Deployment at k = 4Stored per pagenDCG@5
Learned code alone (stage 1)1.0 KB0.640
Learned code + generative read-out (GLIE)1.0 KB0.655
Learned code + original vectors kept for re-scoring1.0 KB + 257.8 KB0.782

When the original vectors can be kept, re-scoring the shortlist with them is the stronger second stage (the row the paper calls the shortlist oracle). GLIE targets the case where they cannot be kept: at one million pages they take 258 GB, against 1.0 GB for the GLIE index. The index can shrink further: with the stored code quantized to 4 bits per value, without retraining, a page takes 280 bytes (0.28 GB at one million pages) and the score of GLIE does not change (within 0.001, table in our reply to reviewer Q8Xh, W4). Working memory at query time is the same in both cases (20 pages of full-size vectors, about 5 MB), and the read-out adds about 2 ms per query at every corpus size. We will add this table to the paper.

W3. The link between the geometry and the method.

We agree that the intrinsic dimension does not set k. Its role is to explain why a few vectors per page can work at all. The part of the geometry that the method is built on is the sphere. Page vectors have unit norm, so Proposition 1 gives the score that averaging loses. This predicts the gain of spherical anchoring and its decay at large k (+0.093 at k = 4, +0.030 at k = 64), and it is why the decoder keeps its outputs on the sphere. On text, with the late-interaction model ColBERTv2 on two BEIR test sets, the intrinsic dimension is low as well (TwoNN 2.9 on SciFact and 2.8 on NFCorpus, against 25.7 and 25.3 for a Gaussian with the same covariance). Below we show that the prediction of Proposition 1 also holds there.

nDCG@10, ColBERTv2k = 4k = 8k = 16k = 32uncompressed
SciFact, raw k-means0.3420.4600.5260.5960.692
SciFact, normalized k-means0.4500.5440.5850.633
NFCorpus, raw k-means0.1800.2160.2650.2960.344
NFCorpus, normalized k-means0.2120.2450.2890.312

Normalizing the k-means centroids improves nDCG@10 at every budget, and the gain shrinks with k as Proposition 1 predicts (+0.108 at k = 4 and +0.037 at k = 32 on SciFact). We will revise the geometry section so that each measurement is followed by the design choice it supports, with the intrinsic dimension presented as motivation.

W4. Presentation.

We agree with all four points and will make these edits.

  • One name per object: "spherical anchoring" for the step and "normalized k-means" for the training-free code it produces. "Normalized clustering" is removed.
  • Section 3.1 will state that k-means is the compression step before its centroids are analyzed.
  • Figure 4 will start its y-axis at 0 and print the value of each component.
  • "Training can only move it where the objective improves" becomes: "Training starts from it. On the source holdout the objective can only improve. On a transferred corpus the learned code can fall below it, as the reviewer notes for DocVQA and InfoVQA."

I thank the authors for their rebuttal. The clarifications addressed my main concerns.

I believe clarifying the motivation of the method and its important baselines, as done in the rebuttal, will improve the presentation in the final version (i.e. more about the geometry of page vectors, rather than the generative part as said by the authors).

I am raising my score from 2 (weak reject) to 3 (weak accept).

Reviewer_tiqYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the storage cost of visual document retrieval. GLIE compresses approximately 1,031 vectors per page into a small set without retraining the original encoder. It normalizes cluster centroids, learns compact vectors for retrieval, and regenerates vectors to rerank the top 20 candidate pages. At aggressive storage budgets, the method improves nDCG@5 over the evaluated compression baselines.

Critical strengths
  1. Clear and practically relevant problem. The method compresses embeddings from an existing encoder, avoiding the cost of retraining the encoder and re-encoding the corpus.
  2. Effective spherical normalization. This simple modification yields a substantial improvement, and the ablation study makes its contribution clear.
  3. Evaluation across several settings. The paper reports results on ViDoRe v1 and v2, a second retrieval encoder, and multiple vector budgets.
Critical weaknesses
  1. The contribution of the generative module should be framed more narrowly. Its performance gain is limited relative to the other components. In the ViDoRe v1 ablation at k=4k=4, spherical normalization improves nDCG@5 by 0.093, whereas generative reranking raises it only from 0.640 to 0.655, a gain of 0.015. In this setting, the largest measured gain comes from spherical normalization.

  2. Lack of end-to-end efficiency evaluation. GLIE achieves substantial retrieval gains under tight storage budgets, but offline indexing requires clustering and refinement for each page, while query-time inference requires vector generation and reranking for shortlisted pages. The paper reports storage size and model training time, but not per-page indexing time, query latency, or throughput. A comparison of both retrieval quality and runtime costs for normalized kk-means, the learned compressed vectors alone, and the full GLIE system would help establish whether the gain from generative reranking justifies its additional cost.

Questions for authors
  1. Could the authors report the indexing and query-time costs of full GLIE, the learned-vector-only stage, and normalized kk-means? How do these costs compare with the storage savings, and are they acceptable in a realistic deployment?
  2. Could the authors directly analyze which retrieval-critical local details remain recoverable after compression? For example, how do per-query-token best-match scores change after compression and regeneration, and what types of queries benefit or suffer?
  3. Why does “Ours” at k=4k=4 score 0.677 in Table 10 but 0.655 in Table 3? Do the tables differ in training data, model selection, or evaluation protocol
2 comments

We thank the reviewer for the constructive comments. We answer each point below.

W1. The contribution of the generative module.

The reviewer's numbers are right, and they are the paper's own decomposition (Table 7). The scope of the paper is the geometry of page vectors, the training-free correction that follows from it, and a minimal generative read-out. We believe decoder design is a large topic of its own, so we kept the read-out minimal and limited the paper to two results that show this direction is promising: (1) the design of the decoder matters (Table 10, re-run with three seeds under Q3), and (2) a headroom of 0.13 nDCG@5 at k = 4 remains between GLIE and the original vectors on the same shortlist (Figure 4). The generative read-out is therefore a first step in this direction, and we will reflect this scope more clearly in the revision.

W2 / Q1. End-to-end efficiency: indexing time, query latency and throughput.

The indexing and query-time costs are small next to the storage saving, and they are acceptable for a realistic deployment. We timed the three systems and exact MaxSim over the uncompressed index on one A100 (1,000 real queries, one at a time, L = 20), with the corpus replicated to 10K, 100K and 1M pages.

System at k = 4nDCG@5Storage per pageIndexing after encodingP50 latency at 10K pages100K pages1M pages
Uncompressed, exact MaxSim0.836257.8 KBnone13.3 ms133.8 msdoes not fit on the GPU
Normalized k-means0.6051.0 KB0.45 ms0.42 ms0.81 ms5.5 ms
Learned code alone (stage 1)0.6401.0 KB0.59 ms0.42 ms0.81 ms5.5 ms
Full GLIE0.6551.0 KB0.59 ms2.5 ms2.6 ms7.1 ms

Indexing is cheap: encoding a page with ColPali takes 42 ms, and k-means (on GPU) plus the refiner add 0.59 ms (1.4%). At query time the generative read-out adds about 2 ms (1.6 to 2.1 ms) at every corpus size, because it only regenerates the 20 shortlisted pages. At one million pages full GLIE answers a query in 7.1 ms (about 140 queries per second) from a 1.0 GB index, where the 258 GB uncompressed index does not fit on the GPU. We will add this table to the paper.

Q2. Which local details remain recoverable, and which queries benefit or suffer?

Details that compression weakens are mostly recovered, and details that it removes are recovered only in part. We measured this with the per-query-token best-match scores that the reviewer suggests. For each query token we took its best-match score on the relevant page, how much of that score is lost when the page is compressed to four vectors, and how much of the loss the read-out restores. We then sorted the tokens by the size of their loss and split them into three equal groups.

Query tokens in three equal groups (ViDoRe v1, k = 4, one seed)Score lost after compressionShare of the loss restored by the read-out
Tokens with a small loss0.07570%
Tokens with a medium loss0.19449%
Tokens with a large loss0.40036%
All tokens0.22344%

The read-out restores most of a small loss and about a third of a large one.

Which queries benefit or suffer depends on where the relevant page sits before the read-out:

  • Queries benefit when the relevant page is in the shortlist but not ranked first (35% of the queries). The read-out adds +0.075 nDCG@5 on them (re-scoring with the original vectors adds +0.477).
  • Queries whose relevant page is already ranked first lose 0.025. They lose the same when re-scored with the original vectors (0.026).
  • By content, queries with numbers gain less than the others (+0.016 against +0.023), so numbers are the hardest detail to regenerate.

We will add this analysis to the appendix.

Q3. Why does Ours score 0.677 in Table 10 but 0.655 in Table 3?

Both tables use the same evaluation (all ten ViDoRe v1 subsets, all queries, L = 20, model selection on the source only). Table 10 came from an earlier development run: a separate implementation of the training loop, fitted on 2,000 source pages with one seed. We re-ran all six decoders in the main pipeline under the Table 3 protocol (5,000 source pages, same objective, three seeds). The Ours row is now identical to Table 3, and the new table replaces Table 10.

Decoder (nDCG@5, ViDoRe v1, mean of 3 seeds)Built on the stored code vectorsk=2k=4k=8k=16
Oursyes0.6100.6550.7110.755
Cross-Attention + MLPyes0.6040.6560.7080.751
Dot-Productyes0.6030.6450.6980.751
Cross-Attention + Local Mixingyes0.6060.6560.7040.750
Cross-Attentionno0.5470.5910.6240.645
MLPno0.5480.5410.5150.517
Reference: normalized k-means (training-free, no decoder)0.5520.6050.6840.736
Reference: shortlist oracle (original vectors)0.7690.7820.8070.821

The table shows that the design of the decoder matters, and that it is one place where results can improve. Decoders that are not built on the stored code vectors fall below training-free normalized k-means, and the four that are differ by up to 0.011 at k = 4 while all stay about 0.13 below the shortlist oracle.

Reviewer_Q8XhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Late-interaction models (e.g., ColPali, ColQwen2) for visual document retrieval achieve great retrieval accuracy with MaxSim, but keeping 1,000 vectors per page costs too much storage. Existing pruning and pooling methods fail badly at tiny budgets (k<16k < 16), while fine-tuning backbones is slow and invalidates existing indexes. The authors observe that visual token embeddings have an intrinsic dimension of only 5–6 and lie on the unit sphere. Standard k-means centroids land inside the sphere, artificially shrinking dot-product scores. Re-normalizing centroids to the sphere ("spherical anchoring") fixes this for free (+0.093 nDCG@5 at k=4k=4). Building on this, the paper proposes GLIE (Generative Late-Interaction Embeddings): a 415K-parameter zero-initialized cross-attention layer refines the anchor vectors offline, and at query time, a lightweight generative decoder reconstructs the full NN vectors on-demand for only the top-LL (L=20L=20) candidates for reranking. On ViDoRe v1 and v2, GLIE preserves ~78–83% of uncompressed retrieval quality with just 4 stored vectors per page (~250x compression).

Critical strengths
  1. The geometric explanation of centroid shrinkage is brilliant and immediately useful. Showing that Euclidean centroids sit inside the unit sphere and drag down MaxSim scores is mathematically clean (Proposition 1). Normalizing them back to the sphere costs zero compute and gives a massive +0.03 to +0.09 nDCG@5 boost across benchmarks. It's a great practical finding for the whole community.

  2. Zero-initializing the refiner guarantees Stage 1 starts no worse than normalized k-means, and passing anchors unchanged into slot 0 guarantees that Stage 2 reconstruction will never lower the first-stage MaxSim score (MaxSim(Q,gψ(C))≥MaxSim(Q,C)MaxSim(Q, g_\psi(C)) \ge MaxSim(Q, C)). It’s a very clean, low-risk architectural design.

  3. The entire refiner and decoder add only 415K parameters, and fitting on cached embeddings takes under 3 GPU-minutes on 5,000 pages without touching the heavy 3B backbone.

Critical weaknesses
  1. In Table 7, most of the gain comes from normalizing centroids (+0.093 at k=4k=4) and Stage 1 refinement (+0.035), while the Stage 2 generative decoder only adds +0.015. At larger budgets (k≥16k \ge 16), the generative gain is almost negligible. Doing on-the-fly vector decoding and rescoring adds real serving latency, but the paper provides no wall-clock numbers (QPS, P99 latency).

  2. Training a 3B-parameter model via LoRA for just 1.5 GPU hours on 4,000 pages is practically starving it. Light-ColPali was designed for ~72 GPU hours on 130K queries. Comparing a tiny codec against an undertrained 3B model doesn't prove the codec beats backbone fine-tuning; it only proves it converges faster.

  3. For large collections (millions of pages), running iterative k-means with multiple restarts per page can be much slower than simple uniform pooling or patch slicing. The paper doesn't report offline indexing throughput.

  4. ColPali deployments rely on vector engines like PLAID or EMVB with residual quantization. The paper doesn't test whether GLIE's anchor vectors or regenerated vectors play nicely with scalar/product quantization.

Questions for authors
  1. What is the actual wall-clock search latency (P50/P99 in ms) and QPS? How much overhead does Stage 2 decoding add compared to Stage 1 alone?

  2. How does GLIE compare against Light-ColPali's officially trained checkpoints at overlapping budgets (k≥16k \ge 16) rather than your 1.5-hour undertrained reproduction?

  3. How many pages per second can your indexing pipeline process (including k-means and the refiner) on a single A100 GPU?

  4. Why does GLIE flatline (+0.003) when expanding the shortlist LL from 20 to 100, while the oracle gains +0.041? Is the decoder hallucinating false-positive patch features?

  5. Have you checked intrinsic dimension and spherical normalization on older text-only ColBERTv2 models, or does this geometric property only hold for vision-language models?

2 comments

We thank the reviewer for the constructive comments. We answer each point below.

W1. The gain of the generative read-out.

The reviewer's numbers are right, and they are the paper's own decomposition (Table 7). The scope of the paper is the geometry of page vectors, the training-free correction that follows from it, and a minimal generative read-out. We believe decoder design is a large topic of its own, so we kept the read-out minimal and limited the paper to two results that show this direction is promising: (1) the design of the decoder matters (Table 10, re-run with three seeds in our reply to reviewer tiqY), and (2) a headroom of 0.13 nDCG@5 at k = 4 remains between GLIE and the original vectors on the same shortlist (Figure 4). The generative read-out is therefore a first step in this direction, and we will reflect this scope more clearly in the revision.

W1 / Q1. Latency (P50 and P99), throughput and the overhead of stage 2.

Late interaction is limited by storage before speed, so the question is whether GLIE pays for its storage saving with latency. It does not. We timed the full search end to end on one A100 (1,000 real queries, one at a time, L = 20), with the corpus replicated to 10K, 100K and 1M pages.

Corpus sizeSystem at k = 4P50 (ms)P99 (ms)QPS (one query at a time)
10K pagesUncompressed, exact MaxSim13.2713.3775
10K pagesLearned code alone (stage 1)0.420.432,383
10K pagesFull GLIE (stage 1 + generative read-out)2.502.63399
100K pagesUncompressed, exact MaxSim133.83134.017.5
100K pagesLearned code alone (stage 1)0.810.871,235
100K pagesFull GLIE (stage 1 + generative read-out)2.572.68389
1M pagesUncompressed, exact MaxSimdoes not fit on the GPU (258 GB)
1M pagesLearned code alone (stage 1)5.535.63181
1M pagesFull GLIE (stage 1 + generative read-out)7.107.19141

Stage 2 adds about 2 ms to stage 1 alone (2.1, 1.8 and 1.6 ms at the three corpus sizes). This overhead does not grow with the corpus, because the read-out only regenerates the 20 shortlisted pages. At one million pages full GLIE answers a query in 7.10 ms at P50 and 7.19 ms at P99, about 140 queries per second, from a 1.0 GB index. We will add this table to the paper.

W3 / Q3. Indexing throughput, including k-means and the refiner.

The indexing pipeline processes 23.5 pages per second on one A100, including k-means and the refiner, against 23.8 for ColPali encoding alone. We measured each step.

Indexing step (one A100, k = 4)Pages per secondTime per page
ColPali encoding (needed by every method)23.842 ms
k-means (batched on the GPU)2,2280.45 ms
Refinerabout 7,0000.14 ms
k-means + refinerabout 1,7000.59 ms

Indexing is bound by the encoder. k-means and the refiner add 0.59 ms to the 42 ms of encoding (1.4%). The timed k-means is a batched GPU implementation of the same algorithm and gives the same accuracy as the CPU implementation (within 0.007 nDCG@5 for k up to 16). We will add this table to the paper.

W2 / Q2. Comparison with Light-ColPali.

We agree that our short reproduction measures what a small training budget buys, not the published model. To our knowledge the checkpoints have not been released, so we compare against the numbers published for Light-ColQwen2, with GLIE fitted on ColQwen2. Each entry is the share of that system's own uncompressed score that is kept.

Vectors per pageLight-ColQwen2, published (backbone fine-tuned, 72 GPU-hours, 130K queries)GLIE on ColQwen2 (frozen backbone, 2.7 minutes, 5K pages)Normalized k-means on ColQwen2 (training-free)
1694.6%93.3%91.7%
31 (Light-ColQwen2), 32 (GLIE)96.3%96.1%95.4%
85 (Light-ColQwen2), 64 (GLIE)98.2%98.3%97.6%

The published model is ahead by 1.3 points at 16 vectors, and GLIE is level with it from 32 vectors with a frozen backbone and 2.7 minutes of training. We will add this comparison to the paper and label our reproduction as a reduced-budget one.

W4. Quantization as in PLAID and EMVB.

PLAID and EMVB reduce the bytes per vector. GLIE reduces the number of vectors per page, so the two combine. We tested this with scalar quantization of the stored code after training, with no retraining. The regenerated vectors are never stored, so only the code needs to be quantized.

Stored code quantized after trainingChange in nDCG@5 against the paper's 16-bit code, k = 4 to 64
8 bits per value, all three systemswithin 0.0004
4 bits per value, normalized k-means-0.001 to -0.002
4 bits per value, learned code alone-0.002 to -0.003
4 bits per value, full GLIE0.000 to +0.001

At 8 bits nothing changes. At 4 bits the stored code loses at most 0.003 and full GLIE loses nothing, so the read-out absorbs the quantization error of the code. A page at k = 4 then takes 280 bytes instead of 1,040, so the index is more than 900 times smaller than the uncompressed one (about 250 times in the paper). We will add this table to the paper.

Q4. Why does GLIE stay flat when the shortlist grows from 20 to 100? Is the decoder hallucinating?

GLIE stays flat because the pages that enter at rank 21 to 100 are pages whose stored code misses the evidence for the query. The read-out regenerates each page from that same code without seeing the query, so it cannot add the missing evidence, and the ranking barely changes. On the full protocol at k = 4, growing the shortlist from 20 to 100 adds +0.002 for the read-out and +0.041 for the oracle. We traced the queries that this concerns, those whose relevant page sits at stage-1 rank 21 to 100.

Relevant page at stage-1 rank 21 to 100 (13.9% of queries, k = 4)Generative read-outOriginal vectors
Score gained by the relevant page over the pages ranked ahead of it+0.017+0.104
Relevant page moved into the top 54.4%61.8%

The read-out raises the relevant page and the pages ahead of it by almost the same amount (+0.107 and +0.091), so the order stays. The original vectors carry evidence that the four stored vectors do not, so they move the relevant page up.

The read-out is not hallucinating false positives. A hallucinated feature would give a non-relevant page evidence that it does not have. That would raise its score above its true value and move it ahead of the relevant page. We measured both, and the read-out does the opposite:

  • Scores: at k = 4 the decoded scores of non-relevant shortlisted pages stay 0.023 below their true values.
  • Order: at k = 4 the relevant page is ranked ahead of a non-relevant shortlisted page in 86.9% of the pairs with the stored code, 88.6% after the read-out, and 96.6% with the original vectors. The read-out moves the order toward the true one at every budget.
  • Design: every regenerated vector stays within 37° of a stored vector of its page, so the read-out fills in around what the page stores.

So the read-out is limited by evidence that it does not recover, not by false positives. We will add this analysis to the appendix.

Q5. Do the low intrinsic dimension and spherical normalization also hold for text-only ColBERTv2?

Yes, both hold. We measured text-only ColBERTv2 (colbert-ir/colbertv2.0 weights) on two BEIR test sets, SciFact and NFCorpus, with the same estimator and the same null controls as in the paper. Each control is built per document with that document's own token count and dimension, so a low reading cannot come from estimator bias. Token embeddings are unit-norm, and the intrinsic dimension sits far below both the linear spread and a Gaussian with the same covariance.

ColBERTv2, median over 1,000 documents with at least 100 tokensSciFactNFCorpus
TwoNN, real tokens2.92.8
TwoNN, covariance-matched Gaussian25.725.3
TwoNN, uniform on the sphere56.556.4
Participation ratio21.219.8

Proposition 1 needs only unit-norm tokens, so its consequence should carry over as well. It does. Normalizing per-document k-means centroids improves nDCG@10 at every budget, and the gain shrinks with k as the proposition predicts.

nDCG@10, ColBERTv2k = 4k = 8k = 16k = 32uncompressed
SciFact, raw k-means0.3420.4600.5260.5960.692
SciFact, normalized k-means0.4500.5440.5850.633
NFCorpus, raw k-means0.1800.2160.2650.2960.344
NFCorpus, normalized k-means0.2120.2450.2890.312

In this paper we focus on visual document retrieval, because storage is the binding constraint there (about a thousand vectors per page). The experiment above shows that the same geometry, and the gain that follows from it, carry over to text. We will add both tables to the appendix.

Comments on the paper

1 comment

We thank the AC and the reviewers for their time and constructive comments. Each question is answered in the reply under its review, and the new results are now in the revised PDF (changes in blue).

Contribution of the generative read-out (tiqY, Q8Xh, KcQP). The main concern of the reviews was the gain of the generative read-out next to spherical anchoring. To address it, we added normalized k-means to the main comparison and made the scope of the read-out clearer throughout the paper. (Abstract, Sections 1, 4.2 and 5, Tables 3 and 4, Figure 1)

Cost and deployment (tiqY, Q8Xh, KcQP). The reviews raised the missing indexing and query costs, the memory trade-off of stage 2, and compatibility with quantization. To address them, we measured the full pipeline and showed that indexing is bound by the encoder, that the read-out lets the index keep no original vectors at a small query-time overhead that does not grow with the corpus, and that quantization combines with GLIE, shrinking the index further at no cost in score. (Sections 3.6, 4.2 and 4.5, Appendix A.6)

Analysis of the read-out (tiqY, Q8Xh). The reviews raised which local details the read-out recovers and why a larger shortlist does not help it. To address them, we showed with a per-query-token analysis and an error analysis of the shortlist that the read-out recovers most of what compression weakens and is limited by the evidence in the stored code. (Section 4.5, Appendix A.4 and A.5)

Baselines and design choices (KcQP, Q8Xh). The reviews raised the choice of k-means and the comparison with the published Light-ColPali. To address them, we compared normalized k-means with other training-free ways to choose the vectors, including spherical k-means. We also compare with the published Light-ColQwen2 numbers, which GLIE matches at moderate budgets with a frozen backbone. (Sections 3.3 and 4.3, Appendix A.3 and A.7)

Geometry (KcQP, Q8Xh). The reviews raised the link between the geometry and the method, and whether the geometry holds beyond vision-language encoders. To address them, we tied each geometric measurement to the design choice it supports, and showed on a text-only encoder, ColBERTv2, that the same geometry and the same gain from spherical anchoring hold. (Section 3.1, Appendix A.7)

We hope these results address the reviewers' concerns, and we thank the AC and all reviewers again for their time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.