- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies query-independent compression of late-interaction visual document indexes. Its method, PARQ, retains every patch and allocates variable-length residual-quantization prefixes under a per-page code budget. Additional stages are assigned greedily according to marginal reconstruction-error reduction. A grouped lookup-table implementation evaluates MaxSim directly over these variable-depth codes. The paper provides a reconstruction-based bound on score distortion and proves greedy optimality under diminishing, nonnegative marginal gains. Experiments cover nine ViDoRe subsets and two frozen retrievers. At 64× code-payload compression, PARQ improves average nDCG@5 over fixed-depth RQ by 6.32 points for ColPali and 5.77 points for ColQwen2. Further experiments examine residual-error concentration, codebook transfer, and execution overhead.
Critical strengths
-
The main allocation comparison isolates a useful design choice and shows substantial gains at tight budgets. Tables 1 and 2 retain the same patch set and match code budgets across fixed-depth, random, norm-based, and marginal-gain allocations. At 64×, PARQ improves nDCG@5 from 57.16 to 63.48 on ColPali and from 63.05 to 68.82 on ColQwen2, with improvements across all nine reported subsets. Random and norm-based allocations perform worse than fixed depth, making the result more informative than simply showing that variable-length codes can work. Achieving these gains without query supervision or encoder fine-tuning gives the method practical value.
-
The residual-distribution analysis goes beyond an aggregate reconstruction-versus-retrieval correlation. Table 3 compares allocations with the same code budget and approximately matched average distortion. Reducing tail residuals improves nDCG@5 from 61.40 to 63.60 and reduces rank reversals from 15.0% to 10.5%. This is useful evidence that average reconstruction error alone does not characterize retrieval fidelity. The paper also appropriately limits its theoretical claims: Equation (8) bounds score error, while Appendix C establishes surrogate optimality only under stated assumptions and reports empirical violations of those assumptions.
Critical weaknesses
-
The comparison with the strongest external baseline is not specified precisely enough to establish the claimed advantage. Appendix B.3 describes Light-ColPali/ColQwen2 using “pooling factors” and allows either official checkpoints or reproductions on frozen embeddings, without identifying which configuration produced each reported row. The original Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings (2025) defines the final Light method using semantic clustering after the projector together with fine-tuning. It is therefore important to distinguish that method from training-free pooling or clustering variants. At 64×, PARQ’s reported advantages over Light are only 0.27 and 0.26 nDCG points, with no uncertainty estimates. I cannot determine whether these differences represent a reliable advantage over the published method. This limits the external comparison, although it does not invalidate the larger gains over fixed-depth RQ.
-
The physical-storage result is not yet auditable from the described layout. Appendix B.2.1 states that all required structures are included, but Table 5’s totals equal the listed patch-code bytes plus codebooks exactly at the displayed precision. Grouping patches by depth removes the need for an explicit depth field per patch, but decoding variable-size page/bucket groups still requires boundary information or an equivalent encoding. Its representation and byte cost are not explained. The listed codebook sizes also correspond approximately to a single 12-stage family with 16-bit coordinates, whereas the accuracy experiments fit domain-specific families. The paper needs to identify the collection and families used for this storage measurement. These omissions leave the claimed 25.51×/63.86× physical compression less well established than the nominal payload ratios.
Questions for authors
-
Which exact checkpoints and merging procedures produced the Light rows in Tables 1 and 2? Were these the fine-tuned semantic-clustering models, or frozen-embedding adaptations? Please also report any available variability for the 64× comparisons. A faithful baseline configuration and a reproducible difference would strengthen the practical comparison; otherwise, I would treat these results as approximate parity with a different training requirement.
-
How can the serialized index in Table 5 recover every page and depth-bucket boundary? Please provide the encoding and byte breakdown, identify the codebook families included, and clarify whether auxiliary bytes are contained within the “Patch code IDs” row. A self-contained layout that reconciles with the measured totals would resolve my storage concern.