Budget-Aware Multi-Vector Index Compression for Visual Document Retrieval

Visual Document Retrieval (VDR) has become an essential capability for document intelligence systems, yet the growing storage cost of multi-vector indexes has become a critical barrier to large-scale deployment. Compressing VDR indexes introduces two fundamental challenges: 1) how to allocate a limited storage budget between preserving document-side matching opportunities and reducing representation cost; 2) how to maintain reliable late-interaction scoring when aggressive compression distorts the geometry of compressed representations. In this work, we propose BaMIC (Budget-aware Multi-Vector Index Compression), a coverage-first compression method. We identify a structural asymmetry between compression dimensions: reducing vector cardinality directly removes potential matching opportunities, whereas numerical compression preserves matching coverage but may introduce geometric distortions in the extreme compression regime. Based on this observation, BaMIC prioritizes document-side vector coverage over per-vector precision and adapts representation rates to the available storage budget, and applies geometry-calibrated scoring, which requires no additional training or storage, to mitigate compression-induced radial bias. Extensive experiments across multiple VDR benchmarks, vision-language backbones, and compression budgets demonstrate that BaMIC achieves favorable storage-quality trade-offs, retaining 94.1% of full-precision Hit@1 on average with ColQwen3.5 at 64$\times$ compression and improving Hit@1 by up to 34.5 points over the strongest vector-count baseline under equal storage budgets.