- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes AC-Scoreq, a fixed-codebook residual quantization method for efficient vector retrieval. It combines coordinate-level 2-bit phase codes with pair-level 2-bit amplitude-shape codes in the final residual stage, using jointly fitted block scales to improve reconstruction. Database vectors are encoded independently, and queries undergo the same orthogonal transform, enabling direct similarity computation from compressed codes for inner product, cosine similarity, and L2 distance. Experiments on FIQA and DBPedia-Entity with three embedding models evaluate retrieval quality, storage, and full-corpus GPU latency.
Critical strengths
- The phase–amplitude–scale design is well motivated and cleanly separated. Amplitude coupling is integrated into the final reconstruction rather than applied as post-processing, and the structured scoring identity supports direct similarity computation from packed codes without a full-precision reranking vector.
- All codebooks are fixed analytically; database vectors are encoded independently with no corpus-level training or query knowledge required.
- The paper reports actual bytes per vector, explicitly frames T=3 as a nearby-budget comparison, averages over 50 seeds with confidence intervals, and includes equal-amplitude and same-budget extra-phase controls.
Critical weaknesses
-
The specific shape codebook, pairwise coordinate grouping, and final-stage-only amplitude encoding are geometrically motivated but not systematically compared against alternative codebooks, grouping sizes, or stage allocations. Consequently, the benefits of amplitude coupling are more convincing than the necessity of the particular design.
-
The theoretical analysis provides useful geometric intuition but does not fully characterize the effects of shared or quantized block scales, interactions across residual stages, or errors in inner-product and ranking computations. The connection between the theoretical analysis and downstream retrieval performance therefore remains partly empirical.
-
Evaluation is limited to FIQA and DBPedia-Entity, without validation on standard large-scale approximate nearest-neighbor benchmarks. Although the paper includes useful ablations, some retrieval differences are small, and more detailed statistical comparisons and effect-size analyses would help establish the robustness of the reported improvements.
Questions for authors
-
Can the authors provide quality–storage Pareto curves at closely matched budgets, with consistent accounting for scales and auxiliary information? This would clarify whether the observed gains arise from the representation itself or partly from differences in storage allocation.
-
Have alternative shape codebooks, coordinate grouping sizes, or stage allocations been evaluated? In particular, why is amplitude-shape encoding restricted to adjacent complex-coordinate pairs in the final residual stage?
-
How sensitive is the alternating scale-fitting and code-refinement procedure to initialization and the number of refinement sweeps?
-
Do the comparisons against the same-budget extra-phase control show consistent retrieval improvements across embedding models and similarity metrics? Are there cases where lower reconstruction MSE does not translate into better retrieval quality?
6 comments
Response to Weakness 3 and Question 1: Benchmarks and Storage–Quality–Latency Trade-offs
(Weakness 3)
Benchmark Coverage and Effect-Size Evaluation
We added angular-distance retrieval on NYTimes, with 290,000 database vectors and 10,000 queries. The results are shown in Table 3.
Table 3. Storage, distortion, and exact-neighbor recall on a standard ANN dataset.
| Dataset | Method | B/vector | MSE ↓ | NN Recall@10, % ↑ |
|---|---|---|---|---|
| NYTimes | AC, | 58 | 1.04693e−3 | 65.517 |
| NYTimes | AC, | 100 | 3.13131e−4 | 81.127 |
| NYTimes | AC, | 142 | 9.67977e−5 | 89.092 |
For text retrieval, we further use the official 648 FIQA and 400 DBPedia test queries for paired statistical analysis. The original AC and extra-phase control are evaluated with 50 and three seeds on the two datasets, respectively. The complete intervals are reported in Table 10 of this response.
(Question 1)
Quality–Storage–Latency Trade-offs and Storage Accounting
We place the default stage counts and storage-sensitivity configurations in the same table to compare operating points using actual B/vector. are independently encoded configurations; phase codes, final amplitude shapes, and scales jointly determine the payload.
As shown in Table 4, in addition to the stage count, scale precision and block size also change the trade-off between storage and retrieval quality.
Table 4. Actual storage–quality operating points on FIQA. Each cell reports "B/vector / IP NDCG@10." All AC configurations retain amplitude coupling in the final stage.
| Configuration | ModernBERT | Qwen-1.5B | Qwen-7B |
|---|---|---|---|
| AC, | 170 / 0.458340 | 338 / 0.530670 | 786 / 0.615047 |
| AC, | 292 / 0.498751 | 580 / 0.536494 | 1348 / 0.624395 |
| AC, , default | 414 / 0.502031 | 822 / 0.539079 | 1910 / 0.627198 |
| AC, , 4-bit scales | 378 / 0.501224 | 750 / 0.535855 | 1742 / 0.624320 |
| AC, , block size 32 | 378 / 0.499455 | 750 / 0.539346 | 1742 / 0.627430 |
To also show computational cost, Table 5 reports quality and latency produced by the same set of timing runs.
Table 5. Storage–quality–latency operating points corresponding to the same actual retrieval outputs on DBPedia/Qwen-7B. 400 official test queries, A800, document chunks of 50,000 vectors, top-1000; times are means over five runs after two warmup runs.
| Method | B/vector | IP NDCG@10 ↑ | Cosine NDCG@10 ↑ | IP seconds ↓ | Cosine seconds ↓ |
|---|---|---|---|---|---|
| FP32 reference | 14336 | 0.470776 | 0.470776 | 0.853490 | 0.881974 |
| PQ-16x | 896 | 0.467410 | 0.471366 | 0.201670 | 0.229450 |
| PQ-8x | 1792 | 0.472264 | 0.470284 | 0.232437 | 0.260153 |
| AC, | 786 | 0.469197 | 0.469896 | 0.198047 | 0.226133 |
| AC, | 1348 | 0.470316 | 0.470793 | 0.201881 | 0.229622 |
| AC, | 1910 | 0.472470 | 0.472178 | 0.208448 | 0.236383 |
These operating points reflect the method's main purpose: selecting a configuration according to storage, retrieval quality, and query cost, rather than solely pursuing the highest accuracy at a single compression ratio.
Response to Questions 2–3: Design Choices and Refinement Sensitivity
(Question 2)
Alternative Shapes, Group Sizes, Pairing, and Stage Placement
We compared four types of design choices.
First, the narrower shapes use , the default shapes use , and the wider shapes use single-coordinate endpoints at ; all three retain the zero and equal-amplitude shapes.
As shown in Table 6, the default shapes achieve the lowest reconstruction error in this comparison, but the configuration with the highest NDCG is not the same for all models.
Table 6. Sensitivity to the amplitude-shape range. FIQA, seed 42; each cell reports "MSE ↓ / IP NDCG@10 ↑," with identical storage and optimization settings.
| Shape range | ModernBERT | Qwen-1.5B | Qwen-7B |
|---|---|---|---|
| Narrower, | 0.0493293 / 0.500462 | 1.66214e−5 / 0.538566 | 7.10733e−6 / 0.627135 |
| Default, | 0.0481763 / 0.502031 | 1.62303e−5 / 0.539079 | 6.93949e−6 / 0.627198 |
| Endpoints, | 0.0541217 / 0.500323 | 1.82242e−5 / 0.539670 | 7.79336e−6 / 0.627389 |
Second, under a consistent greedy initialization protocol, we compared placing AC at different residual stages, as well as pairwise shapes versus four-coordinate shapes. The four-coordinate control uses 16 fixed support patterns with 4 bits per group, while the pairwise configuration uses 2 bits per pair, keeping the total shape-field budget the same.
Table 7. Comparison of stage placement and grouping at the same shape budget. FIQA, seed 42, three refinement sweeps; each cell reports "MSE ↓ / IP NDCG@10 ↑."
| Setting | ModernBERT | Qwen-1.5B | Qwen-7B |
|---|---|---|---|
| Pairwise AC at stage 1 | 0.0625284 / 0.498180 | 2.10056e−5 / 0.538486 | 8.98813e−6 / 0.625520 |
| Pairwise AC at stage 2 | 0.0557714 / 0.500552 | 1.87560e−5 / 0.539195 | 8.02267e−6 / 0.626032 |
| Pairwise AC at stage 3 | 0.0534314 / 0.501382 | 1.79783e−5 / 0.537900 | 7.68710e−6 / 0.626457 |
| Four-coordinate shapes at stage 3 | 0.0592836 / 0.50080 | 1.99304e−5 / 0.53784 | 8.52514e−6 / 0.62535 |
Finally, we compared adjacent, fixed-shuffle, and strided pairing, as shown in Table 8.
Table 8. Retrieval sensitivity to coordinate pairing. FIQA, seed 42, IP NDCG@10; higher is better. All other encoding settings are held constant.
| Pairing method | ModernBERT | Qwen-1.5B | Qwen-7B |
|---|---|---|---|
| Adjacent pairing | 0.502031 | 0.539079 | 0.627198 |
| Pairing after a fixed random shuffle | 0.500516 | 0.540734 | 0.626885 |
| Strided pairing | 0.500872 | 0.539512 | 0.627328 |
These results support the default design as a choice that is simple to implement and requires no additional learning of coordinate pairings, with its scope determined by empirical comparisons.
(Question 3)
Initialization and Refinement-Sweep Sensitivity
We compared mean-amplitude initialization, projection initialization, and one, three, and five refinement sweeps. The default encoder selects among initialization candidates based on the reconstruction objective, without using queries or qrels.
As shown in Table 9, increasing the number of sweeps from one to three reduces reconstruction error for all three models; the further improvement from increasing to five sweeps is smaller, and relevance-based retrieval does not improve consistently.
Table 9. Sensitivity to initialization and the number of refinement sweeps. FIQA, seed 42; each cell reports "MSE ↓ / IP NDCG@10 ↑." All configurations have the same storage budget.
| Configuration | ModernBERT | Qwen-1.5B | Qwen-7B |
|---|---|---|---|
| Mean initialization only, 3 sweeps | 0.0481771 / 0.502031 | 1.62303e−5 / 0.539079 | 6.93949e−6 / 0.627198 |
| Projection initialization only, 3 sweeps | 0.0508483 / 0.500308 | 1.71437e−5 / 0.538016 | 7.32697e−6 / 0.625102 |
| Default initialization, 1 sweep | 0.0498999 / 0.500424 | 1.68134e−5 / 0.539125 | 7.18826e−6 / 0.626474 |
| Default initialization, 3 sweeps | 0.0481763 / 0.502031 | 1.62303e−5 / 0.539079 | 6.93949e−6 / 0.627198 |
| Default initialization, 5 sweeps | 0.0478743 / 0.501930 | 1.61279e−5 / 0.537564 | 6.89569e−6 / 0.627279 |
Three sweeps already capture the main reconstruction gains; further optimization does not guarantee corresponding improvements in retrieval quality, so we retain the same three-sweep configuration.
Response to Weaknesses 1–2: Amplitude Design, Scale Fitting, and Retrieval
Thank you for recognizing the method's structure and configurability. To address design choices, theoretical connections, and evaluation scope, we added comparisons of amplitude codebooks and grouping methods, sensitivity analyses of stage placement and initialization, ablations of joint scale fitting, paired query statistics, and results on standard ANN benchmarks and actual storage–quality–latency trade-offs.
We added factorial experiments that separately control the zero shape and asymmetric shapes. As shown in Table 1, adding asymmetric amplitude patterns still substantially reduces reconstruction error when the zero shape is retained.
Table 1. Controlled comparison of amplitude-shape composition. FIQA, mean MSE over three random seeds; lower is better. Encoding fields and actual byte counts are identical within each model.
We also evaluated narrower and wider shapes, four-coordinate grouping, nonadjacent pairing, and different stage placements. The complete results are presented in Tables 6–8 under Question 2 of this response. These controls support local amplitude adaptation as a design mechanism and treat the specific codebook and grouping choices as testable implementation choices.
We added conditional analyses at three levels.
First, with the discrete codes fixed, let C be the complex basis matrix containing the phase codes of each stage and the final amplitude shapes, and let s be the vector of real-valued scales. Joint scale fitting satisfies
=\operatorname{Re}(C^{H}z), \qquad \eta=2B\lambda.$$ The off-diagonal terms explicitly describe correlations between stages. This solution minimizes the regularized objective with fixed codes; when the regularization term is nonzero, we do not directly equate this conclusion with minimizing unregularized MSE. Second, if the stored scales are $\widehat{s}$, the additional error introduced by scale quantization satisfies $$\|z-C\widehat{s}\|_2 \le \|z-Cs^{*}\|_2+ \|C\|_2\|\widehat{s}-s^{*}\|_2.$$ This connects scale quantization error to the actual stage basis matrix without requiring different stages to be orthogonal. Finally, let $e_i=\widehat{x}_i-x_i$. For IP retrieval, if the exact score margin satisfies $$q^{\top}(x_i-x_j)> \|q\|_2\|e_i-e_j\|_2,$$ then the relative order of documents $i$ and $j$ is preserved after reconstruction. This is a sufficient condition that depends on the query and score margin. We also added a control that disables joint scale fitting during both initialization and refinement. As shown in Table 2, restoring joint fitting improves exact-neighbor preservation in all nine settings. **Table 2. NN Recall@10 gains of full AC over disabled joint scale fitting. FIQA, three seeds; values are in percentage points, and higher is better.** | Model | IP | Cosine | L2 | |---|---:|---:|---:| | ModernBERT | **+6.816 [6.108, 7.536]** | **+1.590 [0.977, 2.207]** | **+2.459 [1.857, 3.061]** | | Qwen-1.5B | **+3.102 [2.567, 3.642]** | **+1.101 [0.628, 1.574]** | **+1.343 [0.859, 1.836]** | | Qwen-7B | **+1.651 [1.224, 2.094]** | **+0.756 [0.386, 1.137]** | **+1.070 [0.694, 1.456]** | Brackets contain adjusted paired query bootstrap intervals: the three seeds are first averaged within each query, followed by 100,000 resamples. The correction family comprises three models, three similarities, and four controls: zero + equal amplitude, independent 1-bit amplitudes, disabled joint fitting, and extra phase. This gives 36 comparisons with a nominal family confidence level of 95%. **The conditional analyses explain how errors affect scores and rankings, while the independent ablation validates the practical retrieval benefits of joint fitting.**