- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to incrementally refine a quantized LLM already cached at the client, so as to achieve a higher quality target with less transmission than replacing the model. RECAP combines two correction representations with different communication costs: one transmits coefficients under a shared activation-eigenvector basis, which both endpoints reconstruct from a public calibration context; the other transmits low-rank factors to correct the residual error remaining after decoding. The paper proves the minimax optimality of the shared basis when the residual is unobserved, provides an approximate continuous rate-allocation rule, and derives a joint distortion lower bound for the unquantized problem under a common metric.
Critical strengths
-
The paper formulates a meaningful incremental communication problem and provides extensive experimental support. RECAP distinguishes algebraic rank from full transmission cost and exploits a locally reconstructable activation basis to reduce the transmitted payload.
-
The theoretical scope and ablation studies are fairly rigorous. The paper clearly separates exact subspace identities, approximate low-bit rate allocation, and unquantized optimality under a common metric. Component comparisons, cross-corpus evaluation on C4, and endpoint-agreement tests help explain both the benefits and limitations of the design, strengthening the credibility of the reported gains.
Critical weaknesses
-
The source of RECAP’s full-system advantage is not fully isolated. Appendix C.2–C.3 indicates that RECAP and the baselines use different factor precisions, scaling, and fitting procedures, so these comparisons do not fully separate the benefit of the shared basis from that of the factor coding and fitting choices. In Table 13, complement-only actually achieves lower perplexity in two GPTQ-2 settings.
-
Table 12 shows that for Qwen-2.5-7B-Instruct GPTQ-2, RECAP perplexity varies from 21.04 to 23.12 depending on the numerical environment, indicating that the exact reconstruction configuration matters greatly. Please clarify how sensitive the reported gains are to calibration and eigensolver settings choices.
Questions for authors
-
Appendix D.6 states that allocating one quarter of the budget to the shared subspace was selected after component comparisons. How was this ratio selected?
-
Section 4.1 allocates bits according to , the product of the activation eigenvalue and the correction coefficient energy, whereas Table 7 shows that allocation based only on the activation eigenvalues, , achieves even better results in some settings. Could the authors explain this observation?
-
GlowQ (An et al., 2026) also corrects quantization error using shared factors. Please discuss how RECAP differs from GlowQ. Given the differences in sharing mechanisms and optimization objectives, a comparison under settings suitable for both methods could help clarify their relative quality–communication trade-offs.
-
Figure 1 needs clearer presentation. Please redraw it with consistent typography, cleaner alignment, and clearly labeled components and connections.
Reference: An, S., Suh, I., & Kim, Y. (2026, April). GlowQ: Group-Shared LOw-Rank Approximation for Quantized LLMs. In International Conference on Learning Representations (Vol. 2026, pp. 124445-124486).
2 comments
Response to Reviewer nv7F (2/2): Questions 2–4
Question 2: S_j against λ_j alone
The two signals tie, and the reason is measurable: the eigenvalues vary far more across columns than the coefficient energies, so they dominate the product S_j = λ_j‖A*_j‖² and both rules allocate almost the same rates (Appendix D.4).
The tie. Across the four settings of Table 12 (Table 7 in the submission), S_j is better in three (by at most 0.15) and worse in one (by 0.08).
The reason. We measured both factors on two layers of each Table 12 setting. Across the columns, the eigenvalues vary far more than the coefficient energies, so λ_j alone explains about 99% of the variation in log S_j, and the two rules assign the same bit rate to 98% of the columns on SINQ and 86–87% on GPTQ. Table 12 shows the same from the other side: allocating by coefficient energy alone is worse than equal column rates in every setting, while λ_j alone beats them.
Why we keep S_j. It is the quantity the allocation analysis yields, and it needs no extra statistic. What matters more than the choice between the two is the rate: water-filling's gain over equal column rates grows as the rate falls, from at most 0.6 perplexity at four bits to as much as 9.1 at one bit, and it is what lets the subspace code trade coefficient precision for basis dimension: with equal rates, one bit at d = 2048 is worse than four bits at d = 512 on both SINQ bases (Table 11).
Question 3: GlowQ
GlowQ fits its shared factor to the quantization errors and stores it for serving speed, whereas RECAP's shared basis is rebuilt by the client and never sent; compared in RECAP's setting, quality per transmitted byte on a frozen 2-bit base, RECAP is 13–29% lower in perplexity on all six settings (Table 20; Appendix D.10).
We thank the reviewer for the pointer. We now cite GlowQ (An et al., 2026) and compare with it (Section 2; Table 20; Appendix D.10).
How the two differ. Both methods use the fact that projections reading the same input (q, k and v; gate and up) can share an input-side factor, and RECAP groups them the same way. GlowQ fits its shared factor to the group's quantization errors, so the factor is part of the stored correction, and its purpose is serving efficiency: computing the shared projection once per group lowers latency. RECAP's shared basis depends only on the deployed checkpoint and public calibration text, so the client rebuilds it and it never crosses the link. What is transmitted is the residual's coefficients in that basis, coded at one to two bits, plus a low-rank complement. GlowQ also corrects a model when it is quantized, whereas RECAP refines a checkpoint that is already deployed, under a byte budget.
Comparison. GlowQ reports perplexity, accuracy and serving latency at 4 bits, so we compared the two in RECAP's setting: quality per transmitted byte on a frozen 2-bit base. We re-implemented GlowQ's closed form (one shared factor per group, sent once, and one factor per module) with 8-bit factors, as for the other baselines, at RECAP's payload of about 350 MB (WikiText-2 perplexity):
| Base | QERA | GlowQ | RECAP |
|---|---|---|---|
| Llama SINQ-2 | 10.86 | 10.66 | 9.30 |
| Llama HQQ-2 | 11.89 | 11.69 | 9.48 |
| Llama GPTQ-2 | 24.81 | 24.54 | 18.50 |
| Qwen SINQ-2 | 17.00 | 17.23 | 13.63 |
| Qwen HQQ-2 | 17.46 | 17.93 | 13.47 |
| Qwen GPTQ-2 | 30.51 | 29.76 | 21.03 |
Sharing the factor helps slightly over per-module QERA on four of the six settings. The two ideas also compose: GlowQ's cached group projection is a serving optimization that RECAP's factorized layout could adopt.
Question 4: Figure 1
We have redrawn Figure 1 as a vector figure in one consistent typeface, on a single aligned grid.
Each component is labeled by where it lives (server, link, client), and the connections show what the server computes, what crosses the link, and what the client rebuilds from its cached checkpoint and the public calibration text.
We hope these responses resolve the reviewer's concerns. The revision adds the shared-basis control, the specified numerical protocol, the validation-selected split, the GlowQ comparison and the redrawn Figure 1 discussed above. If the concerns are resolved, we would be grateful if the reviewer would consider raising the score, and we are happy to answer any further questions.
Response to Reviewer nv7F (1/2): Weaknesses 1–2 and Question 1
We thank the reviewer for recognizing the incremental communication problem, the distinction between algebraic rank and transmission cost, and the rigor of the theoretical scope and ablations. Table and appendix numbers below refer to the revised PDF, where new material is in blue.
Weakness 1: isolating the source of the advantage
A control that differs from RECAP only in the shared basis isolates it: with exactly RECAP's 4-bit factor code, per-row scaling and rounding-aware refits given to QERA, the shared basis still lowers perplexity by 17–71% on the 1.58-bit bases and by 1–6% on the naive 2-bit bases, and on GPTQ-2, where it pays little, the validation-selected split moves RECAP to its complement code or near it (Tables 3, 20 and 21).
The control is QERA's closed-form initialization with RECAP's factor code, at RECAP's payload (the "complement code alone" of Tables 3 and 20; Table 20 was Table 13 in the submission). At about 350 MB (WikiText-2 perplexity):
Matching the factor procedure improves QERA substantially (837.6 to 199.2 on Llama SINQ-1.58), and the shared basis then lowers it to 96.5.
Why GPTQ is the exception. GPTQ's error feedback moves the activation-weighted residual out of the leading activation directions, which hold only 25–26% of it, against 64–65% on SINQ (Appendix D.8; D.6 in the submission). The complement code alone is RECAP's d = 0 case, so the revision selects the split on validation data with d = 0 among the candidates: RECAP falls back to it on Qwen GPTQ-2 and selects d = 64 on Llama GPTQ-2, within 0.07 of it (Table 21).
Why 8 bits for QERA and 4 bits for RECAP. The better precision depends on the rounding: under QERA's own one-shot rounding, 4-bit factors at twice the rank are worse than its published 8 bits on every naive 2-bit base (Llama SINQ-2: 20.76 against 10.86), while with RECAP's rounding-aware refits the same 4-bit payload reaches 9.53.
Weakness 2: sensitivity to the numerical environment
Because the client rebuilds the shared basis, the protocol now fixes the numerics that determine it (an exact double-precision eigendecomposition and a float32 calibration pass at both endpoints): every audited client lands within 0.004 perplexity of the server, where the gap reached 2.1 before, and even before the fix RECAP kept its lead over the best baseline in every audited setting.
We thank the reviewer for raising this: it led us to trace the variation in Table 17 (Table 12 in the submission) to two causes in our reference implementation, and we have now fixed both.
The causes. The widest inputs used a randomized eigensolver whose draws differ between devices, and the CPU calibration pass ran at a different arithmetic precision from the GPU pass.
The fix. The reviewer's 2.1 gap, 23.12 against 21.04 on Qwen GPTQ-2, closes under the protocol (Appendix C.4; third block of Table 17): a CPU client now rebuilds the GPU server's correction exactly, so what remains is evaluation noise.
The reported gains. Even before the fix, RECAP's worst audited case was 23.12 against the best baseline's 25.21 on Qwen GPTQ-2. As a safeguard, the server sends a fingerprint of each basis, and a client that cannot reproduce it receives the basis at 4 bits instead, at a cost of at most 0.24 perplexity.
Question 1: how the quarter split was chosen
The quarter rule came from comparing a small and a larger subspace on the component grid, using the test perplexities the paper reports, and because the best split depends on the base, the server now selects it on held-out validation data, which matches the best test configuration in three of the four Table 20 settings and comes within 0.07 perplexity in the fourth (Table 21).
The choice. We compared a small fixed subspace (d = 64) with a larger one (d = 256) at about 350 MB on the twelve 2- and 3-bit 7B settings; the larger one gave lower perplexity in ten, losing only on the two GPTQ-2 settings. A fixed d = 256 does not fit the smallest payloads, so we made the rule proportional: the subspace code takes a quarter of the payload. It is one rule for all bases, and these comparisons used the test perplexities the paper reports.
The revision. Because the best split depends on the base, and to keep the choice off the test data, the split is now selected at the server on held-out validation data, with the complement code alone (d = 0) among the candidates (Appendix C.2; Table 21).