- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ShareSVD groups Key heads using activation-aware subspace distances, shares an input basis obtained through joint SVD while retaining head-specific output factors, reallocates Q/K/V capacity using task-aware direction scores and a heuristic allocation rule, and performs Fisher-weighted local refinement. Evaluation covers three VLMs on ScienceQA-IMG and SEED-Bench-IMG, with the strongest reported gains at low retention ratios. Additional results examine two Qwen3 models, quantization compatibility, and materialized low-rank KV-cache allocation. The appendix provides a conditional least-squares result, group statistics, and a refinement ablation.
Critical strengths
-
A useful shared representation with targeted support for its design choices. Activation-aware grouping, a shared Key input factor, and head-specific output factors provide a practical structural extension of per-head low-rank factorization. Table 2 reports higher average accuracy for geometry-based grouping on all three models while matching group-size profiles. Proposition 1 supports the initialization's conditional optimality for a fixed subspace. Together, these provide meaningful support beyond an unexamined change in factorization.
-
Reported low-budget benefits extend across models and datasets. At ρ=0.3, Table 1 improves on WSVD in all six VLM–dataset combinations by approximately 8.57 accuracy percentage points on average. Improvements on SEED-Bench-IMG after ScienceQA calibration provide some cross-dataset evidence; Figure 6 reports lower perplexity for two Qwen3 models at most evaluated retention ratios. Table 4 also directly supports approximately 14.7% lower materialized low-rank cache allocation on average at the measured configuration. These constitute the main empirical case for acceptance, conditional on verification of the realized budgets.
Critical weaknesses
-
The relationship between the equal-budget reference and the claimed sharing savings is not coherently specified. Equation 10 defines using the cost of an independent per-head direction. The following text then selects the top group candidates. Under the literal reading , the shared cost is , exceeding that reference budget. The saving in Eq.11 instead compares against a different reference in which every member head retains directions. The sharing identity itself is correct, but the conversion between these references is missing. This affects the central reallocation mechanism and interpretation of equal-budget performance. Page 6 and Algorithm 1 state that realized costs are finally reconciled, so this establishes a problem in the written accounting rather than proving that the implementation or experiments exceed their budgets.
-
The contributions of sharing and rank reallocation are not sufficiently separated. The Share-Q/K/V and random-grouping comparisons support design choices but also involve capacity allocation. The paper does not clearly compare adjusting Q/K/V ranks alone, sharing with fixed per-projection budgets, and the full method under the same realized parameter budget, direction scoring, and refinement setup. The appendix does provide a refinement ablation: Table 7 increases SmolVLM accuracy from 16.16 to 47.23 at ρ=0.3, showing that refinement matters substantially. The evidence supports the reported combined pipeline but does not yet sufficiently attribute its gains to the proposed sharing-and-reallocation mechanism. A bounded component comparison would materially affect the assessment of its knowledge contribution.
-
The memory–latency trade-off of actual inference is not sufficiently established. Section 4.3, Figure 4, and Table 4 do support approximately 14.7% lower materialized BF16 low-rank KV-buffer allocation at . However, the paper reports neither peak memory nor latency/throughput for actual prefill and decoding, and does not specify whether latent caches are retained throughout, whether per-head K/V tensors are temporarily reconstructed, or which attention implementation is used. Equation 9 shares input factors while retaining head-specific output factors, making runtime compute and temporary storage implementation-dependent. For a method motivated by compression and cache savings, this limits assessment of its deployment value. The appropriate request is to quantify benefits and costs, not to assume it must be faster. If the claims are restricted to the measured buffers, the concern limits the scope and practical evidence rather than invalidating the existing measurements.
Questions for authors
-
How does the reference direction count in Eq.10 map to group ranks and final direction counts in Eqs.11–12? Please use a two-head group to illustrate how Algorithm 1 enforces the final budget, and report realized Q/K/V factor parameter counts and deviations from target budgets for representative configurations.
-
Under matched realized QKV budgets, sharing structures, calibration, and refinement, how does the proposed rule compare with uniform budget allocation and importance-based allocation? Please report accuracy and Q/K/V rank assignments.
-
Could you jointly report accuracy, actual GPU memory usage, and inference throughput across representative compression settings to characterize the accuracy–efficiency trade-off?
If the authors address these concerns satisfactorily, I would be willing to raise my rating.
I thank the authors for their time in responding to my concerns. I have decided not to increase my score because my main concerns remain unresolved. First, the introduction states that the primary motivation for this method is that it provides improved parameter compression rates by leveraging the robustness of the key heads. As shown in Figure 1, this is in fact not the case in two of the three models tested. If the authors aim to instead emphasize that this approach is a more performant KV cache compression method over prior work then the authors should instead state that in the introduction and include relevant KV cache compression baselines. Currently, the KV cache comparisons are presented as secondary.
Second, as mentioned in my earlier comment, the steps taken in the various stages in Section 3 lack formal justification. For instance, the authors compute the principal angles using Cholesky factors but not provide a formal justification for this approach. Moreover the authors refer to their decomposition as “activation aware” but also do not provide formal justification for this approach over naive non-activation aware approaches.