- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DoseRank, a training-free rank allocation method for converting existing attention models to MLA. Instead of assigning the same latent rank to every layer or relying on spectral energy, DoseRank directly measures each layer's accuracy sensitivity around a compressed operating point and uses these measurements to allocate a fixed rank budget. Across four models, DoseRank substantially improves accuracy over uniform allocation at matched KV-cache budgets and also consistently improves over CARE's spectral scheduler when applied to the CARE decomposition. The paper further studies task-transfer of the calibration signal and provides statistical tests and failure diagnostics.
Critical strengths
- Clean and consequential empirical result: The paper isolates rank allocation from decomposition and compares methods at matched KV budgets. DoseRank improves over uniform allocation at 24/26 tested budgets and also improves the CARE decomposition's spectral scheduler. This makes a convincing case that measured downstream sensitivity is substantially more useful than spectral energy for this allocation problem.
- Careful experimental diagnosis: The paper does more than report aggregate gains: it includes paired confidence intervals, multiple-comparison correction, alternative sensitivity proxies, held-out calibration-task experiments, and explicit analysis of regimes where the probe signal becomes unreliable. The held-out-task results are particularly useful in reducing the concern that DoseRank simply memorizes the calibration items.
Critical weaknesses
- Generalization beyond the evaluation suite remains unclear: DoseRank's default calibration data come from the same seven task families used for evaluation, and the sensitivity statistic is closely aligned with multiple-choice accuracy. The held-out-task ablation is helpful, but it still stays within the same benchmark suite. More importantly, generation and long-context behavior are not evaluated, despite long-context serving being a central motivation.
- The baseline set does not fully resolve how much novelty comes from the compressed operating point itself: The related-work section discusses other calibration/sensitivity-based layer-allocation approaches such as ASVD and MatryoshkaKV, but the main comparisons are against uniform allocation, spectral schedulers, and proxy ablations. An adapted task-aware allocation/search baseline under the same MLA decomposition and compute budget would make the central methodological claim substantially sharper.
These issues limit my enthusiasm for a clear accept, but I do not think they invalidate the main empirical finding.
Questions for authors
- Can the authors evaluate an allocation calibrated on the current multiple-choice suite on a genuinely different target distribution, preferably a generative or long-context task? This would clarify whether DoseRank identifies broadly important layers or mainly benchmark-specific sensitivity.
- Can the authors compare against an adapted calibration-driven allocation/search baseline of the type discussed in Section 2, under the same decomposition, KV budget, and comparable calibration compute? This would help isolate the benefit of measuring sensitivity specifically at the compressed operating point.
We thank you for your review
Major Weakness 1 / Question 1, generalization beyond the suite. Direct answer: an allocation calibrated on the multiple-choice suite transfers to held-out multiple-choice task families and, on TransMLA's decomposition, to needle retrieval. It does not transfer to GSM8K or LongBench.
Held-out families, gpt-oss-120B: calibrating on knowledge QA raises the held-out commonsense tasks by +2.21 pp at r̄=128 and +1.63 pp at r̄=256. MMLU alone raises the other six tasks by +2.16 pp (Section 5, Table 3b). Llama-3.1-8B, TransMLA, r̄=768 (Table 2): the suite-calibrated allocation improves the needle by 15 points over uniform ranks and leaves LongBench (9.1) and GSM8K (46.1 against 45.9) where they were. Calibrating on long-context items is what moves generation: GSM8K +5.8 [+3.2, +8.5] and LongBench +3.1. Calibrating on both sets keeps an accuracy gain of +1.3 pp and gives GSM8K +7.2. Our reading: the sensitivity DoseRank measures follows the calibration items, so they should match the target. The abstract and contribution 2 now say this, in place of the submission's general transfer claim. Caveat: the suite-calibrated allocation was solved with a rank cap of 1,536, the other two with 1,984 (Section 5). Major Weakness 2 / Question 2, a calibration-driven search baseline. The two baselines under Reviewer 4Gqb, Major Weakness 1, share our decompositions and budgets: ASVD's rule on all 52 cells of Table 1 (Table 15), and a MatryoshkaKV-style greedy search on Llama-3.1-8B (Table 16). Compute is not matched. ASVD's probes are 2.5 to 12 times cheaper than ours. The greedy search takes 1.9 to 3.8 hours against our 1.4 to 1.6, on 175 calibration items against our 2,044. The comparison that isolates the operating point is the full-rank-probe ablation, with the same statistic, items and solver (Table 3a). The operating point matters on TransMLA's decomposition and at r̄=192 on CARE's, and not at 256 or 384.