- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies whether learned changes of tensor basis improve spectral compression after charging for the transformation itself. BND combines a native-basis anchor, a bounded-plane Cayley rotation, and rank search. Experiments on 1,337 matrices, including synthetic controls, find modest additional structure in caches and weights, with much larger gains on planted examples. Storage and matrix-product costs are evaluated separately.
Critical strengths
The cost accounting includes transformation overhead and distinguishes storage costs from matrix-product costs, avoiding the misleading assumption that rank reduction necessarily translates into execution savings. The spectrum-matched controls and component ablations are informative, and the paper reports negative results that help distinguish the benefits of the learned basis from those of compression itself.
Critical weaknesses
W1. The incremental contribution on real matrices is limited. The native anchor and bounded planes are reasonable adaptations of the tensor-disentanglement framework of Wei et al. (2025), but their additional benefit on real matrices is modest. Table 2 reports storage compression factors of 1.18 versus 1.16 for SVD on caches and 1.14 versus 1.13 on weights. The large gains on gradients and state-space matrices require no learned rotation, while compression factors for frontier weights remain near 1.00. The strongest improvements occur on planted controls constructed using this type of rotation. Moreover, the no-worse-than-SVD guarantee follows from retaining SVD as a candidate and does not, by itself, establish the value of basis optimization. W2. A practically useful quality–cost operating point has not been established. Table 1 reports matrix-product speedups of approximately 1.00× for caches and 1.01× for weights, and Section 6 acknowledges that the search is unsuitable for transient caches. Appendix G provides downstream evaluations, but these primarily concern plain SVD. The BND experiment on distilgpt2 (Table 19) reduces storage by only about 0.2–0.4% relative to SVD, while perplexity reaches 81.5 and 174.3 at the two tested budgets, compared with 47.0 for the original model. These results do not demonstrate worthwhile compression while preserving model quality. The isolated transfer measurements also leave factor preparation and the complete inference workflow unevaluated.
Questions for authors
- Which operating point demonstrates a useful advantage of BND over SVD at comparable downstream quality? Please distinguish stored size, device memory, and inference latency, and account for factor preparation and the cost of learning the basis, including how that cost is amortized.
- Equation (9) defines squared relative error, but Section 3.3 compares it directly with the budget used for unsquared relative error in Equation (1). Does the implementation compare the squared tail with the square of the budget? Please clarify the actual certification criterion.
Q1/W2 Fair criticism. We understand that all reviewers consider at least one useful operating point necessary to warrant a higher score. We expand on the promising results on LLM caches by focusing on large, idle (non-transient) key caches of long-running agentic LLM systems with 13k-25k tokens. We tune ϵ to the lossless regime of SVD (ϵ=0.10−0.15, ΔF1 -0.002 [-0.010, +0.005] not statistically significant). Within this regime, BND stores 2.6-3.7% fewer key parameters than SVD using the procedure described in our paper. Device memory and per-token latency are unchanged, since keys are decompressed once on resume; factor preparation takes 0.7 ms (BND) vs 0.3 ms (SVD) for an average 17k-token history, and resume time is on par with SVD from host memory and faster from SSD or network storage by up to its 2.7–3.6% byte saving. Learning the basis takes about 1–4 CPU-minutes per 2048 × 128 block, run offline while the agent is idle and parallel across blocks; it amortizes over the time a cache stays stored, which we state as the method's main limitation.
W1 Fair criticism. We would merely argue that the low magnitude of the findings is an artifact of our study approach, since it favors generality (one single method for all matrix types) over establishing a single practically useful operating point, which we happily accept as a weakness. Since this is, to our knowledge, the first study searching for deeper matrix structure and locating it on a compression scale between random noise and planted structure, we also believe the admittedly large portion of negative results could be interesting to the community.
Q2 The implementation certifies the unsquared relative Frobenius error, so the term in Section 3.3 is wrong. Thank you for catching this!