- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RosKV proposes a KV cache compression method to reduce GPU memory usage for time-series foundation models. The method first applies a fixed rotation to feature dimensions, followed by a real Fourier transform along the temporal dimension. It then selects high energy frequency bins and quantizes the coefficients using a mixture of 4-bit and 2-bit precision, assigning 4-bit precision to the higher-energy retained bins, while using 2 bits for remaining. The results on forecasting datasets show modest changes in accuracy with savings in memory.
Critical strengths
The compression strategy is motivated by properties of time-series representations. The paper empirically shows that KV energy is concentrated in a relatively small number of frequency bins, suggesting that spectral compression can preserve temporal patterns distributed across many historical positions. This provides a concrete rationale for the method, supported by ablations showing that removing the Fourier transform worsens forecasting accuracy, particularly on TimeMoE.
Critical weaknesses
The practical benefit of the memory–latency trade-off is insufficiently demonstrated. In the evaluated TimeMoE setting, RosKV saves around 30 MB of persistent cache memory per sequence, which is a small fraction compared to the model weight footprint of TimeMOE-200M. Achieving these savings require reconstructing the historical keys and values at every decoding step, including dequantization, inverse Fourier transforms, and inverse rotation. Average latency consequently increases from 240 ms with full caching to 383 ms, over a 50% increase. Together, this limits the evidence that the method provides a practically valuable improvement over full caching.
Questions for authors
-
Figure 1 demonstrates concentration of KV energy in the frequency domain, but the claim that forecasting-relevant information is delocalized in time is less directly established. Beyond the forecasting comparisons and component ablations, what evidence supports this claim? In particular, how do you distinguish spectral energy concentration from the importance of those components to forecasting accuracy?
-
On GIFT-Eval, RosKV improves substantially over Full-Cache on Moirai and TimesFM. The paper attributes these gains to spectral denoising, but the accuracy improvements alone do not establish this explanation. Can the authors provide evidence isolating this effect?
2 comments
(2/2)
Question 1.
Figure 1 demonstrates concentration of KV energy in the frequency domain, but the claim that forecasting-relevant information is delocalized in time is less directly established. Beyond the forecasting comparisons and component ablations, what evidence supports this claim? In particular, how do you distinguish spectral energy concentration from the importance of those components to forecasting accuracy?
Response. Thank you for your comment. We agree that Figure 1 shows energy concentration only, and we ran a direct test (new App. E.5, Table 18). On TimeMoE we keep the RosKV pipeline and change only which bins are kept, at equal (average MSE over ETTh1, ETTm1, ECL and Weather):
| Kept bins | FP32 coefficients | NF4/NF2 codec |
|---|---|---|
| Energy top‑ (RosKV) | 0.411 | 0.401 |
| Low‑pass | 0.393 | 0.427 |
| Lowest energy | 0.801 | – |
| Random | 0.852 | – |
- Energy carries importance information. The highest‑energy bins halve the error of the lowest‑energy or random bins.
- Energy ranking is more robust. Since TimeMoE's energy sits at low frequencies, a plain low‑pass support performs similarly without quantization, but with the codec it degrades on Weather (0.855 vs. 0.668) and is less accurate on average.
- Axis at matched storage. Selecting time positions instead of RFFT bins raises TimeMoE MSE from 0.760 to 0.965 (Table 3).
Together these support the operational claim we now state in App. E.1: an energy‑ranked frequency support preserves forecasts better than the time‑position supports we tested. We state the claim in this comparative form in App. E.1, rather than as forecast‑optimality of the energy‑ranked bins.
Question 2.
On GIFT-Eval, RosKV improves substantially over Full-Cache on Moirai and TimesFM. The paper attributes these gains to spectral denoising, but the accuracy improvements alone do not establish this explanation. Can the authors provide evidence isolating this effect?
Response. Thank you for catching this inconsistency in Table 4. The gap reflects the evaluation protocol rather than the cache. For Moirai and TimesFM, the Full‑Cache, SnapKV, PyramidKV and FreqKV entries used 20 channels per series, while the RosKV and KIVI entries came from 5‑channel runs, with RosKV also at . TimesFM's first 96 predictions precede any compressed‑cache read, so at RosKV must equal Full‑Cache; on identical inputs it does.
We therefore no longer attribute it to denoising, and we reran every method on every backbone under one protocol (20 windows × 20 channels per series, ). New App. Tables 19 and 20 supersede Table 4 and App. Table 13. Average MSE:
| Full‑Cache | SnapKV | KIVI | PyramidKV | FreqKV | RosKV | |
|---|---|---|---|---|---|---|
| TimeMoE | 0.458 | 0.565 | 0.697 | 0.539 | 0.506 | 0.513 |
| Chronos | 0.801 | 0.926 | 0.828 | 1.222 | 0.974 | 0.973 |
| Moirai | 0.657 | 0.792 | 0.717 | 0.816 | 0.995 | 0.735 |
| TimesFM | 0.674 | 0.700 | 0.696 | 0.699 | 0.761 | 0.713 |
| Total | 0.647 | 0.746 | 0.735 | 0.819 | 0.809 | 0.733 |
As expected, all compressors remain above Full‑Cache. RosKV has the lowest total among compressors and is within 0.02 of the best compressor on TimeMoE, Moirai and TimesFM, while KIVI is strongest on Chronos. App. E.6 now describes RosKV's out‑of‑distribution accuracy as on par with the strongest baselines. We thank you again for catching this.
Thank you again for your review. We hope the corrected GIFT‑Eval results, the direct bin‑selection test and the re‑measured efficiency results address your concerns, and we would be glad to answer any further questions.
(1/2) Thank you for your careful review, and for recognizing that the compression strategy is motivated by properties of time‑series representations, that KV energy is empirically concentrated in a small number of frequency bins, and that the ablations show the role of the Fourier transform, particularly on TimeMoE. Below we quote each of your comments and respond to it. The main text of the revised PDF is unchanged; all additions and corrections are collected in the new Appendix E and marked in blue.
Weakness 1.
Response. Thank you for your comment. Your accounting is right for a single sequence: about 33 MB per sequence is small next to ≈0.8 GB of weights. The value lies in multi‑sequence serving, because TimeMoE decodes channels independently. One 862‑channel Traffic forecast at H=720 holds ≈77.8 GB of Full‑Cache KV versus ≈49.4 GB for RosKV. We now measure this directly.
While doing so, we found that our efficiency harness had run the eager‑reconstruction variant of §3.5, which keeps a dense copy of the prefill for the whole decode, so the 240 vs. 383 ms figures in the submitted Table 2 are superseded by a controlled re‑measurement of all methods in one setting. The method as described, and as released, rebuilds one layer per step and frees it. Forecasts are bit‑identical, so accuracy is unaffected. All time and memory results are now re‑measured on the per‑step path in the new App. E.3, which supersedes the latency figures of Table 2 (TimeMoE‑200M, ECL):
The trade‑off is compute:
RosKV therefore trades compute for memory. Its practical value lies in memory‑bound serving: when memory, not compute, caps how many sequences can be resident, it fits 1.3–1.7× more of them, with the best accuracy among compressors on the dense‑token TimeMoE (+3.6% vs. +24–43% for the others, Table 1). App. E.1 updates the latency statements of the abstract and §4.3 and states this scope.