acceptodds
Under review as a conference paper at ICLR 2027

RosKV: Robust Spectral Key-Value Cache Compression for Time-Series Foundation Models

Abstract

Autoregressive time-series foundation models (TSFMs) enable zero-shot long-horizon forecasting, but real-world deployment on resource-constrained hardware is bottlenecked by a growing key–value (KV) cache. While caching past states eliminates quadratic recomputation, persistent memory scales linearly with look-back length and horizon. Furthermore, TSFM prefill importance is delocalized in time yet localized in frequency: seasonal cycles span many look-back tokens, so token eviction and uniform time-domain quantization are ill-matched, whereas a small fraction of RFFT bins carries most K/V energy. We present , a memory-first, training-free spectral compressor for frozen TSFMs designed for memory-constrained deployments where persistent cache capacity is the primary barrier. RosKV (i) applies a fixed head-dimension rotation so NF4/NF2 block scales are not dominated by outliers on the few kept bins, (ii) transforms along the look-back with an RFFT, (iii) filters K and V separately by spectral energy, and (iv) stores adaptive mixed-precision coefficients, reconstructing an approximate dense prefill at every decode step before attention. Across nine datasets, four horizons, and four backbones (TimeMoE, Chronos, Moirai, TimesFM), RosKV consistently outperforms NLP token eviction (SnapKV, PyramidKV), coordinate quantization (KIVI), and frequency baselines (FreqKV), staying within of Full-Cache aggregate MSE (with matching rankings confirmed on GIFT-Eval). On dense-token architectures where persistent KV storage forms a critical inference bottleneck (TimeMoE), RosKV slashes persistent cache memory by on average ( MB vs.  MB) and up to at short horizons ( MB vs.  MB), while maintaining competitive decoding latency ( ms vs. No-Cache  ms).

Primary area:
learning on time series and dynamical systems
Keywords:
KV Cache, TSFM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xbC9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RosKV proposes a KV cache compression method to reduce GPU memory usage for time-series foundation models. The method first applies a fixed rotation to feature dimensions, followed by a real Fourier transform along the temporal dimension. It then selects high energy frequency bins and quantizes the coefficients using a mixture of 4-bit and 2-bit precision, assigning 4-bit precision to the higher-energy retained bins, while using 2 bits for remaining. The results on forecasting datasets show modest changes in accuracy with savings in memory.

Critical strengths

The compression strategy is motivated by properties of time-series representations. The paper empirically shows that KV energy is concentrated in a relatively small number of frequency bins, suggesting that spectral compression can preserve temporal patterns distributed across many historical positions. This provides a concrete rationale for the method, supported by ablations showing that removing the Fourier transform worsens forecasting accuracy, particularly on TimeMoE.

Critical weaknesses

The practical benefit of the memory–latency trade-off is insufficiently demonstrated. In the evaluated TimeMoE setting, RosKV saves around 30 MB of persistent cache memory per sequence, which is a small fraction compared to the model weight footprint of TimeMOE-200M. Achieving these savings require reconstructing the historical keys and values at every decoding step, including dequantization, inverse Fourier transforms, and inverse rotation. Average latency consequently increases from 240 ms with full caching to 383 ms, over a 50% increase. Together, this limits the evidence that the method provides a practically valuable improvement over full caching.

Questions for authors
  1. Figure 1 demonstrates concentration of KV energy in the frequency domain, but the claim that forecasting-relevant information is delocalized in time is less directly established. Beyond the forecasting comparisons and component ablations, what evidence supports this claim? In particular, how do you distinguish spectral energy concentration from the importance of those components to forecasting accuracy?

  2. On GIFT-Eval, RosKV improves substantially over Full-Cache on Moirai and TimesFM. The paper attributes these gains to spectral denoising, but the accuracy improvements alone do not establish this explanation. Can the authors provide evidence isolating this effect?

2 comments

(1/2) Thank you for your careful review, and for recognizing that the compression strategy is motivated by properties of time‑series representations, that KV energy is empirically concentrated in a small number of frequency bins, and that the ablations show the role of the Fourier transform, particularly on TimeMoE. Below we quote each of your comments and respond to it. The main text of the revised PDF is unchanged; all additions and corrections are collected in the new Appendix E and marked in blue.


Weakness 1.

The practical benefit of the memory–latency trade-off is insufficiently demonstrated. In the evaluated TimeMoE setting, RosKV saves around 30 MB of persistent cache memory per sequence, which is a small fraction compared to the model weight footprint of TimeMOE-200M. Achieving these savings require reconstructing the historical keys and values at every decoding step, including dequantization, inverse Fourier transforms, and inverse rotation. Average latency consequently increases from 240 ms with full caching to 383 ms, over a 50% increase. Together, this limits the evidence that the method provides a practically valuable improvement over full caching.

Response. Thank you for your comment. Your accounting is right for a single sequence: about 33 MB per sequence is small next to ≈0.8 GB of weights. The value lies in multi‑sequence serving, because TimeMoE decodes channels independently. One 862‑channel Traffic forecast at H=720H{=}720 holds ≈77.8 GB of Full‑Cache KV versus ≈49.4 GB for RosKV. We now measure this directly.

While doing so, we found that our efficiency harness had run the eager‑reconstruction variant of §3.5, which keeps a dense copy of the prefill for the whole decode, so the 240 vs. 383 ms figures in the submitted Table 2 are superseded by a controlled re‑measurement of all methods in one setting. The method as described, and as released, rebuilds one layer per step and frees it. Forecasts are bit‑identical, so accuracy is unaffected. All time and memory results are now re‑measured on the per‑step path in the new App. E.3, which supersedes the latency figures of Table 2 (TimeMoE‑200M, ECL):

Full‑CacheRosKV
Peak memory, batch 1 (H=96H{=}96 / 720720)1918 / 1946 MB1888 / 1922 MB
Peak memory, batch 128 (H=96H{=}96 / 720720)10.7 / 14.4 GB6.6 / 11.0 GB
Max. sequences under 16 GiB (H=96H{=}96 / 720720)216 / 152368 / 200

The trade‑off is compute:

  • Batch 1 (idle A100, all methods interleaved): RosKV takes 2.2× (H=96H{=}96) to 1.8× (H=720H{=}720) the decode time of Full‑Cache.
  • Batch 128: RosKV takes 1.37–1.46× the time; No‑Cache takes 1.8–8.0×.
  • Each method at its largest batch under 16 GiB: RosKV sustains 0.84× (H=96H{=}96) and 0.70× (H=720H{=}720) of Full‑Cache throughput (App. Table 15).

RosKV therefore trades compute for memory. Its practical value lies in memory‑bound serving: when memory, not compute, caps how many sequences can be resident, it fits 1.3–1.7× more of them, with the best accuracy among compressors on the dense‑token TimeMoE (+3.6% vs. +24–43% for the others, Table 1). App. E.1 updates the latency statements of the abstract and §4.3 and states this scope.

(2/2)

Question 1.

Figure 1 demonstrates concentration of KV energy in the frequency domain, but the claim that forecasting-relevant information is delocalized in time is less directly established. Beyond the forecasting comparisons and component ablations, what evidence supports this claim? In particular, how do you distinguish spectral energy concentration from the importance of those components to forecasting accuracy?

Response. Thank you for your comment. We agree that Figure 1 shows energy concentration only, and we ran a direct test (new App. E.5, Table 18). On TimeMoE we keep the RosKV pipeline and change only which kk bins are kept, at equal kk (average MSE over ETTh1, ETTm1, ECL and Weather):

Kept binsFP32 coefficientsNF4/NF2 codec
Energy top‑kk (RosKV)0.4110.401
Low‑pass0.3930.427
Lowest energy0.801–
Random0.852–
  • Energy carries importance information. The highest‑energy bins halve the error of the lowest‑energy or random bins.
  • Energy ranking is more robust. Since TimeMoE's energy sits at low frequencies, a plain low‑pass support performs similarly without quantization, but with the codec it degrades on Weather (0.855 vs. 0.668) and is less accurate on average.
  • Axis at matched storage. Selecting time positions instead of RFFT bins raises TimeMoE MSE from 0.760 to 0.965 (Table 3).

Together these support the operational claim we now state in App. E.1: an energy‑ranked frequency support preserves forecasts better than the time‑position supports we tested. We state the claim in this comparative form in App. E.1, rather than as forecast‑optimality of the energy‑ranked bins.

Question 2.

On GIFT-Eval, RosKV improves substantially over Full-Cache on Moirai and TimesFM. The paper attributes these gains to spectral denoising, but the accuracy improvements alone do not establish this explanation. Can the authors provide evidence isolating this effect?

Response. Thank you for catching this inconsistency in Table 4. The gap reflects the evaluation protocol rather than the cache. For Moirai and TimesFM, the Full‑Cache, SnapKV, PyramidKV and FreqKV entries used 20 channels per series, while the RosKV and KIVI entries came from 5‑channel runs, with RosKV also at r=0.95r{=}0.95. TimesFM's first 96 predictions precede any compressed‑cache read, so at H=96H{=}96 RosKV must equal Full‑Cache; on identical inputs it does.

We therefore no longer attribute it to denoising, and we reran every method on every backbone under one protocol (20 windows × 20 channels per series, r=0.5r{=}0.5). New App. Tables 19 and 20 supersede Table 4 and App. Table 13. Average MSE:

Full‑CacheSnapKVKIVIPyramidKVFreqKVRosKV
TimeMoE0.4580.5650.6970.5390.5060.513
Chronos0.8010.9260.8281.2220.9740.973
Moirai0.6570.7920.7170.8160.9950.735
TimesFM0.6740.7000.6960.6990.7610.713
Total0.6470.7460.7350.8190.8090.733

As expected, all compressors remain above Full‑Cache. RosKV has the lowest total among compressors and is within 0.02 of the best compressor on TimeMoE, Moirai and TimesFM, while KIVI is strongest on Chronos. App. E.6 now describes RosKV's out‑of‑distribution accuracy as on par with the strongest baselines. We thank you again for catching this.


Thank you again for your review. We hope the corrected GIFT‑Eval results, the direct bin‑selection test and the re‑measured efficiency results address your concerns, and we would be glad to answer any further questions.

Reviewer_TuzxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

RosKV proposes training-free KV-cache compression for frozen TSFMs using rotation, spectral filtering, and quantization. It reports 55% average persistent-cache savings on TimeMoE.

Critical strengths
  • The idea to do spectral compression for time-series KV cache is interesting.
  • The method requires no retraining and the evaluations are comprehensive.
Critical weaknesses
  • Whether KV-cache is a real bottleneck needs some justification as there are several non-transformer based backbones fort TSFMs too like TTM, TiRex-2. So this is a narrow problem. Authors may consider establishing the practical importance of the problem.

  • All compression components like Quantization and outlier-reducing rotations are established approaches (KIVI, Zirui et.al; QuaRos, Ashboos et. al). Novelty seems thin. Please clearly justify methodological novelty.

  • Long horizon evals where is could be a real problem is limited. It would be useful to characterize the impact of fixed prefil and growing decode.

Questions for authors

(Please see weakness)

  • Line 262 has a broken referen, "QLora (?)"
3 comments

(1/2) Thank you for your careful review, and for recognizing that spectral compression of the time‑series KV cache is an interesting idea, that the method requires no retraining, and that the evaluations are comprehensive. Below we quote each of your comments and respond to it. The main text of the revised PDF is unchanged; all additions and corrections are collected in the new Appendix E and marked in blue.


Weakness 1.

Whether KV-cache is a real bottleneck needs some justification as there are several non-transformer based backbones fort TSFMs too like TTM, TiRex-2. So this is a narrow problem. Authors may consider establishing the practical importance of the problem.

Response. Thank you for your comment. We agree that the scope is narrower than all TSFMs, and the new App. E.1 states it. TSFMs with a fixed‑size state, such as TTM (MLP‑Mixer) and TiRex (xLSTM), accumulate no KV cache and are outside our scope; both are now cited. Among attention‑based TSFMs, the prefill cache is large mainly for dense‑token designs: at L=512L{=}512, TimeMoE exposes 257 RFFT bins along the look‑back, versus 9–20 for the patch‑based Chronos, Moirai and TimesFM (§2.3).

For these models the cost multiplies with the number of sequences, because TimeMoE forecasts channels independently.

  • Scale. TimeMoE‑200M persists 90.2 MB per sequence at H=720H{=}720, so one 862‑channel Traffic forecast holds ≈77.8 GB of Full‑Cache KV, versus ≈49.4 GB under RosKV.
  • Measured capacity. Under a 16 GiB memory cap (ECL), Full‑Cache fits 216 concurrent sequences at H=96H{=}96 and 152 at H=720H{=}720; RosKV fits 368 and 200 (1.70× and 1.32×; new App. Table 15).
  • Trade‑off. At batch 128, RosKV takes 1.37–1.46× the decode time of Full‑Cache, so it is best suited to deployments where memory, not compute, is the limit. For a single sequence the ≈0.8 GB of weights dominate, and we make no single‑stream claim.

This is also the regime where existing compressors struggle: on TimeMoE, SnapKV, PyramidKV, KIVI and FreqKV raise MSE by 24–43%, versus 3.6% for RosKV (Table 1), and RosKV keeps the lowest average MSE in each of three held‑out window blocks (App. Table 10).

Weakness 2.

All compression components like Quantization and outlier-reducing rotations are established approaches (KIVI, Zirui et.al; QuaRos, Ashboos et. al). Novelty seems thin. Please clearly justify methodological novelty.

Response. Thank you for your comment. Rotation and low‑bit coding are established, and QuaRot is now cited and discussed in App. E.1. The difference is the axis. KIVI and QuaRot quantize every time‑domain KV entry, whereas RosKV changes which coefficients are stored at all: a sparse RFFT support along the look‑back, with low bits spent only there. Rotation matters because after top‑kk only a few dd‑dimensional vectors carry the signal, so a single outlier coordinate inflates a whole NF block scale. Evidence from one pipeline:

  • Axis. Selecting time positions instead of RFFT bins, at matched storage and with the same rotation, energy rule and codec, raises Total MSE from 0.744 to 0.815 (TimeMoE 0.760 → 0.965; Table 3).
  • Rotation. Removing only the rotation raises Total MSE to 0.777 (Table 3).
  • Time‑domain quantization. KIVI gives +33.4% on TimeMoE, versus +3.6% for RosKV (Table 1).

The contribution is therefore a TSFM‑specific design in which the compressed axis and the order of operations matter, not a new quantization primitive.

(2/2)

Weakness 3.

Long horizon evals where is could be a real problem is limited. It would be useful to characterize the impact of fixed prefil and growing decode.

Response. Thank you for your comment. RosKV compresses only the prefill; decoded tokens stay in an exact buffer. The absolute saving is therefore constant at about 33–36 MB per TimeMoE sequence, while the relative saving shrinks as HH grows:

HH961923367201440
Full‑Cache (MB)42.547.261.990.2141.6
RosKV (MB)9.514.229.057.3105.6
Saving−78%−70%−53%−36%−25%
  • New long‑horizon run. We added H=1440H{=}1440, where decoding is 2.8× longer than the look‑back (new App. Table 16). Accuracy holds: MSE is 0.745 for Full‑Cache and 0.759 for RosKV (+1.9%), ahead of PyramidKV (0.781), SnapKV (0.787) and FreqKV (0.792). Decode time there is 1.76× that of Full‑Cache for RosKV and 2.02× for No‑Cache.
  • Configuration. Rows for H≤720H\le720 use the Table 2 codec (NF4‑only at r=0.9r{=}0.9), now stated in App. E.2; H=1440H{=}1440 uses NF4‑only at r=0.5r{=}0.5.

RosKV therefore fits best when the look‑back is long relative to the horizon. When H≫LH\gg L the exact decode buffer dominates; compressing it as well is left to future work, and App. E.3 states this limitation.


Question.

Line 262 has a broken referen, "QLora (?)"

Response. Thank you for catching this. The bibliography entry was missing from the submitted source; the citation is fixed in the revised PDF.


Thank you again for your review. We hope the clarified scope, the capacity measurements and the long‑horizon analysis address your concerns, and we would be glad to answer any further questions.

My concern about the narrow scope of KV compression remains. I am not convinced that this is a real bottleneck in TSFMs and feel like the current approach is a design optimization of autoregressive transformer based forecasting models (which only constitute of one type of TSFMs). Hence I retain my score.

Reviewer_nBuSOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RosKV, a training-free KV cache compression method for time-series foundation models. The main idea is to exploit the concentration of KV cache energy in the frequency domain rather than directly removing historical tokens. RosKV combines orthogonal rotation, adaptive spectral filtering, and mixed-precision quantization to reduce memory usage while preserving forecasting accuracy. The authors evaluate the method on nine datasets and four foundation models, showing competitive results against several KV cache compression baselines. They also report substantial savings in persistent cache memory, along with ablation studies and additional evaluations on GIFT-Eval.

Critical strengths
  1. The paper addresses an important practical issue of reducing KV cache memory usage in time series foundation models. The idea of exploiting spectral energy concentration for cache compression is reasonable, and the method can be applied to pretrained models without additional training.
  2. The authors evaluate RosKV on nine datasets and four time series foundation models, showing competitive forecasting performance against several compression baselines. They also provide ablation studies and additional experiments on GIFT-Eval to further examine the effectiveness and generalization of the proposed method.
Critical weaknesses
  1. RosKV mainly combines existing techniques, including orthogonal rotation, spectral filtering, and low-bit quantization, so the overall novelty seems somewhat limited. While the authors introduce adaptive K/V frequency allocation and mixed-precision quantization, they do not compare these designs with simpler fixed strategies, making it difficult to tell how much the adaptive mechanisms actually contribute.
  2. Although RosKV significantly reduces persistent KV cache memory, it reconstructs the dense K/V states at every decoding step, which adds extra computation and temporary memory overhead. The paper does not provide a clear evaluation of peak GPU memory usage, and the reported decoding latency is considerably higher than Full-Cache. This makes it hard to judge how much benefit RosKV would offer in real-world deployments.
Questions for authors
  1. In Eq. (7), the adaptive allocation seems to run into an issue when the number of frequency bins is small. For example, with F=9F=9, r=0.5r=0.5, and δ=0.02\delta=0.02, the lower bound becomes larger than the upper bound, making the constraint infeasible. How do the authors handle this case in practice, especially for TimesFM?
  2. RosKV reconstructs the dense K/V cache at every decoding step, which introduces additional computation and temporary memory usage. While the paper reports substantial savings in persistent cache memory, could the authors also provide the actual peak GPU memory usage, including the temporary reconstruction buffers?
  3. The proposed method uses adaptive frequency allocation between K and V, as well as adaptive NF4/NF2 precision assignment. However, Table 3 only evaluates removing the main components rather than testing these adaptive designs separately. Could the authors compare them with simpler alternatives, such as a fixed K/V allocation and a fixed NF4/NF2 ratio, to show whether the added complexity is actually necessary?
2 comments

(1/2) Thank you for your careful review, and for recognizing the practical importance of reducing KV‑cache memory in time‑series foundation models, the training‑free design that builds on spectral energy concentration, and the breadth of the evaluation across nine datasets and four foundation models, with ablations and GIFT‑Eval. Below we quote each of your comments and respond to it. The main text of the revised PDF is unchanged; all additions and corrections are collected in the new Appendix E and marked in blue.


Weakness 1.

RosKV mainly combines existing techniques, including orthogonal rotation, spectral filtering, and low-bit quantization, so the overall novelty seems somewhat limited. While the authors introduce adaptive K/V frequency allocation and mixed-precision quantization, they do not compare these designs with simpler fixed strategies, making it difficult to tell how much the adaptive mechanisms actually contribute.

Response. Thank you for your comment. Each primitive is indeed known: Hadamard‑type rotation (QuaRot, now cited and discussed in App. E.1), NF codebooks (QLoRA) and frequency‑domain KV (FreqKV). Our claim is that for TSFM prefills, the axis that is compressed and the order of composition decide whether low‑bit storage works. Both are tested inside one pipeline (Table 3):

  • Axis. With the same rotation, energy rule and codec, but selecting time positions instead of RFFT bins at matched storage (iso‑memory within 1%, App. C.4), Total MSE rises from 0.744 to 0.815, and TimeMoE MSE from 0.760 to 0.965.
  • Rotation. Removing only the rotation raises Total MSE to 0.777.
  • Closest prior method. FreqKV transferred to TSFMs gives +43.1% on TimeMoE, versus +3.6% for RosKV (Table 1).

On the adaptive mechanisms, we ran the comparison you asked for (Question 3 below). Over three held‑out window blocks, fixed rules perform on par with the adaptive ones, so the method can use the simpler rules without loss of accuracy (App. E.4).

Weakness 2.

Although RosKV significantly reduces persistent KV cache memory, it reconstructs the dense K/V states at every decoding step, which adds extra computation and temporary memory overhead. The paper does not provide a clear evaluation of peak GPU memory usage, and the reported decoding latency is considerably higher than Full-Cache. This makes it hard to judge how much benefit RosKV would offer in real-world deployments.

Response. Thank you for your comment. It led us to re‑examine our efficiency measurements, and we found that the efficiency harness had run the eager‑reconstruction variant of §3.5, which rebuilds the prefill once and keeps a dense copy for the whole decode. The method as described, and as released, rebuilds one layer's prefill per step and frees it after attention. Forecasts of the two paths are bit‑identical, so no accuracy number changes. All time and memory results are now re‑measured on the per‑step path in the new App. E.3, which supersedes the latency figures of Table 2 (TimeMoE‑200M, default codec):

  • Batch 1 (idle A100, all methods interleaved): RosKV takes 2.2× (H=96H{=}96) to 1.8× (H=720H{=}720) the decode time of Full‑Cache, versus up to 1.2× for the other compressors.
  • Batch 128 (ECL): RosKV takes 1.37× (H=96H{=}96) and 1.46× (H=720H{=}720) the time, with 38% and 23% less peak memory. No‑Cache takes 1.8× and 8.0×.
  • Each method at its largest batch under 16 GiB: RosKV sustains 0.84× (H=96H{=}96) and 0.70× (H=720H{=}720) of Full‑Cache throughput (App. Table 15).

RosKV therefore trades compute for memory: its benefit is the number of sequences that fit in a fixed memory budget (Question 2), and App. E.1 updates the latency statements of the abstract and §4.3 accordingly.

(2/2)

Question 1.

In Eq. (7), the adaptive allocation seems to run into an issue when the number of frequency bins is small. For example, with [F=9F=9], [r=0.5r=0.5], and [δ=0.02\delta=0.02], the lower bound becomes larger than the upper bound, making the constraint infeasible. How do the authors handle this case in practice, especially for TimesFM?

Response. Thank you for your comment. You are right that the interval is empty in this case. It is [⌈4.32⌉,⌊4.68⌋]=[5,4][\lceil 4.32\rceil,\lfloor 4.68\rfloor]=[5,4], and the experiments then use the equal split kK=kV=⌈Fr⌉=5k_K{=}k_V{=}\lceil Fr\rceil{=}5. On Chronos and Moirai (F=20F{=}20) the interval is [10,10][10,10], which forces the same split. Adaptive K/V allocation is therefore active only on TimeMoE (F=257F{=}257, k∈[124,133]k\in[124,133]), and the reported numbers for the other three backbones already are the fixed‑split variant. App. E.2 now states the fallback and the per‑backbone regime, and the released codec.py uses the same fallback.

Question 2.

RosKV reconstructs the dense K/V cache at every decoding step, which introduces additional computation and temporary memory usage. While the paper reports substantial savings in persistent cache memory, could the authors also provide the actual peak GPU memory usage, including the temporary reconstruction buffers?

Response. Thank you for your comment. Peak allocated memory, including weights, activations and the temporary reconstructions, is now reported in App. E.3, Tables 14–15 (TimeMoE‑200M, ECL):

Full‑CacheRosKV
Peak, batch 1 (H=96H{=}96 / 720720)1918 / 1946 MB1888 / 1922 MB
Peak, batch 128 (H=96H{=}96 / 720720)10.7 / 14.4 GB6.6 / 11.0 GB
Max. sequences under 16 GiB (H=96H{=}96 / 720720)216 / 152368 / 200

The temporary reconstruction covers one layer at a time, so it is far smaller than the persistent cache it replaces. Under the same cap, SnapKV fits 272 / 168 and No‑Cache 352 / 88 sequences. (Our KIVI baseline stores dequantized tensors, so its capacity, 216 / 136, understates a packed kernel.) TimeMoE decodes channels independently, so one 321‑channel ECL forecast is already 321 resident sequences.

Question 3.

The proposed method uses adaptive frequency allocation between K and V, as well as adaptive NF4/NF2 precision assignment. However, Table 3 only evaluates removing the main components rather than testing these adaptive designs separately. Could the authors compare them with simpler alternatives, such as a fixed K/V allocation and a fixed NF4/NF2 ratio, to show whether the added complexity is actually necessary?

Response. Thank you for your comment. We ran this ablation on TimeMoE, the one backbone where the rules are active (new App. E.4, Table 17). Gap to Full‑Cache MSE:

adaptive (paper)tied K/V splitfixed 20:80tied + fixed 20:80NF4‑onlyNF2‑only
1 block, 9 datasets+3.4%+6.2%+4.3%+5.6%+6.1%+27.4%
3 blocks, 8 datasets+3.5 ± 2.3%+3.5 ± 1.5%+3.6 ± 1.4%+4.0 ± 1.5%

On one window block the adaptive rules score best, and over three held‑out blocks all four allocation variants are within one standard deviation of each other. The tied split with a fixed 20:80 share therefore performs on par with the adaptive rules and is a simpler choice; App. E.4 now says this. What is necessary is the NF4 tier itself: NF2‑only raises the gap to +27%. We keep the adaptive rules as the default so that every reported number uses one configuration.


Thank you again for your review. We hope the re‑measured efficiency results and the new ablation address your concerns, and we would be glad to answer any further questions.

Comments on the paper

1 comment

Dear Reviewers, thank you once again for your constructive feedback. With the discussion period nearing its end, we would greatly appreciate knowing whether our rebuttal and new experimental results have resolved your concerns. We remain available to clarify any remaining questions before the deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.