- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
HorizonKV assigns historical layer/head/scale KV groups to native precision, INT8, or eviction. Offline calibration estimates future attention demand and state-dependent distortion, and a Lagrangian allocation schedules irreversible transitions under a persistent historical-cache byte budget. Experiments on Infinity-2B/8B report improved consistency with full-cache outputs compared with pruning baselines and largely preserved GenEval/DPG scores at budgets down to 1%. Component ablations, calibration-size sensitivity, and peak-memory/latency measurements provide additional validation.
Critical strengths
-
Calibrating future demand separately from quantization sensitivity gives the allocation more information than an attention-importance score alone. The component ablation shows complementary gains from horizon awareness and the three-state representation, supporting the value of the proposed combination.
-
Physical budgets include quantization metadata, and matched prompts and seeds make the output-consistency comparison meaningful. On Infinity-8B, HorizonKV at 1% and HeatKV at 4% both obtain consistency FID 2.71. Preserved GenEval/DPG scores and measured persistent-memory savings substantiate a useful memory–quality improvement within this model family.
Critical weaknesses
-
The main experiments compare against pruning policies, whereas the proposed method jointly uses quantization and eviction. MiKV and DiffKV are discussed but not experimentally adapted, and a tuned pruning-plus-INT8 control is also needed. The Native/Drop ablation establishes that INT8 helps this framework; it does not establish an advantage over competitive alternative combined policies. This leaves the incremental value of the specific scheduler insufficiently isolated.
-
Equation (4) aggregates normalized single-group perturbations measured while every other group remains native. Simultaneously compressing many groups changes attention normalization and downstream representations, so their joint effect need not follow that additive ranking. This is particularly consequential at 1% storage. The observed quality gains are encouraging, but neither direct validation of surrogate rankings under joint compression nor evidence beyond the Infinity family establishes the broader scheduling principle.
Questions for authors
-
Does the gain persist over a tuned pruning-plus-INT8 policy or a compatible mixed-precision/retention baseline at equal physical bytes, and what latency trade-off accompanies that comparison?
-
How well does the aggregate cost rank actual joint-compression distortion? A measured correlation or ranking comparison at the 1% budget would directly address the surrogate's reliability.
6 comments
Q2. How well does the aggregate cost rank actual joint-compression distortion? A measured correlation or ranking comparison at the 1% budget would directly address the surrogate's reliability.
We directly test how reliably the cost defined in Eq. (4) orders the real distortion caused by different joint-compression schedules, using Infinity-8B with a 1% KV budget.
(1) Candidate schedule construction.
Starting from HorizonKV's original schedule, we generate 99 perturbed schedules, yielding 100 candidates in total. Each perturbation swaps complete compression-state trajectories of compatible Layer–Head–Scale KV Groups, while keeping the same physical byte budget and respecting the irreversible transition constraint Native → K8V8 → Drop.
(2) Predicted cost and measured distortion.
For each candidate schedule, we compute:
-
Predicted cost: the aggregate cost in Eq. (4), using the offline calibrated future attention demand A and compression sensitivity D.
-
Measured distortion: mean LPIPS against the corresponding Full Cache outputs over the same 50 prompt/seed pairs for every schedule.
Each candidate provides one predicted cost and one measured mean LPIPS. The evaluation comprises 100 × 50 = 5,000 images, yielding 100 paired cost–distortion observations for rank-correlation analysis.
(3) Rank-correlation results.
We compute Spearman and Kendall correlations across the 100 cost/distortion pairs.
Table W3. Correlation between predicted cost and measured LPIPS
| Rank metric | Coefficient | 95% CI |
|---|---|---|
| Spearman ρ | 0.8462 | [0.8240, 0.8672] |
| Kendall τ-b | 0.6586 | [0.6295, 0.6792] |
The 95% confidence intervals are obtained by 20,000 paired bootstrap resamples: we keep the 100 schedules fixed, jointly resample the common 50 prompt/seed cases, recompute the mean LPIPS per schedule, and recalculate the correlations.
To check whether rank agreement is driven solely by the perturbation magnitude, we group schedules by the percentage of modified KV Groups and compute Spearman correlation within similar perturbation ranges.
Table W4. Within-range Spearman correlation by perturbation magnitude
| Modified KV groups | Spearman ρ |
|---|---|
| 5%–10% | 0.599 |
| 10%–15% | 0.548 |
| 15%–20% | 0.606 |
The within-range correlations remain positive (0.548–0.606), although they are lower than the overall Spearman coefficient of 0.8462. Thus, the association between predicted cost and measured LPIPS is not solely explained by differences in perturbation magnitude. The correlation results are in Section 4.3 (Surrogate Cost Reliability; Fig. 6(a)); candidate construction and bootstrap procedures are given in Appendix D.6 (Surrogate Cost Reliability).
(4) Additional cross-model evaluation.
We further evaluate applicability beyond Infinity, including the original VAR [2], Switti-AR [3], and FlexVAR [4], against HeatKV [1] at matched KV-budget caps.
Table W5. Fidelity across visual autoregressive models
| Model | KV budget | Method | PSNR vs Full ↑ | LPIPS vs Full ↓ | FID vs Full ↓ |
|---|---|---|---|---|---|
| VAR-d36 | 10% | HeatKV | 18.337 | 0.25248 | 6.1575 |
| HorizonKV | 20.994 | 0.16641 | 4.1546 | ||
| VAR-d36 | 4% | HeatKV | 16.620 | 0.32767 | 8.5699 |
| HorizonKV | 17.664 | 0.27794 | 6.7094 | ||
| Switti-AR | 50% | HeatKV | 22.121 | 0.16657 | 3.7791 |
| HorizonKV | 25.209 | 0.12099 | 2.6917 | ||
| Switti-AR | 30% | HeatKV | 19.577 | 0.21664 | 5.2048 |
| HorizonKV | 21.583 | 0.15045 | 3.6674 | ||
| FlexVAR-d20 | 50% | HeatKV | 18.414 | 0.22232 | 6.2599 |
| HorizonKV | 19.938 | 0.18082 | 5.2329 | ||
| FlexVAR-d20 | 30% | HeatKV | 16.238 | 0.32508 | 8.1090 |
| HorizonKV | 18.622 | 0.21480 | 5.8901 | ||
| FlexVAR-d24 | 50% | HeatKV | 17.710 | 0.24859 | 6.7657 |
| HorizonKV | 19.885 | 0.18153 | 5.3508 | ||
| FlexVAR-d24 | 30% | HeatKV | 16.137 | 0.32331 | 7.977 |
| HorizonKV | 18.577 | 0.21498 | 6.080 |
Note: FID is computed against the Full Cache image set.
For VAR-d36, we calibrate on 10 ImageNet classes and evaluate 5,000 images from the remaining 990 classes; Switti-AR uses paired COCO-5000 text-conditioned evaluation.
Across the evaluated architectures and KV budgets, HorizonKV has higher PSNR and lower LPIPS and FID than HeatKV. The LPIPS reductions include 34.1% for VAR-d36@10%, 30.6% for Switti-AR@30%, and 33.9% for FlexVAR-d20@30%. The comparison is reported in Section 4.2 (Main Results) and Appendix D.4 (Cross-Architecture Generalization).
We thank the reviewer for the suggestions concerning a matched joint-compression baseline, online latency, and the ranking reliability of the surrogate cost.
Q1. Does the gain persist over a tuned pruning-plus-INT8 policy or a compatible mixed-precision/retention baseline at equal physical bytes, and what latency trade-off accompanies that comparison?
(1) Relation to existing joint-compression methods.
MiKV [5] uses importance-aware mixed precision to store tokens at different fidelities. DiffKV [6] jointly considers Key/Value sensitivity and attention-head characteristics when compressing and managing the cache. Both target long-context LLM inference; their original designs do not directly address historical KV management in cross-scale visual autoregressive generation.
Existing VAR-oriented KV methods, including HeatKV [1], have primarily focused on pruning or selective retention. To the best of our knowledge, HorizonKV is the first framework to jointly schedule KV quantization and pruning for cross-scale visual autoregressive generation under a unified physical-byte budget. It assigns historical KV to
Native,K8V8, orDropstates, jointly controlling retention and precision.(2) Tuned pruning-plus-INT8 baseline.
We construct HeatKV+K8V8, extending HeatKV's official S-CAS importance ranking from
Native/DroptoNative/K8V8/Drop. It uses exactly the same 10 calibration prompt/seed pairs, K8V8 implementation, three-state allocation solver, and physical KV-byte budgets as HorizonKV, while protecting the first three historical scales at Native precision. S-CAS is recomputed using the official procedure on the shared calibration set.The two policies differ in scheduling costs: HeatKV+K8V8 uses static costs derived from S-CAS importance ranks, whereas HorizonKV uses future attention demand A and compression sensitivity D. This allows a comparison under matched quantization, pruning options, and byte constraints.
(3) Generation fidelity at matched physical budgets.
We compare all methods on Infinity-2B/8B at 1% and 4% KV budgets, using 5,000 paired images per configuration, with PSNR, LPIPS, and FID measured relative to Full Cache.
Table W1. Generation fidelity at matched physical KV budgets (PSNR ↑, LPIPS/FID ↓)
HeatKV+K8V8 improves on original HeatKV in PSNR, LPIPS, and FID at all four settings. HorizonKV obtains the best values of all three metrics in each setting.
For Infinity-8B at 1%, relative to HeatKV+K8V8, HorizonKV improves PSNR by 2.42 dB, reduces LPIPS by 40.8%, and lowers FID from 4.50 to 2.71. At 4%, its PSNR increases by 1.70 dB, LPIPS decreases from 0.111 to 0.075, and FID drops from 2.61 to 1.74.
The improvement over HeatKV+K8V8 remains after matching the quantization implementation, compression states, and physical byte budgets. The two policies differ in the calibrated costs used for allocation.
(4) Inference-latency trade-off.
We also benchmark online inference latency for the three methods at Infinity-8B with a 4% KV budget.
Table W2. Online latency on Infinity-8B at 4% (s/image, mean ± std.)
Relative to HeatKV+K8V8, HorizonKV incurs +3.1%, -0.9%, and +4.5% per-image latency at B=1, 4, and 16, respectively.
In this comparison, the observed per-image latency difference from HeatKV+K8V8 ranges from -0.9% to +4.5%; the corresponding quality measurements are reported in Table W1. The joint-compression quality comparison is included in Section 4.3 (Scheduling Strategy Analysis); the three-method latency measurements are in Appendix D.5 (Scheduling Strategy Analysis). Additional inference-efficiency results appear in Section 4.4 (Practical Inference Efficiency).