acceptodds
Under review as a conference paper at ICLR 2027

HorizonKV: Three-State KV Cache Compression under a Shrinking Generation Horizon for Visual Autoregressive Models

38 views

Abstract

Visual autoregressive (VAR) KV caches consume massive GPU memory—over 80 GB for Infinity-8B at 1024×1024 resolution. Existing compression relies on binary pruning: each KV state is either kept at full precision or discarded entirely. Our analysis shows that this forced choice is unnecessary: preserving all tokens at lower precision often proves more effective than discarding a fraction of them, demonstrating that fidelity reduction is not information deletion. This insight points to a different path: rather than deciding only whether to keep a state, we should also decide how faithfully to keep it as its future role evolves. To this end, we propose HorizonKV, which replaces binary decisions with a three-state lifecycle Native (full precision), K8V8 (8-bit quantized), and Drop (released) governed by the monotonically shrinking remaining generation horizon. Specifically, a small offline calibration first decouples future demand from compression sensitivity for each state. Afterwards, we allocate a global physical-byte budget via Lagrangian relaxation, enforcing irreversible transitions as future responsibility decreases. Extensive experiments across Infinity-2B and Infinity-8B show that HorizonKV consistently improves the memory–quality frontier: at only 1% KV storage, it approaches full-cache visual quality and achieves roughly 4× additional compression over the state-of-the-art pruning baseline HeatKV at comparable fidelity, while preserving semantic generation quality on GenEval and DPG-Bench.

Primary area:
generative models
Keywords:
Visual Autoregressive Models, KV Cache Compression, KV Cache Quantization, Multi-State Memory Management, Horizon-Aware Scheduling, Memory-Efficient Inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_s1wCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HorizonKV assigns historical layer/head/scale KV groups to native precision, INT8, or eviction. Offline calibration estimates future attention demand and state-dependent distortion, and a Lagrangian allocation schedules irreversible transitions under a persistent historical-cache byte budget. Experiments on Infinity-2B/8B report improved consistency with full-cache outputs compared with pruning baselines and largely preserved GenEval/DPG scores at budgets down to 1%. Component ablations, calibration-size sensitivity, and peak-memory/latency measurements provide additional validation.

Critical strengths
  1. Calibrating future demand separately from quantization sensitivity gives the allocation more information than an attention-importance score alone. The component ablation shows complementary gains from horizon awareness and the three-state representation, supporting the value of the proposed combination.

  2. Physical budgets include quantization metadata, and matched prompts and seeds make the output-consistency comparison meaningful. On Infinity-8B, HorizonKV at 1% and HeatKV at 4% both obtain consistency FID 2.71. Preserved GenEval/DPG scores and measured persistent-memory savings substantiate a useful memory–quality improvement within this model family.

Critical weaknesses
  1. The main experiments compare against pruning policies, whereas the proposed method jointly uses quantization and eviction. MiKV and DiffKV are discussed but not experimentally adapted, and a tuned pruning-plus-INT8 control is also needed. The Native/Drop ablation establishes that INT8 helps this framework; it does not establish an advantage over competitive alternative combined policies. This leaves the incremental value of the specific scheduler insufficiently isolated.

  2. Equation (4) aggregates normalized single-group perturbations measured while every other group remains native. Simultaneously compressing many groups changes attention normalization and downstream representations, so their joint effect need not follow that additive ranking. This is particularly consequential at 1% storage. The observed quality gains are encouraging, but neither direct validation of surrogate rankings under joint compression nor evidence beyond the Infinity family establishes the broader scheduling principle.

Questions for authors
  1. Does the gain persist over a tuned pruning-plus-INT8 policy or a compatible mixed-precision/retention baseline at equal physical bytes, and what latency trade-off accompanies that comparison?

  2. How well does the aggregate cost rank actual joint-compression distortion? A measured correlation or ranking comparison at the 1% budget would directly address the surrogate's reliability.

6 comments

We thank the reviewer for the suggestions concerning a matched joint-compression baseline, online latency, and the ranking reliability of the surrogate cost.

Q1. Does the gain persist over a tuned pruning-plus-INT8 policy or a compatible mixed-precision/retention baseline at equal physical bytes, and what latency trade-off accompanies that comparison?

(1) Relation to existing joint-compression methods.

MiKV [5] uses importance-aware mixed precision to store tokens at different fidelities. DiffKV [6] jointly considers Key/Value sensitivity and attention-head characteristics when compressing and managing the cache. Both target long-context LLM inference; their original designs do not directly address historical KV management in cross-scale visual autoregressive generation.

Existing VAR-oriented KV methods, including HeatKV [1], have primarily focused on pruning or selective retention. To the best of our knowledge, HorizonKV is the first framework to jointly schedule KV quantization and pruning for cross-scale visual autoregressive generation under a unified physical-byte budget. It assigns historical KV to Native, K8V8, or Drop states, jointly controlling retention and precision.

(2) Tuned pruning-plus-INT8 baseline.

We construct HeatKV+K8V8, extending HeatKV's official S-CAS importance ranking from Native/Drop to Native/K8V8/Drop. It uses exactly the same 10 calibration prompt/seed pairs, K8V8 implementation, three-state allocation solver, and physical KV-byte budgets as HorizonKV, while protecting the first three historical scales at Native precision. S-CAS is recomputed using the official procedure on the shared calibration set.

The two policies differ in scheduling costs: HeatKV+K8V8 uses static costs derived from S-CAS importance ranks, whereas HorizonKV uses future attention demand AA and compression sensitivity DD. This allows a comparison under matched quantization, pruning options, and byte constraints.

(3) Generation fidelity at matched physical budgets.

We compare all methods on Infinity-2B/8B at 1% and 4% KV budgets, using 5,000 paired images per configuration, with PSNR, LPIPS, and FID measured relative to Full Cache.

Table W1. Generation fidelity at matched physical KV budgets (PSNR ↑, LPIPS/FID ↓)

ModelKV budgetMethodPSNR ↑LPIPS ↓FID vs Full ↓
Infinity-2B1%HeatKV18.310.2466.39
HeatKV+K8V819.180.2185.85
HorizonKV20.620.1633.36
Infinity-2B4%HeatKV22.140.1272.70
HeatKV+K8V822.380.1252.65
HorizonKV23.290.1042.23
Infinity-8B1%HeatKV18.380.2034.72
HeatKV+K8V818.890.1994.50
HorizonKV21.310.1182.71
Infinity-8B4%HeatKV21.780.1112.71
HeatKV+K8V822.290.1112.61
HorizonKV23.990.0751.74

HeatKV+K8V8 improves on original HeatKV in PSNR, LPIPS, and FID at all four settings. HorizonKV obtains the best values of all three metrics in each setting.

For Infinity-8B at 1%, relative to HeatKV+K8V8, HorizonKV improves PSNR by 2.42 dB, reduces LPIPS by 40.8%, and lowers FID from 4.50 to 2.71. At 4%, its PSNR increases by 1.70 dB, LPIPS decreases from 0.111 to 0.075, and FID drops from 2.61 to 1.74.

The improvement over HeatKV+K8V8 remains after matching the quantization implementation, compression states, and physical byte budgets. The two policies differ in the calibrated costs used for allocation.

(4) Inference-latency trade-off.

We also benchmark online inference latency for the three methods at Infinity-8B with a 4% KV budget.

Table W2. Online latency on Infinity-8B at 4% (s/image, mean ± std.)

Batch sizeHeatKVHeatKV+K8V8HorizonKV
B=12.158 ± 0.0652.785 ± 0.0972.870 ± 0.099
B=41.899 ± 0.0192.043 ± 0.0232.025 ± 0.028
B=161.730 ± 0.0071.774 ± 0.0081.854 ± 0.016

Relative to HeatKV+K8V8, HorizonKV incurs +3.1%, -0.9%, and +4.5% per-image latency at B=1, 4, and 16, respectively.

In this comparison, the observed per-image latency difference from HeatKV+K8V8 ranges from -0.9% to +4.5%; the corresponding quality measurements are reported in Table W1. The joint-compression quality comparison is included in Section 4.3 (Scheduling Strategy Analysis); the three-method latency measurements are in Appendix D.5 (Scheduling Strategy Analysis). Additional inference-efficiency results appear in Section 4.4 (Practical Inference Efficiency).

Q2. How well does the aggregate cost rank actual joint-compression distortion? A measured correlation or ranking comparison at the 1% budget would directly address the surrogate's reliability.

We directly test how reliably the cost defined in Eq. (4) orders the real distortion caused by different joint-compression schedules, using Infinity-8B with a 1% KV budget.

(1) Candidate schedule construction.

Starting from HorizonKV's original schedule, we generate 99 perturbed schedules, yielding 100 candidates in total. Each perturbation swaps complete compression-state trajectories of compatible Layer–Head–Scale KV Groups, while keeping the same physical byte budget and respecting the irreversible transition constraint Native → K8V8 → Drop.

(2) Predicted cost and measured distortion.

For each candidate schedule, we compute:

  • Predicted cost: the aggregate cost in Eq. (4), using the offline calibrated future attention demand A and compression sensitivity D.

  • Measured distortion: mean LPIPS against the corresponding Full Cache outputs over the same 50 prompt/seed pairs for every schedule.

Each candidate provides one predicted cost and one measured mean LPIPS. The evaluation comprises 100 × 50 = 5,000 images, yielding 100 paired cost–distortion observations for rank-correlation analysis.

(3) Rank-correlation results.

We compute Spearman and Kendall correlations across the 100 cost/distortion pairs.

Table W3. Correlation between predicted cost and measured LPIPS

Rank metricCoefficient95% CI
Spearman ρ0.8462[0.8240, 0.8672]
Kendall τ-b0.6586[0.6295, 0.6792]

The 95% confidence intervals are obtained by 20,000 paired bootstrap resamples: we keep the 100 schedules fixed, jointly resample the common 50 prompt/seed cases, recompute the mean LPIPS per schedule, and recalculate the correlations.

To check whether rank agreement is driven solely by the perturbation magnitude, we group schedules by the percentage of modified KV Groups and compute Spearman correlation within similar perturbation ranges.

Table W4. Within-range Spearman correlation by perturbation magnitude

Modified KV groupsSpearman ρ
5%–10%0.599
10%–15%0.548
15%–20%0.606

The within-range correlations remain positive (0.548–0.606), although they are lower than the overall Spearman coefficient of 0.8462. Thus, the association between predicted cost and measured LPIPS is not solely explained by differences in perturbation magnitude. The correlation results are in Section 4.3 (Surrogate Cost Reliability; Fig. 6(a)); candidate construction and bootstrap procedures are given in Appendix D.6 (Surrogate Cost Reliability).

(4) Additional cross-model evaluation.

We further evaluate applicability beyond Infinity, including the original VAR [2], Switti-AR [3], and FlexVAR [4], against HeatKV [1] at matched KV-budget caps.

Table W5. Fidelity across visual autoregressive models

ModelKV budgetMethodPSNR vs Full ↑LPIPS vs Full ↓FID vs Full ↓
VAR-d3610%HeatKV18.3370.252486.1575
HorizonKV20.9940.166414.1546
VAR-d364%HeatKV16.6200.327678.5699
HorizonKV17.6640.277946.7094
Switti-AR50%HeatKV22.1210.166573.7791
HorizonKV25.2090.120992.6917
Switti-AR30%HeatKV19.5770.216645.2048
HorizonKV21.5830.150453.6674
FlexVAR-d2050%HeatKV18.4140.222326.2599
HorizonKV19.9380.180825.2329
FlexVAR-d2030%HeatKV16.2380.325088.1090
HorizonKV18.6220.214805.8901
FlexVAR-d2450%HeatKV17.7100.248596.7657
HorizonKV19.8850.181535.3508
FlexVAR-d2430%HeatKV16.1370.323317.977
HorizonKV18.5770.214986.080

Note: FID is computed against the Full Cache image set.

For VAR-d36, we calibrate on 10 ImageNet classes and evaluate 5,000 images from the remaining 990 classes; Switti-AR uses paired COCO-5000 text-conditioned evaluation.

Across the evaluated architectures and KV budgets, HorizonKV has higher PSNR and lower LPIPS and FID than HeatKV. The LPIPS reductions include 34.1% for VAR-d36@10%, 30.6% for Switti-AR@30%, and 33.9% for FlexVAR-d20@30%. The comparison is reported in Section 4.2 (Main Results) and Appendix D.4 (Cross-Architecture Generalization).

Reviewer_SaJDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

HorizonKV addresses the growing memory cost of historical KV caches in visual autoregressive generation. Instead of deciding only whether to retain or discard a cache group, it assigns groups to three states: full precision (Native), 8-bit quantization (K8V8), or removal (Drop). Offline calibration separately estimates future demand and sensitivity to compression, and a schedule fixed before generation allocates a physical-byte budget through irreversible transitions as the remaining generation horizon shrinks. Experiments on Infinity-2B and Infinity-8B compare fidelity, semantic quality, and resource use against pruning baselines. The paper reports substantial fidelity improvements at very small KV budgets and argues that 1% historical KV storage can approach full-cache quality, with roughly fourfold additional compression over HeatKV at comparable fidelity.

Critical strengths
  1. Fidelity gains at very small budgets are substantial. At 1% in Table 1, FID improves over HeatKV from 6.39 to 3.36 on Infinity-2B and from 4.72 to 2.71 on Infinity-8B, alongside LPIPS and PSNR improvements. Quantization metadata is included in the budget.

  2. The evaluations address both fidelity and semantic performance. Figure 6 supports complementary benefits from horizon awareness and three-state allocation. GenEval and DPG results suggest that average semantic performance is largely preserved.

Critical weaknesses
  1. The practical trade-off at comparable quality remains unclear. On Infinity-8B, HorizonKV@1% matches HeatKV@4% in FID but is slightly worse in LPIPS (0.118 versus 0.111) and PSNR (21.31 versus 21.78). At 4%, batch size 1, both use about 22.57 GiB of peak GPU memory, while time rises from 2.199 to 2.869 seconds per image (Appendix E.1). The important missing evidence is resource usage at 1%, together with the severity of residual visual degradation. Average fidelity and selected examples do not fully establish the claim of approaching full-cache quality.

  2. The scheduling contribution is not separated from the benefit of quantization. HeatKV already considers future scales. Figure 6(b) starts from an internal static baseline with FID 4.63, whereas HeatKV achieves 2.70 in the same setting. A matched HeatKV-plus-K8V8 comparison is needed to determine what the proposed schedule adds.

Questions for authors
  1. Please provide time and peak GPU memory for the Table 1 configurations on matched hardware, especially at 1%. Separate online generation from calibration costs and include repeated timing measurements.

  2. How severe is degradation at 1%? Paired per-image differences, severe-failure rates, and a small blinded comparison with full-cache outputs would help. At comparable quality to HeatKV, what memory, time, and batch-size benefits remain?

  3. How does HorizonKV compare with HeatKV using the same K8V8 quantization, byte budget, and calibration data at 1% and 4%?

  4. Which assumptions limit applicability to HART, the original VAR, or Switti-AR? Clarifying compatibility is the priority; a representative experiment would strengthen broader claims.

I would consider raising my score if the response establishes the schedule’s benefit over quantized HeatKV and demonstrates a meaningful quality–resource trade-off at 1%.

11 comments

We thank the reviewer for the comments on online memory and latency, low-budget fidelity, joint compression, and architectural coverage.

Q1. Please provide time and peak GPU memory for the Table 1 configurations on matched hardware, especially at 1%. Separate online generation from calibration costs and include repeated timing measurements.

(1) Online inference efficiency.

We benchmark Infinity-2B/8B on an NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GiB) at KV budgets of 1%, 4%, 10%, and 20%, with B=1/4/16, and include a Full Cache reference. Methods use matched hardware and generation configurations. Each configuration has three warm-up batches followed by ten timed batches; we report mean and standard deviation of seconds per image and peak total GPU memory.

Table S1. Infinity-2B: Runtime and peak GPU memory (latency: s/image, mean ± std.; memory: GiB)

KV budgetMethodB=1 Lat.B=1 Peak mem.B=4 Lat.B=4 Peak mem.B=16 Lat.B=16 Peak mem.
100%Full Cache1.037 ± 0.00315.650.798 ± 0.00142.87OOMOOM
1%HeatKV1.414 ± 0.01513.620.856 ± 0.04123.960.656 ± 0.05544.41
1%HorizonKV1.595 ± 0.09113.620.907 ± 0.05324.020.673 ± 0.05844.49
4%HeatKV1.440 ± 0.04013.800.883 ± 0.06024.790.668 ± 0.05948.86
4%HorizonKV1.875 ± 0.08113.840.977 ± 0.06225.000.717 ± 0.05848.92
10%HeatKV1.494 ± 0.02414.110.901 ± 0.05825.190.682 ± 0.05755.60
10%HorizonKV1.995 ± 0.06214.121.045 ± 0.06625.170.784 ± 0.06055.47
20%HeatKV1.553 ± 0.01914.570.916 ± 0.04925.910.701 ± 0.06064.78
20%HorizonKV2.078 ± 0.09414.611.092 ± 0.06326.120.839 ± 0.06364.81

Table S2. Infinity-8B: Runtime and peak GPU memory (latency: s/image, mean ± std.; memory: GiB)

KV budgetMethodB=1 Lat.B=1 Peak mem.B=4 Lat.B=4 Peak mem.B=16 Lat.B=16 Peak mem.
100%Full Cache2.042 ± 0.00436.851.843 ± 0.00994.91OOMOOM
1%HeatKV1.953 ± 0.06329.461.689 ± 0.00237.331.585 ± 0.00769.93
1%HorizonKV2.403 ± 0.08729.471.776 ± 0.02337.421.617 ± 0.00969.19
4%HeatKV2.158 ± 0.06530.081.899 ± 0.01938.521.730 ± 0.00779.11
4%HorizonKV2.870 ± 0.09929.762.025 ± 0.02838.931.854 ± 0.01679.51
10%HeatKV2.269 ± 0.08030.511.974 ± 0.02241.381.790 ± 0.00692.22
10%HorizonKV3.195 ± 0.06329.942.121 ± 0.02641.081.928 ± 0.00792.71
20%HeatKV2.391 ± 0.08331.562.160 ± 0.02341.76OOMOOM
20%HorizonKV3.353 ± 0.10931.042.245 ± 0.02542.30OOMOOM

Note: Peak GPU memory denotes the overall peak, not just persistent historical KV. OOM indicates out-of-memory.

Full Cache runs out of memory in several large-batch settings where compressed-cache methods remain feasible. HorizonKV has higher latency than HeatKV at matched KV budgets, although the relative difference narrows with batch size. For Infinity-8B at 1%, the difference is 23.0% at B=1 and 2.0% at B=16.

(2) Offline calibration and scheduling costs.

Future-demand estimation, sensitivity evaluation, and scheduling optimization for HorizonKV are performed offline and accounted for separately from online generation.

Table S3. Offline calibration and schedule optimization time

MethodOffline calibrationSchedule solverTotal
HeatKV67.400 s1.002 s68.402 s
HorizonKV264.613 s7.265 s271.878 s

HorizonKV's offline calibration includes distortion evaluation (219.182 s), future-demand estimation (30.980 s), and model-loading/setup costs.

Total offline preparation requires 271.878 s for HorizonKV and 68.402 s for HeatKV. The resulting schedules are reused without per-image calibration or online optimization. The benchmarking protocol is specified in Section 4.1 (Experimental Setup), with online results in Section 4.4 (Practical Inference Efficiency) and full batch-wise and offline measurements in Appendix E.1–E.3 (Memory Accounting and Runtime Details).

Q2. How severe is degradation at 1%? Paired per-image differences, severe-failure rates, and a small blinded comparison with full-cache outputs would help. At comparable quality to HeatKV, what memory, time, and batch-size benefits remain?

On Infinity-8B, HorizonKV at 1% and HeatKV at 4% both achieve approximately 2.71 FID relative to Full Cache, while HorizonKV uses only one quarter of the historical KV budget. We characterize their fidelity–efficiency trade-offs using paired image metrics, two blinded human studies, and runtime measurements.

(1) Paired fidelity and human studies.

For the same 5,000 prompt/seed pairs, we compare each compressed result against its corresponding Full Cache image and compute PSNR, LPIPS, and per-image LPIPS quantiles.

Table S4. Infinity-8B: Paired fidelity of HeatKV at 4% and HorizonKV at 1%

Metric (vs Full Cache)HeatKV@4%HorizonKV@1%
PSNR ↑21.7821.31
LPIPS ↓0.1110.118
LPIPS P50 ↓0.110710.11501
LPIPS P70 ↓0.135730.14092
LPIPS P99 ↓0.208610.21280
Lower LPIPS per image86.88%13.12%

The mean and tail-distortion differences are small: HorizonKV at 1% is lower by 0.47 dB PSNR, higher by 0.007 mean LPIPS, and higher by 0.00419 P99 LPIPS. HeatKV at 4% has marginally lower LPIPS for most paired samples, although their mean and tail distributions remain close.

To assess perceptual differences directly, we select the 500 samples with the largest HorizonKV@1% LPIPS and ask five raters to complete two separate blinded evaluations.

Study A — Similarity to Full Cache. Five raters each compare 500 image pairs against the corresponding Full Cache reference, yielding 2,500 blinded choices. HorizonKV@1% is preferred in 44.80% ± 1.90% of cases and HeatKV@4% in 55.20% ± 1.90% (mean ± SD across raters), corresponding to approximately 1,120 and 1,380 choices, respectively. Both methods receive substantial preference shares, with HeatKV@4% preferred somewhat more often.

Study B — Severity of visual errors. Raters separately categorize the anonymized results as no visible error, mild local degradation, visible structural error, or severe generation failure.

Table S5. Blinded visual-error assessment on 500 high-LPIPS samples (mean ± SD)

Visual error severityHorizonKV@1%HeatKV@4%
No visible error42.52% ± 1.72%39.08% ± 2.24%
Mild local degradation57.48% ± 1.72%60.92% ± 2.24%
Visible structural error0.00% ± 0.00%0.00% ± 0.00%
Severe generation failure0.00% ± 0.00%0.00% ± 0.00%

The share labeled "no visible error" is 42.52% for HorizonKV@1% versus 39.08% for HeatKV@4%. Neither method received a structural-error or severe-failure label within these selected 500 cases; observed differences mainly concern mild blurring or local edge degradation.

(2) Resource efficiency at comparable quality.

We further compare HeatKV@4% and HorizonKV@1% on Infinity-8B.

Table S6. Resource efficiency at comparable quality on Infinity-8B

Batch sizeMethodLatency (s/image) ↓Peak GPU memory (GiB) ↓
B=4HeatKV@4%1.899 ± 0.01938.52
HorizonKV@1%1.776 ± 0.02337.42
B=16HeatKV@4%1.730 ± 0.00779.11
HorizonKV@1%1.617 ± 0.00969.19

At B=4 and B=16, HorizonKV@1% reduces seconds per image by approximately 6.5% relative to HeatKV@4%. The reduction in overall peak GPU memory grows from 2.9% at B=4 to 12.5% at B=16; throughput increases by approximately 7.0% at B=16.

Across 5,000 paired images, the two methods have similar mean LPIPS (0.118 vs. 0.111), P99 LPIPS (0.21280 vs. 0.20861), and FID (approximately 2.71 for both). In Study A, preference shares are 44.80% for HorizonKV@1% and 55.20% for HeatKV@4%. In Study B, the observed differences are mainly mild local degradations; neither method is labeled as a severe generation failure among the selected images.

Against this quality comparison, HorizonKV@1% uses one quarter of the historical KV budget of HeatKV@4%. It reduces per-image latency by approximately 6.5% at B=4 and B=16; at B=16, peak total GPU memory falls by 12.5% and throughput rises by approximately 7.0%.

The quality and system results are in Sections 4.2 and 4.4; the paired and blinded evaluations are detailed in Appendix D.7 (Low-Budget Fidelity and Human Evaluation).

Q3. How does HorizonKV compare with HeatKV using the same K8V8 quantization, byte budget, and calibration data at 1% and 4%?

(1) Controlled joint-compression baseline.

We construct S-CAS-based Quantized HeatKV (HeatKV+K8V8), extending HeatKV's official S-CAS importance ranking with INT8 quantization and KV eviction. Rather than using a separately precomputed ranking, we recompute S-CAS from exactly the same 10 calibration prompt/seed pairs used by HorizonKV. The baseline supports Native, K8V8, and Drop, protects the first three historical scales at Native precision, and uses the same K8V8 implementation, three-state allocation solver, and physical KV-byte budget as HorizonKV.

The difference lies in the scheduling costs: HeatKV+K8V8 derives static compression costs from S-CAS importance ranks, whereas HorizonKV uses future attention demand AA and compression sensitivity DD. Both methods use the same quantization and pruning options under matched storage constraints.

(2) Joint-compression quality comparison.

We evaluate Infinity-2B/8B at 1% and 4% KV budgets, with 5,000 paired images per configuration.

Table S7. Joint-compression fidelity at matched physical KV byte budgets

ModelKV budgetMethodPSNR ↑LPIPS ↓FID vs Full ↓
Infinity-2B1%HeatKV18.310.2466.39
HeatKV+K8V819.180.2185.85
HorizonKV20.620.1633.36
Infinity-2B4%HeatKV22.140.1272.70
HeatKV+K8V822.380.1252.65
HorizonKV23.290.1042.23
Infinity-8B1%HeatKV18.380.2034.72
HeatKV+K8V818.890.1994.50
HorizonKV21.310.1182.71
Infinity-8B4%HeatKV21.780.1112.71
HeatKV+K8V822.290.1112.61
HorizonKV23.990.0751.74

HorizonKV achieves better PSNR, LPIPS, and FID than HeatKV+K8V8 in all four configurations. For Infinity-8B at 1%, HorizonKV gains 2.42 dB PSNR, reduces LPIPS by 40.8%, and reduces FID from 4.50 to 2.71 relative to HeatKV+K8V8.

With the K8V8 codec, three-state solver, calibration examples, and physical byte budgets matched, HorizonKV retains lower LPIPS and FID than the S-CAS-based allocation in all four settings. The baseline construction and quality comparisons are reported in Section 4.3 (Scheduling Strategy Analysis) and Appendix D.5 (Scheduling Strategy Analysis).

Reviewer_udxdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HorizonKV, a training-free KV cache compression methodfor Visual Autoregressive (VAR) models. Existing methods primarily rely onbinary pruning, i.e., fully retaining or directly deleting key-value (KV)information. HorizonKV, however, introduces three storage states: Native (fullprecision), K8V8 (8-bit quantized), and Drop (released), dynamically adjustingthe storage precision and retention state of KV based on the remaininggeneration stages. This method proposes an offline calibration approach toevaluate the future demand and compression sensitivity of each KV on acalibration dataset, and then uses Lagrange optimization to formulate a cachescheduling strategy within the memory budget. Experiments show thatHorizonKV outperforms existing pruning methods on both Infinity-2B andInfinity-8B models, maintaining good image generation quality while using only1% KV cache, and achieving approximately four times the compression performance of HeatKV at similar quality levels.

Critical strengths
  1. Large KV-Cache memory is a long-standing and important research problem in AR generation.

  2. HorizonKV is training-free and highly effective, making it easily scalable to other VAR models and delivering significant efficiency improvements. With 1% KV cache storage only, the generation quality is comparable with base model.

  3. This paper observes that KV cache importance varies across groups and proposes future demand and compression sensitivity to quantify their importance and distortion, respectively. It further develops a Lagrangian-based optimization strategy for budget-aware three-state allocation and thoroughly validates the effectiveness of its key components through ablation experiment. To my best of knowleadge, these observations are novel.

Critical weaknesses
  1. The paper uses 10 fixed prompt/seed pairs, but does not adequately explain how these samples were selected or whether they represent different image content and generation patterns.
  2. The selection criteria for calibration data are unclear. Pls provide more details for this part.
  3. Limited Validation of the Calibration Metric. Although the authors carefully design future demand and compression sensitivity and demonstrate the effectiveness of the overall framework, it remains unclear how well there sulting distortion cost correlates with actual image quality. A direct analysis of this relationship would help validate the proposed objective.
  4. Limited Evidence for the Effectiveness of the Proposed Criteria.
  5. As described in Appendix C.1, HorizonKV provides a mechanism to determine when historical KV groups should retain native precision, when lower fidelity is sufficient, and when they can be dropped. However, compared with existing methods such as HeatKV, the paper does not sufficiently demonstrate how much the proposed future demand and compression sensitivity criteria contribute to the final generated image quality, making their practical benefits difficult to assess.
  6. Provide more discussion about related works like [1] and [2].

[1] Entropy-Aware Dynamic KV Cache Sparsification for Autoregressive Image Generation and Editing. ICML'26 [2] Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models. Arxiv 26'5

Questions for authors
  1. Could the authors provide the performance of HorizonKV on more VAR models, such as Meissonic?
  2. The core contribution of HorizonKV lies in proposing an offline calibration method to control and balance KV compression. However, the choice ofcalibration set, its working principle, and its generalization ability remainunclear. Could the authors explain why the offline calibration operation can generalize to a large number of other samples?
  3. HorizonKV only compresses KV during image generation. However, inTables 3 and 4, HorizonKV, retaining only 1% of the KV budget, outperforms the 100% KV budget setting in some metrics. Could the authors explain the reason for this?
  4. As described in Appendix E.1, although HorizonKV significantly compresses the KV cache, at a 4% cache budget, the generation latency per image increases from 2.101 seconds to 2.869 seconds compared to HeatKV when the batch size is 1. Could the authors explain how HorizonKV affects latency and what latency overhead it would ideally incur?
9 comments

We thank the reviewer for the comments on model coverage, calibration, quality metrics, and inference latency.

Q1. Could the authors provide the performance of HorizonKV on more VAR models, such as Meissonic?

Meissonic [1] uses a non-autoregressive generation mechanism based on masked image modeling, unlike the cross-scale autoregressive historical KV reuse studied in this paper.

We evaluate HorizonKV on VAR-d36 [2], Switti-AR [3], and FlexVAR-d20/d24 [4], with HeatKV [6] as the baseline at matched KV-budget caps.

Table U1. Generation fidelity across visual autoregressive architectures

ModelKV budgetMethodPSNR vs Full ↑LPIPS vs Full ↓FID vs Full ↓
VAR-d3610%HeatKV18.3370.252486.1575
HorizonKV20.9940.166414.1546
VAR-d364%HeatKV16.6200.327678.5699
HorizonKV17.6640.277946.7094
Switti-AR50%HeatKV22.1210.166573.7791
HorizonKV25.2090.120992.6917
Switti-AR30%HeatKV19.5770.216645.2048
HorizonKV21.5830.150453.6674
FlexVAR-d2050%HeatKV18.4140.222326.2599
HorizonKV19.9380.180825.2329
FlexVAR-d2030%HeatKV16.2380.325088.1090
HorizonKV18.6220.214805.8901
FlexVAR-d2450%HeatKV17.7100.248596.7657
HorizonKV19.8850.181535.3508
FlexVAR-d2430%HeatKV16.1370.323317.977
HorizonKV18.5770.214986.080

Note: FID is computed against the Full Cache image set.

HorizonKV improves PSNR, LPIPS, and FID in all settings. LPIPS decreases by approximately 34.1% on VAR-d36 at 10% and 30.6% on Switti-AR at 30%; on FlexVAR-d20 at 30%, PSNR increases by 2.38 dB and LPIPS decreases by 33.9%.

The evaluated models differ from Infinity in visual-token representation, residual prediction, or conditioning. HorizonKV obtains lower LPIPS and FID than HeatKV at each tested budget after model-specific cache adaptation and calibration. These results appear in Section 4.2 (Main Results) and Appendix D.4 (Cross-Architecture Generalization) of the revised manuscript.

Q2. The core contribution of HorizonKV lies in proposing an offline calibration method to control and balance KV compression. However, the choice of calibration set, its working principle, and its generalization ability remain unclear. Could the authors explain why the offline calibration operation can generalize to a large number of other samples?

(1) Calibration-set construction and selection

We use randomly sampled captions from the public Flickr30k dataset [7] rather than LLM-generated calibration prompts. This avoids additional uncertainty from a generative model and potential selection bias from manually crafting or choosing prompts. Specifically, we read caption1 and caption2 in the source dataset's original row order, remove duplicates, and sample 10 distinct English captions without replacement using random.Random(87). The selected captions are fixed before evaluation, are not filtered by generation quality or compression performance, and are reused across all Infinity KV budgets.

The protocol is described in Section 4.1 (Experimental Setup); the sampling procedure and all ten captions are provided in Appendix A.2 (Calibration of Future Demand and Compression Sensitivity).

(2) Working Principle and Generalization

HorizonKV uses calibration examples to estimate the future attention demand A and compression sensitivity D of each Layer–Head–Scale group, and constructs a fixed three-state compression schedule based on the aggregated statistics. This design builds upon the relatively stable information-processing structures within pretrained models. Different Transformer layers, attention heads, and historical generation scales serve distinct information-processing and transmission functions, resulting in different dependencies on historical KV states and different sensitivities to compression.

Prior studies have also observed these structural differences from different perspectives. ScaleKV [5] and HeatKV [6] both adopt a small number of calibration examples for offline importance estimation. ScaleKV further evaluates calibration sets of varying sizes (1–128 prompts) randomly sampled from LAION-Art, with FID consistently remaining at 2.53. Similarly, HeatKV's calibration-size ablation shows that FID varies only from 1.66 to 1.67 as the number of calibration prompts increases from 1 to 10. Furthermore, Forcing-KV [9] reports that different attention heads exhibit distinct functional roles, which remain relatively stable across different generation samples and denoising steps. Together, these studies provide empirical evidence that relatively stable structural patterns within pretrained generative models can be effectively captured using a small number of calibration examples.

Building on this observation, HorizonKV uses a small number of randomly selected calibration prompts to estimate the future information demand and compression sensitivity of different Layer–Head–Scale groups, aggregating them into model-level statistics. Since these statistics characterize the information-processing properties of different model components rather than the content of specific inputs, the resulting compression schedule can be reused across different inputs without re-estimating importance or optimizing the compression strategy for each individual prompt.

Figure 6(d) further examines the sensitivity of generation quality to the calibration set size. With 1, 2, 5, and 10 calibration examples, the resulting FID scores are 2.220, 2.230, 2.217, and 2.230, respectively, with a maximum absolute difference of only 0.013. Even with a single calibration example, HorizonKV achieves generation quality comparable to that obtained with more calibration examples, indicating that its scheduling performance is relatively insensitive to the calibration set size.

Furthermore, we evaluate HorizonKV on different visual autoregressive architectures, including Switti-AR, VAR-d36, and FlexVAR, further demonstrating its applicability across different models.

Q3. HorizonKV only compresses KV during image generation. However, in Tables 3 and 4, HorizonKV, retaining only 1% of the KV budget, outperforms the 100% KV budget setting in some metrics. Could the authors explain the reason for this?

A possible explanation is redundancy in historical KV states and an implicit regularization effect induced by compression.

Historical KV states differ substantially in their relevance to future autoregressive predictions and contain redundant information. Even with a 1% historical KV budget, retaining the most consequential dependencies can sustain downstream generation; therefore, memory reduction need not cause a proportional degradation in image quality.

Selective compression may also suppress some redundant or distracting attention information, resulting in effects resembling denoising or implicit regularization. Moreover, compressing KV changes subsequent token distributions and can alter the final autoregressive generation trajectory. These factors may explain why a few quality metrics improve under compression.

Similar non-monotonic behavior has been reported in HeatKV [6]. Implicit regularization is one possible explanation, but its causal role has not been established. The corresponding semantic-quality results appear in Section 4.2 (Main Results) and Appendix D.2 (Detailed GenEval Results) and Appendix D.3 (Detailed DPG-Bench Results).

Comments on the paper

1 comment

We thank the reviewers for their comments. We have uploaded a revised manuscript and appendix during the rebuttal period. The revision includes the experiments, methodological details, and related-work discussion summarized below. Locations of individual changes are given with the corresponding responses.

The main additions are:

  1. Cross-architecture evaluation: We evaluate HorizonKV on VAR-d36, Switti-AR, and FlexVAR-d20/24 against HeatKV under matched KV-budget caps.

  2. Controlled joint-compression baseline: We introduce HeatKV+K8V8, combining HeatKV's official S-CAS importance ranking with INT8 quantization and KV eviction. The baseline uses exactly the same calibration samples, K8V8 implementation, three-state allocation solver, and physical KV-byte budgets as HorizonKV, allowing the two scheduling policies to be compared under matched conditions.

  3. Low-budget fidelity: We report paired image metrics and tail-distortion statistics on 5,000 images, as well as two blinded evaluations of 500 selected high-LPIPS samples at a 1% KV budget.

  4. System measurements: We report per-image latency, peak GPU memory, and offline calibration cost on an RTX PRO 6000 across batch sizes, together with CPU/CUDA profiling.

  5. Surrogate-cost validation: Across 100 candidate joint-compression schedules, the predicted cost and measured mean LPIPS have Spearman ρ=0.8462.

Detailed results and the corresponding manuscript revisions are described in the responses below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.