acceptodds
Under review as a conference paper at ICLR 2027

ValueDiff: Value-Geometric KV Cache Eviction for Sink-Suppressed LLMs

Abstract

Modern LLMs with QK-normalization, gated attention, learned attention sinks, or logit softcapping exhibit weaker persistent attention sinks, on which existing KV cache eviction methods primarily rely. We observe that across these models, weaker sinks co-occur with greater value-vector dispersion relative to key-vector dispersion. Motivated by this value-side dispersion, we present ValueDiff, a value-geometric eviction that ranks tokens by the L2 deviation of their value vectors from the cache mean. The same score arises as the minimal-disturbance eviction under a max-entropy assumption about future attention. We evaluate under fixed cache budgets, with eviction at every block boundary during prefill and at every decoding step during generation. On RULER at a tight 2k token budget, ValueDiff retains 88–99% of dense across seven sink-suppressed models (best on 6 out of 7). On LongBench at the 4k budget, ValueDiff averages 92% retention across sink-suppressed models versus 83% for the strongest prior baseline. On MATH-500, ValueDiff is the strongest non-dense method on every sink-suppressed model tested at the 25% cache budget, outperforming prior methods by up to 20 points on gated-attention models. Across all three benchmarks, value geometry emerges as the more reliable query-invariant eviction signal for sink-suppressed models.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
KV Cache Eviction

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_9WEAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

ValueDiff proposes a training-free KV-cache eviction rule that retains tokens whose value vectors lie farthest from the cache mean. Motivated by an observed association between weaker attention sinks and greater relative value dispersion, the method avoids relying on observed attention weights. Experiments under continuous cache eviction show strong performance on several sink-suppressed architectures across long-context and mathematical reasoning tasks.

Critical strengths
  1. The method is simple, query-independent, and requires neither training nor auxiliary memory modules. Its low implementation complexity makes the empirical improvements practically appealing.
  2. The evaluation enforces bounded cache capacity during both prefill and generation. This provides a meaningful test of irreversible eviction decisions when later context and future queries remain unavailable.
  3. The paper evaluates diverse architectural families and includes matched key/value geometry comparisons, direction-versus-magnitude ablations, and an attention-weighted counterpart. These comparisons help identify which scoring components contribute to performance.
Critical weaknesses
  1. The architectural explanation remains observational. Models differ simultaneously in training, attention structure, head sharing, gating, and local-context mechanisms. Consequently, weaker sinks and stronger ValueDiff performance may share other causes rather than reflect the proposed relationship directly.

  2. The maximum-entropy derivation relies on assumptions that are not established by sink suppression. Removing persistent sink tokens does not imply that future attention is uniform or exchangeable; attention may remain highly selective over content. The derivation therefore motivates the score without showing that its assumptions describe the evaluated models.

  3. Single-token eviction optimality does not justify retaining the highest-scoring set during bulk compression. Individually distinctive values may be mutually redundant, while removing many near-mean values can collectively discard important information. Repeated selection also changes the centroid on which subsequent decisions depend.

Questions for authors

Can controlled comparisons within a model family isolate sink suppression from other architectural differences and establish whether it predicts the advantage of value-based eviction?

How closely do actual future attention distributions satisfy these assumptions, and how accurately does ValueDiff predict reconstruction error as attention departs from the symmetric setting?

How large is the gap between independent ranking and joint subset selection, and does sequentially updating scores improve performance on clustered or multimodal value distributions?

3 comments

Thank you for your careful review and thoughtful questions. Your comments helped us clarify the limits of the theoretical and architectural interpretation and motivated the additional reconstruction-error, joint-selection, and sequential ValueDiff evaluations.

Response to W1 / Q1

We agree that the architectural explanation is observational. Our architectural groups organize the empirical comparison; they do not isolate the effects of sink suppression from training, gating, head sharing or other model differences.

We further examined checkpoints within the Qwen3.5 and Gemma2 families. We measured dense-prefill attention on the same 150 RULER-8K prompts per model, averaging five corresponding input positions, query heads and softmax-attention layers. Initial-token mass measures sink strength at token zero; normalized entropy measures attention flatness, with one corresponding to uniform attention. Performance gaps average all 13 RULER tasks at a 2K cache budget; positive values favor ValueDiff.

ModelInitial-token massNormalized entropyVD−KeyDiff (pp)
Qwen3.5-4B2.92%0.561+3.90
Qwen3.5-9B3.12%0.548+3.70
Gemma2-2B13.38%0.461−5.95
Gemma2-9B17.81%0.408+0.37
  • Qwen3.5: Both checkpoints exhibit weak initial-token sinks and similar ValueDiff advantages over KeyDiff.
  • Gemma2: The 9B checkpoint has stronger initial-token sinks and more concentrated attention than the 2B checkpoint, yet the VD−KeyDiff gap improves from −5.95 pp to +0.37 pp. Sink strength alone therefore does not explain the performance difference.

These within-family comparisons provide additional empirical context but do not isolate sink suppression. We will revise the manuscript to present the architectural explanation as an interpretation of the observed results and clarify that a causal test requires controlled interventions.

Response to W2

We agree that weaker attention sinks do not establish uniform attention or exchangeability of future token usage. Our study starts from empirical eviction results; the analysis helps interpret the algorithm's behavior, as reflected in the Section 3.3 title, “A Max-Entropy Interpretation of ValueDiff.”

We measured dense-prefill attention on 150 RULER-8K prompts per model. Initial-token mass measures the strength of the attention sink at token zero; normalized entropy measures how evenly attention is distributed over available tokens, with one corresponding to uniform attention.

ModelInitial-token massNormalized attention entropy
Llama3.1-8B57.68%0.3043
Qwen3-4B-Instruct37.22%0.3607
Qwen3.5-4B2.92%0.5612

The evaluated Qwen checkpoints have weaker initial-token sinks and flatter overall attention than Llama3.1, while attention remains nonuniform. These observations provide empirical context for the interpretation without establishing uniformity or exchangeability.

We will clarify in the manuscript that uniform attention and the symmetric Dirichlet prior are analytical reference settings, rather than consequences of sink suppression.

Response to Q2

Our primary objective is to preserve downstream task accuracy under a fixed cache budget. Reconstruction error is an analytical surrogate; a change in the attention output does not necessarily change the generated answer. We do not claim that centroid distance exactly predicts reconstruction error under general attention.

We directly compared ValueDiff and actual-output CAOTE using the same dense Q/K/V on 15 RULER-8K prompts per Qwen checkpoint: five each from retrieval, variable tracking and QA. Both policies use a 2K cache, sink1/recent128 protection and 128-token prefill chunks. Evictions are irreversible and applied after each chunk.

For an attention row tt, let mtiMm^M_{ti} indicate whether method MM retains token ii. The dense and masked outputs are

ot=∑i≤tαtivio_t=\sum_{i\leq t}\alpha_{ti}v_i

otM=∑i≤tmtiMαtivi∑i≤tmtiMαtio_t^M=\frac{\sum_{i\leq t}m^M_{ti}\alpha_{ti}v_i} {\sum_{i\leq t}m^M_{ti}\alpha_{ti}}

EtM=∥ot−otM∥22.E_t^M=\Vert o_t-o_t^M\Vert _2^2.

We normalize the pooled squared error by the same dense-output energy for both methods:

EM=∑t∈A∥ot−otM∥22∑t∈A∥ot∥22\mathcal{E}_M= \frac{\sum_{t\in\mathcal{A}}\Vert o_t-o_t^M\Vert _2^2} {\sum_{t\in\mathcal{A}}\Vert o_t\Vert _2^2}

where A\mathcal{A} contains eviction-active query/head/layer rows across the measured prompts. Lower reconstruction error is better. The separate downstream scores average all 13 RULER tasks at a 2K cache budget.

ModelVD reconstruction E\mathcal{E}CAOTE reconstruction E\mathcal{E}VD RULERCAOTE RULER
Qwen3-4B-Instruct5.95%4.81%91.2072.62
Qwen3.5-4B2.55%1.57%92.9656.71

CAOTE has smaller local reconstruction error on both Qwen checkpoints, while ValueDiff achieves higher downstream accuracy. Lower attention-output reconstruction error does not necessarily translate into better downstream task performance. Query-adaptive sparse attention can select different KVs for each query from a retained cache. Token eviction, however, irreversibly removes KVs under a restricted cache budget and must preserve information that may be useful during future generation. This distinction helps explain why minimizing local reconstruction error need not yield the best eviction policy.

Response to W3 / Q3

We agree that single-token optimality does not establish optimality of bulk selection. Our batch analysis instead shows that fixed-centroid ranking minimizes an upper bound on disturbance under uniform attention.

For nn cached values, let RR be a removed set of size m<nm<n and μ=n−1∑i=1nvi\mu=n^{-1}\sum_{i=1}^n v_i the original full-cache centroid. Under uniform attention, the original and renormalized survivor outputs are

o=μo=\mu

o∖R=nμ−∑i∈Rvin−m.o_{\setminus R}=\frac{n\mu-\sum_{i\in R}v_i}{n-m}.

Their exact squared difference is therefore

E(R)=∥o−o∖R∥22=∥∑i∈R(vi−μ)∥22(n−m)2.E(R)=\left\Vert o-o_{\setminus R}\right\Vert _2^2 =\frac{\left\Vert \sum_{i\in R}(v_i-\mu)\right\Vert _2^2}{(n-m)^2}.

Writing di=vi−μd_i=v_i-\mu and si=∥di∥2s_i=\Vert d_i\Vert _2 makes the interactions explicit:

E(R)=∑i∈Rsi2+2∑i<j,i,j∈Rdi⊤dj(n−m)2.E(R)=\frac{ \sum_{i\in R}s_i^2 +2\sum_{\substack{i<j,\quad i,j\in R}}d_i^\top d_j }{(n-m)^2}.

Thus, many individually small deviations can reinforce one another, while larger deviations can cancel. Individual ranking does not optimize these cross-token terms. Applying the triangle inequality and Cauchy–Schwarz gives

E(R)≤(∑i∈R∥di∥2)2(n−m)2≤m(n−m)2∑i∈Rsi2.E(R) \leq\frac{\left(\sum_{i\in R}\Vert d_i\Vert _2\right)^2}{(n-m)^2} \leq\frac{m}{(n-m)^2}\sum_{i\in R}s_i^2.

For fixed mm, removing the smallest full-centroid scores minimizes this upper bound, rather than the exact joint error. The implementation uses the eligible-token centroid cc, excluding protected sink/recent tokens. The corresponding bound is

E(R)≤(m∑i∈R∥vi−c∥22+m∥c−μ∥2)2(n−m)2.E(R)\leq \frac{\left( \sqrt{m\sum_{i\in R}\Vert v_i-c\Vert _2^2} +m\Vert c-\mu\Vert _2 \right)^2}{(n-m)^2}.

The centroid-offset term is constant for a fixed cache and removal count, so selecting the smallest implemented scores minimizes this bound too.

Joint subset selection

To assess the selection gap without assuming clustered geometry, we compared three policies:

  • Ordinary VD: ranks tokens once by distance from a fixed centroid and removes the lowest-scoring tokens.
  • Sequential VD: removes the token closest to the current centroid, then recomputes the centroid before the next removal.
  • Exact joint enumeration: evaluates every subset of the required removal size and selects the one minimizing the joint reconstruction error E(R)E(R) under uniform attention.

We compared these policies on 16-token miniature caches sampled from saved text-only WikiText values. Each policy removes four tokens without protection. The table reports mean reconstruction error normalized by the original centered value variance:

E~(R)=E(R)1n∑i=1n∥vi−μ∥22.\widetilde E(R)= \frac{E(R)}{\frac1n\sum_{i=1}^n\Vert v_i-\mu\Vert _2^2}.

ModelMiniature cachesVDSequential VDExact joint
Gemma2-9B360.0210040.0182910.013441
Gemma3-4B360.0174840.0165000.013047
GPT-OSS-20B720.0173440.0159930.009548
Qwen3-4B-Instruct720.0179330.0171000.010912

Sequential updates lower mean reconstruction error in all four sampled model groups, but leave a gap to exact joint selection.

Exact enumeration requires evaluating (nm)\binom{n}{m} subsets when removing mm of nn tokens, making it computationally intractable at benchmark cache sizes. We therefore evaluated sequential VD on all 13 RULER-8K tasks with 50 examples per task, using Qwen3-4B-Instruct and a 2K cache.

ModelPlain VDSequential VDSequential−VD (pp)
Qwen3-4B-Instruct91.2091.51+0.31

The observed overall difference is small: sequential updates improve two tasks, regress on two and tie on nine. RULER measures task accuracy with a 2K cache, while the miniature-cache experiment measures reconstruction error under uniform attention.

Reviewer_zQCjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ValueDiff is a query-independent KV-cache eviction method that retains tokens according to the Euclidean distance between their value vectors and the current cache mean. Its motivation is that attention scores observed at eviction time may poorly predict future token utility, particularly during blockwise prefill, where decisions are irreversible and later queries are unavailable. The authors connect its effectiveness to architectural changes associated with weaker attention sinks. They show that its score minimizes single-token output disturbance under uniform attention and expected squared disturbance under a symmetric Dirichlet prior. Experiments on RULER, LongBench, and MATH-500 demonstrate strong results on several sink-suppressed models, although the advantage depends on the architecture and evaluation protocol.

Critical strengths
  1. ValueDiff requires only cached value vectors, avoiding query-dependent scoring and explicit attention-weight computation for eviction.

  2. The score follows from an explicit output-disturbance objective under stated assumptions.

  3. The experiments cover multiple architectural families and include useful comparisons between blockwise and one-shot prefill.

Critical weaknesses
  1. The connection between sink suppression and the theoretical assumptions remains insufficiently supported. The symmetric-prior derivation is valid, but weaker sinks do not imply approximately uniform attention or exchangeability of future token usage.

  2. The geometric explanation requires more careful interpretation. The quantity called value dispersion is the standard deviation of distances from the centroid, rather than total vector variance. It can therefore be small even for directionally diverse vectors with similar radii.

  3. The incremental benefit of centroid distance over simpler value-based scores is not fully established. Table 2 shows that norm-only scoring outperforms ValueDiff on Qwen3.5-4B, while direction-only scoring outperforms it on Gemma3-4B.

  4. The theoretical optimality results concern single-token eviction and should not be interpreted as batch-eviction guarantees; interactions between removed tokens remain unaddressed.

Questions for authors
  1. Do attention nonuniformity and output-to-centroid discrepancy predict the performance gap between ValueDiff and attention-weighted scoring?

  2. Does ValueDiff retain its advantage when the attention-weighted comparator uses actual attention outputs rather than the value mean?

  3. Does protecting sink tokens across all methods substantially reduce the advantage of key-based scoring on strong-sink models?

4 comments

Response to W1

Our study starts from empirical eviction results. The analysis helps interpret the algorithm’s behavior, as reflected in the Section 3.3 title, “A Max-Entropy Interpretation of ValueDiff.” We agree that weaker attention sinks do not establish uniform attention or exchangeability of future token usage.

Nevertheless, our measurements show weaker initial-token sinks and flatter overall attention on the evaluated Qwen checkpoints than on Llama3.1.

We measured attention on 150 RULER-8K prompts per model. Initial-token mass measures the strength of the attention sink at token zero; normalized entropy measures how evenly attention is distributed over all available tokens, with one corresponding to uniform attention.

ModelInitial-token massNormalized attention entropy
Llama3.1-8B57.68%0.3043
Qwen3-4B-Instruct37.22%0.3607
Qwen3.5-4B2.92%0.5612

These observations provide context for the interpretation without establishing uniformity or exchangeability. We will clarify the role and assumptions of the analysis in the manuscript.

Response to W2

The ValueDiff score incorporates both value magnitude and directional alignment with the centroid:

si2=∥vi−vˉ∥22s_i^2 =\Vert v_i-\bar v\Vert _2^2

si2=∥vi∥22−2vi⊤vˉ+∥vˉ∥22s_i^2 =\Vert v_i\Vert _2^2-2v_i^\top\bar v+\Vert \bar v\Vert _2^2

si2=∥vi∥22−2∥vi∥2∥vˉ∥2cos⁡θi+∥vˉ∥22s_i^2 =\Vert v_i\Vert _2^2 -2\Vert v_i\Vert _2\Vert \bar v\Vert _2\cos\theta_i +\Vert \bar v\Vert _2^2

vˉ=1n∑i=1nvi.\bar v=\frac1n\sum_{i=1}^n v_i.

Here, θi\theta_i is the angle between viv_i and vˉ\bar v, when both are nonzero. Thus, vectors with similar value norms can receive different scores because of their alignment with the centroid.

We agree, however, that our aggregate statistic measures the spread of these scalar scores:

σV=Std⁡i(si).\sigma_V=\operatorname{Std}_i(s_i).

This differs from total centered vector variance,

1n∑i=1n∥vi−vˉ∥22=1n∑i=1nsi2.\frac1n\sum_{i=1}^n\Vert v_i-\bar v\Vert _2^2 =\frac1n\sum_{i=1}^n s_i^2.

Vectors with equal centroid distances receive equal scores even when their directions differ. We will replace “value dispersion” with “ValueDiff score spread” and clarify this distinction in the manuscript.

Response to W3

We agree that combining magnitude and direction does not improve every configuration, although ValueDiff outperforms both component ablations in the majority of reported model–budget configurations across RULER, LongBench and MATH-500. Norm-only and direction-only are component ablations of our value-based approach, and the LongBench cases you identify favor an individual component. The table below reports accuracy retention relative to dense (%) at 4K/8K cache budgets:

ModelValueDiffNorm-onlyDirection-only
Gemma3-4B88.1 / 94.582.6 / 93.090.5 / 96.5
Qwen3.5-4B92.2 / 95.592.8 / 97.688.4 / 95.8

We further evaluated the component ablations on a different workload: decoding-heavy mathematical reasoning using the full MATH-500 benchmark. Each method uses a 256-token cache and all 500 problems on the same two models. Gemma3 averages five sampled responses per problem; Qwen3.5 uses one greedy response, following the manuscript protocol. Differences are percentage points with 95% paired bootstrap confidence intervals (20,000 replicates).

ModelValueDiffNorm-onlyDirection-onlyVD−Norm (pp), 95% CIVD−Direction (pp), 95% CI
Gemma3-4B67.4464.6463.16+2.80 [1.28, 4.36]+4.28 [2.52, 6.08]
Qwen3.5-4B73.2066.6064.00+6.60 [3.80, 9.40]+9.20 [5.80, 12.60]

ValueDiff exceeds both component ablations in these full-suite comparisons. Together, these results show that the combined score provides benefits whose extent depends on the model, task and cache budget. We will add the completed comparisons to Appendix G and clarify where the component variants perform better, avoiding a claim that the combined score is universally superior.

Response to W4

We agree that single-token optimality does not establish optimality of bulk selection. Under uniform attention, fixed-centroid ranking minimizes an upper bound on disturbance, while the exact error includes interactions between removed tokens. The derivation below makes this distinction explicit.

1. Exact disturbance for a removed set. For fixed values v1,…,vnv_1,\ldots,v_n, let RR contain m<nm<n removed tokens and let aR=∑i∈Rαi<1a_R=\sum_{i\in R}\alpha_i<1. The original and renormalized survivor outputs are

o=∑i=1nαivio=\sum_{i=1}^n\alpha_i v_i

o∖R=o−∑i∈Rαivi1−aR.o_{\setminus R}=\frac{o-\sum_{i\in R}\alpha_i v_i}{1-a_R}.

Subtracting gives the exact identity

o−o∖R=∑i∈Rαi(vi−o)1−aR.o-o_{\setminus R} =\frac{\sum_{i\in R}\alpha_i(v_i-o)}{1-a_R}.

For general attention, disturbance therefore depends on both the attention weights and how the removed deviations combine.

2. Uniform-attention reference. Set αi=1/n\alpha_i=1/n and define the original full-cache centroid μ=n−1∑ivi\mu=n^{-1}\sum_i v_i. Then o=μo=\mu, aR=m/na_R=m/n, and

o∖R=nμ−∑i∈Rvin−mo_{\setminus R} =\frac{n\mu-\sum_{i\in R}v_i}{n-m}

o−o∖R=∑i∈R(vi−μ)n−m.o-o_{\setminus R} =\frac{\sum_{i\in R}(v_i-\mu)}{n-m}.

Thus the exact squared batch disturbance is

E(R)=∥∑i∈R(vi−μ)∥22(n−m)2.E(R)=\frac{\left\Vert \sum_{i\in R}(v_i-\mu)\right\Vert _2^2}{(n-m)^2}.

For m=1m=1, this reduces to E({j})=∥vj−μ∥22/(n−1)2E(\lbrace j\rbrace )=\Vert v_j-\mu\Vert _2^2/(n-1)^2, so removing the smallest full-cache-centroid distance exactly minimizes single-token disturbance in this reference.

3. Why bulk selection differs. Write di=vi−μd_i=v_i-\mu and si=∥di∥2s_i=\Vert d_i\Vert _2. Expanding the numerator gives

E(R)=∑i∈Rsi2+2∑i<j,i,j∈Rdi⊤dj(n−m)2.E(R)=\frac{\sum_{i\in R}s_i^2 +2\sum_{\substack{i<j,\quad i,j\in R}}d_i^\top d_j}{(n-m)^2}.

Positive cross terms reinforce disturbance; negative cross terms cancel it. Individual score ranking does not optimize these interactions, so it need not minimize the exact joint error.

4. What fixed-centroid ranking guarantees. The exact error contains ∥∑i∈Rdi∥22\Vert \sum_{i\in R}d_i\Vert _2^2, the squared norm of the summed deviations. To obtain a bound in terms of individual ValueDiff scores si=∥di∥2s_i=\Vert d_i\Vert _2, first apply the triangle inequality:

∥∑i∈Rdi∥2≤∑i∈R∥di∥2.\left\Vert \sum_{i\in R}d_i\right\Vert _2 \leq\sum_{i\in R}\Vert d_i\Vert _2.

Next, apply Cauchy–Schwarz to the mm scalar norms ∥di∥2\Vert d_i\Vert _2 and mm ones:

(∑i∈R∥di∥2⋅1)2≤(∑i∈R∥di∥22)(∑i∈R12)=m∑i∈Rsi2.\left(\sum_{i\in R}\Vert d_i\Vert _2\cdot1\right)^2 \leq \left(\sum_{i\in R}\Vert d_i\Vert _2^2\right) \left(\sum_{i\in R}1^2\right) =m\sum_{i\in R}s_i^2.

Squaring the first inequality, combining the two, and dividing by (n−m)2(n-m)^2 yields

E(R)≤(∑i∈R∥di∥2)2(n−m)2≤m(n−m)2∑i∈Rsi2=:U(R).E(R) \leq\frac{\left(\sum_{i\in R}\Vert d_i\Vert _2\right)^2}{(n-m)^2} \leq\frac{m}{(n-m)^2}\sum_{i\in R}s_i^2 =:U(R).

Thus, individual scores enter through the upper bound; their squared sum does not equal the exact batch error.

For fixed mm and centroid μ\mu, the prefactor is constant. Removing the mm smallest scores therefore minimizes this separable upper bound.

This statement uses distances to the original full-cache centroid μ\mu. The implementation instead scores eligible tokens against their centroid cc, excluding protected sink and recent tokens. Since

∑i∈R(vi−μ)=∑i∈R(vi−c)+m(c−μ)\sum_{i\in R}(v_i-\mu) =\sum_{i\in R}(v_i-c)+m(c-\mu)

the same inequalities give

E(R)≤(m∑i∈R∥vi−c∥22+m∥c−μ∥2)2(n−m)2.E(R)\leq \frac{\left( \sqrt{m\sum_{i\in R}\Vert v_i-c\Vert _2^2} +m\Vert c-\mu\Vert _2 \right)^2}{(n-m)^2}.

For a fixed cache and removal count, the centroid-offset term is constant. Removing the smallest implemented scores therefore minimizes this upper bound too, without establishing exact joint optimality. We will include this derivation and clarify its uniform-attention condition in the manuscript.

Response to Q1

We examined how attention flatness and output-to-centroid discrepancy relate to the VD–FastCAOTE gap. We measure them using normalized attention entropy and normalized distance from the attention output to the value mean:

H=−∑i=1nαilog⁡αilog⁡nH=-\frac{\sum_{i=1}^n\alpha_i\log\alpha_i}{\log n}

D=∥o−vˉ∥21n∑i=1n∥vi−vˉ∥22D=\frac{\left\Vert o-\bar v\right\Vert _2} {\sqrt{\frac1n\sum_{i=1}^n\Vert v_i-\bar v\Vert _2^2}}

o=∑i=1nαivi.o=\sum_{i=1}^n\alpha_i v_i.

Higher HH indicates flatter attention; higher DD indicates greater discrepancy from the centroid. We measured dense-prefill attention on 150 RULER-8K prompts per model, averaging five positions, query heads and softmax-attention layers. The performance gaps below average all 13 RULER tasks at 8K context and a 2K cache budget.

ModelNormalized entropyNormalized discrepancyVD−FastCAOTE (pp)
Llama3.1-8B0.3040.506+8.90
Qwen3-4B0.3610.524+22.54
Qwen3.5-4B0.5610.598+28.37
Gemma2-2B0.4610.519+15.86
Gemma2-9B0.4080.546+23.32

Across the five evaluated checkpoints, higher normalized attention entropy and larger output-to-centroid discrepancy accompany larger VD−FastCAOTE gaps, with Pearson correlations of 0.710 and 0.846, respectively. These observations suggest that attention flatness and centroid discrepancy are relevant to understanding model-dependent performance.

Response to Q2

Yes, on both evaluated checkpoints. We evaluated an actual-output CAOTE comparator using the last available query's attention output,

o=∑i=1nαivio=\sum_{i=1}^n\alpha_i v_i

including the contributions of protected sink and recent tokens. Results cover all 13 RULER tasks at 8K context and a 2K cache budget. Differences are percentage points with 95% paired bootstrap confidence intervals (20,000 replicates).

ModelValueDiffActual-output CAOTEVD−CAOTE (pp), 95% CI
Qwen3-4B-Instruct91.2072.62+18.58 [16.32, 20.88]
Qwen3.5-4B92.9656.71+36.25 [33.12, 39.39]

ValueDiff retains its advantage on both models, with both difference intervals above zero.

Response to Q3

We evaluated KeyDiff (KD) without explicit initial-token protection on both Llama checkpoints at 8K context and 2K/4K cache budgets. RULER-8K results are as follows:

ModelCacheKD, sink=0KD, sink=1Difference (pp)
Llama3.1-8B2K91.7185.86+5.84
Llama3.1-8B4K94.7091.34+3.36
Llama3.2-3B2K86.5985.57+1.02
Llama3.2-3B4K91.9289.65+2.26

KeyDiff retains strong performance without explicit initial-token protection on both strong-sink Llama checkpoints. The sink=0 runs score higher than the historical sink=1 controls, although runtime and template differences prevent attributing the gap solely to sink protection.

Reviewer_zQCjOpenReview ↗

Thanks for the rebuttal

Thank you for the detailed rebuttal. I also appreciate the clarification of the dispersion statistic and the distinction between minimizing a batch-error upper bound and minimizing the exact batch error.

The maximum-entropy derivation provides a valid interpretation under the chosen prior. The entropy measurements and five-checkpoint correlations provide empirical context, but they leave open whether that prior helps explain the method’s relative performance across models.

Q3 also remains open. Running KeyDiff without explicit sink protection shows that it can perform well under that configuration. My question was whether giving KeyDiff and ValueDiff the same sink protection, under otherwise matched conditions, reduces the performance gap between them on strong-sink models. The reported KeyDiff-only comparison cannot estimate that gap.

Reviewer_wFkpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the decoding time KV cache eviction under the given budget and proposes the new ValueDiff metric to identify un-important tokens for future generation. The proposed metric works well for new attention architectures including Qwen3.5 and GPT-OSS, demonstrating that the method is simple and effective. The authors also validate the theoretical optimum of ValueDiff from the max-entropy interpretation under the assumption of the future uniform attention vector.

Critical strengths
  • The proposed metric is simple and effective, especially for the new gated attention and learned sink architecture.
  • Several baselines are evaluated under three widely used long input context and long reasoning benchmarks.
Critical weaknesses
  • The statement "Prior token-eviction work commonly compresses the full-prompt KV cache once before generation, without decoding-time eviction" in the background section is in-accurate, as the KV cache compression methods during both the prefilling and decoding stages stages are widely studied.

  • The theoretical assumption of the future uniform attention vector does not hold in most cases, which challenges the analysis in section 3.3.

Questions for authors
  • Could you please explain why the method does not work well for the standard, logic softcapping, and QK-normalization models?
1 comment

Response to W1

We agree that eviction during decoding is established: TOVA explicitly removes a token at each decoding step once cache capacity is reached (Oren et al., 2024, Section 4). The manuscript also cites KeyDiff and CAOTE as precedents for block prompt processing. Our use of “commonly” refers to one-shot evaluation protocols, rather than a claim of priority for continuous eviction. We will clarify the background wording and our setup: we apply eviction after each prompt block and each decoding step, with the retained cache bounded in both phases.

Response to W2

Our study starts from empirical eviction results. The analysis helps interpret the algorithm’s behavior, as reflected in the Section 3.3 title, “A Max-Entropy Interpretation of ValueDiff.” We agree that attention is generally nonuniform.

Nevertheless, our measurements show weaker initial-token sinks and flatter overall attention on the evaluated Qwen checkpoints than on Llama3.1.

We measured attention on 150 RULER-8K prompts per model. Initial-token mass measures the strength of the attention sink at token zero; normalized entropy measures how evenly attention is distributed over all available tokens, with one corresponding to uniform attention.

ModelInitial-token massNormalized attention entropy
Llama3.1-8B57.68%0.3043
Qwen3-4B-Instruct37.22%0.3607
Qwen3.5-4B2.92%0.5612

These observations provide context for the interpretation without establishing uniformity. We will clarify the role and assumptions of the analysis in the manuscript.

Response to Q1

We would like to clarify that ValueDiff matches or exceeds the reported scores of the strongest comparison baselines on the evaluated QK-normalized models and the logit-softcapped Gemma2-9B. Gemma2-2B is an exception, where KeyDiff performs better. We included the standard Llama checkpoints as controls, where we expected key-based eviction to outperform ValueDiff under our geometric interpretation.

We group models by explicit architectural mechanisms to organize the comparison, independently of the measured statistics. These groups exhibit associations with attention-sink strength and representation geometry, although the statistics and eviction performance also vary within each group. We will clarify that this classification provides context for the empirical analysis, rather than isolating the effect of architecture.

The table reports RULER-8K accuracy retention relative to dense performance (%) at 2K/4K cache budgets. The comparison method is the strongest baseline excluding ValueDiff's norm-only and direction-only ablations.

BOS cosine measures alignment between the initial token’s key and the content-key centroid. σK/σV\sigma_K/\sigma_V compares the standard deviations of ManifoldKV’s key-centroid distance scores and ValueDiff’s value-centroid distance scores. This raw ratio is scale dependent.

FamilyModelBOS cosineRaw σK/σV\sigma_K/\sigma_VValueDiff (2K / 4K)Comparison methodComparison (2K / 4K)
StandardLlama3.1-8B−0.6043.05988.1 / 95.0KeyNorm91.8 / 95.5
StandardLlama3.2-3B−0.5862.53188.8 / 94.7KeyDiff92.9 / 97.4
Logit softcappingGemma2-2B−0.3501.13689.6 / 95.7KeyDiff96.4 / 98.7
Logit softcappingGemma2-9B−0.4241.00099.4 / 100.0KeyDiff99.0 / 99.3
QK-normalizationQwen3-4B+0.1821.02695.2 / 98.0ManifoldKV92.7 / 98.0
QK-normalizationGemma3-4B+0.5551.49387.6 / 96.1ManifoldKV87.1 / 94.6
  • Standard models. The Llama checkpoints favor key-based selection. Both have negative BOS cosine and σK/σV>2.5\sigma_K/\sigma_V>2.5; Llama3.1 also has stronger initial-token attention sinks in our attention measurements. We interpret these observations as consistent with key geometry remaining a useful eviction signal on these checkpoints. Value-centroid distance alone need not identify the tokens most useful to selective future queries.
  • Logit softcapping. Gemma2-2B favors KeyDiff, whereas Gemma2-9B is close to KeyDiff. Both have negative BOS cosine, while their key/value score-spread ratios are 1.136 and 1.000, respectively. This provides descriptive context for their different rankings; softcapping alone does not determine which eviction score works best.
  • QK-normalization. ValueDiff is competitive with the strongest comparison method on Qwen3-4B and Gemma3-4B. Both have positive BOS cosine, although their key/value score-spread ratios differ. The small Gemma3-4B 2K margin should be interpreted cautiously, as its paired confidence interval includes zero.
Reviewer_nVGFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript proposes ValueDiff, a training-free KV-cache eviction method that keeps the tokens whose value vectors lie farthest from the mean of the cached values. The method is aimed at models with suppressed attention sinks. It is evaluated under a fixed-memory inference protocol, with eviction applied during both blockwise prefill and decoding. The main contribution is empirical: in this setting, a value-based, query-independent score outperforms several attention-based and key-based baselines. The authors support this with a theoretical analysis linking eviction error to value dispersion, and with an observed association between attention-sink strength and the relative dispersion of values and keys.

Critical strengths
  1. The proposed method is simple and has been evaluated in a practically relevant setting. ValueDiff needs no training, no access to attention scores and no future queries, so it fits naturally with fused attention kernels and is easy to adopt. It is also evaluated with the memory budget enforced during prefill as well as decoding. Many eviction methods compress only after the full prompt has been processed, so this protocol is more faithful to real memory-constrained deployment. This strengthens the practical value of the reported gains.

  2. The experimental study is comprehensive. The experiments cover several architecture families and complementary benchmarks. They include direct comparisons with attention-based and key-based methods, and the direction-only and norm-only ablations help explain which part of the value geometry drives the gains. Together, these give reasonable confidence that the main effect is not specific to one model or task.

Critical weaknesses
  1. The geometric interpretation is not yet well justified. The association between weaker attention sinks and greater relative value dispersion is interesting, but a weak sink does not imply roughly uniform attention over content tokens, which the analysis seems to assume. More importantly, the ratio σ_V/σ_K is not invariant to representation scale. If the values are multiplied by a constant and the output projection is divided by the same constant, the model’s outputs and the ValueDiff rankings stay the same, but the ratio changes. Keys and queries can be rescaled in the same inverse way. Because the paper uses this ratio to separate “key-dominant” from “value-dominant” regimes and to explain when ValueDiff should be preferred, the claim needs a scale-invariant formulation or stronger justification.

  2. There is a gap between the theory and the implemented algorithm. The theoretical result concerns evicting a single token. The implemented policy evicts a block of tokens at once, ranked against a fixed centroid. The single-token guarantee does not establish optimality, or even a bound, for this block selection. The paper should state clearly which part is guaranteed and which part is heuristic, and it should assess the gap analytically or empirically. This matters because the theory is presented as the main justification for the scoring rule.

  3. The empirical evidence does not yet establish every claimed advantage robustly. The benchmark comparisons report no uncertainty estimates, such as bootstrap confidence intervals over test instances or variation across seeds and prompt orderings. Several margins over the strongest baselines are therefore hard to interpret. In addition, the full MATH-500 evaluation in Appendix G omits KeyDiff and the direction-only and norm-only variants. These are the comparisons most relevant to the paper’s central claim, so the evidence there is incomplete.

Questions for authors
  1. Is the proposed link between representation geometry and eviction performance still informative after controlling for representation scale, for example by using a scale-invariant dispersion measure or by normalising with the output and query projections? In addition, how closely do the measured future attention distributions match the assumptions in the theoretical analysis?

  2. Could the authors report uncertainty estimates for the main benchmark results, and add KeyDiff and the direction-only and norm-only variants to the full MATH-500 evaluation?

4 comments

Response to W1 and Q1

Thank you for distinguishing the empirical findings from their geometric interpretation.

We agree that scaling the value or key vectors can change σV/σK\sigma_V/\sigma_K without changing either method's token ranking. We inspect this ratio to compare the spread of the two eviction scores:

  • ValueDiff: siValueDiff=∥vi−vˉ∥2s_i^{\mathrm{ValueDiff}} = \Vert v_i-\bar{v}\Vert_2, with σV=Stdi(siValueDiff)\sigma_V = \mathrm{Std}_i(s_i^{\mathrm{ValueDiff}}).
  • ManifoldKV: siManifoldKV=∥ki−kˉ∥2s_i^{\mathrm{ManifoldKV}} = \Vert k_i-\bar{k}\Vert_2, with σK=Stdi(siManifoldKV)\sigma_K = \mathrm{Std}_i(s_i^{\mathrm{ManifoldKV}}).

Here, vˉ\bar v and kˉ\bar k are the value and key centroids. The ratio connects the geometry diagnostic to the token-selection rules. To address scale invariance, we further analyze BOS cosine, a scale-invariant metric already used in the paper.

BOS cosine and normalized attention entropy. The paper already investigates BOS key cosine in its experimental analysis and the RULER/LongBench/MATH-500 comparison figure:

BOSCosine=k0⊤kˉcontent∥k0∥2∥kˉcontent∥2\mathrm{BOSCosine} = \frac{k_0^{\top}\bar{k}_{\mathrm{content}}}{\Vert k_0\Vert_2 \Vert\bar{k}_{\mathrm{content}}\Vert_2}

kˉcontent=1n−1∑i=1n−1ki\bar{k}_{\mathrm{content}} = \frac{1}{n-1}\sum_{i=1}^{n-1}k_i

Here, k0k_0 is the initial token's key and nn is the number of cached tokens. BOSCosine measures its alignment with the mean of the remaining keys and is invariant to uniform key rescaling. We complement this existing geometry analysis with normalized attention entropy, defined as attention entropy divided by the logarithm of causal support size. Entropy is unchanged under compensating K/Q rescaling that preserves attention logits.

ModelBOS cosineRaw σV/σK\sigma_V/\sigma_KNormalized entropyVD−KeyDiff (pp)VD−ManifoldKV (pp)VD−FastCAOTE (pp)
Llama3.1-8B−0.6040.3270.304−1.37+26.13+8.90
Qwen3-4B-Instruct+0.1820.9750.361+5.41+2.36+22.54
Qwen3.5-4B+0.1151.3560.561+3.90+4.68+28.37
Gemma2-2B−0.3500.8800.461−5.95+33.55+15.86
Gemma2-9B−0.4241.0000.408+0.37+28.92+23.32

VD outperforms FastCAOTE and ManifoldKV on all five checkpoints. The VD–KeyDiff comparison varies across models:

  • Llama3.1-8B: Negative BOS cosine and a low σV/σK\sigma_V/\sigma_K accompany a KeyDiff advantage.
  • Qwen3-4B: Positive BOS cosine accompanies a VD advantage, even though σV/σK\sigma_V/\sigma_K is slightly below one.
  • Qwen3.5-4B: Positive BOS cosine and σV/σK>1\sigma_V/\sigma_K>1 accompany a VD advantage.
  • Gemma2-2B: Negative BOS cosine and σV/σK<1\sigma_V/\sigma_K<1 accompany a KeyDiff advantage.
  • Gemma2-9B: Negative BOS cosine and σV/σK≈1\sigma_V/\sigma_K\approx1 accompany near parity with KeyDiff.

BOS key alignment and relative score spread provide complementary context for these results. We will update the manuscript to clarify this complementary interpretation and the scale dependence of the raw score-spread ratio.

Response to W1 and Q1 — attention assumptions

We agree that weak attention sinks do not establish uniform or exchangeable attention over content tokens. Uniform attention and the symmetric Dirichlet prior are analytical reference settings for interpreting the ValueDiff score.

We measure attention statistics on 150 RULER-8K prompts. The table averages these measurements across prompts, positions, query heads and softmax-attention layers.

  • Initial-token mass measures the strength of the attention sink at the initial token. It is the attention probability assigned to token zero, averaged across the measured attention rows and reported as a percentage:

    Minitial=100Mean⁡p,ℓ,h,t[αt,0(p,ℓ,h)]M_{\mathrm{initial}} = 100 \operatorname{Mean}_{p,\ell,h,t} \left[\alpha^{(p,\ell,h)}_{t,0}\right] (%).

    Here, pp, ℓ\ell, hh and tt index prompts, softmax layers, query heads and the five sampled query positions. Larger values indicate a stronger initial-token attention sink.

  • Overall entropy measures how evenly attention is distributed over all available tokens, including the initial token:

    Hall=−∑i=0n−1αilog⁡αilog⁡n.H_{\mathrm{all}} =-\frac{\sum_{i=0}^{n-1}\alpha_i\log\alpha_i}{\log n}.

    Higher values indicate flatter overall attention; one means uniform attention.

ModelInitial-token massOverall entropyVD−KeyDiff (pp)
Llama3.1-8B57.68%0.3043−1.37
Qwen3-4B-Instruct37.22%0.3607+5.41
Qwen3.5-4B2.92%0.5612+3.90

In the evaluated checkpoints, the Qwen models exhibit lower initial-token mass (weaker attention sinks) and higher attention entropy (flatter overall attention) than Llama3.1. We hypothesize that weaker attention sinks and more diffuse attention can reduce the usefulness of key geometry as a proxy for token importance, making value-based eviction a more suitable alternative. VD's advantage over KeyDiff on the evaluated Qwen checkpoints is consistent with this motivation, although these comparisons do not isolate its cause.

We will update the manuscript to explicitly frame uniform attention and the symmetric Dirichlet prior as analytical reference settings, rather than consequences of sink suppression, and distinguish the observed flatter attention from the hypothesis about the usefulness of key-based eviction.

Response to W3 and Q2

We agree that small score margins require uncertainty estimates. We have computed paired bootstrap confidence intervals for two close RULER comparisons and completed the requested full MATH-500 comparisons for Gemma3-4B and Qwen3.5-4B.

Close RULER comparisons. At 8K context and a 2K cache budget, we use 20,000 paired bootstrap replicates, resampling matched instances within each of the 13 tasks and averaging task scores equally. Each comparison covers 650 instances. Scores below are raw RULER percentages; differences and 95% confidence intervals are in percentage points.

ModelComparison methodValueDiffComparison scoreVD−comparison (pp), 95% CI
Gemma3-4BManifoldKV80.5280.04+0.49 [−2.14, 3.13]
Gemma2-9BKeyDiff85.8785.51+0.37 [−1.31, 2.01]

Both intervals include zero, so these small observed gains do not establish clear superiority. We will add the uncertainty estimates to the manuscript and revise the table presentation and accompanying claims to make this clear.

Full MATH-500 comparisons. Each method is evaluated on all 500 problems with a 256-token cache. Scores are percentages using the flex-match metric. Gemma3 averages five sampled responses per problem; Qwen3.5 uses one greedy response per problem, following the evaluation procedure in the main manuscript.

ModelValueDiffKeyDiffNorm-onlyDirection-only
Gemma3-4B67.4459.6464.6463.16
Qwen3.5-4B73.2051.0066.6064.00

VD leads KeyDiff and both component ablations on these two full-suite comparisons.

Uncertainty estimates. We use 20,000 paired bootstrap replicates. For each replicate, we resample the same problems for VD and the comparison method, keeping each problem's responses together. The table reports VD minus the comparison method in percentage points, with 95% confidence intervals.

ModelComparison methodVD−comparison method (pp), 95% CI
Gemma3-4BKeyDiff+7.80 [5.72, 9.96]
Gemma3-4BNorm-only+2.80 [1.28, 4.36]
Gemma3-4BDirection-only+4.28 [2.52, 6.08]
Qwen3.5-4BKeyDiff+22.20 [18.40, 26.00]
Qwen3.5-4BNorm-only+6.60 [3.80, 9.40]
Qwen3.5-4BDirection-only+9.20 [5.80, 12.60]

We will update Appendix G with the completed KeyDiff, norm-only and direction-only comparisons and their confidence intervals.

Response to W2

The single-token result establishes exact optimality in the uniform-attention reference setting. For bulk eviction, fixed-centroid ranking minimizes an upper bound on disturbance under the uniform-attention assumption:

∥o−o∖R∥22=∥∑i∈R(vi−μ)∥22(n−m)2≤m(n−m)2∑i∈R∥vi−μ∥22.\left\Vert o-o_{\setminus R}\right\Vert _2^2 =\frac{\left\Vert \sum_{i\in R}(v_i-\mu)\right\Vert _2^2}{(n-m)^2} \leq\frac{m}{(n-m)^2}\sum_{i\in R}\left\Vert v_i-\mu\right\Vert _2^2.

Here, nn is the original number of cached tokens, RR is the removed set of size m<nm<n, and μ=n−1∑ivi\mu=n^{-1}\sum_i v_i is the original full-cache centroid. For fixed mm, removing the mm smallest centroid-distance scores minimizes the right-hand side. We will include this derivation and clarify the scope of these guarantees in the manuscript.

1. Exact disturbance for a removed set. For fixed values v1,…,vnv_1,\ldots,v_n, let RR contain m<nm<n removed tokens and let aR=∑i∈Rαi<1a_R=\sum_{i\in R}\alpha_i<1. The original and renormalized survivor outputs are

o=∑i=1nαivio=\sum_{i=1}^n\alpha_i v_i

o∖R=o−∑i∈Rαivi1−aR.o_{\setminus R}=\frac{o-\sum_{i\in R}\alpha_i v_i}{1-a_R}.

Subtracting gives the exact identity

o−o∖R=∑i∈Rαi(vi−o)1−aR.o-o_{\setminus R} =\frac{\sum_{i\in R}\alpha_i(v_i-o)}{1-a_R}.

For general attention, disturbance therefore depends on both the attention weights and how the removed deviations combine.

2. Uniform-attention reference. Set αi=1/n\alpha_i=1/n and define the original full-cache centroid μ=n−1∑ivi\mu=n^{-1}\sum_i v_i. Then o=μo=\mu, aR=m/na_R=m/n, and

o∖R=nμ−∑i∈Rvin−mo_{\setminus R} =\frac{n\mu-\sum_{i\in R}v_i}{n-m}

o−o∖R=∑i∈R(vi−μ)n−m.o-o_{\setminus R} =\frac{\sum_{i\in R}(v_i-\mu)}{n-m}.

Thus the exact squared batch disturbance is

E(R)=∥∑i∈R(vi−μ)∥22(n−m)2.E(R)=\frac{\left\Vert \sum_{i\in R}(v_i-\mu)\right\Vert _2^2}{(n-m)^2}.

For m=1m=1, this reduces to E({j})=∥vj−μ∥22/(n−1)2E(\lbrace j\rbrace )=\Vert v_j-\mu\Vert _2^2/(n-1)^2, so removing the smallest ValueDiff score exactly minimizes single-token disturbance in this reference.

3. Why bulk selection differs. Write di=vi−μd_i=v_i-\mu and si=∥di∥2s_i=\Vert d_i\Vert _2. Expanding the numerator gives

E(R)=∑i∈Rsi2+2∑i<j,i,j∈Rdi⊤dj(n−m)2.E(R)=\frac{\sum_{i\in R}s_i^2 +2\sum_{\substack{i<j,\quad i,j\in R}}d_i^\top d_j}{(n-m)^2}.

Positive cross terms reinforce disturbance; negative cross terms cancel it. Individual score ranking does not optimize these interactions, so it need not minimize the exact joint error.

4. What fixed-centroid ranking guarantees. The exact error contains ∥∑i∈Rdi∥22\Vert \sum_{i\in R}d_i\Vert _2^2, the squared norm of the summed deviations. To obtain a bound in terms of individual ValueDiff scores si=∥di∥2s_i=\Vert d_i\Vert _2, first apply the triangle inequality:

∥∑i∈Rdi∥2≤∑i∈R∥di∥2.\left\Vert \sum_{i\in R}d_i\right\Vert _2 \leq\sum_{i\in R}\Vert d_i\Vert _2.

Next, apply Cauchy–Schwarz to the mm scalar norms ∥di∥2\Vert d_i\Vert _2 and mm ones:

(∑i∈R∥di∥2⋅1)2≤(∑i∈R∥di∥22)(∑i∈R12)=m∑i∈Rsi2.\left(\sum_{i\in R}\Vert d_i\Vert _2\cdot1\right)^2 \leq \left(\sum_{i\in R}\Vert d_i\Vert _2^2\right) \left(\sum_{i\in R}1^2\right) =m\sum_{i\in R}s_i^2.

Squaring the first inequality, combining the two, and dividing by (n−m)2(n-m)^2 yields

E(R)≤(∑i∈R∥di∥2)2(n−m)2≤m(n−m)2∑i∈Rsi2=:U(R).E(R) \leq\frac{\left(\sum_{i\in R}\Vert d_i\Vert _2\right)^2}{(n-m)^2} \leq\frac{m}{(n-m)^2}\sum_{i\in R}s_i^2 =:U(R).

Thus, individual scores enter through the upper bound; their squared sum does not equal the exact batch error.

For fixed mm and centroid μ\mu, the prefactor is constant. Removing the mm smallest scores therefore minimizes this separable upper bound. The bound does not require updating the centroid after each removal. Fixed protected tokens restrict the eligible subsets without changing the argument.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.