acceptodds
Under review as a conference paper at ICLR 2027

How Active Readout Paths Shape KV-Cache Sensitivity

Abstract

Across 64 shared QASPER document caches on Qwen2.5-1.5B, changing the question produces a median absolute same-entry log-sensitivity change of 0.573. We ask what a score must observe to predict the effect of a fixed KV perturbation. Controlled query substitution isolates reader dependence; an audited tail case gives a -fold change with storage held fixed. Under radial perturbations, static vector magnitude gives a mean within-reader position-rank correlation of , squared current-reader routing reaches 0.747, and the first-order last-query response (S4) reaches 0.778. In an eight-document matched perturbation-family control, S4 falls from 0.780 under radial perturbations to 0.403 under unit-Gaussian directions. In a separate seven-document length control, it falls from 0.807 at 512 tokens to 0.580 at 8,192 tokens; routing remains close. Tracking the same entry across questions requires more path information: on a matched path sample, complete-suffix differentiation raises layer-3 tracking from 0.100 for the last-query downstream-Fisher diagnostic to 0.698, with little gain at layer 23. Actual joint deletion reveals a parallel depth pattern: measured-sensitivity selection reduces output divergence beyond routing at layer 3, while no tested score gives a resolved improvement over routing at layer 23. Current routing supports position ranking; broader path coverage improves early-layer tracking, and joint deletion locates a corresponding early-layer gap in output preservation.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.