- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SCAR repairs the KV cache of the surviving suffix after deleting a middle span. It trains one small corrector per layer that cross-attends from stale suffix states to deleted-span states and predicts residual updates to suffix K/V. All correctors run in parallel, avoiding backbone replay. Evaluated on Qwen3-8B and Llama-3.1-8B across seven task categories, SCAR beats 15%-budget selective recomputation on 13 of 14 backbone–category cells and runs 2.9–4.8× faster than exact recomputation on four timed Qwen settings.
Critical strengths
- Novel architecture: per-layer parallel correctors that decouple repair latency from backbone depth.
- Strong empirical results on suffix-carried-state tasks, especially compared to delete-and-shift and 15%-budget selective recomputation.
- Useful ablation: functional supervision outperforms direct KV MSE.
Critical weaknesses
- No analysis of how positional shift is handled. The corrector reads stale suffix states and deleted-span states and outputs K/V residuals. It is unclear whether it learns joint content+position correction or assumes pre-applied position correction. This is central to the method and needs clarification.
- No theoretical or empirical analysis of inter-layer error propagation. If layer ℓ’s repair is approximate, layer ℓ+1’s corrector still sees the original stale hidden state. Does error compound? A simple comparison of predicted residuals vs. true residuals per layer would help.
- KVEraser comparison is imperfectly controlled. Training SCAR and KVEraser on the same pool, or reporting on KVEraser’s fully covered subsets, would make the comparison fairer.
- Failure modes (natural counting, MuSiQue) are noted but not analyzed.
Questions for authors
- How does SCAR handle the positional shift after deletion? Is position correction applied before the corrector, or does the corrector learn it jointly?
- Can you provide an analysis of inter-layer error propagation? For example, compare SCAR’s predicted K/V residuals to the true residuals (−) at each layer, and show whether the gap grows with depth.
- Can you train KVEraser on the same data pool as SCAR, or at least report SCAR results on the subset where KVEraser’s checkpoint has full coverage?
- What is the exact corrector architecture (parameter count, heads, weight sharing) and training loss? The main text says “cache MSE” but the abstract mentions “functional supervision”.
- Why do natural counting and MuSiQue remain difficult?
2 comments
Thank you for your explanation. I increased my score to 3. Thanks.
Thank you for the detailed and constructive reviews.
Q1. "How does SCAR handle the positional shift after deletion? Is position correction applied before the corrector, or does the corrector learn it jointly?"
Position correction is applied separately from the learned corrector (stated in §3.2, "Separating position changes from learned repair" and Eq. 4). SCAR first removes the original RoPE rotation from the suffix keys (de-RoPE) and predicts the residual in pre-RoPE coordinates. It then re-rotates the corrected key to its edited position (re-RoPE).
Q2. "Can you provide an analysis of inter-layer error propagation? For example, compare SCAR's predicted K/V residuals to the true residuals at each layer, and show whether the gap grows with depth."
We extended App F with per-layer key and value L1, L2 and L∞ residuals, cosine similarity to the exact-recompute cache (exact cos), and correction-direction cosine (Δcos) for every method. Figures A1-A10 show that SCAR's error does not grow with depth: its key L2 stays around 0.2, and its key exact cos stays around 0.99 at every layer, consistent with our insight that KV cache can be repaired independently at each layer. SCAR also has relatively less residuals and better cosine similarity compared with other methods at middle and late layers.
Q3. "Can you train KVEraser on the same data pool as SCAR, or at least report SCAR results on the subset where KVEraser's checkpoint has full coverage?"
The "KVEraser matched" row in Tables 2–3 is already this. It is KVEraser's released recipe retrained on the same training pool as SCAR (stated in §4.1 and App B).
We also made the comparison strictly paired and updated the paper accordingly (§4, Table 2). The earlier 94–99% KVEraser coverage came from several requests where KVEraser(matched) outputs
<EOS>immediately, which our scorer had skipped. They are now counted as empty answers. Its GSM / VT / macro scores change from 0.403 / 0.323 / 0.566 to 0.399 / 0.305 / 0.563, and SCAR's lead on GSM, VT and LiC is now 25–48 points (was 24-48). The difference is small and does not impact SCAR's relative performance.Q4. "What is the exact corrector architecture (parameter count, heads, weight sharing) and training loss? The main text says 'cache MSE' but the abstract mentions 'functional supervision'."
These are specified in §3.2–3.3 and summarized here. Each corrected layer has:
That gives about 5.3M parameters per layer: 184.4M on Qwen and 163.4M on Llama, about 2% of the backbone. Training loss uses per-layer suffix-cache MSE. The ablations (Table 4) vary the query proxy, the span readout and the MLP depth.
Q5. "Why do natural counting and MuSiQue remain difficult?"
Natural counting and MuSiQue are hard for every approximate method for mainly two reasons. First, the required repair is spread across the suffix rather than concentrated on a few tokens, which limits selective recomputation methods: even exact states for half of the suffix (ISR r=.5) reach only 0.447 on Natural (Qwen). Second, both tasks require a larger repair than the tasks that dominate SCAR's training data (tally, NIAH, GSM, and VT). We acknowledge this limitation and variants of SCAR that target such tasks are an important focus of future work.