acceptodds
Under review as a conference paper at ICLR 2027

CACHET: How to Make KV-Cache Sharing Carry What the Larger Model Read

Abstract

Collaborative inference lets a small language model use the understanding of a larger one. Rather than passing that understanding through lossy text, recent methods share the larger model's KV cache, reshaped by a learned translator for the small model. When both models read the same prompt, accuracy rises, as if the small model had received what the larger one read. Yet the state-of-the-art C2C fuser still retains half of its gain when handed the cache of an unrelated question, and its updates track the question rather than the cache. The gain is largely an effect of added parameters adapting the small model's own cache, which the fuser also reads. The collaboration is then capped at what an adapter can add, however much the larger model understood. We propose CACHET, a translator designed so that the small model's answer depends on what the larger model read. It reads only the larger model's cache, closing that shortcut, and is trained to reproduce, jointly across all heads, keys, and values, the cache the small model would have built from the same input. Because most of the key cache is common to all inputs, this training emphasizes the part that differs between inputs. The loss is kept throughout answer training, which alone would pull the transmitted cache off target. When only the larger model sees the passage that determines the answer, a 0.6B model assisted by a 4B one reaches 97.3% without training on this test and drops to 1.0% when given the cache of a passage supporting a different answer. On four benchmarks, CACHET recovers 87.5% of the accuracy gap to the larger model, against 36.0% for the C2C fuser.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, cross-model communication, KV cache translation, representation alignment

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_ueWbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies how to transfer a larger model’s KV cache to a smaller model while preserving input-dependent information. Through cache-replacement experiments and update analysis, the authors show that a substantial portion of C2C’s gains persists without the corresponding source cache. They propose CACHET, which jointly processes source keys and values across heads, introduces a K/V-aware mean–residual alignment objective, and retains alignment during task training. Experiments show improved multiple-choice accuracy and successful transfer of passage information unavailable to the receiver.

Critical strengths
  1. Useful analysis of source dependence. The cache-replacement controls distinguish accuracy improvements from reliance on the sharer’s current input. The counterfactual passage experiments provide complementary evidence that the transferred cache enables the receiver to answer questions about unseen passages.
  2. Substantial empirical support for the design choices. The paper evaluates multiple model pairs and random seeds, and includes controlled studies of alignment retention, residual weighting, and translator architecture. The parameter-matched separated-translator comparison is particularly useful for assessing the benefit of joint source processing.
Critical weaknesses
  1. The methodological advance over DLC appears incremental. Receiver-native cache reconstruction, source-only translation, and two-stage training are already present in closely related work. CACHET’s contribution primarily consists of architectural and objective refinements. Although the ablations support these refinements, DLC-SI and the plain-alignment variant already succeed on the counterfactual passage task.
  2. The evaluation leaves practical generation capabilities unclear. The reported tasks use multiple-choice scoring through answer-token logits after a forced answer prefix, rather than free-form generation. This establishes answer-selection performance but does not establish whether transferred caches support coherent explanations, faithful open-ended answers, or sustained generation. Consequently, the practical scope remains limited.
Questions for authors
  1. Free-form generation: Can the authors compare option-logit scoring with freely generated answers on the same evaluation items? Reporting answer accuracy, format compliance, and explanation consistency would help establish whether the gains extend beyond constrained answer selection.
  2. Scaling and layer mapping: What explains the accuracy decline when increasing the sharer from 14B to 32B? Could convergence diagnostics and reconstruction errors help identify whether the limitation comes from training budget, translator capacity, or the fixed layer mapping?
  3. Practical efficiency: How does CACHET compare with using the sharer directly in terms of both accuracy and end-to-end latency? The comparison with text relay is useful, but a sharer-only baseline would clarify the practical trade-off.
  4. Multi-agent applications: How well does CACHET work in a multi-agent task where agents have access to different documents and must combine their information to answer a question? A comparison with text-based communication in terms of task performance and end-to-end latency would help establish its practical value.
5 comments
AuthorsOpenReview ↗

Response to Reviewer ueWb (1/2): Contribution and scope

Thank you for recognizing the source-dependence analysis and architectural controls.

W1. CACHET connects the diagnosis of source-insensitive gains to a cache-alignment objective and validates the resulting design. We share the source-only interface, native reconstruction and two-stage training with DLC; the revision states this in §5. Our starting point is different: whether an interface's accuracy gain reflects use of the corresponding source input, or adaptation that persists without it. The common-pool Mean controls make this measurable: a constant prefix retains about 8 pp over the receiver alone, while the matched cache adds about 29 pp (R1-A). The receiver-native target then shows why total reconstruction error is not the right objective for the matched part: the batch-mean component accounts for 79.7% of K energy but 25.7% of V energy. CACHET therefore supervises mean and residual components separately, with separate K/V residual scales.

Three controls validate the design:

  • Joint K/V processing: a parameter-matched separated translator obtains PGR 62.0 ± 14.2%, versus 87.8 ± 2.2% for CACHET.
  • Alignment retention: DLC's generation phase uses only the generation loss. CE-only can match four-benchmark accuracy after LR tuning, while CACHET maintains 71.7–74.0% over a 30× LR range and scores 31.23 ± 0.62% versus 27.77 ± 0.83% on MMLU-Pro (7 seeds, higher in every seed; R1-A).
  • Mean–residual objective: on MMLU-Pro, PGR improves from 49.5 to 61.3 (83.8 to 87.8 on the four benchmarks) while total reconstruction error rises from 0.2932 to 0.3120; preserving input-specific variation differs from minimizing overall reconstruction error.

Against the strongest DLC setting we evaluated, CACHET scores 71.66% versus 66.91% on 4B→0.6B (7 seeds), and 77.97% (same-input) / 77.62% (mixed) versus 75.05% on 8B→1.7B (common seeds; R1-C). As you note, DLC-SI and plain alignment also pass the passage test; that test diagnoses input dependence, and the comparison between methods rests on these benchmarks and ablations.

Q4. Multi-agent tasks with different documents. Your proposed task would test evidence integration beyond our single-source experiments; combining distinct agents' documents remains untested. The new passage-only ranking (GR2-A) and two-turn prefix appending (UVDt Q3) address prerequisites for that setting: context can be transferred without the source seeing the question, and extended across turns without reprocessing.

AuthorsOpenReview ↗

Response to Reviewer ueWb (2/2): Generation, scaling and cost

W2/Q1. We evaluated generated answers on short-answer QA without a forced answer prefix. We agree that forced option scoring alone does not establish the accuracy, format compliance or explanation consistency of actual answer generation. We therefore kept the questions and options but removed the forced answer prefix on the same 2,764 items, using greedy generation capped at 64 tokens (3 seeds). After correcting termination training, generated-answer accuracy is 70.9 ± 3.5% under flexible extraction, close to the same model's forced accuracy of 72.93 ± 1.83%. Under this extraction rule, much of the short-answer selection performance carries over to generation.

This evaluation also exposed an omission in our training-target construction. The original targets lacked EOS, and the model often continued producing output after answering. Since 70.8% of targets were truncated at 64 tokens, we appended EOS only to the naturally ending 29.2% and retrained with the other Stage-2 settings unchanged.

Generation before and after the target correction (accuracy/termination %, mean ± seed SD):

ConditionForced accuracyFree: original extractionFree: flexible extractionEOS termination
Before73.20 ± 0.9357.75 ± 8.7971.7 ± 1.519.9 ± 10.7
EOS appended72.93 ± 1.8355.9 ± 13.970.9 ± 3.599.6 ± 0.5

DLC-SI, plain alignment and CE-only (LR 2e-6) trained on the same targets also terminate in only 1.3%, 19.8% and 26.1% of cases, respectively; a receiver-native cache through the same injection path terminates in 99.9%.

The original extraction rule remains primary; flexible extraction is a post-hoc diagnostic accepting unambiguous forms such as "B. [option text]". The correction improves termination rather than generated-answer accuracy. EOS termination rises from 19.9% to 99.6%, and from 0% to 93–98% on the 256-token explanation test.

We assessed format and explanation consistency separately. Pre-fix extraction rates are 80.9% under the original rule and 97.1% under the flexible one. A blinded LLM-assisted audit of 40 pre-fix outputs finds explanations supporting the final choice in 25/29/29 cases across seeds, versus 34 for the source; this is not a human evaluation. We thus report short-answer selection and termination, with explanation quality limited to this small auxiliary audit.

Q2. The submitted runs used a common mapping rule and training budget without 32B-specific optimization. With the 0.6B receiver fixed, seven-seed accuracy is 80.84 ± 1.06% for the 14B source and 77.93 ± 3.45% for 32B. We investigated whether the fixed mapping from the 32B source's 64 layers to the 28-layer receiver limits performance. For the lowest and median seeds from the original seven, we held translator architecture fixed and changed only the map or update budget:

32B settingLowest seedMedian seed
Submitted: 28 evenly spaced layers71.9579.21
All 64 layers, adjacent-layer averages78.8081.94
Last 28 layers83.0082.68
Submitted map, 2× updates83.0080.07

Both mapping changes and additional updates improve both seeds. The last-28 map reaches 83.00/82.68%, supporting the importance of mapping choice when using the larger source. This is an exploratory diagnostic on two seeds selected after the original results. Systematic mapping and training-budget optimization across model scales remains future work.

Q3. Handoff reduces fixed-length generation latency by up to 23.1%. At 256 output tokens, end-to-end latency is 5,022.1/6,533.9 ms (CACHET/source), a 23.1% reduction; at 64 tokens it is 1,277.9/1,475.9 ms, a 13.4% reduction. These measurements use two B200 GPUs at batch 1 (GR2-B). In the separate option-scoring task, CACHET achieves 73.20 ± 0.93% versus 78.37% for the source, or 93.4% of source accuracy. This does not measure the quality of the fixed-length outputs. Short scoring, naturally ending answers, memory and communication costs are reported in GR2-B.

AuthorsOpenReview ↗

Follow-up to Reviewer ueWb

Thank you again for your review. As the discussion period closes, we briefly point to what in the revision bears on your two main concerns.

Contribution relative to DLC (W1). On MMLU-Pro, removing any one design component costs 8–28 PGR points (Table 4, 4B→0.6B). From 61.3 for CACHET, PGR falls to 33.1 without joint K/V processing, 49.5 without the mean–residual objective, 53.1 without K/V-specific residual scales, and 49.4 without alignment retention even after tuning its learning rate. DLC-SI reaches 40.3.

Generation (W2/Q1). The receiver can generate from the transferred cache without a forced answer prefix. Outputs originally failed to terminate because our training targets lacked EOS; after correcting the targets, 99.6% of outputs stop on their own, with no loss in answer-selection accuracy (Appendix E).

We would be grateful to hear whether these address your concerns, and we are happy to answer any remaining questions.

Reviewer_hxqXOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks how to make KV-cache sharing actually transfer information from a larger model to a smaller one. The authors find that an existing method still retains substantial gains when given another question’s cache. This suggests that some improvement comes from the fusion module adapting the smaller model, rather than using information from the larger model’s current input. They propose CACHET, a translator that reads only the larger model’s cache and converts it into a form the smaller model can use. Its training emphasizes differences between inputs and keeps a cache-alignment objective active during answer training, helping preserve input-specific information. Experiments cover 12 Qwen3 model pairs and five standard benchmarks. With a 4B model assisting a 0.6B model, average accuracy across four benchmarks rises from 36.0% to 73.1%. More importantly, when only the larger model sees the answer-determining passage, the smaller model reaches 97.3% accuracy. Replacing that cache with one supporting a different answer reduces accuracy against the original answer to 1.0%, showing that its answers depend on the transmitted content. Overall, the results and ablations support the main claims: CACHET makes cache-based information transfer more effective, and its key design choices help.

Critical strengths

(1) Convincing evidence of actual information transfer. The paper goes beyond accuracy gains by testing whether the smaller model uses information available only to the larger model. The paired counterfactual passages keep the receiver’s question fixed while changing the answer-determining content, providing strong evidence that its answers depend on the transmitted cache.

(2) Well-controlled ablations supporting the method’s design. The paper systematically tests continued alignment, residual-aware weighting, and joint K/V processing. Comparisons from the same pretrained checkpoint and between parameter-matched architectures make the effects of these choices more convincing, while evaluations across multiple model pairs and training seeds strengthen the overall findings.

Critical weaknesses

(1) Limited evidence for transfer reliability beyond controlled retrieval tasks. The passage experiments convincingly establish input dependence, but primarily test structured factual retrieval. On the harder eight-option task, CACHET exhibits substantial variation across seeds (88.3% ± 12.6%). Evaluation on more diverse passages requiring evidence integration would strengthen confidence in the method’s reliability and broader usefulness. This limits the scope of the demonstrated benefits, although it does not invalidate the central information-transfer result.

Questions for authors

I have no more questions.

2 comments
AuthorsOpenReview ↗

Response to Reviewer hxqX

Thank you for recognizing the source-dependence analysis and controlled ablations.

W1. Transfer reliability across more diverse passages. Your concern is how consistently the demonstrated transfer supports broader evidence-integration tasks, particularly given the seed variation on the eight-option task. We added controls that strengthen the evidence for access to the source's passage information.

The paired passage task was designed to isolate input dependence: the receiver's question stays fixed while the answer-determining passage available only to the source changes. We extended this test by removing both the question and the options from the source input. With the source reading only the passage, the receiver reaches 85.3 ± 5.2% candidate-value ranking accuracy, versus 23.8% without the cache (GR2-A). The evaluator compares the full-sequence likelihoods of four candidate values, which are not listed in either model's prompt. This shows that the receiver can recover passage information without the source processing the question or seeing answer labels.

The matched self-source control provides complementary evidence: with identical translator architecture and training conditions, the 4B source exceeds the 0.6B self-source by 25.38 pp on average, with positive differences in all 28 seed×benchmark cells (GR1-A).

As you point out, the eight-option result of 88.3 ± 12.6% over seven seeds shows substantial variation. We will present the mean and seed variability together in the main text. Your proposed evaluation on more diverse passages requiring evidence integration is an important direction for assessing transfer reliability, and we will discuss this extension concretely in the revised manuscript.

Good luck to you. Personally, I think this is a good paper.

Reviewer_UVDtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CACHET studies how to make cross-model KV-cache sharing convey information specific to the source input. With both language models frozen, a source-only translator jointly processes source attention heads and K/V branches to construct a prefix for the smaller receiver. Training first aligns this prefix to the receiver's native cache for the same text, using a K/V-aware mean–residual objective, and then adds answer supervision while retaining alignment. Evaluation covers same-input multiple-choice benchmarks, passage tasks in which only the source model sees the supporting context, cache-substitution tests, and training and architecture ablations. Across Qwen3 model pairs, the paper reports improved accuracy over DLC-SI and transfer of passage information without additional training on those tasks. It also measures latency relative to text relay.

Critical strengths
  1. The paired passage experiment provides convincing evidence of source-dependent information transfer. The receiver's question and options remain fixed while the source passage changes the correct answer. On 4B→0.6B, accuracy is 97.3% with the matched cache and 1.0% against the original answer after a partner-passage swap (Table 3). The receiver never reads the passage directly, and the checkpoints receive no additional training for this task. This directly supports the paper's central claim that the interface can carry information absent from the receiver's input.

  2. The benchmark gains are consistent and supported by meaningful ablations. CACHET improves on DLC-SI across all nine principal model pairs on the four-benchmark aggregate and eight of nine on MMLU-Pro, with seven training seeds for each method. For 4B→0.6B, the improvements are 6.2 and 6.0 accuracy percentage points, respectively (Table 2). The parameter-matched Separated comparison supports joint source processing, and removing alignment during Stage 2 substantially reduces performance (Table 4). Together, these results support the empirical value of the proposed training and translator design.

Critical weaknesses
  1. The larger model's contribution to the accuracy gains is not clearly isolated. The alignment target is the receiver's own KV cache. If it were reconstructed exactly, the translated prefix and the native-cache prefix would produce the same predictions under the same execution path. The Task-only and alignment-removal ablations show that alignment matters, but they do not establish how much of the additional accuracy depends on the larger model's representations. Results after each training stage, a native-cache prefix baseline, and a matched translator that takes receiver-generated caches as input would help separate this contribution from the benefit of training the prefix pathway.

  2. The efficiency comparison does not establish a practical benefit. The reported 5.2–8.8× speedup is measured against text relay, which requires generating an intermediate message. This does not show an advantage over using the larger model directly. CACHET still incurs the cost of the larger model's prefill and cache translation. Without a comparison of answer quality, latency, and memory usage against direct larger-model inference, it remains unclear which workloads would benefit from this combined system.

Questions for authors
  1. Mechanism and attribution of the gain. How do receiver-native alignment and Stage 2 answer training jointly produce the observed improvement, and what evidence attributes part of it specifically to larger-model representations? A mechanistic account supported by targeted controls would be sufficient. For a representative pair, please report accuracy after each stage and compare with a native-cache prefix and a translator using receiver-generated source caches under matched architecture, data, and training budget. On passage tasks, direct passage input to the receiver would help separate information access from the benefit of the larger model. Stage-wise accuracy alone would not settle this attribution.

  2. Efficiency and intended workload. In the reported setup, how does CACHET compare with direct larger-model inference in answer quality, total latency, and peak memory, including source prefill, translation, and receiver processing? If the expected advantage depends on longer outputs or prefill reuse, please identify that assumption and provide a cost breakdown for a representative workload.

  3. Scope clarification: repeated interaction. Does the intended use require multi-turn cache reuse? If so, how are new messages or tool results incorporated, is the translated prefix appended or replaced, and must the larger model reprocess the full history? If the scope is single-transfer inference, stating this explicitly is sufficient. A new agent benchmark is not requested.

  4. Positional alignment. Appendices A.1–A.2 specify pre-RoPE source features, no explicit position input, and direct post-RoPE key prediction. When the first source layer is selected, identical tokens at different positions can have identical source features but different post-RoPE targets. Is this an intentional approximation, or is a position-handling step omitted from the description? Please clarify the source of positional information and the implications for alignment at these layers.

4 comments
AuthorsOpenReview ↗

Response to Reviewer UVDt

Thank you for identifying the controls needed to distinguish a learned prefix interface from the larger source's contribution. We added those controls and measured an explicit two-device handoff.

Q1. With a matched translator, the 4B source adds 25.4 pp over the self-source. You are right that the original ablations did not isolate the larger source's contribution. We therefore added a matched self-source control, alongside native-prefix and stage-wise evaluations (GR1-A). With identical translator architecture, initialization and training recipe, the 4B source exceeds the 0.6B self-source by 25.38 ± 2.96 pp, positive in all 28 seed×benchmark cells. Swapping caches costs 42.04 pp with 4B versus 4.20 pp with the self-source, so the larger-source configuration improves both accuracy and dependence on the corresponding input. Native-prefix accuracy is 38.71% and Stage 1 almost always predicts A: reconstructing the native target alone does not explain the final gain.

For your information-access control, the receiver scores 25.00% without the passage and 95.50% with it; CACHET scores 97.32 ± 5.82% over 7 seeds. The receiver can solve this task when given the information directly. Further, a source seeing only the passage, without question or options, supports 85.3 ± 5.2% value ranking versus 23.8% for the receiver alone (GR2-A): the receiver recovers passage information from the cache without any source-side question processing.

Q2. Handoff reduces fixed-length generation latency by up to 23.1%. At 256 output tokens, end-to-end latency is 5,022.1 ms for CACHET handoff versus 6,533.9 ms for the source alone, a 23.1% reduction. At 64 tokens, the corresponding times are 1,277.9 versus 1,475.9 ms, a 13.4% reduction. In a separate option-scoring task, accuracy is 73.20 ± 0.93% versus 78.37%, or 93.4% of source accuracy. This accuracy ratio does not measure the quality of the fixed-length outputs.

We agree that speedup over text relay alone does not establish an advantage over direct source inference. We therefore compared them on one server with two B200 GPUs, NVLink and batch 1 (GR2-B). The large source performs prefill and cache translation, while the smaller receiver decodes on a separate device; large-device computation is 30.6 ms per request at 64 output tokens and 32.1 ms at 256 tokens. This measures a heterogeneous prefill–decode handoff in which the smaller model continues from the larger model's computed state.

Faster receiver decoding amortizes the initial handoff cost, so latency reductions increase with output length over the measured range. Phase-specific batching and overlapping requests are extension directions. Component timings, memory, payload and short-output results are in GR2-B. We will restrict the earlier 5.2–8.8× claim to the comparison with text relay.

Q3. A two-turn check preserves performance while appending the prefix without reprocessing earlier history. After the passage turn, only the new question tokens are prefilled, translated and appended to the existing prefix.

Second-turn processingSource reprocesses prior historyLetter accuracy (%)Value ranking (%)
Translate and append new question tokensNo99.7586.67
Retranslate the whole historyYes99.7587.08

Letter accuracy is unchanged; the value-ranking difference is 0.41 pp. This supports incremental prefix appending in the tested two-turn setting. The main benchmark results use single-transfer inference; tool-result merging was not evaluated.

Q4. Position. Your point about the first layer is correct: identical tokens have position-independent source features there, so position-dependent targets cannot be represented exactly. This is an intended approximation, with no rotation step omitted, and we will state the restriction. Predicting pre-RoPE keys and then applying receiver RoPE gives 70.11 ± 2.66% versus 73.33 ± 0.92% for the current design (3 seeds, same evaluator).

AuthorsOpenReview ↗

Follow-up to Reviewer UVDt

Thank you again for your review. As the discussion period closes, we briefly point to what in the revision bears on your two main concerns.

Contribution of the larger model (W1/Q1). A small source cannot stand in for the larger one. With the 0.6B model as its own source, under the same translator and training recipe, accuracy is 47.7%; with the 4B source it is 73.1% (+25.4 pp, positive in all 28 seed×benchmark cells). The receiver is therefore drawing on what the larger model provides, not only on the trained translator. This control is now Table 5 in the main text.

Practical benefit (W2). In a two-GPU handoff, the larger model is occupied for about 30 ms per request and the receiver decodes; end-to-end latency falls by 13.4% at 64 tokens and 23.1% at 256 relative to the source alone, and the receiver stops on its own in 99.6% of outputs (GR2-B, GR2-C; Appendix E).

We would be grateful to hear whether these address your concerns, and we are happy to answer any remaining questions.

Thank you for the additional experiments. With the translator architecture and training setup held fixed, switching the cache source from the 0.6B model to the 4B model improved average accuracy across the four benchmarks by 25.4 percentage points. This addresses my main question about whether the larger model contributes to the accuracy gains. However, when training starts from the same Stage 1 checkpoint, using only the answer loss in Stage 2 achieves comparable average accuracy once the learning rate is tuned. I therefore remain uncertain how much additional benefit comes from retaining the alignment loss on these benchmarks.

My concern about practical efficiency has been only partly addressed. For short answers that end naturally, CACHET and the larger model alone have nearly identical latency. CACHET does reduce latency for longer outputs, but it remains unclear how the two approaches compare in answer quality under those same conditions. The additional experiments give me greater confidence that the transferred cache carries useful information. However, I still find the methodological advance and the demonstrated practical benefits limited, so I am keeping my original rating.

AuthorsOpenReview ↗

Response to Reviewer UVDt's follow-up

Thank you for reading the additional experiments and for stating clearly what they do and do not resolve.

On the alignment loss: we agree that on the four benchmarks, answer-loss-only Stage 2 reaches comparable accuracy once its learning rate is tuned. The difference appears on the harder MMLU-Pro subset, where retaining alignment is higher in all seven seeds (PGR 60.4 ± 4.6 vs. 49.4 ± 3.0; § 4.3). Retaining alignment also keeps accuracy stable over a 30× learning-rate range.

On efficiency: our comparison for longer outputs measures cost, and answer quality at those lengths is outside the current evaluation (§6). Although the larger model alone is more accurate on the four benchmarks (78.4% vs. 73.2%), CACHET keeps the sharer's GPU busy for only about 30 ms per request instead of for the entire generation, which frees it to serve other requests.

Thank you again for your time and careful reading.

Reviewer_uAk7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether learned KV-cache transfer from a larger frozen sharer to a smaller frozen receiver actually carries the sharer's input. An audit of the released C2C fuser (Qwen3-4B→0.6B) replaces the sharer's cache with its own (Matched), another question's (Swap), or a training average (Mean). It suggests much of C2C's gain is receiver-side adaptation.

CACHET is a source-only, token-wise MLP gθg_\theta (≈260M params). It maps the sharer's pre-RoPE K/V from all heads jointly to a receiver prefix T^i=gθ(Si)\hat T_i = g_\theta(S_i). Stage 1 aligns the prefix to the receiver's own cache Ti=Cr(xi;Pi)T_i = C_r(x_i;P_i) with a K/V-aware mean–residual (MR) loss that upweights residuals (λres=12\lambda_{\text{res}}=12). Stage 2 keeps that loss and adds answer loss: Lalign+0.3Ltask\mathcal L_{\text{align}} + 0.3 \mathcal L_{\text{task}}.

Evaluation covers 12 Qwen3 pairs on ARC-C, OpenBookQA, MMLU-Redux, C-Eval, and MMLU-Pro subsets, using zero-shot letter-logit scoring. On 4B→0.6B, CACHET recovers 87.5% of the receiver–sharer accuracy gap (PGR). This compares with 73.0% for DLC-SI (the authors' reimplementation of DLC) and 36.0% for C2C. On a counterfactual passage task where only the sharer sees the passage, it scores 97.3% Matched vs. 1.0% Swap.

Critical strengths

S1. A clean causal test of input dependence. The paired counterfactual passages keep the receiver's tokens identical; only the sharer's passage differs. Together with the Matched/Swap/Mean controls, this isolates transfer better than accuracy gains alone: 97.3% vs. 1.0%, and 88.3% vs. 8.6% on eight-option retrieval (chance 12.5%). The protocol is reusable whether or not CACHET is the best design.

S2. Consistent, transparently reported gains. CACHET beats DLC-SI on the four-benchmark average in all 9 pairs with 0.6B/1.7B receivers, over seven seeds. A parameter-matched per-head variant shows joint K/V access matters (PGR 62.0 vs. 87.8). The paper also reports results against itself: DLC-SI also passes the counterfactual test, and reconstruction error doesn't predict accuracy. Finally, it shows a source-only prefix can work, whereas C2C's projection-only ablation fell to ~21%.

Critical weaknesses

W1. The core claim — that gains come from what the sharer read, not from adaptation or answer relay — is not established.

  • The alignment target carries no sharer information. With the same input, the optimum T^i=Ti\hat T_i = T_i only re-encodes xix_i, which the receiver already reads. Missing controls: Stage-1-only accuracy, a native-prefix oracle (prepend TiT_i itself), and a 0.6B→0.6B self-pair. C2C's own same-model control kept most of its gain.
  • The adapter shortcut is shrunk, not closed. A fixed output-mean prefix still adds +11.8/+6.8/+7.9+11.8/+6.8/+7.9 points (ARC/OBQA/Redux, Table B.4), 75–83% of C2C's absolute input-mean gain. Stage 2 mainly moves the input-independent component (Eμ,K ⁣:0.068→0.110E_{\mu,K}\colon 0.068\to0.110, Eμ,V ⁣:0.072→0.126E_{\mu,V}\colon 0.072\to0.126; residual errors change ~2%, Table D.2). The two Mean controls also differ: input-side vs. output-side, 2,000 rows vs. 50 prompts.
  • Answer relay is not ruled out.
    • On answer-only multiple choice, the sharer alone is better (78.4 vs. 73.1) and cheaper: CACHET runs the same sharer prefill plus a translator and a receiver.
    • The text-relay (T2T) baseline uses C2C's prompt, which forbids giving the answer.
    • For every sharer, the 0.6B receiver matches or beats the 1.7B receiver, as expected if it mostly reads out the sharer's answer.
    • In the passage task the sharer also sees the question and options, and no receiver-only or receiver-with-passage results are given.

Weakness: this is the title claim and the main difference from C2C.

W2. Comparisons are confounded, and the novelty is incremental.

  • DLC-SI looks under-tuned.
    • Its learning rates sit at the grid edge (12/12 pairs in Phase 1, 7/12 in Phase 2) and were selected after only 150 updates.
    • The passage-task gap to CACHET is 24.8 points in pairs where Phase 2 picked 10−310^{-3}, but 0.9 points where it picked 2×10−42\times10^{-4}.
    • With DLC's own mixed Phase-II training, 8B→1.7B gives 75.05 vs. CACHET's 75.10 (App. A.5); Table 2 reports 75.7 vs. 68.3.
  • The other baselines are weak.
    • C2C was retrained on 10,468 rows; the original recipe used 500k.
    • C2C-C, which recovers 76–86% of the gap on the same pair (C2C arXiv v1, Table 8; different protocol), is omitted.
    • Task-only training (2.7±26.12.7\pm26.1 PGR) is untuned.
  • CACHET's hyperparameter selection is unstated, while ARC-C and OBQA "informed development" (App. A.6).
  • Novelty. DLC already has source-only translation, receiver-native targets, two-phase training, and transfer without receiver context. LCF-X already handles differing contexts. The abstract compares against C2C (36.0%), not the strongest baseline (73.0%).

Weakness: the headline margins rest on these baselines.


Other issues

  • Table B.1 lists 33.9±1.933.9\pm1.9 for both 8B→0.6B and 14B→0.6B (MMLU-Pro); possible copy error.
  • Inconsistent numbers: Table 1 (0.2657) vs. Table D.2 (0.2597); T2T token counts 22–48 (Table 2 caption) vs. 22.4–43.1 (Table B.3); DLC-SI+align 94.8→98.5 appears only in the text; Table 3 mixes 7- and 3-seed means (88.3 vs. 91.33).
  • Undefined terms: "sectioned prefill", "stem positions", "PLAN", "teacher-format failures".
  • §2.2 points to Appendix C for Table B.4, which is in Appendix B. "Tables 4 and 3" should be "Tables 3 and 4"; "(Chen et al., 2026) (Appendix A)" has a doubled parenthetical.
  • Abstract: "this test" should be "task", and "which alone" is ambiguous. Contribution 3 and the Conclusion have a dangling modifier.
  • Inconsistent names: MR&KV12 vs. MR & KV12; Phase 2 vs. Phase-II; solo / standalone gap.
  • Lester et al. (2021) is prompt tuning, not prefix tuning. OpenHermes-2.5 and WikiText-103 are not cited.
  • Reference titles lose capitalization after "?". Table A.4 has a stray ";". Limitations deserve their own section.
Questions for authors

Questions

  1. Where does the gain come from? Please report Stage-1-only accuracy, a native-prefix oracle (prepend TiT_i itself), and a 0.6B→0.6B self-pair. If the self-pair recovers most of the gain, the "larger model's knowledge" framing does not hold.
  2. Comparable controls. Please apply the same Mean control to both methods, and report how much gain the output-mean prefix retains with and without MR.
  3. Answer relay. Please report:
    • the sharer-alone accuracy and latency;
    • a baseline that relays the sharer's answer letter as text;
    • how often CACHET agrees with the sharer, and copies the sharer's errors;
    • bytes transferred per example.
  4. Passage task. Please rerun it with the sharer seeing only the passage, and add receiver-only and receiver-with-passage results. How was C2C applied to longer sharer inputs? The truncation in App. C.2 may drop the passage.
  5. Baselines.
    • Widen DLC-SI's learning-rate grid and select on final validation performance.
    • Run DLC with mixed Phase-II training on all Table 2 pairs.
    • Check DLC-SI against at least one number reported in the DLC paper.
    • Train C2C at its original data scale, and include or address C2C-C.
  6. Selection and statistics.
    • How were the learning rates, λres\lambda_{\text{res}}, λtask\lambda_{\text{task}}, and width chosen?
    • Do the results hold on benchmarks not used during development?
    • Does CE-only Stage 2 still collapse at smaller learning rates? Removing the alignment term changes AdamW's effective step size.
    • Please add item-level paired CIs and results on the unfiltered evaluation sets.

Clarifications

  1. Positions. The translator gets position-free (pre-RoPE) inputs but predicts post-RoPE keys at positions 0,…,ni−10,\dots,n_i-1. How is that position-dependent target learnable? Please ablate applying the receiver's RoPE afterward; CACHET's Stage-1 Eμ,KE_{\mu,K} is 3.3× DLC-SI's.
  2. Singleton coordinates. At batch size 16, 21.4% of coordinates have only one valid example, so they get only mean-term supervision. Per unit error they weigh ≈λresdmqdu⋅n−1n≈16.5×\approx\frac{\lambda_{\text{res}} d_m}{q d_u}\cdot\frac{n-1}{n}\approx 16.5\times (V) and 63×63\times (K) less than residual errors. The constants were calibrated at batch size 4 but training uses 16. Is this intended, and why not dataset-level per-position means?
  3. Geometry. "Updates track the question rather than the cache" conflicts with Dsrc=0.33D_{\text{src}}=0.33 (keys) / 0.410.41 (values), and the PCA explains only 16–17% of variance. Please give a receiver / source / interaction decomposition of Δij\Delta_{ij}, and report the C-Eval control from App. C.2.

Suggestions

  • Add generation, long-context, and multi-hop tasks where only the sharer sees the context, plus one cross-family pair (C2C supports Llama/Gemma sharers).
  • Try a functional alignment loss, e.g., matching attention outputs, or KL between next-token distributions with native vs. translated prefixes.
  • Discuss: LCF-X (a natural baseline for §4.2), XKV (concurrent), CIPHER, activation communication, LatentMAS, cache augmentation, KV cache steering, CALM, gisting/ICAE/xRAG, routing and speculative decoding.
7 comments
AuthorsOpenReview ↗

Response to Reviewer uAk7 (1/4): Attribution and training (Q1, Q2, Q6)

Q1/Q2 and the CE-only part of Q6 are addressed here; Q3/Q4 in R1-B, comparisons in R1-C, and Q7–Q9 in R1-D. Thank you for the targeted controls.

Q1. The larger source adds more than the self-source pathway. GR1-A reports Stage-1-only, native-prefix and matched self-source controls: +25.38 pp for 4B over self-source, positive in all 28 cells. The self-source recovers +11.73 pp but still loses 4.20 pp under swaps; we therefore do not equate that entire gain with source-independent adaptation.

Q2. A common prefix helps, but matched input contributes substantially more. We used the same first 2,000 valid training items for all Mean controls (seeds 0–2; released C2C: one checkpoint):

MethodMatchedSwapInput-MeanOutput-Mean
CACHET73.2031.2743.9943.77
Plain alignment retained71.5231.5046.8745.16
MR, residual weight 171.1429.4543.6445.74
Task-only, submitted LR37.0936.9839.7139.99
Released C2C52.4344.5046.55—
Equal-exposure C2C51.2240.4540.04—

Relative to receiver-alone 35.96%, a constant prefix retains about 8 pp; matched-minus-Mean is about 29 pp. Source-only processing prevents direct transformation of receiver caches, not this common-prefix benefit. Output-Mean retains 21.0% of the gain with MR versus 25.9% with plain alignment. Averaging C2C's full output erases the receiver question too; preserving R_i and averaging only Δ gives 46.12%/47.44% (released/equal-exposure).

MR is motivated by native-target geometry: batch means account for 79.7% of K energy and 25.7% of V energy. In the objective ablation, K/V residual errors decrease from 0.0948/0.3537 to 0.0732/0.3146, while plain reconstruction error increases from 0.2932 to 0.3120 and PGR improves from 83.8 to 87.8. This is the distinction between reconstruction error and accuracy that you noted in S2.

Q6. Lowering the LR rescues CE-only; alignment broadens stability and improves MMLU-Pro. We compared common Stage-1 checkpoints on the following Stage-2 grid (3 seeds). This is test-set sensitivity analysis, not validation selection.

LRMacro: CE-only / CACHETMMLU-Pro: CE-only / CACHETRecon. error: CE-only / CACHET
2e-667.7 ± 0.9 / 67.2 ± 2.625.5 ± 0.5 / 24.8 ± 0.80.39 / 0.29
2e-573.2 ± 1.6 / 73.1 ± 0.328.2 ± 0.9 / 27.9 ± 0.91.25 / 0.30
6e-565.4 ± 3.8 / 74.0 ± 0.425.9 ± 3.0 / 31.1 ± 0.41.86–2.13 / 0.31
1e-457.4 ± 4.0 / 72.4 ± 1.122.2 ± 3.2 / 31.2 ± 1.72.44–3.09 / 0.31
2e-452.3 ± 7.6 / 73.2 ± 0.919.7 ± 3.7 / 31.1 ± 0.65.55 / 0.31
6e-4— / 71.7 ± 1.6— / 29.1 ± 0.3— / 0.33

You were right that the reported CE-only drop depended on LR. At 2e-5, its 7-seed macro is 73.05 ± 1.04%, matching the submitted CACHET at 2e-4 (73.07 ± 0.88%). With alignment, macro stays at 71.7–74.0% over 2e-5–6e-4. At each method's best tested MMLU-Pro LR, 7-seed scores are 27.77 ± 0.83% versus 31.23 ± 0.62%, with CACHET higher in every seed. We will replace the blanket "collapse" claim with this LR-dependent result. Table 4's CE-only PGR 29.5 used an earlier smaller translator and plain Stage-1 initialization; the matched-configuration value is 38.6 ± 17.9, which we will correct.

Retuned Task-only without Stage 1 reaches PGR 17.3 ± 3.4, versus 87.8 ± 3.7 for CE-only from aligned initialization.

AuthorsOpenReview ↗

Response to Reviewer uAk7 (2/4): Relay and passage controls (Q3, Q4)

Q3. Cache transfer outperforms the tested text relays; content ranking provides a separate diagnostic. On the four benchmarks:

ConditionMacro accuracy
Source alone78.37
CACHET, seeds 0–273.20 ± 0.93
T-score: source-selected letter as text44.36
T-gen: source-generated answer as text57.93

T-score follows the source on only about 48% of items, so the gap reflects the receiver's use of the message, not absence of answer information in text. Source/CACHET agreement is 79.2–82.5%. Among source-wrong items, CACHET repeats the same error in 56.9–62.3% and answers correctly in 21.7–23.7%; among source-correct items it is wrong in 11.9–14.6%. Cache/text payloads are 9.09 MB/about 31 B; latency and memory are in GR2-B.

Agreement alone does not determine the mechanism. In GR2-A, a source that reads only the passage, with neither question nor options, supports 85.3 ± 5.2% value ranking versus 23.8% for the receiver alone. The receiver can thus recover passage information without source-side question processing or option labels. This evaluator-assisted ranking is distinct from free generation.

Q4. Passage-only succeeds under value ranking, but not under the original letter readout. GR2-A includes the requested passage-only, receiver-only and direct-passage controls. Partner-passage swapping in the original question-aware condition gives 0.17% against the original gold answer; a receiver-native passage cache through the same injection path gives 98.00%.

Passage-only scores 25.00% with letter scoring versus 85.3% with value ranking. With a fixed question-aware cache and only receiver options reordered, the output follows the source's original letter 99.83% of the time. This supports your concern about label dependence in the original readout, and the 0.6B-versus-1.7B pattern you noted points the same way. We will report it alongside the label-free content diagnostic and will not use letter accuracy alone to claim content use.

For C2C's passage control, there was no truncation: the answer span is in every source input but outside every fused window. We will describe this window restriction rather than infer a general inability of C2C to transfer content.

AuthorsOpenReview ↗

Response to Reviewer uAk7 (3/4): Baselines and selection (Q5, Q6)

Q5. The advantage remains after reselection, including comparison with DLC's stronger earlier mixed setting. We reselected learning rates using final-checkpoint option-NLL on 500 held-out items (seed 0), for DLC-SI, DLC-mixed and an expanded CACHET grid.

Setting4B→0.6B (3 seeds)8B→1.7B (seed count)
CACHET, submitted73.20 ± 0.9375.68 ± 1.23 (7)
CACHET, NLL-selected71.88 ± 0.9676.96 ± 1.79 (7)
CACHET, mixed—77.62 ± 1.04 (3)
DLC-SI, NLL-selected65.98 ± 0.4471.97 ± 0.33 (7)
DLC-mixed, NLL-selected67.13 ± 1.2171.65 ± 0.80 (7)
DLC-mixed, earlier LR—75.05 ± 0.61 (3)
C2C, equal data exposure, NLL-selected53.89 ± 1.82—

DLC-SI improves on 8B→1.7B from 68.33% to 71.97%. NLL selection lowers DLC-mixed accuracy, so we compare against its earlier 75.05%: on common seeds 0–2, CACHET scores 77.97% (NLL-selected same-input; +2.92 pp) and 77.62% (mixed; +2.57 pp). Your 75.05-versus-75.10 comparison used mixed DLC and submitted same-input CACHET; both recipes will be shown.

On 4B→0.6B over 7 seeds, NLL-selected CACHET (71.66%) exceeds NLL-selected DLC-mixed (66.79%) by +4.87 pp [3.72, 6.05], and the submitted DLC-SI (66.91%), the highest DLC value on this pair, by +4.75 pp. Candidate counts are 46/12/10 (CACHET/DLC-SI/DLC-mixed); the DLC search spans Phase-1 LR 1.667e-5–1.8e-2 and Phase-2 LR 2.222e-5–3e-3. The selection criterion is shared; tuning budgets are unequal.

These are paper-based DLC adaptations using CACHET's data and update budget, not numerical reproductions of DLC's original training and generation-scoring protocol; no DLC implementation is currently public, and overlapping pairs (8B→4B, 14B→4B) do not make the protocols identical. C2C likewise uses equal data exposure; the original 500k-scale recipe and C2C-C were not run. Mixed-recipe reselection covers two representative pairs.

Q6. We will document the development history and correct the untouched-benchmark statement. Submitted LR/λ_res/λ_task were inherited from development without recorded validation selection; width 4096 followed a four-benchmark sweep, and MMLU-Pro was also evaluated during development. We will correct App. A.6 and report the new comparison as validation-selected evidence on previously used benchmarks.

Full ARC-C/OBQA split macro is 78.57 ± 0.74% / 76.65 ± 1.86% (submitted/NLL-selected) for 4B→0.6B and 81.89 ± 0.28% / 84.14 ± 1.98% for 8B→1.7B. Seed variation and paired item intervals will be reported separately.

Contribution. The source-only interface, native reconstruction and two-stage training overlap with DLC. Our contribution connects Mean controls and target geometry to joint K/V processing, mean–residual alignment and its retention during task training (R1-A); position handling is tested in R1-D. The abstract will include the DLC comparison.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response (1/2): Where the gains come from

We thank the reviewers. We summarize the key questions raised in the reviews and the answers from our additional experiments:

  • Does the larger source account for the gain? Replacing the 0.6B source with 4B adds 25.4 pp under a matched translator and training recipe.
  • Does the cache convey passage content without answer labels? The receiver recovers passage information from the cache of a source that saw neither the question nor the options: 85.3% candidate-value accuracy, versus 23.8% alone (GR2-A).
  • Does handoff offer a cost advantage over the source alone? The two-GPU handoff reduces end-to-end latency by up to 23.1% for fixed-length generation (256 tokens), with a 13.4% reduction at 64 tokens (GR2-B).
  • Can the receiver generate answers without a forced answer prefix? After correcting the training targets, the receiver generates short answers from the cache without a forced answer prefix and stops on its own in 99.6% of outputs, while largely preserving its previous forced-scoring accuracy (R4 W2/Q1).
  • Does the advantage persist after baseline retuning? CACHET remains ahead of DLC on both evaluated pairs after validation-based reselection. Comparisons with the stronger earlier DLC-mixed setting are also reported (R1-C).

Macro averages ARC-C/OBQA/MMLU-Redux/C-Eval on the same 2,764 items; ± is training-seed SD; brackets are paired item-bootstrap 95% intervals.

A. Does the gain come from the larger source or from training an additional translator? (uAk7 Q1/Q2, UVDt Q1). Trained rows use 7 seeds with identical translator architecture (260.1M parameters), initialization per seed, data/order, optimizer, 1,320+1,320 updates and layer-map rule.

ConditionFour-benchmark macro (%)
Receiver alone35.96
Receiver-native prefix38.71
Stage 1, 4B / self-source24.20 ± 0.07 / 24.18 ± 0.01
Stage 2, 0.6B self-source47.69 ± 2.90
Stage 2, 4B source (CACHET)73.07 ± 0.88

The 4B–self-source difference is +25.38 ± 2.96 pp [23.63, 27.02], positive in all 28 seed×benchmark cells. Swapping in another item's cache costs 42.04 pp with 4B and 4.20 pp with the self-source, so the self-source's +11.73 pp is not wholly source-independent. Stage 1 almost always predicts A; reconstruction alone does not provide answering ability, and the native prefix is not a performance ceiling.

B. What do Stage-1 alignment and retaining alignment in Stage 2 each contribute? (uAk7 Q6, ueWb W1). Without Stage-1 alignment, task-only training reaches PGR 17.3 ± 3.4 at its best tested LR, versus 87.8 ± 3.7 for CE-only starting from the aligned checkpoint. Retaining alignment in Stage 2 keeps accuracy at 71.7–74.0% across a 30× LR range and gives 31.23 ± 0.62% on MMLU-Pro versus 27.77 ± 0.83% without it (7 seeds, CACHET higher in every seed). The common-grid comparison is in R1-A.

These controls are intervention-specific contrasts, not an additive causal decomposition. With the translator and training conditions matched, changing only the source to 4B adds 25.4 pp on average, with positive differences in all 28 cells.

AuthorsOpenReview ↗

General Response (2/2): Content, handoff cost and generation

Relevant to uAk7 Q3/Q4, UVDt Q1/Q2, hxqX, and ueWb W2/Q1/Q3/Q4.

A. Can the receiver recover passage content from the cache without answer labels? Passage content is accessible without source-side question processing. On 400 passage items (seeds 0–2), we removed the options and then the question from the source input. Candidate ranking scores the full token sequence of four candidate values; the no-option prompts list no candidates for either model. This was a post-hoc diagnostic, with prompts/scoring fixed before execution.

Source input / controlLetter scoringCandidate ranking
Passage + question + options99.75*85.8 ± 0.8
Passage + question—88.2 ± 4.7
Passage only25.0085.3 ± 5.2
Receiver alone25.0023.8
Receiver reads passage95.50100.0

*99.75 uses seeds 0–2; the 7-seed result is 97.32 ± 5.82.

The receiver recovers passage information from the transferred cache even when the source has seen neither the question nor the options: candidate ranking reaches 85.3%, versus 23.8% for the receiver alone. Source-side question processing is therefore not required for the transfer. On 138 items with a distractor sharing the answer's leading digit, accuracy is 79.0 ± 3.3%. The same cache gives 25% under letter scoring: content access and label readout differ (R1-B). Candidate ranking supplies four candidates through the evaluator and is distinct from free generation.

B. Does handoff offer a cost advantage over using the source alone? Fixed-length generation latency falls by up to 23.1% at 256 tokens, with a 13.4% reduction at 64 tokens. We measured large-model prefill and small-model decode on one server with two B200 GPUs (A: 4B + translator; B: 0.6B), NVLink, batch 1, 128 items × 3 repetitions. Fixed-length generation, median ms:

Output tokensSource aloneCACHET handoffLatency reductionDevice A computation, handoff
16370.5342.27.7%29.9
641,475.91,277.913.4%30.6
2566,533.95,022.123.1%32.1

Source prefill takes 23.4–25.7 ms, translation 4.4, pack, transfer and unpack 2.2, and receiver question processing 20.0. Per-token decode is 23.0–25.5 ms for the source versus 19.3–19.5 for the receiver. Faster decoding amortizes the initial handoff cost; these component timings imply a break-even at about 4.2–7.0 output tokens, not a direct measurement at those lengths.

Payload is 9.09 MB; peak memory is A 9.44 + B 1.46 GiB versus 8.73 GiB for the source alone in the same session. In the separate option-scoring evaluation, accuracy is 73.20 ± 0.93% for CACHET versus 78.37% for the source, or 93.4% of source accuracy. Accuracy is the four-benchmark macro on 2,764 items; the table measures fixed-length generation latency.

Short option scoring takes 117.0/121.8 ms (source/handoff), versus 218.6–321.1 ms for text relays. With the termination-corrected model, naturally ending short answers take similar time, 180.4/178.8 ms (source/handoff). We therefore distinguish the fixed-length latency reductions from naturally short responses. These are batch-1, in-server measurements.

C. Can the receiver generate answers beyond forced option scoring? On short-answer QA with questions and options provided, we removed the forced answer prefix and evaluated answer accuracy, format compliance and explanation consistency. Under flexible extraction, answer-selection performance approaches forced scoring. Results for both extraction rules and the before/after comparison correcting the missing termination token in the training targets are in R4 W2/Q1.

AuthorsOpenReview ↗

Revision uploaded: summary of changes

We have uploaded a revised manuscript. Changes are marked in blue. Table numbers 1–4 are unchanged; Table 5, Section 6 and Appendix E are new.

  • Abstract: "rather than the cache" → "the dominant source of variation (61.7% / 82.5%)"; "closing that shortcut" → "removing direct receiver-cache access"; DLC-SI added to the headline comparison.
  • §3.3, Table 1: rerun from the CACHET Stage-1 checkpoint under the Table D.2 protocol; the conclusion is unchanged.
  • §4.2: eight-option results with seed SD; the letter readout is reported alongside the passage-only, label-free ranking result (85.3% vs. 23.8%).
  • §4.3, Table 4: the CE-only row is corrected (PGR 38.6 ± 17.9; MMLU-Pro unchanged). A new paragraph reports the Stage-2 LR search: CE-only and CACHET are on par on the four benchmarks, and CACHET stays ahead on MMLU-Pro.
  • §4.3, Table 5 (new): matched self-source control (47.69% → 73.07%; +25.4 pp, positive in all 28 cells).
  • §5, §6: expanded Related Work; new Limitations section.
  • Appendix A: first-layer position restriction (A.2), development history (A.6), definitions of "sectioned prefill", "stem positions", "PLAN" and "teacher-format failures", citations and naming.
  • Appendix E (new): Mean controls on a common pool, Stage-2 LR grid, baseline reselection, passage-only ranking, generated answers with EOS supervision, two-GPU handoff, 32B layer-map diagnostic, position and calibration checks.

Four values in our earlier comments were corrected after re-checking against logs: the 16-token latency reduction is 7.7% (was 7.6%); the 2.2 ms component covers packing, transfer and unpacking; reconstruction error at LR 2e-5 is 0.30 (was 0.31); the 26.1% CE-only termination rate is at LR 2e-6.

We thank the reviewers again and are happy to answer further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.