- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies latent communication between a larger frozen "sharer" model and a smaller frozen "receiver". It proposes common-state conditioning (CS-QKV). Per-layer MLPs map the sharer's key–value cache to a message m, which is added to the receiver's hidden state before its attention-input RMSNorm. The receiver's native Q/K/V projections are then applied to N(h+m), while the residual branch keeps h. Communication happens only during prefix processing; afterwards the receiver generates on its own. One interface is trained on OpenHermes for each of 15 larger-to-smaller Qwen3 pairs; the core No-H variant has about 0.51B parameters. The interfaces are evaluated on four multiple-choice tasks (OBQA, ARC-C, MMLU-Redux, C-Eval).
The Full variant exceeds default Cache-to-Cache (C2C) on 13 of 15 pairs. For 32B→0.6B, it raises the four-task mean from 32.40% (native receiver) to 75.84%. CS also outperforms a control that directly predicts Q/K/V increments on two anchor pairs. The evaluation is completed by cross-family pairs, a locked SciQ test, prompt-sensitivity checks and latency measurements.
Critical strengths
S1: The scale study is complete and transparently reported.
All 15 larger-to-smaller Qwen3 pairs are reported against the native receiver, default C2C and the sharer alone, including the pairs where CS loses. The prompts, answer extractor, confidence intervals and a locked SciQ test are documented in detail. This makes the empirical picture easy to check.
S2: The receiving-rule comparison is carefully controlled.
CS is compared with an interface that directly predicts Q/K/V increments. The two are trained with the same budget and have approximately matched parameter counts. On 8B→0.6B, the No-H advantage holds over three seeds (+5.16 ± 1.43 points). This is a reusable observation about how to inject external features into a frozen model.
Critical weaknesses
W1: The performance advantage is shown only against default C2C, not against text communication or the sharer itself.
The accuracy gains are measured against three references: the native receiver, default C2C, and the paper's own interface variants. Against the simplest alternatives, CS is less accurate:
- Sharer alone. CS is below the sharer on all 15 pairs, by 3.52 to 12.74 points (Table 1). For example, on 32B→0.6B, CS reaches 75.84% while the sharer alone reaches 88.58%.
- Text communication. When the sharer's short answer (about 6.6 tokens) is passed to the receiver as text, the receiver reaches:
- 77.44% on 8B→0.6B, versus 73.65% for CS;
- 76.74% on 8B→1.7B, versus 73.88% for CS (§7.1, Appendix D.5).
What the paper says about text communication. The paper states that "Text guidance exceeds CS on these task scores" and that "The comparison does not assign the latency of one protocol to the accuracy of another" (Appendix D.5). Neither an accuracy advantage nor a latency advantage over text communication is therefore established. Text communication is also tested only with the 8B sharer and is not mentioned in the abstract.
Even against C2C, only the default variant is compared credibly. The stronger C2C-C variant, as reimplemented here, scores 49.27% on 8B→0.6B, below the reproduced default C2C (51.31%). In the C2C paper, C2C-C is far stronger than the default (with a 4B sharer and an 8-token protocol). The authors evaluate their reconstruction "rather than reproducing the original paper's Table 9 configuration" (Appendix C.1).
The one favorable setting has no quality measurement. CS might pay off in long generation with a large sharer, where Figure 3 shows lower latency. But Figure 3 is measured with replayed token trajectories, and its caption states that "replay does not measure its answer quality". All quality evaluations are multiple-choice tasks with at most 64 generated tokens.
W2: Closely related latent-communication methods are neither cited nor compared.
The paper does not cite KVComm (Shi et al., ICLR 2026; arXiv:2510.03346), LatentMAS (Zou et al., arXiv:2511.20639), Interlat (arXiv:2511.09149) or Thought Communication (NeurIPS 2025; arXiv:2510.20733). KVComm and LatentMAS are designed for agents that share the same model or base model, so they do not apply directly to the larger-to-smaller pairs studied here. However, the paper already evaluates a self-sharer setting, in which the same model is both source and receiver (Table 13), and these methods could be compared there. At a minimum, the paper should discuss them. In addition, residual-stream activation communication (Ramesh and Li, 2025) is cited but not used as a baseline, although it is the most direct alternative to injecting the message before normalization.
W3: The native baseline is evaluated under a protocol that depresses it, which inflates the headline gains.
The scores are implausibly low. Under the T0 protocol (greedy decoding, at most 64 tokens, required answer format), native Qwen3-0.6B scores:
- 20.13% on C-Eval, below chance for four options;
- 32.95% on MMLU-Redux (Table 2).
For comparison, the Qwen3 technical report lists 42.6 and 44.6 for non-thinking Qwen3-0.6B on these benchmarks, under its own protocol.
Answer-format failures explain much of the gap. On the 1,024-question diagnostic panel, 17.97% of native 0.6B answers cannot be resolved under T0, versus 0.49% and 0.59% under T1 and T2 (Appendix D.4).
The gain depends on the template. Across the three templates, the gain of CS over native 0.6B ranges from 28.91 to 39.06 points, while CS itself varies by less than 0.6 points (Table 19, Appendix D.4). The authors acknowledge this protocol sensitivity and note that T0 uses a different batching mode (Appendix D.4).
The abstract's "+43.44 points over the native receiver" therefore partly reflects answer-format failures of the native model rather than communication.
Questions for authors
Q1: In what setting does CS beat both text communication and the sharer alone?
This is a requested experiment that could change my assessment. Please report text communication for all pairs, including the 32B sharer, and measure the latency of text communication and CS under the same protocol. Please also evaluate answer quality on at least one long-generation or reasoning task, where the latency advantage in Figure 3 would matter.
Q2: How does CS compare with KVComm, LatentMAS and residual-stream injection?
This is a requested experiment. Please compare against KVComm and LatentMAS in the self-sharer setting (Table 13), where they apply, and against residual-stream injection as in Ramesh and Li (2025) on the anchor pairs. Please also discuss why these methods are or are not applicable to the larger-to-smaller pairs.
Q3: How large are the gains over the native receiver and C2C under a protocol that does not penalize format failures?
This is a requested experiment. For the headline pairs, please report the native receiver, default C2C and CS under T1/T2 or under likelihood-based scoring, so that the gains are not driven by unresolved answers.
Q4: Why does the reimplemented C2C-C score below default C2C, contrary to the original paper?
This is a clarification request. Please explain the discrepancy. Ideally, please compare against C2C-C using the original configuration or the official checkpoints.
Q5: Is the receiver mostly reading out the sharer's answer?
This is a requested experiment. Please report the agreement rate between CS and the sharer, CS accuracy split by whether the sharer is correct, and a control in which the sharer's KV cache comes from a different question. On SciQ, CS finishes within 0.80 points of the sharer (l. 412–413), which is consistent with this interpretation.
9 comments
W1 & Q1. Practical value relative to text communication and the standalone sharer (2/2)
Across these four pairs, Full CS improves on default C2C by an absolute 13.12–24.87%, while taking less time than native sharer inference or text communication. For 32B→0.6B, it also improves on the native receiver in both accuracy (63.84% to 67.85%) and time (3.292 s to 2.005 s). At 1.7B, the receiver already reasons more accurately on its own: 32B→1.7B Full CS reaches 75.59% versus 82.79% for standalone receiver inference, with 40.3% less time. Its 2.246 s response time is also 72.2% lower than the 32B sharer's 8.089 s.
Text communication across model pairs. We evaluated text communication on OpenBookQA, ARC-C, MMLU-Redux, and C-Eval, referred to below as the main benchmarks. Text (≤256 tokens) passes a source solution; Text (≤512 tokens) passes a structured source message. Both conditions cover all 15 original Qwen3 pairs under T0 and five pairs under each corrected prompt, T1 and T2. T0 is the original answer instruction; T1 requests an option letter, and T2 requests Final answer: <letter>. Table 3 shows the three 32B-source pairs under T2 generated-answer scoring.
Table 3. Text and native-sharer comparisons on the main benchmarks with T2 generated-answer scoring. Cells show macro ACC (%) / Time (ms); Time is mean end-to-end latency. Each benchmark receives equal weight in ACC. Native sharer is the standalone reference.
| Receiver | Text (≤256 tokens) | Text (≤512 tokens) | Full CS | No-H CS | Native sharer | Sharer |
|---|---|---|---|---|---|---|
| 0.6B | 86.16 / 370.73 | 85.72 / 219.71 | 82.95 / 217.68 | 82.22 / 203.24 | 88.94 / 150.89 | 32B |
| 1.7B | 84.61 / 370.02 | 84.13 / 220.12 | 80.57 / 130.20 | 82.32 / 128.87 | 88.94 / 150.89 | 32B |
| 4B | 78.70 / 392.60 | 77.84 / 252.65 | 84.24 / 190.33 | 84.20 / 201.17 | 88.94 / 150.89 | 32B |
| … | … | … | … | … | … | … |
Both prompts and all five model pairs are reported in Appendix E.1 (Table 31) of the revised paper.
On 32B→4B, Full CS is more accurate and faster than both text conditions. On the two smaller receivers, text is more accurate and CS is faster, while native 32B has the highest accuracy in all three rows. Each text condition exceeds Full CS on 14 of the 15 T0 pairs; 32B→4B is the exception. Generating a text message adds source-decoding cost; CS transfers source features directly to the receiver. The revised abstract uses the 32B→4B example.
We have added the budget analysis to Section 7.1 and Table 7 of the revised paper, and the accuracy–time cases to Figure 3 in its main text. Appendix E.1–E.2 (Tables 29–32) gives the detailed text comparisons; Appendix E.3–E.4 (Tables 33–36) gives the reasoning results, selected configurations, and text-message feasibility.
W2 & Q2. Related work, communication baselines, and residual injection (1/2)
We have added the requested self-sharer and activation-transfer comparisons. Section 8 now discusses KVComm, LatentMAS, Interlat, and Thought Communication, including their communication representations and model-compatibility requirements.
Self-sharer comparison. We compared KVComm, LatentMAS, and Self-CS on the main benchmarks at 0.6B and 1.7B. Self-CS trains a CS interface with the same pretrained model in both roles. We followed KVComm's selective-sharing procedure and LatentMAS's four-role workflow.
Table 4. Self-sharer comparison on the main benchmarks with T1 generated-answer scoring. Native ACC is standalone macro accuracy; communication cells show macro ACC (%) / Time (ms), with mean end-to-end latency.
| Pair (Sharer→Receiver) | Native ACC (%) | KVComm | LatentMAS | Self-CS |
|---|---|---|---|---|
| 0.6B→0.6B | 41.69 | 42.57 / 60.44 | 41.48 / 510.39 | 46.21 / 161.35 |
| 1.7B→1.7B | 61.82 | 61.16 / 45.14 | 50.87 / 462.93 | 58.17 / 169.71 |
Self-CS improves on LatentMAS in accuracy by an absolute 4.74% and 7.30%, while reducing response time by 68.4% and 63.3%, respectively. At 0.6B, it also raises accuracy from KVComm's 42.57% to 46.21%. Among these communication methods, KVComm is fastest at both sizes and most accurate at 1.7B.
Activation communication and instruction sensitivity. AC transfers an intermediate source activation to the receiver. We select its source layer, receiver layer, and addition or replacement operation by negative log-likelihood on independent OpenHermes validation data, then freeze the configuration before testing. AC(W) learns a linear map from source to receiver activations using a mean-squared-error objective. Table 5 compares these methods on all four 8B/32B→0.6B/1.7B pairs using candidate-likelihood scoring, which selects the legal formatted answer with the highest summed token log probability.
Table 5. Activation-communication baselines under T1 and T2. Macro ACC (%) on the main benchmarks with candidate-likelihood scoring. Native receiver is the standalone reference.
| Receiver | Prompt | Native receiver | AC | AC(W) | Full CS | No-H CS | Sharer |
|---|---|---|---|---|---|---|---|
| 0.6B | T1 | 41.67 | 43.59 | 78.92 | 73.37 | 74.13 | 8B |
| 0.6B | T2 | 46.73 | 45.39 | 46.49 | 73.98 | 73.75 | 8B |
| 0.6B | T1 | 41.67 | 25.58 | 25.47 | 82.62 | 77.57 | 32B |
| 0.6B | T2 | 46.73 | 45.45 | 46.77 | 83.21 | 82.39 | 32B |
| 1.7B | T1 | 61.68 | 57.47 | 67.42 | 73.89 | 74.76 | 8B |
| 1.7B | T2 | 64.07 | 63.95 | 64.06 | 73.67 | 75.50 | 8B |
| 1.7B | T1 | 61.68 | 23.67 | 26.24 | 81.09 | 82.48 | 32B |
| 1.7B | T2 | 64.07 | 64.02 | 64.06 | 80.39 | 82.16 | 32B |
Full CS exceeds AC(W) in seven of the eight comparisons. Its four-task macro accuracy changes by less than an absolute 1% between T1 and T2 on each of the four pairs. On 8B→0.6B, AC(W) leads under T1 but falls from 78.92% to 46.49% under T2, while Full CS stays near 74%.
AC(W) transfers a mapped activation from the source's final input token. In an 8B→0.6B diagnostic on a 1,024-question subset of the main benchmarks, keeping the receiver at T1 and changing only the source from T1 to T2 reduces AC(W)'s candidate-likelihood macro accuracy from 79.49% to 23.34%. This diagnostic indicates that changes in the source instruction can affect receiver predictions through the transferred activation. When both models switch together between T1 and T2, Full CS retains similar accuracy with the interface fixed (Table 5).
The GSM8K comparison uses the common reasoning instruction and generated-answer scoring:
Table 6. GSM8K comparisons with a 0.6B receiver and a common reasoning instruction. Cells show ACC (%) / Time (s); Time is mean end-to-end latency. ACC scores generated answers.
| Receiver | AC(W) | Full CS | Sharer |
|---|---|---|---|
| 0.6B | 64.75 / 3.324 | 64.22 / 2.012 | 8B |
| 0.6B | 65.28 / 3.515 | 67.85 / 2.005 | 32B |
| … | … | … | … |
All four model pairs and the full baseline comparison are in Appendix E.3 (Table 33) of the revised paper.
For 8B→0.6B, Full CS reaches 64.22% versus AC(W)'s 64.75%, using 39.5% less time. For 32B→0.6B, it is both more accurate (67.85% versus 65.28%) and faster (42.9% less time). The efficiency advantage therefore also appears when evaluating generated reasoning responses.
We sincerely thank the reviewer for the careful assessment and detailed experimental suggestions. The revised paper includes corrected answer-format evaluation, additional communication baselines, reasoning and latency measurements, and source-information controls.
W1 & Q1. Practical value relative to text communication and the standalone sharer (1/2)
CS improves accuracy under a tight response-time budget. On GSM8K math word problems, Full CS achieves higher accuracy than text communication and the standalone sharer on all three fixed model pairs in Table 1 under a 2-second mean response-time budget. Full CS constructs messages from source KV features and the normalized receiver state; No-H CS uses source KV alone.
Table 1. GSM8K under a 2-second mean response-time budget. Cells show ACC (%) / Time (s); Time is mean end-to-end latency. Text passes the sharer's numerical answer with a 32-token source-message limit. Native sharer is the standalone reference.
Full fixed-model results are in Appendix E.4 (Table 35) of the revised paper.
For 8B→1.7B, Full CS reaches 72.63% at 1.735 s, compared with 39.27% for the native sharer, 31.77% for text communication, and 55.34% for default C2C. CS obtains source features in one prefill pass and uses the smaller receiver to generate reasoning.
How configurations were selected. For each fixed method/model, we searched answer-only and concise reasoning prompts with output limits of 128, 512, and 2,048 tokens. We selected the most accurate configuration within the mean time budget on held-out GSM8K training data and froze it before testing. For native 8B, validation selection favors answer-only (48.83% at 0.665 s): 128-token reasoning is less accurate (35.55% at 1.955 s), while 512-token reasoning exceeds the budget (93.75% at 2.956 s).
The text search included source-message limits of 32, 256, and 2,048 tokens; only the 32-token condition had a feasible validation configuration for these pairs. The selected Native and Text configurations produce answer-only responses; C2C and CS generate reasoning. Timing includes source computation, communication, and receiver generation, measured with a common subset, batching, and H200 allocation.
We also selected configurations across the evaluated model sizes and text-message conditions. At the 2-second budget, this gives test accuracies of 71.42% for No-H CS, 40.79% for native inference, and 39.88% for text communication. At 4 seconds, native reaches 94.31%, text 90.75%, and No-H CS 84.84%. The tighter budget favors generating reasoning with a smaller receiver; a larger budget lets the stronger standalone model produce more accurate reasoning responses.
Reasoning quality with a common instruction. We also evaluated generated reasoning responses with the same instruction and a 2,048-token receiver output limit. Table 2 reports answer accuracy and measured end-to-end response time.
Table 2. GSM8K with generated reasoning responses. Cells show ACC (%) / Time (s); Time is mean end-to-end latency. Text token limits apply to the source message; the receiver output limit is 2,048 tokens. Native columns are standalone references.
The comparison including AC and AC(W) is in Appendix E.3 (Table 33) of the revised paper.