acceptodds
Desk rejected at ICLR 2027

Beyond Tokens: Model Communication through Common States

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Language models can communicate through internal representations, but transfer depends on how the receiver uses them. We introduce common-state conditioning (CS-QKV): a feed-forward interface maps sharer key-value features to a continuous message added before a frozen receiver's attention-input normalization. Native projections construct the conditioned queries, keys, and values, while the direct residual carries the current layer input. Communication occurs during prefix processing; the receiver then generates independently. Across 15 larger-to-smaller Qwen3 pairs, Full CS exceeds default Cache-to-Cache (C2C) in 13. At the largest size gap, 32B0.6B, it raises four-task mean accuracy from 32.40% to 75.84%, gaining 43.44 points over the native receiver and 19.55 over C2C. Under the primary T0 protocol, single-run matched controls with sharer-KV-only inputs (No-H) gain 3.80 and 2.14 points over directly predicting Q/K/V increments. Receiving controls examine how native computations construct these updates; a separately trained first-order variant reaches 75.23% on 8B0.6B. Selected cross-family applications and a locked SciQ test extend the evidence, while prompt and readout comparisons delimit relative advantages. The results support using native normalization and projections to parameterize cross-model attention updates.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Cross-model knowledge transfer, KV representations, common-state conditioning, latent communication, LLMs

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XunmOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies latent communication between a larger frozen "sharer" model and a smaller frozen "receiver". It proposes common-state conditioning (CS-QKV). Per-layer MLPs map the sharer's key–value cache to a message m, which is added to the receiver's hidden state before its attention-input RMSNorm. The receiver's native Q/K/V projections are then applied to N(h+m), while the residual branch keeps h. Communication happens only during prefix processing; afterwards the receiver generates on its own. One interface is trained on OpenHermes for each of 15 larger-to-smaller Qwen3 pairs; the core No-H variant has about 0.51B parameters. The interfaces are evaluated on four multiple-choice tasks (OBQA, ARC-C, MMLU-Redux, C-Eval).

The Full variant exceeds default Cache-to-Cache (C2C) on 13 of 15 pairs. For 32B→0.6B, it raises the four-task mean from 32.40% (native receiver) to 75.84%. CS also outperforms a control that directly predicts Q/K/V increments on two anchor pairs. The evaluation is completed by cross-family pairs, a locked SciQ test, prompt-sensitivity checks and latency measurements.

Critical strengths

S1: The scale study is complete and transparently reported.

All 15 larger-to-smaller Qwen3 pairs are reported against the native receiver, default C2C and the sharer alone, including the pairs where CS loses. The prompts, answer extractor, confidence intervals and a locked SciQ test are documented in detail. This makes the empirical picture easy to check.

S2: The receiving-rule comparison is carefully controlled.

CS is compared with an interface that directly predicts Q/K/V increments. The two are trained with the same budget and have approximately matched parameter counts. On 8B→0.6B, the No-H advantage holds over three seeds (+5.16 ± 1.43 points). This is a reusable observation about how to inject external features into a frozen model.

Critical weaknesses

W1: The performance advantage is shown only against default C2C, not against text communication or the sharer itself.

The accuracy gains are measured against three references: the native receiver, default C2C, and the paper's own interface variants. Against the simplest alternatives, CS is less accurate:

  • Sharer alone. CS is below the sharer on all 15 pairs, by 3.52 to 12.74 points (Table 1). For example, on 32B→0.6B, CS reaches 75.84% while the sharer alone reaches 88.58%.
  • Text communication. When the sharer's short answer (about 6.6 tokens) is passed to the receiver as text, the receiver reaches:
    • 77.44% on 8B→0.6B, versus 73.65% for CS;
    • 76.74% on 8B→1.7B, versus 73.88% for CS (§7.1, Appendix D.5).

What the paper says about text communication. The paper states that "Text guidance exceeds CS on these task scores" and that "The comparison does not assign the latency of one protocol to the accuracy of another" (Appendix D.5). Neither an accuracy advantage nor a latency advantage over text communication is therefore established. Text communication is also tested only with the 8B sharer and is not mentioned in the abstract.

Even against C2C, only the default variant is compared credibly. The stronger C2C-C variant, as reimplemented here, scores 49.27% on 8B→0.6B, below the reproduced default C2C (51.31%). In the C2C paper, C2C-C is far stronger than the default (with a 4B sharer and an 8-token protocol). The authors evaluate their reconstruction "rather than reproducing the original paper's Table 9 configuration" (Appendix C.1).

The one favorable setting has no quality measurement. CS might pay off in long generation with a large sharer, where Figure 3 shows lower latency. But Figure 3 is measured with replayed token trajectories, and its caption states that "replay does not measure its answer quality". All quality evaluations are multiple-choice tasks with at most 64 generated tokens.

W2: Closely related latent-communication methods are neither cited nor compared.

The paper does not cite KVComm (Shi et al., ICLR 2026; arXiv:2510.03346), LatentMAS (Zou et al., arXiv:2511.20639), Interlat (arXiv:2511.09149) or Thought Communication (NeurIPS 2025; arXiv:2510.20733). KVComm and LatentMAS are designed for agents that share the same model or base model, so they do not apply directly to the larger-to-smaller pairs studied here. However, the paper already evaluates a self-sharer setting, in which the same model is both source and receiver (Table 13), and these methods could be compared there. At a minimum, the paper should discuss them. In addition, residual-stream activation communication (Ramesh and Li, 2025) is cited but not used as a baseline, although it is the most direct alternative to injecting the message before normalization.

W3: The native baseline is evaluated under a protocol that depresses it, which inflates the headline gains.

The scores are implausibly low. Under the T0 protocol (greedy decoding, at most 64 tokens, required answer format), native Qwen3-0.6B scores:

  • 20.13% on C-Eval, below chance for four options;
  • 32.95% on MMLU-Redux (Table 2).

For comparison, the Qwen3 technical report lists 42.6 and 44.6 for non-thinking Qwen3-0.6B on these benchmarks, under its own protocol.

Answer-format failures explain much of the gap. On the 1,024-question diagnostic panel, 17.97% of native 0.6B answers cannot be resolved under T0, versus 0.49% and 0.59% under T1 and T2 (Appendix D.4).

The gain depends on the template. Across the three templates, the gain of CS over native 0.6B ranges from 28.91 to 39.06 points, while CS itself varies by less than 0.6 points (Table 19, Appendix D.4). The authors acknowledge this protocol sensitivity and note that T0 uses a different batching mode (Appendix D.4).

The abstract's "+43.44 points over the native receiver" therefore partly reflects answer-format failures of the native model rather than communication.

Questions for authors

Q1: In what setting does CS beat both text communication and the sharer alone?

This is a requested experiment that could change my assessment. Please report text communication for all pairs, including the 32B sharer, and measure the latency of text communication and CS under the same protocol. Please also evaluate answer quality on at least one long-generation or reasoning task, where the latency advantage in Figure 3 would matter.

Q2: How does CS compare with KVComm, LatentMAS and residual-stream injection?

This is a requested experiment. Please compare against KVComm and LatentMAS in the self-sharer setting (Table 13), where they apply, and against residual-stream injection as in Ramesh and Li (2025) on the anchor pairs. Please also discuss why these methods are or are not applicable to the larger-to-smaller pairs.

Q3: How large are the gains over the native receiver and C2C under a protocol that does not penalize format failures?

This is a requested experiment. For the headline pairs, please report the native receiver, default C2C and CS under T1/T2 or under likelihood-based scoring, so that the gains are not driven by unresolved answers.

Q4: Why does the reimplemented C2C-C score below default C2C, contrary to the original paper?

This is a clarification request. Please explain the discrepancy. Ideally, please compare against C2C-C using the original configuration or the official checkpoints.

Q5: Is the receiver mostly reading out the sharer's answer?

This is a requested experiment. Please report the agreement rate between CS and the sharer, CS accuracy split by whether the sharer is correct, and a control in which the sharer's KV cache comes from a different question. On SciQ, CS finishes within 0.80 points of the sharer (l. 412–413), which is consistent with this interpretation.

9 comments

We sincerely thank the reviewer for the careful assessment and detailed experimental suggestions. The revised paper includes corrected answer-format evaluation, additional communication baselines, reasoning and latency measurements, and source-information controls.

W1 & Q1. Practical value relative to text communication and the standalone sharer (1/2)

CS improves accuracy under a tight response-time budget. On GSM8K math word problems, Full CS achieves higher accuracy than text communication and the standalone sharer on all three fixed model pairs in Table 1 under a 2-second mean response-time budget. Full CS constructs messages from source KV features and the normalized receiver state; No-H CS uses source KV alone.

Table 1. GSM8K under a 2-second mean response-time budget. Cells show ACC (%) / Time (s); Time is mean end-to-end latency. Text passes the sharer's numerical answer with a 32-token source-message limit. Native sharer is the standalone reference.

ReceiverC2CText (≤32 tokens)Full CSNo-H CSNative sharerSharer
0.6B40.94 / 1.79139.88 / 0.43365.88 / 1.78367.10 / 1.79140.79 / 0.31332B
1.7B55.34 / 1.68531.77 / 1.72472.63 / 1.73571.42 / 1.75439.27 / 0.5598B
1.7B58.15 / 1.76932.15 / 1.82173.69 / 1.73073.31 / 1.75440.79 / 0.31332B
…………………

Full fixed-model results are in Appendix E.4 (Table 35) of the revised paper.

For 8B→1.7B, Full CS reaches 72.63% at 1.735 s, compared with 39.27% for the native sharer, 31.77% for text communication, and 55.34% for default C2C. CS obtains source features in one prefill pass and uses the smaller receiver to generate reasoning.

How configurations were selected. For each fixed method/model, we searched answer-only and concise reasoning prompts with output limits of 128, 512, and 2,048 tokens. We selected the most accurate configuration within the mean time budget on held-out GSM8K training data and froze it before testing. For native 8B, validation selection favors answer-only (48.83% at 0.665 s): 128-token reasoning is less accurate (35.55% at 1.955 s), while 512-token reasoning exceeds the budget (93.75% at 2.956 s).

The text search included source-message limits of 32, 256, and 2,048 tokens; only the 32-token condition had a feasible validation configuration for these pairs. The selected Native and Text configurations produce answer-only responses; C2C and CS generate reasoning. Timing includes source computation, communication, and receiver generation, measured with a common subset, batching, and H200 allocation.

We also selected configurations across the evaluated model sizes and text-message conditions. At the 2-second budget, this gives test accuracies of 71.42% for No-H CS, 40.79% for native inference, and 39.88% for text communication. At 4 seconds, native reaches 94.31%, text 90.75%, and No-H CS 84.84%. The tighter budget favors generating reasoning with a smaller receiver; a larger budget lets the stronger standalone model produce more accurate reasoning responses.

Reasoning quality with a common instruction. We also evaluated generated reasoning responses with the same instruction and a 2,048-token receiver output limit. Table 2 reports answer accuracy and measured end-to-end response time.

Table 2. GSM8K with generated reasoning responses. Cells show ACC (%) / Time (s); Time is mean end-to-end latency. Text token limits apply to the source message; the receiver output limit is 2,048 tokens. Native columns are standalone references.

ReceiverNative receiverC2CText (≤256 tokens)Text (≤2,048 tokens)Full CSNo-H CSNative sharerSharer
0.6B63.84 / 3.29245.41 / 3.08583.55 / 6.85792.80 / 8.01264.22 / 2.01264.37 / 2.09093.48 / 4.9308B
0.6B63.84 / 3.29242.99 / 2.54386.73 / 9.73295.00 / 11.18167.85 / 2.00567.40 / 1.95195.30 / 8.08932B
1.7B82.79 / 3.76159.36 / 2.02090.14 / 7.22993.33 / 8.34572.48 / 1.97673.39 / 2.15593.48 / 4.9308B
1.7B82.79 / 3.76161.71 / 2.08092.87 / 9.95995.15 / 11.44675.59 / 2.24674.60 / 2.10295.30 / 8.08932B
………………………

The comparison including AC and AC(W) is in Appendix E.3 (Table 33) of the revised paper.

W1 & Q1. Practical value relative to text communication and the standalone sharer (2/2)

Across these four pairs, Full CS improves on default C2C by an absolute 13.12–24.87%, while taking less time than native sharer inference or text communication. For 32B→0.6B, it also improves on the native receiver in both accuracy (63.84% to 67.85%) and time (3.292 s to 2.005 s). At 1.7B, the receiver already reasons more accurately on its own: 32B→1.7B Full CS reaches 75.59% versus 82.79% for standalone receiver inference, with 40.3% less time. Its 2.246 s response time is also 72.2% lower than the 32B sharer's 8.089 s.

Text communication across model pairs. We evaluated text communication on OpenBookQA, ARC-C, MMLU-Redux, and C-Eval, referred to below as the main benchmarks. Text (≤256 tokens) passes a source solution; Text (≤512 tokens) passes a structured source message. Both conditions cover all 15 original Qwen3 pairs under T0 and five pairs under each corrected prompt, T1 and T2. T0 is the original answer instruction; T1 requests an option letter, and T2 requests Final answer: <letter>. Table 3 shows the three 32B-source pairs under T2 generated-answer scoring.

Table 3. Text and native-sharer comparisons on the main benchmarks with T2 generated-answer scoring. Cells show macro ACC (%) / Time (ms); Time is mean end-to-end latency. Each benchmark receives equal weight in ACC. Native sharer is the standalone reference.

ReceiverText (≤256 tokens)Text (≤512 tokens)Full CSNo-H CSNative sharerSharer
0.6B86.16 / 370.7385.72 / 219.7182.95 / 217.6882.22 / 203.2488.94 / 150.8932B
1.7B84.61 / 370.0284.13 / 220.1280.57 / 130.2082.32 / 128.8788.94 / 150.8932B
4B78.70 / 392.6077.84 / 252.6584.24 / 190.3384.20 / 201.1788.94 / 150.8932B
…………………

Both prompts and all five model pairs are reported in Appendix E.1 (Table 31) of the revised paper.

On 32B→4B, Full CS is more accurate and faster than both text conditions. On the two smaller receivers, text is more accurate and CS is faster, while native 32B has the highest accuracy in all three rows. Each text condition exceeds Full CS on 14 of the 15 T0 pairs; 32B→4B is the exception. Generating a text message adds source-decoding cost; CS transfers source features directly to the receiver. The revised abstract uses the 32B→4B example.

We have added the budget analysis to Section 7.1 and Table 7 of the revised paper, and the accuracy–time cases to Figure 3 in its main text. Appendix E.1–E.2 (Tables 29–32) gives the detailed text comparisons; Appendix E.3–E.4 (Tables 33–36) gives the reasoning results, selected configurations, and text-message feasibility.

W2 & Q2. Related work, communication baselines, and residual injection (1/2)

We have added the requested self-sharer and activation-transfer comparisons. Section 8 now discusses KVComm, LatentMAS, Interlat, and Thought Communication, including their communication representations and model-compatibility requirements.

Self-sharer comparison. We compared KVComm, LatentMAS, and Self-CS on the main benchmarks at 0.6B and 1.7B. Self-CS trains a CS interface with the same pretrained model in both roles. We followed KVComm's selective-sharing procedure and LatentMAS's four-role workflow.

Table 4. Self-sharer comparison on the main benchmarks with T1 generated-answer scoring. Native ACC is standalone macro accuracy; communication cells show macro ACC (%) / Time (ms), with mean end-to-end latency.

Pair (Sharer→Receiver)Native ACC (%)KVCommLatentMASSelf-CS
0.6B→0.6B41.6942.57 / 60.4441.48 / 510.3946.21 / 161.35
1.7B→1.7B61.8261.16 / 45.1450.87 / 462.9358.17 / 169.71

Self-CS improves on LatentMAS in accuracy by an absolute 4.74% and 7.30%, while reducing response time by 68.4% and 63.3%, respectively. At 0.6B, it also raises accuracy from KVComm's 42.57% to 46.21%. Among these communication methods, KVComm is fastest at both sizes and most accurate at 1.7B.

Activation communication and instruction sensitivity. AC transfers an intermediate source activation to the receiver. We select its source layer, receiver layer, and addition or replacement operation by negative log-likelihood on independent OpenHermes validation data, then freeze the configuration before testing. AC(W) learns a linear map from source to receiver activations using a mean-squared-error objective. Table 5 compares these methods on all four 8B/32B→0.6B/1.7B pairs using candidate-likelihood scoring, which selects the legal formatted answer with the highest summed token log probability.

Table 5. Activation-communication baselines under T1 and T2. Macro ACC (%) on the main benchmarks with candidate-likelihood scoring. Native receiver is the standalone reference.

ReceiverPromptNative receiverACAC(W)Full CSNo-H CSSharer
0.6BT141.6743.5978.9273.3774.138B
0.6BT246.7345.3946.4973.9873.758B
0.6BT141.6725.5825.4782.6277.5732B
0.6BT246.7345.4546.7783.2182.3932B
1.7BT161.6857.4767.4273.8974.768B
1.7BT264.0763.9564.0673.6775.508B
1.7BT161.6823.6726.2481.0982.4832B
1.7BT264.0764.0264.0680.3982.1632B

Full CS exceeds AC(W) in seven of the eight comparisons. Its four-task macro accuracy changes by less than an absolute 1% between T1 and T2 on each of the four pairs. On 8B→0.6B, AC(W) leads under T1 but falls from 78.92% to 46.49% under T2, while Full CS stays near 74%.

AC(W) transfers a mapped activation from the source's final input token. In an 8B→0.6B diagnostic on a 1,024-question subset of the main benchmarks, keeping the receiver at T1 and changing only the source from T1 to T2 reduces AC(W)'s candidate-likelihood macro accuracy from 79.49% to 23.34%. This diagnostic indicates that changes in the source instruction can affect receiver predictions through the transferred activation. When both models switch together between T1 and T2, Full CS retains similar accuracy with the interface fixed (Table 5).

The GSM8K comparison uses the common reasoning instruction and generated-answer scoring:

Table 6. GSM8K comparisons with a 0.6B receiver and a common reasoning instruction. Cells show ACC (%) / Time (s); Time is mean end-to-end latency. ACC scores generated answers.

ReceiverAC(W)Full CSSharer
0.6B64.75 / 3.32464.22 / 2.0128B
0.6B65.28 / 3.51567.85 / 2.00532B
…………

All four model pairs and the full baseline comparison are in Appendix E.3 (Table 33) of the revised paper.

For 8B→0.6B, Full CS reaches 64.22% versus AC(W)'s 64.75%, using 39.5% less time. For 32B→0.6B, it is both more accurate (67.85% versus 65.28%) and faster (42.9% less time). The efficiency advantage therefore also appears when evaluating generated reasoning responses.

Reviewer_9Ey3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a transformer to transformer model communication method, namely Common-State Conditioning (CS-QKV), that addresses how a stronger model's (Sharer) information can be transferred the smaller model (Receiver) during the prefill stage of LLMs. The idea is, for a layer of Sharer and Receiver respectively, the prefix text is passed into the Sharer, the key and value are computed, then passed into a trainable feed-forward network (Message Network) to get message embedding mm, and then added to the Receiver pre-QKV embedding hh and then normalized N(h+m)N(h+m). Then it is passed into the attention module and finally the layer's prefix KV cache is computed and saved for the Receiver to continue to the cached decoding phase.

Results on pairs of Receiver and Sharer models of the Qwen3 class show that the CS method outperforms the C2C method and closes the gap to the Sharer model on its own (Sharer decoding). Section 6 then compares how the result changes if the way the embedded Sharer KV cache and the Receiver pre-attention embeddings are combined. Section 7 then examines how the results change if the Receiver and Sharer come from different model classes.

Critical strengths
  • The idea is straightforward and the results seem strong, especially compared to C2C that was published at ICLR in 2026 (last year) and the performance gap is large in Figure 2 and Table 1. The datasets chosen are the same as the ones in C2C.
  • An adequate amount of ablation is done, especially over different Receiver and Sharer models, different ways to combine Sharer KV embeddings with Receiver embeddings
Critical weaknesses

I'm not familiar with the literature in this area so I could not spot any major issues with the methodology or results. But there are Clarity issues:. Overall, the clarity needs to be improved if it is to be presented as an ICLR 2027 publication. A lot of sentences are not written in a clearly understandable or logical manner (often why something is done is not explained) and often jargon is not explained before its introduction:

  • For example, the abstract is a little confusing. For example, "Full CS", "sharer", "receiver" are all undefined and could confuse a reader who are not familiar with the terminology. And "primary T0 protocol", "No-H". I'm guessing "predicting Q/K/V increments" also refers to the C2C fused KV caches (or not?). "how native computations construct these updates" - I'm not sure what this sentence means.
  • The introduction does not clearly describe the research question and motivation, for example the problem of Text2Text and Cache2Cache communication, why you would want to do this (e.g. information loss) and limitations of existing methods. NN is not defined in the introduction also. The advantage of the proposed method over existing methods is also not described clearly: why do this and what is there to gain?
Questions for authors
  • Table 1 is not so clear. What is the receiver used in the results? Why display half a table of No-H accuracy on each dataset only when there is no reference number to compare to? Might as well replace the left half with four-task macro-accuracy for the 0.6B receiver, and the right half with 1.7B receiver
4 comments

We thank the reviewer for the time and care devoted to the review. Your specific examples of unclear wording and terminology helped us identify where the presentation needs improvement.

Weaknesses

W1. Computation, design choices, and terminology

We agree that these terms need clearer definitions at first use.

Exposition and design rationale. CS (common-state conditioning) aims to improve a small model's answers using a larger model's internal representations. It assigns answer generation to the smaller model to reduce decoding cost. The source model (sharer) processes the input once and supplies its key–value (KV) features; the answer-generating model (receiver) incorporates these features through a learned interface and generates the answer. In 8B→0.6B, the sharer is Qwen3-8B and the receiver is Qwen3-0.6B. Both pretrained models stay frozen; only the interface is trained.

Full CS and No-H CS. The default Full CS interface constructs the message from the source's KV features and the receiver's normalized hidden state. No-H CS uses only the source's KV features as inputs to the message network. Both variants add the resulting message to the receiver's current hidden state before attention-input normalization.

T0 and the corrected evaluation. T0, T1, and T2 name prompt templates. We corrected T0's ambiguous answer-format instruction using T1 (an option letter) and T2 (Final answer: <letter>). Generated-answer scoring extracts the model's answer; candidate-likelihood scoring selects the legal formatted answer with the highest summed token log probability.

Direct-QKV, C2C, and the native computation. “Predicting Q/K/V increments” refers to our Direct-QKV control, which predicts additive changes to the receiver's query, key, and value (Q/K/V) vectors. C2C projects and fuses source and receiver KV caches. In CS, the interface predicts a message mm in the receiver's hidden space. For the current layer input hh, the receiver applies its attention-input normalization NN to h+mh+m, then its frozen WQW_Q, WKW_K, and WVW_V projections. These existing normalization and projection layers are what we meant by “native computations.”

We have revised the abstract and Section 1 to introduce the motivation and model roles, and Section 3 to define the message, normalization, attention projections, and direct residual.

W2. Research question, motivation, and measured benefit

Research question. How should source-model information enter a frozen receiver so that it improves the receiver's answers? This motivates our comparison of different ways to incorporate the same source features into the receiver. The revised Introduction states this question before introducing the architecture.

Why communicate; text and cache alternatives. Communication gives a smaller model access to information computed by a larger model for the current request. Text communication passes the source's generated answer or explanation, so the exchanged information is limited to that text and incurs the cost of producing and processing it. Cache communication provides access to internal source features after input processing. C2C learns to project and fuse KV caches, requiring its interface to construct KV states that the receiver can use. CS predicts a message in the receiver's hidden space and uses the receiver's own normalization and projections to construct the attention inputs.

Definition of N. NN is the receiver's frozen attention-input normalization, implemented as root-mean-square normalization (RMSNorm) in the Qwen3 models studied here. The revised Introduction defines it at first use.

Concrete benefit. On OpenBookQA, ARC-C, MMLU-Redux, and C-Eval, CS with a 32B source raises the 0.6B receiver's macro accuracy from 46.73% to 83.21% under T2 candidate-likelihood scoring; C2C reaches 58.09%.

On these benchmarks with T2 generated-answer scoring, 32B→4B Full CS reaches 84.24% at 190 ms, compared with 78.70% at 393 ms for Text (≤256 tokens) and 77.84% at 253 ms for Text (≤512 tokens). CS improves both accuracy and response time, with 2.06× and 1.33× speedups, respectively.

On GSM8K with a common reasoning prompt and a 2,048-token receiver output limit, 32B→0.6B Full CS reaches 67.85% generated-answer accuracy at 2.005 s mean end-to-end time, compared with 42.99% at 2.543 s for default C2C.

The revised abstract now includes the accuracy and text-communication examples. In the main text, Table 1 reports candidate-likelihood accuracy and Section 7.1 presents the text comparison in Figure 3 and the GSM8K results. The full GSM8K comparison is in Appendix E.3 (Table 33).

Questions

Q1. Receiver identity and comparable scores in Table 1

Receiver identity and macro layout. In 8B→0.6B, Qwen3-0.6B is the receiver that generates the answer. Table 1 below reproduces the corrected T2 comparison from Table 1 of the revised paper, with the receiver in the first column and the sharer in the last.

Table 1. Revised main-table layout with T2 candidate-likelihood scoring. Macro ACC (%) weights each of the four main benchmarks equally. Receiver ACC and Sharer ACC are standalone references.

ReceiverReceiver ACCC2CFull CSNo-H CSSharer ACCSharer
0.6B46.7353.4073.9873.7581.988B
0.6B46.7358.0983.2182.3988.9032B
━━━━━━━━━━━━━━━━━━━━━
1.7B64.0763.3973.6775.5081.988B
1.7B64.0765.9480.3982.1688.9032B
━━━━━━━━━━━━━━━━━━━━━
4B77.0477.0684.3984.2688.9032B

Per-task references. Table 1 in the revised main text organizes the five-pair T2 candidate-likelihood comparison by receiver. The complete 15-pair generated-answer scale study is retained in Appendix B.1 (Table 9). The task-level scores and native references are in Appendix B.1–B.2 (Tables 10–13), with T1/T2 macro comparisons in Table 14. Appendix E.1 (Table 31) reports generated-answer accuracy and latency under both corrected prompts.

Reviewer_9Ey3OpenReview ↗

Response to rebuttal

Many thanks to the authors for improving the manuscript, it reads a lot clearer now!

The results look strong compared to established baselines and on well-known benchmarks and I'm happy to raise the score to 3.

AuthorsOpenReview ↗

Response to Reviewer 9Ey3

Thank you for your helpful feedback and positive reassessment of our revised manuscript.

Reviewer_2YXROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how internal representations from one language model can be effectively incorporated into another model for latent communication. The authors propose Common-State Conditioning (CS-QKV), which maps sharer KV features into continuous messages injected before the receiver's attention-input normalization, allowing its native projections to construct the attention updates. Experiments across Qwen3 model pairs demonstrate improvements over existing communication methods, particularly for smaller receivers. The paper further investigates different receiving mechanisms through controlled comparisons and ablation studies.

Critical strengths

Clear and valuable research question. The paper clearly identifies an important problem in latent model communication: how the receiver should incorporate information from another model. This is a well-defined research question with meaningful implications for model collaboration.

Well-motivated methodology. The connection between the motivation and the proposed mechanism is clear and reasonable. The method builds on observations about native attention computations and provides a concrete approach to addressing the identified problem.

Relevant baseline comparisons. The authors compare their method with recent and relevant approaches, including C2C, and introduce controlled comparisons with Direct-QKV. These experiments provide evidence supporting the proposed receiving mechanism.

Insightful analyses. Beyond the main performance results, the paper investigates alternative receiving rules, normalization effects, and different communication configurations. These analyses offer useful insights into why and how the proposed approach works.

Critical weaknesses

I have two minor concerns that could be addressed to further improve the paper.

Presentation. The title appears somewhat broader than the specific contribution of the paper. In addition, the abstract does not sufficiently explain the motivation behind the proposed approach, particularly why the receiving mechanism deserves separate investigation. A clearer explanation would help readers understand the significance of the research question.

Baseline coverage. Although the paper includes relevant comparisons, the experimental evaluation could benefit from additional baselines, such as Latent Cache Flow (LCF). While LCF follows a different technical approach, comparing against it and discussing the potential complementarity between the two methods would provide a more comprehensive picture of the proposed framework.

Questions for authors

Could the authors clarify the motivation for the proposed receiving mechanism more explicitly in the abstract?

Could the authors provide additional comparisons or discussion involving other recent latent communication approaches, such as LCF?

4 comments

Weaknesses

W1. Title scope and motivation

Title and scope. The title's “Beyond Tokens” refers to communication through internal model representations. We have revised the abstract and Section 1 to state the specific contribution: conditioning a frozen receiver during prefix processing so that its native projections construct the attention inputs.

Why study the receiving computation. Latent communication allows a smaller model to use information computed by a stronger model while generating the answer itself. The receiving operation determines how that information changes its computation: the same source features can affect attention differently depending on where they enter. This matters when both models are frozen, since the receiver must use external features through its existing normalization and projections. We study whether common-state conditioning (CS), which changes the state from which the receiver computes queries, keys, and values (Q/K/V), provides an effective way to use those features.

W2. LCF comparison and complementarity

Matched-capacity comparison. We compared LCF with small CS interfaces at approximately 0.03B and 0.07B parameters. Our LCF baseline follows its joint-KV compression and gated cache-update design, with bottleneck widths 128 and 256; small CS uses source KV alone (No-H CS) with narrower message networks. Both methods follow our paper's training setup, with matched data and training budgets.

Table 1. LCF and CS at matched interface capacities. Macro ACC (%) across OpenBookQA, ARC-C, MMLU-Redux, and C-Eval under T0 generated-answer scoring. Each benchmark receives equal weight. The final CS column reports the scale-study reference from Table 9 in Appendix B.1 of the revised paper.

ReceiverLCF-128 (~0.03B)Small CS (~0.03B)LCF-256 (~0.07B)Small CS (~0.07B)Original No-H CS (~0.5B)Sharer
0.6B48.1454.0847.3852.5673.658B
0.6B49.7254.7551.4463.5271.6232B
1.7B61.5661.4561.6264.8973.728B

Table 2. End-to-end latency for the matched LCF/CS comparison. Mean Time (ms), including source prefill, interface execution, and receiver answer generation, measured on a common subset of the main benchmarks.

ReceiverLCF-128 (~0.03B)Small CS (~0.03B)LCF-256 (~0.07B)Small CS (~0.07B)Sharer
0.6B132.05151.21166.34163.078B
1.7B158.36156.48148.08146.788B

Accuracy and interface capacity. CS exceeds LCF on all three pairs at approximately 0.07B parameters and on two at approximately 0.03B. The small interfaces preserve CS's use of the receiver's frozen normalization and projections, showing that this receiving rule remains effective when the message network is substantially reduced. On 32B→0.6B, increasing capacity from approximately 0.03B to 0.07B improves CS by 8.77%, compared with 1.71% for LCF. Here, a wider message network improves how the same receiver uses the same source features. At approximately 0.07B, the two timed 8B-source pairs also have similar LCF/CS latency, so the accuracy gains come at comparable interface size and inference cost.

Complementarity. LCF compresses joint KV features and reconstructs receiver-cache updates; CS conditions the state used by the receiver's own attention projections. An LCF-style compressed representation could therefore supply the CS message map, combining a compact communication channel with this receiving operation.

We have added the LCF comparison and capacity analysis to Section 5 (Table 2), latency results and implementation details to Appendix B.3 (Table 15), and the complementarity discussion to Section 8 of the revised paper.

Questions

Q1. Make the abstract motivation explicit

Revised abstract. We have updated the abstract with the following text:

Latent communication allows a smaller language model to use a larger model's internal representations while generating the answer itself. Effective transfer depends on how the frozen receiver incorporates source features into its own computation. We introduce common-state conditioning (CS), a feed-forward interface that maps source key–value features to a message added before the receiver's attention-input normalization. The receiver's native projections construct the conditioned queries, keys, and values, while a residual connection carries the current layer input unchanged. Communication occurs while processing the input prefix; the receiver then generates independently. With candidate-likelihood scoring on OpenBookQA, ARC-C, MMLU-Redux, and C-Eval, CS raises Qwen3-0.6B's mean accuracy from 46.73% to 83.21% using Qwen3-32B as the source, with absolute accuracy gains of 36.48% over the receiver and 25.12% over default Cache-to-Cache (C2C). Across five Qwen3 source–receiver pairs, the mean absolute accuracy gain over C2C is 15.55%. For generated answers on these benchmarks, 32B→4B CS reaches 84.24% accuracy at 190 ms, compared with 78.70% at 393 ms for solution text (≤256 source tokens) and 77.84% at 253 ms for structured text (≤512 source tokens), giving 2.06× and 1.33× speedups, respectively. CS trains only the feed-forward interface and keeps both pretrained models frozen.

Q2. Additional recent latent communication approaches

Additional baseline comparison. Alongside LCF in Tables 1–2, Table 3 compares CS with default C2C, activation communication (AC) [1], and AC(W). AC transfers an intermediate source activation to the receiver. We select its source layer, receiver layer, and addition or replacement operation by negative log-likelihood on independent OpenHermes validation data, then freeze the configuration before testing. AC(W) learns a linear map from source to receiver activations using a mean-squared-error objective.

Table 3. Additional latent-communication baselines on the main benchmarks. Macro ACC (%) with candidate-likelihood scoring, which selects the legal answer with the highest summed token log probability. T1 requests an option letter; T2 requests Final answer: <letter>. Full CS uses source KV and the normalized receiver state; No-H CS uses source KV alone.

ReceiverPromptC2CACAC(W)Full CSNo-H CSSharer
0.6BT152.8843.5978.9273.3774.138B
0.6BT253.4045.3946.4973.9873.758B
0.6BT155.7225.5825.4782.6277.5732B
0.6BT258.0945.4546.7783.2182.3932B
1.7BT162.0657.4767.4273.8974.768B
1.7BT263.3963.9564.0673.6775.508B
1.7BT164.5923.6726.2481.0982.4832B
1.7BT265.9464.0264.0680.3982.1632B

Full CS exceeds default C2C in all eight conditions in Table 3 and AC(W) in seven. Its advantage over both baselines grows with source size under T2. Increasing the source from 8B to 32B raises Full CS accuracy by an absolute 9.22% for the same 0.6B receiver, compared with 4.68% for C2C and 0.28% for AC(W). The same ordering holds for the 1.7B receiver. In these comparisons, CS makes more effective use of the larger source while keeping the receiver size fixed.

The revised paper includes the AC comparisons in Appendix B.2 (Tables 13–14) and their implementation and validation-selection details in Appendix B.3 (Table 16). We have also expanded Section 8 to discuss these approaches alongside LCF.

References

[1] Vignav Ramesh and Kenneth Li. Communicating Activations Between Language Model Agents. ICML, 2025. (https://proceedings.mlr.press/v267/ramesh25a.html)

Thanks for the response and additional results. Although I cannot further improve the rating to 4, I would like to restate my support this paper for acceptance.

We sincerely thank you for your time, effort, and thoughtful feedback, which have helped us make the paper clearer and stronger.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their feedback and suggestions. In response to their comments, we have added experiments and revised the main text to make the research question clearer and the paper easier to follow. Below, we summarize the main updates, with further details provided in our responses to each reviewer.

Main updates in response to reviewer feedback:

  1. Clearer Writing and Research Motivation: We revised the main text to explain more directly what problem we address and why we study the receiving computation separately. We also clarified the roles of the sharer and receiver and how the CS operation works, with changes to the abstract, introduction, and method description. (2YXR-W1/Q1, 9Ey3-W1/W2)

  2. Related Methods and Communication Baselines: We added LCF comparisons at comparable interface sizes, activation-communication baselines, and residual-injection controls. We also included self-sharer comparisons with KVComm and LatentMAS. The revised discussion explains how these approaches relate to CS and where they apply. (2YXR-W2/Q2, Xunm-W2/Q2)

  3. Evaluation Clarity and Practical Benefits: We revised the answer-format evaluation and made the model identities and standalone references explicit in the main results. We also expanded the text-communication comparisons and added reasoning experiments on GSM8K. For efficiency, we now report measured end-to-end latency and comparisons under fixed mean response-time budgets. (9Ey3-W2/Q1, Xunm-W1/W3/Q1/Q3)

  4. Contribution of Source Information: We added analyses of sharer–receiver agreement and conditional accuracy, along with answer-text controls and wrong-question KV interventions, to examine the contribution of information supplied by the sharer. (Xunm-Q5)

Overall, the revised manuscript explains our central question more directly: how the receiving computation uses information supplied by a sharer. The additional comparisons and controls help readers assess CS alongside related methods, while the reasoning and latency experiments provide a broader view of its practical benefits.

We thank all reviewers again for helping us improve the paper. We would appreciate further feedback on whether the revisions and our responses address the concerns raised in the reviews. We are happy to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.