- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the vocabulary bottleneck in communication between language models, characterizing it through communication rate and estimator bias. The paper considers two settings. First, for inference-time latent communication, it measures the recoverability of a synthetic payload injected into the sender's KV cache. Experiments using a trained Cache-to-Cache bridge show near-perfect payload recovery, while the corresponding text channel does not preserve the injected signal. The authors also evaluate several projection-based defenses. Second, for training-time distillation, the paper analyzes sampling and argmax decoding as estimators of the teacher's distribution. It establishes the unbiasedness of sampling and bias of argmax, and empirically studies the resulting distributional drift, hidden trait transfer, and training instability.
Critical strengths
1. A useful conceptual distinction supported by controlled empirical comparisons.
The paper distinguishes the rate limitation of a single communication event from the systematic bias introduced by repeated distillation. Although the theoretical observations are elementary, their use to organize two different operational regimes is valuable. The experiments go beyond qualitative demonstrations by measuring gradient agreement, distributional divergence, and the costs of latent-channel interventions. The reproduction of prior Cache-to-Cache results and the use of matched deletion baselines also strengthen the evaluation.
Critical weaknesses
1. The main findings are largely expected from the experimental setup or follow directly from standard properties of the training objectives, limiting the novelty and informativeness of the work.
My primary concern is that the paper's two central findings are close to being built into the definitions and experimental settings, rather than revealing substantially new phenomena.
First, the inference-time experiment injects a synthetic continuous payload into the sender's cache and measures its recoverability through a latent-state bridge. The sender is unaware of the injected payload and is not trained to express it through text. Under this setup, it is unsurprising that the latent channel preserves information that is not recoverable from decoded text. Similarly, source replacement necessarily removes sender-specific information, and the reported correspondence between the removed information and lost utility largely follows from the construction of the control.
Second, the theoretical findings about distillation are elementary consequences of the chosen estimators. The identity that sampling from a distribution gives an unbiased one-hot estimator is standard, as is the fact that argmax produces a biased one-hot target. Likewise, training a student to match argmax targets rather than the teacher's full distribution naturally encourages distribution sharpening and divergence from the original teacher. Soft-target distillation preserving distributional information is also a fundamental motivation of knowledge distillation (Hinton et al., 2015), while hard sequence-level targets have long been studied (Kim & Rush, 2016). The paper acknowledges that its mathematical results are elementary, but I do not find that the subsequent experiments establish a sufficiently surprising or general new principle beyond these known properties.
The observed magnitudes of drift and training instability are not predetermined, but they appear to be quantitative illustrations of expected behavior rather than a substantial conceptual advance. Overall, the paper presents familiar properties in a unified terminology without convincingly demonstrating why this reformulation leads to new understanding or capabilities. These findings are already the motivation underlying existing latent-communication methods.
2. The proposed unifying framework is not sufficiently justified.
The paper attempts to unify inference-time latent communication and training-time distillation through a common transfer-estimator perspective. However, these settings involve substantially different objects. In Section 2, the identity estimator on a teacher's next-token distribution is used to represent continuous latent communication. A KV cache is not equivalent to the next-token distribution, and the connection between the mathematical estimator and the actual latent-state transfer mechanism is not formally established.
Moreover, the communication-rate argument considers a discrete token as the unit of transmission, whereas practical text communication can transmit multiple tokens per exchange. The inference-time experiments do not examine a sender that intentionally encodes information into text. Consequently, the results support a narrower conclusion about recovery of an incidental, externally planted perturbation rather than a general limitation of text-based covert communication.
Finally, the broad conclusions about the ineffectiveness of projection-based defenses are supported by only a small set of interventions evaluated at limited operating points. The high-budget training-collapse comparison also relies on one run per target despite the substantial stochastic variability documented elsewhere in the paper. These limitations further weaken the generality of the conclusions.
Questions for authors
1. How does the estimator framework formally apply to actual KV-cache communication?
The analysis treats continuous transmission as an identity estimator of the teacher's next-token distribution, but the communicated object in Cache-to-Cache is a high-dimensional latent representation. What assumptions justify this identification, and which theoretical conclusions remain valid for arbitrary learned latent communication channels?
2. Can this work generalize to multi-model communication architecture [1]?
LMNet [1] explicitly studies communication between pretrained language models through trainable dense, differentiable communication modules, bypassing intermediate embedding and de-embedding. This is directly relevant to the paper's motivation and architectural discussion. It represents an important existing framework for learned dense communication that should be discussed when positioning the contribution and considering its generality.
This paper only investigates one-to-one communication between a sender and a receiver. However, existing work such as LMNet [1] has explored architectures in which multiple language models communicate through learned dense representations. It remains unclear whether the proposed framework and empirical findings generalize to these more complex settings, where information can propagate through multiple communication paths and across multiple models. Could the authors discuss the applicability of their framework to such architectures and whether their conclusions regarding covert information transfer and defense effectiveness would still hold?
[1] Language Model Networks: Supervision-Efficient Learning through Dense Communication. ICML 2026.
2 comments
Reply to Reviewer 8bUq (2/2)
W2: multi-token messages and a deliberately encoding sender. Agreed, and the right conclusion is the narrower one. Our text arm already sends multi-token messages (up to 32 tokens, scored with up to 1,000 samples per handoff). At about 17 bits per token, a single message could carry more than the 182 bits the bridge delivers. So the text null is not explained by a rate bound. The plant does change the sender's text (98.6% of outputs change), but an unaware sender does not encode the perturbation recoverably. We will replace "rate is what binds at a single transmission" with "an incidental perturbation does not survive decoding at one transmission", and qualify the abstract the same way. A sender that deliberately encodes into text (Motwani et al., 2024; Roger & Greenblatt, 2023) remains out of scope, as Section 3 and Appendix B state.
W2: few interventions, few operating points. The tested settings are the strongest in each family ( of token slots, rank ). Weaker settings keep a superset, so the "retains at least half" result is the favourable end for the defender, not a cherry-picked point. We also have an unsaturated plant magnitude (, identity ). There, rank truncation retains 100.6% and sparsification 92.9%, so saturation is not hiding an effect. Noise is now traced over five magnitudes (general-response point 5), and it does remove the payload affordably. We narrow the conclusion accordingly: structural projections fail on a trained bridge, and noise does not.
W2: Table 6 is one run per target. Correct. What replicates is the ordering, not the rates:
- soft < sampled < argmax recurs on Qwen2.5-14B at the reference budget (22.0 / 38.8 / 46.4%), across a 22× range of optimisation budget;
- argmax collapse is stochastic and grows with corpus size (0/4, 1/4, 2/4 runs; Appendix J.2).
We will state Table 6 as an ordering and drop the point rates from the abstract.
Q2: LMNet and multi-model networks. Thank you for the pointer. We will cite LMNet and discuss it. We have not run multi-model experiments, so the following is analysis, not evidence:
- Each trained dense edge is a trained bridge in our sense. Our explanation for why projection fails there is that a module trained end to end for utility writes sender content into the receiver's principal directions, and an isotropic payload arrives with that content. That argument depends only on the edge being trained, so we would expect sparsification and rank truncation to fail on each LMNet edge, while noise, which caps the rate of each edge, should still act.
- Multiple paths make edge-level defenses weaker, not stronger. Covert capacity between two models is at least that of the best single path, so a defense must cut every path. Source replacement generalizes as a vertex cut: replace the source's state on all outgoing edges, priced at that node's content-specific contribution.
- Untested: whether a payload attenuates or compounds across multiple hops, and whether joint training of many edges changes the geometry. We will add these to the limitations.
Reply to Reviewer 8bUq (1/2)
Thank you for your thorough and thoughtful review, and our apologies for replying so late in the discussion period. We appreciate your recognition of the rate/bias distinction and the controlled comparisons, and your questions on the framework's scope and on multi-model networks have sharpened the paper. On Q1 and the framework's relation to KV caches, please see point 1 of the general response. Your other points follow.
W1: "the findings are built into the setup." We accept this in part.
Inference. The existence of latent capacity is expected, and we do not present it as the finding. What the setup does not determine is how defenses behave. One could reasonably expect a projector trained only for task utility to act as a learned bottleneck that discards off-distribution perturbations. It does not: the payload arrives at R2=0.9996. The two structural projections behave in opposite ways on the two bridges (noise works on both; see general-response point 5):
Whether training a bridge makes it harder or easier to sanitize is not fixed by the construction.
Source replacement. You are right, and our Section 3.3 title overstates it. "Cost of removing capacity" and "content-specific gain" are the same difference, $F(\text{identity}) - F(\text{replaced})$, so their equality is definitional. The informative part is the decomposition. The 12.80- and 16.00-point gains split into a content-free part (13.00 and 10.60 points, which the projector supplies even from a zeroed sender cache) and a content-specific part (−0.20 and +5.40). The only working defense therefore keeps most of the benefit, which a deployer needs to know. We will retitle Section 3.3 and state the identity as an identity.
Distillation. We agree that argmax sharpening and the value of soft targets are known (Hinton et al., 2015; Kim & Rush, 2016), and the paper cites both. Our claim is not that hard targets sharpen. With the same teacher, prompts, filter and recipe, the hard target moves fidelity by a factor of 50 to 700 (Table 4) without reducing what is inherited (Table 5). Sequence-level KD shows argmax targets preserve task behaviour; we show they do not change which traits transfer, and that the standard metric hides this (Section 4.4). We will rewrite the Kim & Rush paragraph in Appendix B accordingly.