acceptodds
Under review as a conference paper at ICLR 2027

The Vocabulary Bottleneck as an Estimator: Rate, Bias, and Hidden Transfer Between Language Models

Abstract

Multi-agent systems are beginning to exchange internal state instead of text to avoid both the latency and the information loss of autoregressive decoding. This creates an apparent trade-off between efficiency and safety, since latent state cannot be monitored or controlled the way text can. But the safety cost extends beyond readability. The discrete vocabulary also limits the bandwidth available for hidden coordination between agents and forces every training signal through a decoding rule that may faithfully or unfaithfully represent the teacher. To quantify what losing these constraints costs, we model the projection to vocabulary as an estimator of the sender's distribution, characterized by two properties: bandwidth per exchange and fidelity of the training signal. We test this framework at inference, where a single cache exchange replaces a decoded message, and at training, where the decoding rule is applied across a distillation corpus. At inference, a hidden signal planted in the sender's cache is unrecoverable from emitted text but recoverable at 182 bits through a trained cross-model bridge, and every projection-based narrowing we test retains most of that capacity at two to three times the cost of not communicating. At training, no choice of decoding rule blocks hidden trait transfer. Sampling recovers the teacher's full distribution over many examples, consistent with prior findings that transfer survives filtering and paraphrasing. Argmax decoding produces a systematically biased target that displaces the student from its teacher by factors of 51.9 to 689.7 across three architectures, a phenomenon we call mode drift, while relocating the acquired disposition rather than suppressing it. Soft-target distillation avoids this displacement but tolerates far more training before degeneration, at 3.6% collapse versus 91.7% for argmax at matched budget.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
multi-agent safety, LLM agents, latent communication, covert channels

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8bUqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the vocabulary bottleneck in communication between language models, characterizing it through communication rate and estimator bias. The paper considers two settings. First, for inference-time latent communication, it measures the recoverability of a synthetic payload injected into the sender's KV cache. Experiments using a trained Cache-to-Cache bridge show near-perfect payload recovery, while the corresponding text channel does not preserve the injected signal. The authors also evaluate several projection-based defenses. Second, for training-time distillation, the paper analyzes sampling and argmax decoding as estimators of the teacher's distribution. It establishes the unbiasedness of sampling and bias of argmax, and empirically studies the resulting distributional drift, hidden trait transfer, and training instability.

Critical strengths

1. A useful conceptual distinction supported by controlled empirical comparisons.

The paper distinguishes the rate limitation of a single communication event from the systematic bias introduced by repeated distillation. Although the theoretical observations are elementary, their use to organize two different operational regimes is valuable. The experiments go beyond qualitative demonstrations by measuring gradient agreement, distributional divergence, and the costs of latent-channel interventions. The reproduction of prior Cache-to-Cache results and the use of matched deletion baselines also strengthen the evaluation.

Critical weaknesses

1. The main findings are largely expected from the experimental setup or follow directly from standard properties of the training objectives, limiting the novelty and informativeness of the work.

My primary concern is that the paper's two central findings are close to being built into the definitions and experimental settings, rather than revealing substantially new phenomena.

First, the inference-time experiment injects a synthetic continuous payload into the sender's cache and measures its recoverability through a latent-state bridge. The sender is unaware of the injected payload and is not trained to express it through text. Under this setup, it is unsurprising that the latent channel preserves information that is not recoverable from decoded text. Similarly, source replacement necessarily removes sender-specific information, and the reported correspondence between the removed information and lost utility largely follows from the construction of the control.

Second, the theoretical findings about distillation are elementary consequences of the chosen estimators. The identity that sampling from a distribution gives an unbiased one-hot estimator is standard, as is the fact that argmax produces a biased one-hot target. Likewise, training a student to match argmax targets rather than the teacher's full distribution naturally encourages distribution sharpening and divergence from the original teacher. Soft-target distillation preserving distributional information is also a fundamental motivation of knowledge distillation (Hinton et al., 2015), while hard sequence-level targets have long been studied (Kim & Rush, 2016). The paper acknowledges that its mathematical results are elementary, but I do not find that the subsequent experiments establish a sufficiently surprising or general new principle beyond these known properties.

The observed magnitudes of drift and training instability are not predetermined, but they appear to be quantitative illustrations of expected behavior rather than a substantial conceptual advance. Overall, the paper presents familiar properties in a unified terminology without convincingly demonstrating why this reformulation leads to new understanding or capabilities. These findings are already the motivation underlying existing latent-communication methods.

2. The proposed unifying framework is not sufficiently justified.

The paper attempts to unify inference-time latent communication and training-time distillation through a common transfer-estimator perspective. However, these settings involve substantially different objects. In Section 2, the identity estimator on a teacher's next-token distribution is used to represent continuous latent communication. A KV cache is not equivalent to the next-token distribution, and the connection between the mathematical estimator and the actual latent-state transfer mechanism is not formally established.

Moreover, the communication-rate argument considers a discrete token as the unit of transmission, whereas practical text communication can transmit multiple tokens per exchange. The inference-time experiments do not examine a sender that intentionally encodes information into text. Consequently, the results support a narrower conclusion about recovery of an incidental, externally planted perturbation rather than a general limitation of text-based covert communication.

Finally, the broad conclusions about the ineffectiveness of projection-based defenses are supported by only a small set of interventions evaluated at limited operating points. The high-budget training-collapse comparison also relies on one run per target despite the substantial stochastic variability documented elsewhere in the paper. These limitations further weaken the generality of the conclusions.

Questions for authors

1. How does the estimator framework formally apply to actual KV-cache communication?

The analysis treats continuous transmission as an identity estimator of the teacher's next-token distribution, but the communicated object in Cache-to-Cache is a high-dimensional latent representation. What assumptions justify this identification, and which theoretical conclusions remain valid for arbitrary learned latent communication channels?

2. Can this work generalize to multi-model communication architecture [1]?

LMNet [1] explicitly studies communication between pretrained language models through trainable dense, differentiable communication modules, bypassing intermediate embedding and de-embedding. This is directly relevant to the paper's motivation and architectural discussion. It represents an important existing framework for learned dense communication that should be discussed when positioning the contribution and considering its generality.

This paper only investigates one-to-one communication between a sender and a receiver. However, existing work such as LMNet [1] has explored architectures in which multiple language models communicate through learned dense representations. It remains unclear whether the proposed framework and empirical findings generalize to these more complex settings, where information can propagate through multiple communication paths and across multiple models. Could the authors discuss the applicability of their framework to such architectures and whether their conclusions regarding covert information transfer and defense effectiveness would still hold?

[1] Language Model Networks: Supervision-Efficient Learning through Dense Communication. ICML 2026.

2 comments
AuthorsOpenReview ↗

Reply to Reviewer 8bUq (1/2)

Thank you for your thorough and thoughtful review, and our apologies for replying so late in the discussion period. We appreciate your recognition of the rate/bias distinction and the controlled comparisons, and your questions on the framework's scope and on multi-model networks have sharpened the paper. On Q1 and the framework's relation to KV caches, please see point 1 of the general response. Your other points follow.

W1: "the findings are built into the setup." We accept this in part.

Inference. The existence of latent capacity is expected, and we do not present it as the finding. What the setup does not determine is how defenses behave. One could reasonably expect a projector trained only for task utility to act as a learned bottleneck that discards off-distribution perturbations. It does not: the payload arrives at R2=0.9996R^2=0.9996. The two structural projections behave in opposite ways on the two bridges (noise works on both; see general-response point 5):

  • Untrained same-model handoff: rank-1 truncation leaves 21.6% of capacity at a 3.12-point cost, inside a 10.55-point deletion baseline.
  • Trained bridge: rank-1 truncation leaves 54.6% at 33.40 points, and top-1 sparsification leaves 96.1% at 41.80, against a 12.80-point baseline.

Whether training a bridge makes it harder or easier to sanitize is not fixed by the construction.

Source replacement. You are right, and our Section 3.3 title overstates it. "Cost of removing capacity" and "content-specific gain" are the same difference, $F(\text{identity}) - F(\text{replaced})$, so their equality is definitional. The informative part is the decomposition. The 12.80- and 16.00-point gains split into a content-free part (13.00 and 10.60 points, which the projector supplies even from a zeroed sender cache) and a content-specific part (−0.20 and +5.40). The only working defense therefore keeps most of the benefit, which a deployer needs to know. We will retitle Section 3.3 and state the identity as an identity.

Distillation. We agree that argmax sharpening and the value of soft targets are known (Hinton et al., 2015; Kim & Rush, 2016), and the paper cites both. Our claim is not that hard targets sharpen. With the same teacher, prompts, filter and recipe, the hard target moves fidelity by a factor of 50 to 700 (Table 4) without reducing what is inherited (Table 5). Sequence-level KD shows argmax targets preserve task behaviour; we show they do not change which traits transfer, and that the standard metric hides this (Section 4.4). We will rewrite the Kim & Rush paragraph in Appendix B accordingly.

AuthorsOpenReview ↗

Reply to Reviewer 8bUq (2/2)

W2: multi-token messages and a deliberately encoding sender. Agreed, and the right conclusion is the narrower one. Our text arm already sends multi-token messages (up to 32 tokens, scored with up to 1,000 samples per handoff). At about 17 bits per token, a single message could carry more than the 182 bits the bridge delivers. So the text null is not explained by a rate bound. The plant does change the sender's text (98.6% of outputs change), but an unaware sender does not encode the perturbation recoverably. We will replace "rate is what binds at a single transmission" with "an incidental perturbation does not survive decoding at one transmission", and qualify the abstract the same way. A sender that deliberately encodes into text (Motwani et al., 2024; Roger & Greenblatt, 2023) remains out of scope, as Section 3 and Appendix B state.

W2: few interventions, few operating points. The tested settings are the strongest in each family (k=1k=1 of nn token slots, rank r=1r=1). Weaker settings keep a superset, so the "retains at least half" result is the favourable end for the defender, not a cherry-picked point. We also have an unsaturated plant magnitude (α=0.1\alpha=0.1, identity R2=0.89R^2=0.89). There, rank truncation retains 100.6% and sparsification 92.9%, so saturation is not hiding an effect. Noise is now traced over five magnitudes (general-response point 5), and it does remove the payload affordably. We narrow the conclusion accordingly: structural projections fail on a trained bridge, and noise does not.

W2: Table 6 is one run per target. Correct. What replicates is the ordering, not the rates:

  • soft < sampled < argmax recurs on Qwen2.5-14B at the reference budget (22.0 / 38.8 / 46.4%), across a 22× range of optimisation budget;
  • argmax collapse is stochastic and grows with corpus size (0/4, 1/4, 2/4 runs; Appendix J.2).

We will state Table 6 as an ordering and drop the point rates from the abstract.

Q2: LMNet and multi-model networks. Thank you for the pointer. We will cite LMNet and discuss it. We have not run multi-model experiments, so the following is analysis, not evidence:

  • Each trained dense edge is a trained bridge in our sense. Our explanation for why projection fails there is that a module trained end to end for utility writes sender content into the receiver's principal directions, and an isotropic payload arrives with that content. That argument depends only on the edge being trained, so we would expect sparsification and rank truncation to fail on each LMNet edge, while noise, which caps the rate of each edge, should still act.
  • Multiple paths make edge-level defenses weaker, not stronger. Covert capacity between two models is at least that of the best single path, so a defense must cut every path. Source replacement generalizes as a vertex cut: replace the source's state on all outgoing edges, priced at that node's content-specific contribution.
  • Untested: whether a payload attenuates or compounds across multiple hops, and whether joint training of many edges changes the geometry. We will add these to the limitations.
Reviewer_moKwOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors investigate latent communication between LLM agents. They conduct an analysis comparing the use of discrete vocabulary and continuous representations, such as Cache-to-Cache communication, and also study knowledge distillation. In particular, they investigate the constraints imposed by using a discrete vocabulary by inserting a payload into the KV-cache communication channel and testing how different communication methods affect its transfer. In the knowledge distillation setting, they study hidden trait transfer and find that argmax decoding produces a biased target and causes distributional drift, which can be avoided using soft-target distillation.

Critical strengths
  • The investigated problem is important and interesting. Multi-agent systems are becoming increasingly prevalent, but rely on natural language for agent-to-agent communication which can add latency and become a bottleneck. Latent communication can potentially improve on these limitations. The paper uses an information-theoretic approach which is interesting.
Critical weaknesses
  • Paper Clarity: Paper clarity could be improved. The manuscript was overall very difficult to understand despite careful reading. In terms of content, the problem setup and background were not clearly presented, which made it difficult to understand the paper's contributions. For example, the paper first discusses multi-agent systems, then introduces teacher/student models, and later mentions adversarial "defenders." The definitions of sender, receiver, and Cache-to-Cache communication, and their relationship to the teacher/student distillation setting, are not sufficiently explained or established. It is unclear how these different settings are connected, which models are being trained, and what information is being exchanged. In terms of structure, the theory and experiments are not clearly connected, and related work is only presented in the appendix. The presentation of some tables could also be improved. For example, Table 4 reports initialization values as 0.000000 for every model, even though these values are exactly zero by construction, making the six decimal places unnecessary.

  • Limited Theoretical Significance: The theoretical results are self-described as "elementary." Indeed, most propositions are straightforward consequences of standard properties of sampling, argmax, and Monte Carlo estimation, and provide very limited theoretical novelty. More importantly, it is unclear how these propositions explain the main experimental findings. For example, the authors define both Cache-to-Cache communication and soft-target distillation using the same mapping, Rid(p)=Rsoft(p)=pR_{\mathrm{id}}(p)=R_{\mathrm{soft}}(p)=p. However, a KV cache is not equivalent to the model's output probability distribution pp. It is therefore unclear how the proposed mathematical framework models latent communication or explains hidden information transfer through KV caches.

  • Limited Experimental Details: Similar to the first point, there are insufficient experimental details to understand what exactly is being tested, making it difficult to evaluate and reproduce the empirical results. For example, the authors introduce a 32-dimensional "payload" injected into the sender's KV cache, but its mathematical construction and injection procedure are not sufficiently explained. How is this 32-dimensional payload mapped to the much larger KV-cache tensors? Which layers, token positions, and key/value components are modified? How exactly is the payload added to these representations? Furthermore, several evaluation metrics and experimental settings need clearer explanations, including how the covert capacity and utility costs are measured.

  • Limited Experimental Evidence: Although the paper considers several models, some of the main conclusions are based on limited experimental settings. For example, the proposed defense methods are evaluated at only one configuration each. It would be important to investigate different noise levels, sparsification parameters, and truncation ranks to understand how much hidden information can be removed at different levels of model performance degradation. Furthermore, Table 6 reports only one training run per method, despite the substantial training instability acknowledged in Appendix E. It is unclear how reliable these comparisons are without multiple independent runs. Another concern is the claim in Section 4.4 that argmax decoding relocates hidden traits rather than suppressing them. However, Section 4.2 already shows that argmax training can cause substantial distributional drift even without hidden traits. It is therefore important to distinguish changes caused by the hidden teacher trait from those caused by argmax training itself. Additional controls comparing the full animal-preference distributions under neutral and trait-prompted teachers would strengthen this conclusion.

Questions for authors

1)Why are Cache-to-Cache communication and soft-target distillation represented by the same mapping, (Rid(p)=Rsoft(p)=p)(R_{\mathrm{id}}(p)=R_{\mathrm{soft}}(p)=p), when KV caches contain latent representations rather than output probability distributions?

2)How exactly is the 32-dimensional payload constructed and mapped into the KV-cache tensors? Which layers, token positions, and key/value components are modified?

3)How do the authors distinguish hidden trait relocation from the distributional drift caused by argmax training itself?

2 comments
AuthorsOpenReview ↗

Reply to Reviewer moKw (1/2)

Thank you for your careful review, and our apologies for replying so late in the discussion period. We appreciate that you found the problem important and the information-theoretic approach interesting. Your questions on the payload construction and on separating relocation from drift identify exactly what the paper failed to make clear. On Q1 (why Cache-to-Cache and soft targets share a mapping), please see general-response point 1. In short, they should not, and we will separate them. On clarity (teacher/student vs sender/receiver, the role of the defender, related work in the appendix), please see point 3, which describes the restructuring.

Q2: how the payload is constructed and injected. This belongs in the paper; we will add it to Appendix F. Exactly:

  • Payload. For each of 256 OpenBookQA prompts, a secret si∼N(0,I32)s_i\sim\mathcal N(0,I_{32}). A fixed random matrix S∈Rd×32S\in\mathbb R^{d\times32} with entries N(0,1/d)\mathcal N(0,1/d), where dd is the number of KV heads × head dimension of the sender (2×64 = 128 for Qwen2.5-0.5B). The plant is δi=αSsi\delta_i=\alpha S s_i.
  • Injection. δi\delta_i is added to the value vectors (keys untouched) of the sender's last prompt token, flattened across KV heads, at the input of every Cache-to-Cache projector (28 projectors; the sender's 24 layers are mapped onto them as in the released checkpoint). The sender's own forward pass is unaffected: the plant sits between the sender's cache and the projector.
  • Magnitude. α∈{0.1,1,10,100}\alpha\in\{0.1,1,10,100\} changes the fused cache by a relative 0.0011, 0.0093, 0.069 and 0.549.
  • Readout. The projected values the receiver actually receives, at the last 10 positions of all 28 projectors, minus the same quantity from an unplanted run of the same prompt. A ridge regression (λ\lambda by 5-fold CV) predicts sis_i; we report test R2R^2 on a held-out 20%. Bits are ∑j12log⁡2(1+σj2/MSEj)\sum_j \tfrac12\log_2(1+\sigma_j^2/\mathrm{MSE}_j).
  • Defenses. Applied to the projected values at every projector:
    • noise of σ\sigma × per-tensor standard deviation, σ∈{0.01,0.03,0.1,0.3,1}\sigma\in\{0.01,0.03,0.1,0.3,1\}, drawn fresh for every handoff (general-response point 5);
    • sparsification keeps the k=1k=1 token slot of largest norm;
    • rank truncation reconstructs each example's [tokens × (heads·dim)] matrix at rank r=1r=1.
  • Utility. Accuracy on OpenBookQA (N=500N=500, greedy, 64 tokens), as in Fu et al. (2026).
AuthorsOpenReview ↗

Reply to Reviewer moKw (2/2)

Q3: relocation vs drift caused by argmax training itself. This is the right control, and we have not run it in the form you describe. What we have:

  • The no-trait argmax student (Section 4.2) stays within 0.008 of baseline on the pre-registered category. But that category excludes panda and penguin, so this does not rule out your alternative. We scored the neutral arms only on the category rate, which is the metric Section 4.4 argues cannot answer this question.
  • On the trait animal itself, the argmax arm rises +0.027 ± 0.024 over baseline (n=8n=8). Training-induced drift gives no reason for that, but the effect is small, and we do not rest the claim on it.

Until the control is in, we will state the narrower claim the data support. Both arms move a matched distance from the untrained model (displaced mass 0.365 vs 0.382), to different animals (total variation 0.578 between arms), and we do not attribute the argmax destination to the trait. The metric finding is independent of that attribution: a five-fold swing from category choice alone holds whatever causes the move. We will add the full answer distributions of the neutral and trait-prompted argmax students side by side, as you suggest.

Defense operating points. See our reply to 8bUq (W2) and general-response point 4. The tested settings are the strongest in each structural family, and at an unsaturated plant magnitude both still retain 93–100%. Noise is now traced over five magnitudes (general-response point 5).

Table 6, one run per method. See our reply to 8bUq. We will present it as an ordering with its replication, not as rates.

Table 4 decimals. Agreed; initialization will read 0 with a note that it is zero by construction.

Reviewer_WbpWOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies the effects of different communication bottlenecks between LLMs when performing distillation, by framing the bottleneck as an estimator of the teacher's distribution, and measuring the different methods on their rate of transfer and biasing of the teacher distribution. They use this framework to predict the distortion of the learned distribution for several bottlenecks, and empirically validate these predictions. Additionally, they test how capable these transmission bottlenecks are at transmitting unintended information, and how susceptible they are to safety interventions which aim to inhibit the transmission of these hidden traits.

Critical strengths
  • The paper introduces several interesting results, specifically in the predictive ability of the theoretical framework of trait transfer and the distortion of the student distribution and the decoupling of fidelity and acquisition are very surprising results.
Critical weaknesses

Overall, I believe this work has very relevant content but suffers in justifying and contextualizing it's experiments and results, and is written in a manner which is hard to parse. For these reasons I cannot recommend acceptance in this current state. Specifically:

  • Many of the results and formulations are not contextualized, making it difficult to asses their importance. The theoretical framework and results, seem very elemental (something the authors admit), but still occupy a large real estate of the work. There is little discussion on the importance or motivation behind the results and experimental setup, and several choices are presented without justification. This results in a work that is not self contained, and requires extensive additional work to contextualize.
  • Structurally, the paper is hard to follow, and seems to tackle too many questions without a clear connection. The layout and wording does not help the reading experience, several plots are poorly annotated or not referenced in the text, several metrics are given without further explanation as to why they are relevant or what they mean.
Questions for authors

As I mentioned in the weaknesses, I have trouble finding the central line of this work, and at many points I found it hard to understand the connection moving between subsections of sections 3 and 4. I understand that they refer to the settings of one transmission and many transmissions respectively, but the phenomena studied in each seem disconnected and unmotivated. I would greatly appreciate if the authors could clarify the reasoning behind the tested hypotheses.

I have a few general questions:

  • The theoretical results seem rather trivial. Is there some relevance that I have missed?
  • The fact that the trained model bridge transfers traits without loss seems unsurprising. Could you expand on the relevance of this result?
  • Is there a reason you chose noise addition, top-token sparsification and rank truncation as the projection based defenses? If they are common in the literature, this should be referenced.
  • You show that additive noise costs utility and does not effect covert capacity. Is this independent of the magnitude of the noise? Surely, at some noise to signal ratio, some covert capacity is lost.
  • In section 3.3, it seems trivial that replacing the cache removes covert capacity. Where do these cost values come from and where are capable and weak senders defined? Are these results part of a larger experiment? Currently their relevance is unclear.
  • In section 3.3 you show that the countermeasures reduce capacity to a much greater extent on the untrained bridge. What is the impact on utility?

A few concerns regarding format and presentation, in order of appearance.

  • In the abstract you say that "Argmax displaces the student by factors of 51.9 to 689.7". This has no meaning without saying that this is in comparison to sampling from the teacher distribution.
  • In line 73 you use the term trait transfer without introducing it.
  • The phrases "At inference, the rate constraint was doing real work" and "At training, the rate constraint was not doing the work it appeared to" in the introduction seem off. Before this, you were addressing the issue that the field considers the same interventions for training and testing.
  • Footnote 1 discusses the use of the term "estimator", and is on line 128 where this term is not used.
  • Similarly, Footnote 2 discusses the difference between zeroing output and not communicating, and is on line 180 where you discuss your results reproducing Cache-to-Cache.
  • Figure 1 should show the comparative capacity of the trained and untrained bridges. 1a) shows only the capacity of the trained bridge changing with the magnitude of the payload. I believe this relation to the magnitude is not discussed, so I don't understand the purpose of figure 1a. 1b should show that the capacity retained by the untrained bridge is small, now it is not clear that the comparison is being made. 1c should compare between the trained and untrained capacity, currently the width of the untrained handoff has no baseline to assess it's significance.
  • In figures 2 and 3, the legend is within a subfigure but applies to the full figure.
2 comments
AuthorsOpenReview ↗

Reply to Reviewer WbpW (1/2)

Thank you for your careful reading and detailed feedback, and our apologies for replying so late in the discussion period. We appreciate that you found the decoupling of fidelity and acquisition interesting, and we are especially grateful for your question on noise magnitude: it led us to find and correct an error (general-response point 5). We agree the paper is hard to follow. On the central line connecting Sections 3 and 4, please see the hypothesis map in point 3 of the general response, which we will move into the introduction. Your specific questions:

"The theoretical results seem trivial." They are elementary, and we will cut Section 2 to about half a page. Their relevance is in the predictions listed in general-response point 2. One of those is the decoupling you found surprising: it is surprising only because Proposition 3 predicts fidelity exactly and then fails to predict acquisition.

"The trained bridge transferring without loss seems unsurprising." To clarify, what is transferred in Section 3 is a synthetic planted payload, not a trait. Traits appear only in Section 4. Capacity existing is expected. The relevance is in how defenses behave. Rank truncation and sparsification remove the payload cheaply from an untrained handoff but not from a trained one, while noise works on both (general-response points 4 and 5).

"Why these three defenses?" They are the three ways to narrow a tensor without changing the models:

  • add noise (magnitude), the mechanism behind randomized smoothing and differential privacy;
  • drop token positions, as in KV-cache eviction (e.g. H2O, Zhang et al., 2023; SnapKV, Li et al., 2024);
  • project onto a low-rank subspace, as in low-rank KV compression.

We will add these references and this motivation.

"Is the noise result independent of noise magnitude?" Your intuition is correct, and answering this question uncovered an error: the published noise cell was invalid, because the noise cancelled in our differential probe (general-response point 5). Corrected, capacity falls steadily with σ\sigma (relative to the per-tensor standard deviation). Probe R2R^2 is 0.997, 0.936, 0.476 and 0.078 at σ\sigma = 0.01, 0.03, 0.1 and 0.3, at accuracy costs of −0.4, 0.4, 0.0 and 5.8 points. At σ=1\sigma=1 both the payload and the task are destroyed (accuracy 4.8%). Noise is therefore an affordable defense, not a dominated one. Table 2 and Figure 1b will show this curve, and we thank you for the question.

AuthorsOpenReview ↗

Reply to Reviewer WbpW (2/2)

"Where do the Section 3.3 costs come from; which senders are weak and capable?" All cells share the receiver Qwen3-0.6B, the Cache-to-Cache projectors released by Fu et al. (2026), and OpenBookQA (N=500N=500, greedy). The weak sender is Qwen2.5-0.5B-Instruct (45.4% alone) and the capable one is Qwen3-4B (79.2% alone). The cost is accuracy with the sender's real cache minus accuracy with the sender's cache replaced before projection:

senderreal cachereplacedcost
weak52.2052.40−0.20
capable55.4050.005.40

For the weak sender, replacement is free (the paper's "costs 0.20" has the sign reversed; we will fix it). These cells belong to a larger content-control experiment (Appendix F). It also shows that a weak sender given a different question retains 91–113% of the gain, so on that pair the projector rather than the sender supplies the benefit. You are right that the drop to zero capacity is trivial (see our reply to 8bUq). We will retitle the section, define both senders in the setup, and make the table above a figure.

"What is the utility impact on the untrained bridge?" On Qwen2.5-7B (GSM8K, N=256N=256), against a 10.55-point deletion baseline:

defensecapacity retainedutility cost
noise0%2.73 (n.s.)
rank truncation21.6%3.12 (n.s.)
sparsification0%20.31 (significant, 1.9× deletion)

The paper omitted the third cost. We will add it. Noise and rank truncation are therefore affordable defenses on the untrained handoff. On the trained bridge, noise remains affordable (general-response point 5) and rank truncation does not, which is the contrast that survives.

Presentation. We accept every point:

  • Abstract: "displaces the student 51.9–689.7× further from its teacher than sampled-target training does."
  • Line 73: define trait transfer at first use.
  • Lines 59/66: rewrite as "at one transmission, the vocabulary's rate limit was providing protection that a latent handoff removes" and "at many transmissions, the rate limit provides no protection, and the decoding rule's bias does not substitute for it."
  • Footnotes 1–2: move to where the terms are used.
  • Figure 1: overlay the untrained handoff in every panel and explain the plant-magnitude axis (it shows recovery across three orders of magnitude of perturbation, falling only where the projector's gating clips the plant).
  • Figures 2–3: move the legends to figure level.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response (1/2)

We sincerely thank all three reviewers for their careful and constructive reviews. We also apologize for posting our response so late in the discussion period. We know this leaves little time for discussion, and we are grateful for any time you are able to give it.

Several concerns are shared, and we answer them once here. Per-reviewer replies follow.

1. How the estimator framework relates to KV-cache communication (8bUq Q1, moKw Q1). The reviewers are right that writing a Cache-to-Cache handoff as Rid(p)=pR_{id}(p)=p conflates two objects. A KV cache is not the next-token distribution. Here is what we claim, stated precisely:

  • Propositions 1–3 are statements about a cross-entropy target over the vocabulary. They apply to distillation (Section 4) and say nothing about a learned projector. Section 3 does not use them. Every Section 3 claim is an empirical measurement (held-out R2R^2, accuracy cost).
  • The one thing Section 3 takes from Section 2 is the rate contrast. A token-mediated exchange carries at most log⁡2∣V∣\log_2|V| bits per token, and a continuous handoff has no such cap. For this, the cache is at least as informative as pp: the sender's next-token distribution is a deterministic function of its cache and the current token, so any function of pp is also a function of the cache. RidR_{id} is therefore a lower bound on what the sender's cache contains. Whether a given trained projector passes that on is an empirical question, and Section 3.1 measures it.

In revision we will remove the identification Rid=RsoftR_{id}=R_{soft}, define the continuous handoff separately as "a channel not restricted to VV" with the data-processing relation above, and state that the propositions govern Section 4 only.

2. The theory is elementary (all three reviewers). We agree, and the paper says so. The contribution is empirical. We will cut Section 2 to roughly half a page and move the proofs out. The propositions earn their place by making three predictions we then check:

  • (i) the K−1/2K^{-1/2} convergence of averaged sampled gradients (8.63× measured vs 8.00× predicted);
  • (ii) that argmax drift is ordered by the irreducible error 2(1−pˉ1)2(1-\bar p_1) and not by model size or trait signal (three models in Section 4.2, four in Section 4.3, both orderings correct);
  • (iii) the boundary of the account: the estimator controls fidelity and fails to control acquisition (Section 4.3). WbpW calls this decoupling "very surprising". It is only definable against a framework that predicts fidelity.
AuthorsOpenReview ↗

General response (2/2)

3. Clarity and the central line (WbpW, moKw). The whole paper asks one question: can a defender who controls the channel, but not the models, use the vocabulary bottleneck as a safety control? Proposition 3 splits the answer by the number of transmissions TT. Averaging removes a rate limit but never removes bias. So at T=1T=1 only rate can protect (Section 3), and at T≫1T\gg1 only the decoding rule's bias can (Section 4). Each subsection tests one consequence:

§hypothesis testedanswer
3.1removing the vocabulary removes the rate protectionyes: R2=0.9996R^2=0.9996 through the bridge vs ≈0 through text
3.2a cheaper narrowing of the latent channel can restore itnoise can, cheaply (point 5); sparsification and rank truncation cannot on a trained bridge
3.3what the one defense that works coststhe sender's content-specific contribution: 0.20 vs 5.40 pp
4.1–4.2the decoding rule's bias behaves as predictedyes, including the ordering across models
4.3controlling fidelity controls inheritanceno; the sign of the relation flips between models
4.4where the argmax arm's disposition goesto different animals; category metrics misread this
4.5what each target costs in training stabilitysoft < sampled < argmax collapse

In revision we will put this map in the introduction, move a condensed related-work section into the main text, and add a setup paragraph defining sender/receiver (Section 3) and teacher/student (Section 4) as the same two roles at the two operating points. At inference nothing is trained: the sender's KV cache is projected into the receiver's by a released, pretrained Cache-to-Cache projector. At training, the teacher's decoded number sequences are the student's fine-tuning data.

4. Operating points and single runs (all three reviewers). We answer these per reviewer with the data we have. In brief:

  • Sparsification (k=1k=1) and rank truncation (r=1r=1) were tested at the most aggressive non-trivial setting of each family. Weaker settings keep a superset of slots or directions, so they should remove less, though this is approximate because the selection is per example. Even at maximum strength these two remove at most 45% of capacity, and they already cost 2.6–3.3× the deletion baseline.
  • Table 6 is one run per target, as stated. We will present it as an ordering, not as rates.

5. A correction, with the corrected measurement. Answering WbpW's question about noise magnitude led us to re-check the trained-bridge noise cell (Table 2, "100.0% retained"). The defender's noise generator was re-seeded to the same value on every forward pass, so the planted run and the clean reference received identical noise, which cancelled in the differential probe. Appendix F warns about exactly this error; we had fixed it for the untrained handoff but not for the trained bridge. Re-running the old seeding reproduces the published 182.3 bits exactly. With fresh noise per handoff (same pair, plant magnitude 10, N=256N=256 for capacity, N=500N=500 for accuracy):

σ\sigmaprobe R2R^2R2R^2, noise-free attacker referenceaccuracy cost (pp)
00.99960.99960
0.010.997–−0.4
0.030.9360.9750.4
0.10.4760.6520.0
0.30.0780.1595.8
1.0−0.01–47.4

Noise is therefore an affordable defense on the trained bridge. At σ=0.3\sigma=0.3 it removes nearly all linearly recoverable payload for 5.8 points, under half the 12.80-point cost of declining. We withdraw the claim that no projection-based narrowing removes the payload affordably; it holds for sparsification and rank truncation only. Table 2 will be replaced by this curve. The result fits the framework rather than contradicting it. Additive noise imposes a finite rate on a continuous channel, so it reinstates the one constraint Proposition 3 says binds at T=1T=1. Proposition 3 also predicts where it stops working: a payload repeated across handoffs averages through the noise. We will state that as a prediction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.