acceptodds
Under review as a conference paper at ICLR 2027

Share the Symbol, Not the Vector: Training-Free Fact Communication Across Language Models

Abstract

Multi-agent systems allow language models to solve tasks collaboratively by sharing complementary information. These systems typically communicate through text, while recent approaches also explore exchanging internal model states. However, sharing these states across architectures often requires learned translators or assumptions about model compatibility. We introduce SYNC (Symbol-Indexed Neural Communication), a word-indexed interface that connects heterogeneous models without fine-tuning or a learned translator. From descriptions only it can see, the sender selects words from a shared list and sends their indices for each object, without seeing the questions. The receiver writes the described facts into its hidden states using its own word directions, vectors fitted once from its activations. We test selective use by swapping one attribute between two objects and requiring correct answers about both changed and unchanged attributes before and after the swap, four answers in total. We choose an early write layer on four development receivers and then fix it. With three different senders and nine newer receivers from five families, four-answer success on new scenes reaches 72.0–98.3%, up to 96 percentage points above middle-layer writing, and the two largest receivers match plain text. A task-free probe predicted in advance which depths each new receiver can use (45 of 45 cases). Without retuning, messages listing every fact keep 89.6–100% success under new layouts, and sender messages reach 23.9–64.0% on a landmark task with multi-token symbols, below text but from 0% without a message. These results show that shared word indices and receiver-local directions can support selective fact use across model families without additional training.

Primary area:
interpretability and explainable AI
Keywords:
multi-agent communication, cross-model communication, large language models, interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9qmDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes SYNC, a training-free framework for communication between heterogeneous language models. The sender selects symbols from a shared inventory, and the receiver writes its own word directions into its hidden states. The authors evaluate the method across different sender—receiver pairs and write depths, and use a “probe” to predict usable write depths.

Critical strengths

The paper conducts a wide range of experiments to evaluate the proposed method, varying senders, receivers, and write depths. The experiments also include several control conditions that help characterize the behavior of the proposed method.

Critical weaknesses
  1. The paper does not clearly position its contribution. As the authors acknowledge that internal writing shows no consistent advantage for facts with equal weights, this paper seems more like a proof of concept, which should be made explicit in the introduction. While the authors say that “advantage is with graded weights”, it is unclear under in what practical scenarios such weights would be useful.

  2. The method relies on a pre-defined list of symbols shared by senders and receivers, which is difficult to maintain in real-world applications. More importantly, real-world communication involves much more than simple object—attribute relationships (also see question 3), making the applicability of the proposed method rather limited.

  3. The presentation lacks clarity and should be improved. Several important experimental settings and methodological details are insufficiently explained in the main text or deferred to the appendix. For example, the “binding probe” in Section 4.3 (Appendix B), despite being presented as a core contribution. The literal prompts are also important, but they are provided only in Appendix H.4, with no direct reference in the main text. While the paragraph in line 305 titles “Why does late writing fail”, the content is largely descriptive, and all explanations are in Appendix G. Some terminology is also confusing, e.g., the “probe” in Section 4.3 differs from its common usage in interpretability research.

  4. Another minor problem: in introduction, the second point of contribution has two “§”.

Questions for authors
  1. In practical applications, is it realistic to assume that senders and receivers share a predefined symbol inventory?

  2. The sender only extracts symbols from private descriptions and does not transmit any model-specific representations. If my understanding is correct, this step could be replaced by a simple rule-based method, as all we need is to transmit symbol indices, especially for the object—attribute tasks considered in this paper. If so, what is the specific contribution of using an LLM sender to cross-model communication?

  3. Real-life cross-model communication can involve much more complex relationships than object—attribute relationship. For example, the same symbols "green" and "sweet" could represent either "the green candy is sweet" or "the green candy is not sweet." How would the proposed method handle such cases?

  4. How was the 221-symbol inventory (Table 16) constructed, and what criteria were used to select these words? In particular, Table 16 includes not only nouns, but also words such as “not”, “if”, “unless”. This raises two questions: (1) Why do you call these words as "symbols"? Is it consistent with the common usage of the term "symbol" in the literature? (2) If the sender assigns “green,” “sweet,” and “not” to the same object, how can the receiver distinguish between “the green candy is not sweet” and “the candy is not green but is sweet”? This question also related to Q3.

  5. If the goal is to achieve good four-answer success, why not simply use a fixed early write depth? If the goal is to select the optimal write depth, why does the probe only predict whether four-answer success exceeds 50%, rather than ranking candidate depths? More importantly, the “binding probe” in Section 4.3 appears structurally similar to the four-answer success evaluation, as both test whether the receiver could associate the object with an attribute, so the high correlation between them is not surprising. To what extent does this reflect the probe's predictive power?

8 comments
AuthorsOpenReview ↗

Reply to Reviewer 9qmD (1/3)

Thank you for noting the wide range of experiments and the control conditions that characterize the method.

W1. "The paper does not clearly position its contribution. [...] it is unclear under in what practical scenarios such weights would be useful."

Our contribution is a training-free channel between unrelated models. Thank you for this; we state the contribution in one sentence before anything else. A receiver acts on facts written as its own word directions, selected via indices from a sender of another family. The depth of the write determines whether the receiver can use those facts, and a probe that fits nothing predicts which depths are usable. The same channel also conveys how strongly a fact is asserted. Before the contribution list, the introduction now states the kind of claim that the paper makes: "SYNC is a training-free interface that delivers facts through hidden states between models of different families, with messages of a few dozen bytes. This paper establishes that capability under controlled conditions, where success without a message is zero by construction." We have taken the scope question into the paper rather than into a definition: the sentence quoted above is now in Section 1, before the contribution list, and it states the conditions under which the capability is established.

On graded weights, the paper establishes a capability: a receiver can act on assertion strength transmitted through the channel. Internal delivery scores 36.1 and 33.8 points higher than printed weights on the two receivers in Table 10, and scores higher on seven of ten additional receivers. The use that we measure is conflict resolution by the receiver itself: when two competing values for one attribute arrive with weights of 255 and of 64 or 128, the receiver answers with the more strongly weighted one at 77.8 to 97.2% across the six receivers in Table R15. We also test signed weights: a fixed rule maps a sender's "not" to a negative weight, which represents a sign rather than a graded level (Appendix P.16). In our experiments these weights are supplied externally, just as a deployment would supply a retrieval score or per-source reliability.

Our experiments test the receiver's ability to act on them.

Across those ten plus the two in Table 10, the ten receivers where the comparison can be read, internal delivery outperforms printed weights on nine. On an additional set of 108 scenes, it outperforms printed weights on five of six receivers, and on three of the four receivers added to that set afterwards (Appendix P.13).

In the internal route the sender emits only a few symbol indices from a declared inventory, with their object assignments and an 8-bit weight each, and the text route spells the same symbols out. Only the sender reads the private descriptions, and it never sees the question or its answer, so the interface fits settings where that source text cannot be handed on, as in privacy-sensitive deployments.

Why not preprocess the text instead. Preprocessing moves the decision to the sender: it keeps each object's and attribute's highest-weighted value and hands the receiver the answer, reaching 100.0% on all six receivers of Table R15 by resolving the conflict before it sends. The arrangement we report is the other one, the two paragraphs above: the strengths travel and the receiver resolves them itself. Delivery is not the cost either: 0.99 to 1.02 times text once a receiver saves its neutral states (Section 6). The scenario is a receiver that has to act on strengths the sender has not already collapsed into an answer.

Table R12 — receiver-side latency with saved neutral states.

ReceiverMedian (ms) TextMedian (ms) Pass repeatedMedian (ms) States savedRatio to text PublishedRatio to text Pass repeatedRatio to text States saved
Qwen3-32B1005128210021.281.281.00
Qwen3-14B-Base4946214941.241.261.00
GLM-4-9B3724693781.191.261.02
Gemma-3-27B-IT93811779381.241.261.00
Qwen3.5-2B274343276-1.251.01
Qwen3.5-4B396495391-1.250.99
Qwen3.5-9B503643502-1.281.00
Qwen3.5-27B124316131243-1.301.00
Qwen3.6-27B122815971227-1.301.00
Granite-4.1-8B337423337-1.251.00
OLMo-3-7B-Instruct287357288-1.241.00
gpt-oss-20B8611096854-1.270.99
Mistral-Small-3.2-24B689879689-1.281.00

Receiver-side latency with saved neutral states. Warm median time (ms) per item and its ratio to the text time. Published is the ratio of the early-write and text medians in the appendix table of receiver-side delivery cost, given for the four development receivers.

AuthorsOpenReview ↗

Reply to Reviewer 9qmD (2/3)

W2. "[...] real-world communication involves much more than simple object—attribute relationships [...]"

Beyond object attributes, the same internal write carries directed relations between objects and comparative words, where 6 of 6 and 5 of 6 receivers meet our prespecified success rule (Appendices P.2 and P.3). Thank you for this; it is why we added new kinds of fact rather than more scenes of the same kind. These two tasks also preserve slot order for each object's symbols, extending the submitted format.

W3 and W4. "The presentation lacks clarity and should be improved."

Thank you for these; every point you listed under them produced a change in the manuscript.

The revision updates the main text on each point that you raised. (1) Binding probe, Section 4.3: "We call this test a probe: it fits nothing, and it differs from the trained classifiers that the term often denotes in interpretability." (2) Literal prompts: Section 2.1 now cites them directly as "(literal prompts: Appendix H.4)".

(3) Why late writing fails, Section 4.2: "A write at layer ℓ can change those keys and values only above ℓ, so the layers below stay neutral. If later tokens read the facts below ℓ, the write cannot reach them." Appendix G presents the diagnostic that tests this mechanism. (4) Notation: "§§" is now written out as "Sections 4.2 to 4.3". Finally, Table 3 (Section 4.4) now reports relational facts, comparative words and negation scope across six receivers.

Q1. "In practical applications, is it realistic to assume that senders and receivers share a predefined symbol inventory?"

Thank you for asking whether a shared inventory is “realistic”; that is the assumption we most wanted to test.

Sharing the paper's 221-word inventory is not required. In our open-vocabulary experiment, none of the 288 words appear in the paper's inventories, and each receiver fits a direction the first time that it encounters a word. Any word on a public word list is sent as its index, while any other word is sent by its spelling. With complete facts, the internal write reaches 79.9 to 100.0% four-answer success, significantly exceeding a control that writes directions of unrelated words on 6 of 6 receivers (Appendix O).

Q2. "[...] what is the specific contribution of using an LLM sender to cross-model communication?"

On the submitted bank a rule can produce the same indices, because the descriptions state the symbols literally. Thank you for this question about the sender specifically. The language-model senders show that models from other families can generate usable messages without tuning. When a Gemma-4 sender receives no word list and chooses its own words, the internal write reaches 83.3 to 98.3% (Appendix O). When each scene's evidence is split between two senders from different families, internal delivery reaches 70.1 to 94.4% across thirteen receivers (Appendix P.5). That is by design: the sender is replaceable, and the receiver requires nothing model-specific from it.

Q3 and Q4(2). "[...] how can the receiver distinguish between 'the green candy is not sweet' and 'the candy is not green but is sweet'?"

Thank you for the worked example; we answer with your own pair of sentences.

A receiver can distinguish "green, not sweet" from "not green, sweet". We write the negated word with a negative weight in its own slot, so the sign of the weight marks the negation scope. Those two meanings produce distinct messages—green (+), sweet (−) versus green (−), sweet (+)—and an audit finds no two meanings that share a message among 888 signed messages (Table R29).

All six receivers then meet our prespecified success rule, reaching 66.0 to 99.3% four-answer success, compared with 0.2 to 3.9% when directions of other words are written at the same edit size. Moving the negative sign to the object's other word flips the receiver's answers: on the questions whose answers change, the internal write leads this control by 99.3 to 100.0 points.

Plain negation meets the rule on 4 of 6 receivers, and temporal order meets it on 4 of 6 (Appendix P.4 reports every receiver and format). We added the scope, negation and temporal-order tasks in response to the reviews. Because the submitted format sorts each object's indices and therefore cannot mark scope, these tasks extend the format; the results hold for the extended format only, using messages generated by rule. When a Gemma-4 sender generates the negation messages from the scene descriptions instead, 4 of 6 receivers meet our prespecified success rule, reaching 73.6 to 81.9% on those four (Appendix P.16).

In an additional test, a Qwen3.6-27B sender generates the scope messages from the scene descriptions, and 5 of 6 receivers meet the same prespecified criteria, reaching 56.2 to 98.6% on those five (Appendix P.20).

AuthorsOpenReview ↗

Reply to Reviewer 9qmD (2/3) — tables

The result tables for the answer above.

Table R6 — negation, scope and temporal order.

ReceiverInternalUnsigned textNo messageReassign.LeadLowConfirms
Negation: 4 of 6 receivers confirm
Qwen3-32B62.563.20.00.998.640.3no
Qwen3.5-27B83.318.80.01.6100.068.1yes
Qwen3.6-27B26.414.60.02.3100.015.3no
Mistral-Small-3.2-24B68.14.20.01.682.650.0yes
Qwen3.5-9B72.922.90.01.297.963.9yes
Granite-4.1-8B84.032.60.00.095.881.9yes
Negation scope: 6 of 6 receivers confirm
Qwen3-32B78.520.10.03.299.372.2yes
Qwen3.5-27B88.20.00.03.9100.086.1yes
Qwen3.6-27B66.09.70.03.7100.062.5yes
Mistral-Small-3.2-24B80.62.10.03.7100.079.2yes
Qwen3.5-9B89.60.00.03.7100.087.5yes
Granite-4.1-8B99.30.70.00.299.398.6yes
ReceiverInternalTextNo messageReassign.LeadLowConfirms
Temporal order: 4 of 6 receivers confirm
Qwen3-32B83.386.80.00.072.980.6yes
Qwen3.5-27B77.891.00.00.759.776.4yes
Qwen3.6-27B87.599.30.00.063.984.7yes
Mistral-Small-3.2-24B8.393.80.00.0-0.74.2no
Qwen3.5-9B26.443.80.00.911.126.4no
Granite-4.1-8B74.366.00.00.063.269.4yes

Negation, scope and temporal order. Four-answer success (%) on 144 scenes per receiver; Reassign. is the mean of three reassignments, Lead is the internal write minus its structural control on the rate at which both changed questions are correct (points), and Low is the weakest registered subclass. Unsigned text prints the word list with "not" as a word and is not a matched comparison (text arms on new banks: the negation and scope appendix). The appendix keeps all of its columns.

Table R7 — two senders, thirteen receivers.

ReceiverFour-answer success InternalFour-answer success TextFour-answer success Reassign.Four-answer success No messageInternal minus Reassign.Internal minus Text
Six receivers of the open-vocabulary experiment
Qwen3-32B91.789.61.40.0+90.3 [+81.2, +97.9]+2.1 [0.0, +4.9]
Qwen3.5-27B94.494.41.40.0+93.1 [+86.1, +98.6]0.0 [0.0, 0.0]
Qwen3.6-27B93.895.12.80.0+91.0 [+83.3, +97.9]-1.4 [-3.5, 0.0]
Mistral-Small-3.2-24B93.193.12.10.0+91.0 [+85.4, +95.8]0.0 [-3.5, +3.5]
Qwen3.5-9B89.691.03.50.0+86.1 [+75.0, +94.5]-1.4 [-5.6, +2.1]
Granite-4.1-8B88.990.30.00.0+88.9 [+79.2, +97.2]-1.4 [-3.5, 0.0]
Seven further receivers
Gemma-3-27B-IT93.193.80.00.0+93.1 [+85.4, +100.0]-0.7 [-2.1, 0.0]
GLM-4-9B74.381.90.00.0+74.3 [+62.5, +85.4]-7.6 [-15.3, +1.4]
GPT-OSS-20B79.993.80.70.0+79.2 [+68.7, +88.9]-13.9 [-23.6, -5.6]
OLMo-3-7B-Instruct70.177.11.40.0+68.8 [+54.9, +81.9]-6.9 [-15.3, +1.4]
Qwen3-14B-Base93.193.12.10.0+91.0 [+84.0, +96.5]0.0 [-2.1, +2.1]
Qwen3.5-2B80.683.32.80.0+77.8 [+71.5, +83.3]-2.8 [-7.6, +2.1]
Qwen3.5-4B92.493.82.80.0+89.6 [+84.0, +94.4]-1.4 [-5.6, +2.1]

Two senders, thirteen receivers. Four-answer success (%) on 144 scenes, and differences in points with 95% intervals. Reassignment is one reassignment of unrelated words at the same edit size. The appendix gives it with every column.

Reviewer_2dPsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SYNC, a protocol that lets a sender communicate facts to an unrelated receiver by transmitting only shared word indices and weights. Using a strict evaluation where zero success without a message is guaranteed by construction, SYNC achieves 72.0–98.3% success across 13 receivers from 7 families and 4 senders, shows that write location determines usability and predicts usable depths with a task-free probe and finds that receivers follow write strength more than printed weights.

Critical strengths
  1. Good evaluation design. Senders receivers were locked and timestamped before outcomes were observed, with strict development separation. Results carry three-seed envelope intervals, two-level cluster bootstrap CI. The “two-world, four-answer” metric converts the null into an arithmetic identity while the reassignment control shows receivers read symbol identity rather than reacting to a perturbation . This is consequential because it makes the core transfer claim unusually robust to leakage, guessing.

  2. The paper reports the total failure of its earlier sparse-extractor system , the failure of its originally locked probe on GLM and its replacement via a dated amendment, the exclusion of two MoE receivers and one sender on failed implementation checks with reasons given, and plainly states that the measurement supports no consistent advantage for equally-weighted facts. This is consequential because it greatly reduces publication bias and shows the positive results survive unusually strong adversarial checks.

Critical weaknesses
  1. The paper does not quantitatively compare against any prior method, and several closely related lines—ROME /MEMIT-style knowledge editing, causal tracing, Geva-style factual recall, and probing methodology are not cited or discussed. This is consequential because the core claims about where a fact must be written to become usable, closely overlap with existing causal-tracing and probing literatures; without positioning against them, the novelty and interpretive contribution are harder to assess.

  2. Limited benchmark scale and narrow generalization evidence. The evaluation relies on a small controlled attribute swapping task with only 20 active dictionary words, binary-choice questions and a fixed message format, and it does not test open-vocabulary items, relational facts, negation, quantity, temporal order, or comparison. This is consequential because the setup is well suited for isolating mechanistic binding failures。

Questions for authors
  1. "0.254 would instead assume independent uniform random guesses for the four answers" What quantity is 0.254?

  2. What architectural property predicts middle-layer write success, which ranges from 0.7% to 82.5% and is not monotone in scale?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer 2dPs (1/3)

Thank you for crediting the paper's evaluation design and disclosures.

W1. "The paper does not quantitatively compare against any prior method, and several closely related lines [...] are not cited or discussed."

The revision adds three quantitative comparisons against prior methods and a new paragraph in related work. Thank you for this one; it set the main work of our discussion period.

Latent communication. On the 576 four-object scenes of Appendix K.1, activation communication reaches at most 0.2% four-answer success across three receivers, whereas SYNC reaches 94.4 to 95.7%. KVComm (Shi et al., 2026) requires a shared model, so its sender is the receiver's own model reading the evidence, whereas SYNC sends messages from a Llama-3.1-8B-Instruct model of another family. Under these conditions, SYNC scores higher in three of nine comparisons—by up to 46.9 points—and shows no detectable difference in five; KVComm scores higher in one, by 4.0 points. A SYNC message averages 37 bytes, compared with 7.9 to 53.9 MB for KVComm. Before running these comparisons, we verified the released code against published numbers, reproducing KVComm's Table 8 within 0.04 and—using the implementation of activation communication bundled with KVComm—reproducing its published Tipsheets score (57.6% against 57%) (Appendix P.1).

We also compared SYNC against weight editing on the paper's two-object task across two receivers. Here the message carries complete facts, and no model acts as sender. This comparison is exploratory and uses 48 development scenes. We fixed the editor settings after an initial set-up fell short on paraphrased questions. The strongest setting in this comparison is MEMIT fitted on four prompt forms at strength 1.5. This setting was fitted on the deployment question, which SYNC's sender never sees. It then shows no detectable difference from SYNC in four-answer success on paraphrased questions. The values are 99.1 against 99.5 on Qwen3-32B, and 100 against 100 on Qwen3.5-27B.

When the editor does not see the deployment question either, it drops to 81.9 against 99.5 on Qwen3-32B while remaining at 100 on Qwen3.5-27B, showing that the gap depends on the receiver. In a confirmation run on Qwen3-32B across 96 scenes where no editing arm had been tested, SYNC reaches 98.8 on paraphrased questions. ROME reaches 44.8, MEMIT 36.9 and the strongest setting 87.0, each significantly below SYNC (Holm-corrected). We also tested a third receiver from another family, Granite-4.2-8B, chosen because SYNC's internal write passes on it. There, the strongest setting again shows no detectable difference from SYNC, while ROME and MEMIT fall 31.5 and 39.4 points below SYNC on paraphrased questions.

To deliver one set of four facts, SYNC writes in under 0.4 s, sends 20 bytes and modifies no weights. On the two Qwen receivers, this MEMIT setting takes 85 to 90 s to edit and restore and modifies 5 to 8 × 10⁸ weight elements. Weight edits can also alter model behaviour on unrelated inputs. In an additional exploratory check across 48 development scenes, we decoded prompts from an unrelated scene, without a message, after each edit. On Granite-4.2-8B, edits with ROME, MEMIT and the strongest setting alter 19.8, 20.5 and 23.6% of those answers, whereas SYNC's write disappears once the message ends and alters none.

On the two Qwen receivers, edits that pass their calibration control alter 1.7 to 6.9% of answers, which we report without making a claim. When the facts are provided in text, no edit turns a correct answer wrong on any receiver (Appendix P.19). Text also reaches 99.1 and 100 on the development scenes (Appendix P.12).

AuthorsOpenReview ↗

Reply to Reviewer 2dPs (1/3) — tables

The result tables for the answer above.

Table R14 — comparison with weight editing.

ArmFour-answer success (%) OriginalFour-answer success (%) ParaphrasedTime (s)Weights changed (10⁸)
Qwen3-32B
SYNC, internal write100.099.50.280
Text98.699.1-0
Editor fitted on the deployment question
ROME94.481.039.81.31
MEMIT100.073.132.97.86
MEMIT, four prompt forms100.088.485.57.86
MEMIT, four prompt forms, strength 1.5100.099.185.57.86
Editor does not see the question
ROME52.854.640.01.31
MEMIT40.348.634.77.86
MEMIT, four prompt forms, strength 1.579.281.984.57.86
Qwen3.5-27B
SYNC, internal write100.0100.00.370
Text100.0100.0-0
Editor fitted on the deployment question
ROMEnot read (calibration control not passed)
MEMIT100.0100.037.15.35
MEMIT, four prompt forms100.0100.090.25.35
MEMIT, four prompt forms, strength 1.5100.0100.090.25.35
Editor does not see the question
ROME63.964.894.20.89
MEMIT100.0100.036.45.35
MEMIT, four prompt forms, strength 1.5100.0100.085.15.35
Granite-4.2-8B, 48 development scenes
SYNC, internal write97.296.80.090
Text98.699.1-0
Editor fitted on the deployment question
ROME75.065.321.00.52
MEMIT77.857.48.42.10
MEMIT, four prompt formsnot read (calibration control not passed)
MEMIT, four prompt forms, strength 1.5100.094.017.82.10
Qwen3-32B, confirmation on 96 scenes
SYNC, internal write100.098.8-0
Text100.098.9-0
Editor does not see the question
ROME57.444.8--
MEMIT36.136.9--
MEMIT, four prompt forms, strength 1.590.787.0--

Comparison with weight editing. Four-answer success on the original and paraphrased questions, with the time and weight change needed to deliver one set of four facts. The question-fitted ROME arm on Qwen3.5-27B is not read, because it did not pass its calibration control. The complete table is in the appendix.

AuthorsOpenReview ↗

Reply to Reviewer 2dPs (2/3)

Continuing W1, quoted at the head of our previous comment — here the second half of it, the related literatures.

Section 5 of the revision adds a paragraph titled "Locating facts and how models bind them", citing Meng et al. (2022, 2023), Geva et al. (2023), Biran et al. (2024), Feng and Steinhardt (2024), Hewitt and Liang (2019) and Belinkov (2022).

Causal tracing and knowledge editing locate and modify facts that a model already stores in its weights (Meng et al., 2022). Our question is a different one: when another model supplies a fact at run time, where must it be written so that the receiver can use it?

An exploratory post hoc analysis across 16 receivers compares the two directly. On average, the half-height restoration window for fact recall extends 0.33 of the model's depth beyond the deepest depth where our internal write stays within 10 points of its peak. The end of the recall window alone therefore does not mark where to write (Appendix P.11).

The binding probe answers that question. We committed it before running the depth scans on the nine newer receivers, and its labels are correct in all 45 depth cells. A rule based on depth alone, fitted on the development cells, mislabels 7 (Table 2).

Related work. That new Section 5 paragraph states the distinction directly: "These studies concern knowledge a model already holds. SYNC delivers facts from another model at run time and edits no weights." The paper's native-state reference is itself activation patching at the message slots—the same type of intervention used in causal tracing.

Reviewer_aTXEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes SYNC, a training-free way for heterogeneous language models to share facts. The sender picks words from a fixed shared list of 221 symbols and sends their indices grouped by object. The receiver writes its own direction for each word, a mean activation difference against a neutral filler fitted once from generic contexts, into reserved slots at an early layer.

Selective fact use is tested with a two-world swap task that requires four correct answers. With settings fixed on development models, three new senders and nine newer receivers from five families reach 72.0 to 98.3% success, while pointing symbols to the wrong directions leaves at most 4.4%. Early writing beats middle-layer writing by up to 96 points, and a task-free probe predicts usable write depths in all 45 held-out cases. The setting also transfers to new layouts and a landmark task, and on two receivers write strength conveys graded weights better than printed numbers.

Critical strengths
  1. Rigorous experimental design. Settings are chosen on development receivers and fixed before confirmation on new senders and receivers, probe predictions are committed before measurement, and the controls are strong: symbol reassignment with matched edit size, complete-fact messages, and a native-state reference, with bootstrap intervals and Holm correction. The results are therefore credible.

  2. Useful findings about where to write. Middle-layer writing fails on most receivers while early writing works, the key-value restoration diagnostic gives a plausible reason, and the task-free probe makes the usable depth predictable for new models.

Critical weaknesses
  1. The benefit over plain text is unclear. For equal-weight facts, text is similar or better on every receiver, by up to 14 points on smaller ones, and leads by 32 to 63 points on the landmark task, while internal delivery is about 1.2 times slower. Appendix K.2 shows that adding input-embedding differences, which the paper notes is equivalent to reading the symbols as text, matches or beats the fitted directions. Since the sender already transmits word indices, the channel is essentially a restricted text channel, and the early-layer write may mostly recreate reading those words. The only clear advantage is graded weights on two receivers, and even there confidence words in text win on one of them.

Questions: What does SYNC enable that sending the same words as text cannot? Is there a setting where internal delivery wins on accuracy, bandwidth or cost? How does it compare with existing latent communication methods, for example activation communication on the shared Countries task?

  1. The task is narrow relative to the claims. Facts come from a closed inventory of single attributes, each scene has two objects and two attributes, and new scenes reuse symbols, question forms and some attribute combinations. Relational facts, open vocabulary and real multi-agent tasks are untested, two MoE receivers failed the implementation check, and senders reading from their own hidden states performed poorly. The title and introduction suggest general fact communication for multi-agent systems, which the experiments do not yet support.

Questions: Can you show results on open-vocabulary or relational facts, or on a downstream collaborative task?

Minor issues: The main text is dense and relies heavily on the appendices for key numbers and definitions, which makes it hard to follow.

Questions for authors

Please see the weaknesses above.

8 comments
AuthorsOpenReview ↗

Reply to Reviewer aTXE (1/4)

Thank you for describing the experimental design as rigorous and the findings about where to write as useful.

Your questions are not numbered in the review — the Questions field reads "Please see the weaknesses above" — so across our replies we number them Q1 to Q4 in the order they appear, and mark which weakness each one belongs to.

W1 · Q1. "What does SYNC enable that sending the same words as text cannot?"

With graded weights, internal delivery outperforms printing the same weights as numbers on nine of the ten receivers where the comparison can be read, by 12.7 to 59.7 points. The two routes carry the same indices, yet the receiver acts on them differently, and we tested both halves of your question. Thank you for putting this question first; we answer both halves of it.

First, the two routes differ when facts carry strength, as stated above. On an additional set of 108 scenes, internal delivery again outperforms printed weights on five of six receivers and matches them on Qwen3.6-27B. We later added four more receivers to this set, and the pattern holds there too. Internal delivery outperforms printed weights on three of them, by 41.7 to 92.6 points. The fourth cannot be read (Appendix P.13).

Second, the routes differ when a fact carries a sign, compared against printing the same words with "not" before the negated word. In the extended format, a negated word receives a negative weight. On a new bank for plain negation using messages from a Gemma-4 sender, the internal write reaches 73.6 to 81.9% on the four receivers that meet the success rule, whereas the same messages reach 24.3 to 50.0% when printed as words with "not" before the negated word.

On the other two receivers, which do not meet the rule, the routes are level on Mistral-Small-3.2 (50.7 against 51.4), and text scores higher on Qwen3.6-27B (25.0 against 34.7, where the control message generated by rule also fails) (Appendix P.16, Table R24). On a new bank that tests negation scope using messages generated by rule, five receivers meet the rule. The internal write outperforms the same message sent as text on four of them, by 19.4 to 45.8 points, and comes close on the fifth (Qwen3.5-27B, 79.9 against 81.9).

On Qwen3.6-27B, where the internal write does not meet the rule (43.1%), text reaches 61.8% (Table R28). These comparisons with text are descriptive: the registered tests compare the internal write against its controls. Answer coverage in the text arm is low on Qwen3-32B (70.5% on negation, 67.9% on scope) and on Granite-4.1-8B (80.9% on scope); on Mistral-Small-3.2 and Qwen3.5-9B, where coverage is at least 94.6%, the internal write leads on scope by 19.4 and 22.2 points.

Where a fact is a name of several tokens, the arms at one position separate as follows: writing the input embedding of the symbol's first, last or mean token at that position gives at most 5.8% four-answer success on Countries, the receiver's own state for the name gives 50.9 to 74.5%, the name sent as text gives 87.1 to 97.2%, and the fitted write at the same position and layer gives 23.9 to 64.0% (Appendix P.7). That comparison is about what one position can be made to carry, not about what text can do; where facts carry strength the two routes do separate, as the comparison in this reply's opening sentence shows.

When facts carry equal weights, the two routes transmit the same indices, and Section 6 gives the comparison with text. Unlike methods that exchange internal states, SYNC needs neither a shared architecture nor a learned translator: the models share a word list, and each receiver fits its own direction for each word.

Across every task that we added, text performs as follows. For open-vocabulary words with complete facts, there is no difference on the three larger receivers, while text scores 9 to 14 points higher on the three smaller receivers. For relational facts, text reaches 63.2 to 100%. With two senders, there is no difference on twelve of thirteen receivers, while text scores 13.9 points higher on gpt-oss-20b. For temporal order, text reaches 43.8 to 99.3%.

For comparative words, the appendix reports text solely as an interpretability check, which is met on two of the five receivers that pass the rule. For count facts with lookup questions, the scene's sentences reach 99.3 to 100% and the word list reaches 59.7 to 79.9%, placing our internal write 17.4 to 47.2 points below the sentences and from 13.2 points below to 2.8 points above the word list. Negation and scope are covered above. On these fact types we establish a capability; the graded setting above is where internal delivery is also more accurate than the same weights printed as numbers.

AuthorsOpenReview ↗

Reply to Reviewer aTXE (2/4)

W1. "The early-layer write may mostly recreate reading those words."

We tested this directly in two comparisons against input embeddings—one with graded weights and one on Countries—and the fitted write scores higher in both. Thank you for stating the alternative explanation precisely, that the write may “mostly recreate reading those words”. These two are the settings that we tested; they do not cover every early-layer write.

With graded weights, we applied identical scaling to input embeddings and to fitted directions. Fitted directions score higher on all nine receivers where the comparison can be read: by 7.9 and 38.9 points on the two receivers in Table 10, and by 15.5 to 31.0 points on seven of ten additional receivers (Holm-corrected). When the competing object's value has weight 128 of 255, the fitted write reaches 95.1% on Qwen3.5-27B, whereas the scaled embedding reaches 29.2%. Two receivers fail our prespecified positive control, so their comparison cannot be read, and one encountered an engineering failure on this arm (Appendices D and P.6).

Without weights, on Countries—where most names span several tokens—writing the fitted direction at one position yields 23.9 to 64.0% four-answer success across four receivers. At the same position and layer, writing the input embedding of the first, last or mean token yields at most 5.8% (Appendix P.7 reports the remaining arms).

Appendix K.2 covers the remaining case—single-token facts at equal weight—where a unit difference in input embeddings is equivalent to reading the word as text; the two results above fall outside that regime. Appendices D, P.6 and P.13 report the text formats that we tested, including text preprocessed before rendering.

W1 · Q2. "Is there a setting where internal delivery wins on accuracy, bandwidth or cost?"

On accuracy, internal delivery outperforms printed weights on nine of the ten receivers where the comparison can be read. Thank you for asking it in three parts, “accuracy, bandwidth or cost”; each is answered separately below. Against confidence words, it scores higher on six of nine and lower on three. Preprocessed text reaches 100.0% on all six receivers in Table R15 because it resolves the conflict before sending: a preprocessing step keeps the highest-weighted value for each object and attribute, so that message carries no weight at all. The comparison therefore separates into two kinds of format. Among formats that do transmit the strengths, the internal write is ahead on most receivers — above printed weights on nine of ten, and above confidence words on six of nine, behind on three, including Qwen3.5-27B by 8.8. The format that reaches 100.0% is not one of them: it resolves the conflict in the sender and transmits a single value. Cost and bandwidth are covered in the last two bullets below.

AuthorsOpenReview ↗

Reply to Reviewer aTXE (2/4) — accuracy, bandwidth and cost

The itemised evidence for the second question of the previous comment.

  • Accuracy. Internal delivery exceeds printed weights on nine of the ten receivers where the comparison can be read. On Qwen3-32B the margin is 36.1 points and on Qwen3.5-27B 33.8 (Table 10); on seven of ten additional receivers it is 12.7 to 59.7 points (Holm-corrected; Appendix P.6). Among those ten, printed weights score higher on one (Qwen3.6-27B, by 14.6), and two cannot be read. Against confidence words—the text format that you point to—internal delivery scores higher on six of the nine receivers where the comparison can be read and lower on three, including Qwen3.5-27B (which you noted) by 8.8. Of the remaining three receivers, two fail the positive control and one encountered an engineering failure on this arm (Appendices D and P.6). On the additional set of 108 scenes, Table R15 also reports preprocessed text. Before rendering, a preprocessing step retains the highest-weighted value for each object and attribute (Appendix D), so the resulting message states complete facts and carries no weight. This format reaches 100.0% on all six receivers. Of the four additional receivers listed in the same appendix, OLMo-3-7B-Instruct is excluded because its positive control—the full-weight write—reaches 89.8%, whereas our prespecified rule requires 90%. The table also reports natural-language confidence words: they match or exceed the internal write on four of the six receivers above, by 0.5 to 3.2 points, whereas the internal write scores higher on the other two, by 25.4 and 6.5 points. Because that column carries no interval, these values are point differences. Preprocessing resolves the conflict upstream by keeping the highest-weighted value for each object and attribute and sending only that one. By contrast, the internal write transmits the strengths, leaving the receiver to resolve the conflict itself at 77.8 to 97.2% across the six receivers of Table R15.
  • Bandwidth. The two routes carry the same indices; Appendix J.6 reports their serialized sizes, which do not form an equal-budget comparison. Relative to latent methods, the gap is large: a SYNC message averages 37 bytes, whereas a KVComm message requires 7.9 to 53.9 MB (Appendix P.1).
  • Cost. When each receiver saves its neutral states once, its warm median latency is 0.99 to 1.02 times that of text across all thirteen receivers (Appendix P.10). This saving holds whenever the neutral prefix is reused (we timed it with complete-fact messages), which requires keeping the object names, slot layout and weight annotations fixed. The paper reports a ratio of 1.19 to 1.28 times (Section 6; medians in Table 31). That ratio reflects a set-up that repeats a neutral forward pass for every message.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Comment to all reviewers (1/2)

We thank the three reviewers and the area chair for your time and comments. Since the submission, we have run the comparisons and tested the fact types that the three reviews asked for; the four results below are the main new evidence. We are glad that the reviewers find the experimental design rigorous, with settings fixed on development receivers before confirmation and strong controls (aTXE). They find the evaluation well designed, which makes the paper's central claim about transfer unusually robust to leakage and guessing (2dPs). They note the wide range of experiments and control conditions (9qmD). We are also glad that Reviewer aTXE finds our results on where to write useful. In detail:

  1. Graded weights. Internal delivery outperforms printing the same weights as numbers on nine of the ten receivers where the comparison can be read (two of which were in the submission).
  2. Prior methods. On 576 four-object scenes, SYNC reaches 94.4 to 95.7%, whereas activation communication reaches at most 0.2%; the first table below lists every comparison.
  3. Fact types. Each fact type that you asked for now has results on six receivers (second table below), and Table 3 of the main text reports three of them.
  4. The probe. With its threshold frozen, the probe correctly labels 24 of 25 depth cells on five additional receivers and 41 of 44 on two question banks that it was not tuned on (Appendices P.8 and P.14).
SYNC compared withSettingResultAppendix
Weights printed as numbersgraded weights; the ten receivers where the comparison can be readinternal delivery higher on 9, by 12.7 to 59.7 points; printed weights higher on 1, by 14.6D, P.6
Scaled input embeddingsgraded weights; the nine receivers where the comparison can be readfitted directions higher on all 9, by 7.9 to 38.9 pointsD, P.6
Activation communication576 four-object scenes, three receivers; and the Countries scenes, Qwen3-32Bat most 0.2%, against 94.4 to 95.7% for SYNC; on Countries 0.0% against 46.5% (SYNC with a Gemma-4 sender, Table 3)P.1; Countries: reply to aTXE only
KVCommthe same scenes; three receivers at three fractions of shared layersSYNC higher in 3 of 9 comparisons, no detectable difference in 5, KVComm higher in 1; 37 bytes per message against 7.9 to 53.9 MBP.1
Input embedding written at one positionCountries, four receivers, names of several tokensfitted direction 23.9 to 64.0%, input embedding of the first, last or mean token at most 5.8%P.7
Text, latencythirteen receivers, warm medians0.99 to 1.02 times text once a receiver saves its neutral states, 1.24 to 1.30 times when a neutral pass is repeated for every messageP.10
Weight editing (ROME, MEMIT, strongest MEMIT setting)confirmation on Qwen3-32B, 96 scenes, paraphrased questions, editor fitted without the deployment question44.8, 36.9 and 87.0%, each significantly below SYNC's 98.8%; in an exploratory comparison the strongest setting, fitted on the deployment question that SYNC's sender never sees, is not detectably different from SYNC on two receivers (99.1 against 99.5, and 100 against 100); SYNC changes no weightsP.12
Added taskReceivers that meet the success ruleInternal write, four-answer successAppendix
Open-vocabulary words6 of 6 significantly above the control79.9 to 100.0%O
Relational facts6 of 684.0 to 98.6%P.2
Comparison5 of 656.2 to 95.1% on those fiveP.3
Negation scope6 of 666.0 to 99.3%P.4
Negation4 of 668.1 to 84.0% on those fourP.4
Temporal order4 of 674.3 to 87.5% on those fourP.4
Count facts with lookup questions4 of 659.7 to 82.6% on those fourP.15
Two senders of different families13 of 13 significantly above the control70.1 to 94.4%P.5

Wherever a reply states that a receiver meets the success rule, that rule was fixed before the run; each rule requires at least 50% four-answer success and a lead over the reassignment control that is significant after Holm correction (full criteria in Appendix P). The rows from relational facts through count facts use an extended message format, which the appendices describe.

AuthorsOpenReview ↗

Comment to all reviewers (2/2)

Cost. SYNC delivers one set of four facts in under 0.4 s without changing any weights. The weight-editing setting in the table takes 85 to 90 s to edit and restore, and modifies 5 to 8 × 10⁸ weight elements (Appendix P.12). SYNC messages average 37 bytes, compared with 7.9 to 53.9 MB for KVComm (Appendix P.1). Text reaches 99.1 and 100% on those scenes: these cost figures are against weight editing, and against text the comparison is the graded setting, where the write is above printed weights on nine of ten receivers.

Appendices O and P of the revision report the new experiments, and Appendix D adds one comparison. In the main text, changes appear in the abstract and in Sections 1, 2.1, 4, 5 and 6, including the lower part of Table 3. We have uploaded the revised manuscript with this response; changes are marked in blue.

AuthorsOpenReview ↗

Author Summary for the Area Chair

We thank the reviewers and the Area Chair. This comment collects four things in one place: what the reviewers agree on, the main new evidence added during the discussion, the points where views still differ, and where the revised PDF changed.

1. What the reviewers appreciate — in their own words

  • Reviewer aTXE — "Rigorous experimental design. Settings are chosen on development receivers and fixed before confirmation on new senders and receivers, probe predictions are committed before measurement, and the controls are strong … The results are therefore credible." aTXE also records "Useful findings about where to write."
  • Reviewer 2dPs — "Good evaluation design. Senders receivers were locked and timestamped before outcomes were observed, with strict development separation. … This is consequential because it makes the core transfer claim unusually robust to leakage, guessing."
  • Reviewer 9qmD — "The paper conducts a wide range of experiments to evaluate the proposed method, varying senders, receivers, and write depths. The experiments also include several control conditions that help characterize the behavior of the proposed method."

2. Main new evidence added during the discussion

Four items, one conclusion and one number each. Every number in this comment already appears in our posted replies.

  1. Comparison with prior methods, which Reviewer aTXE asked for (W1). Against weight editing on a 96-scene confirmation set on Qwen3-32B, SYNC reaches 98.8% four-answer success on paraphrased questions, against 44.8% for ROME and 36.9% for MEMIT, each significantly below SYNC after Holm correction.
  2. Graded weights, the setting the paper claims. Internal delivery exceeds the same weights printed as numbers on nine of the ten receivers where the comparison can be read, by 12.7 to 59.7 points.
  3. Open-vocabulary messages. With no word list supplied, the internal write reaches 79.9 to 100.0% and is significantly above its control on six of six receivers.
  4. Two senders from different families. With each scene's evidence split between two senders of different families, internal delivery reaches 70.1 to 94.4% across thirteen receivers.

3. Points where views still differ

No reviewer has replied to our response during the discussion period, so there is no remaining difference of view to record here. The requests made in the three reviews — a comparison with prior methods, further fact types, the properties that predict usable write depths, and where graded weights are useful — are answered in the comment to all reviewers and in the appendices named there. If a reviewer replies before the discussion closes, any remaining difference will be recorded here.

4. Changes in the revised PDF

Section-level only; every change is marked in blue in the revised PDF. The abstract; Section 1 (introduction); Section 2 (method); Section 4 (results); Section 5 (related work); Section 6 (discussion, limitations and conclusion); and Appendices A, C, D, H, J, N, O and P. The experiments added during the discussion are collected in Appendix P, which also indexes what each one answers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.