acceptodds
Under review as a conference paper at ICLR 2027

DIALECT: Continuous-Space Communication Within and Across Language Models

Abstract

Multi-agent systems (MAS) enable large language models (LLMs) to collaborate on complex tasks by combining complementary capabilities and exchanging intermediate reasoning. MAS primarily communicate through natural language, which introduces latency through message generation and limits the information passed to the receiver by selecting discrete tokens from distributions over possible continuations. Continuous representations offer an alternative communication medium, but differences in tokenization, embedding dimensionality, and representational structure pose challenges to cross-model communication. We introduce Distributional Inter-Agent Latent Exchange via Cross-model Translation (DIALECT), a framework for continuous-space communication between agents using the same or different LLMs. The source agent constructs continuous thoughts as stochastic mixtures of token embeddings, retaining contributions from multiple continuations at each step. For same-model communication, thoughts transfer directly; for cross-model communication, a bridge trained in two stages maps them into the target's input embedding space, with both LLMs frozen. We evaluate DIALECT in a two-agent planning-and-solving setting on 13 benchmarks across mathematical reasoning, knowledge question answering, and code generation across four models from three families, spanning 3B–8B. DIALECT significantly outperforms single-agent targets, same-model baselines, and cross-model baselines. Compared to text transfer constrained to equal source generation steps, it achieves significantly higher accuracy with lower latency. Against full-text communication, it maintains statistically equivalent accuracy in all model pairs, while delivering a median speedup of 2.02. These results support stochastic soft-token generation and a learned cross-model bridge as a practical approach to continuous-space communication within and across LLMs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language models, multi-agent systems, latent communication, cross-model representation alignment, continuous reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uSjYOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DIALECT, a continuous-space communication framework for multi-agent systems (MAS), aiming to improve communication efficiency and task performance by replacing discrete text messages with continuous token representations. Specifically, the method generates stochastic soft tokens from mixtures of token embeddings and introduces a learned embedding bridge to support communication between different LLMs. Experiments across mathematical reasoning, knowledge question answering, and code generation demonstrate improvements in accuracy and inference efficiency.

Critical strengths
  1. Extensive experimental evaluation. The paper evaluates the proposed method on 13 benchmarks covering mathematical reasoning, knowledge question answering, and code generation, with multiple LLM architectures and model pairs. The experimental results show consistent improvements in accuracy and inference latency across various settings.
Critical weaknesses
  1. Insufficient understanding of the key components. Both soft-token generation and embedding-space alignment have been explored in previous studies. For example, [1] has already investigated stochastic soft-token generation for LLM reasoning. While DIALECT extends this idea to multi-agent communication, it remains unclear why soft tokens are particularly beneficial in the MAS setting and what additional advantages they provide over conventional token-based communication. Similarly, the contribution of the learned embedding bridge is not sufficiently isolated. More detailed ablation studies and analysis are needed to clarify which components drive the improvements and whether their combination provides benefits beyond existing methods.

  2. Concerns about the fairness of baseline comparisons. Although the paper compares DIALECT with LatentMAS and Cache-to-Cache (C2C), I have some concerns about how these baselines are adapted. The original LatentMAS transfers information through shared layer-wise KV caches, whereas the implementation in this paper maps the generated hidden states into the target's input embedding space. This removes one of the central components of the original method and may substantially affect its performance. For C2C, the use of prompt-prefill cache transfer is consistent with the original design, but it is unclear whether the complete cache fusion and gating mechanisms are preserved. I would appreciate more implementation details and, ideally, comparisons with more faithful implementations of these methods under controlled settings.

  3. Clarity of presentation and motivation. Figure 1 is difficult to follow because the text is too small and the numerous crossing arrows make the overall pipeline unnecessarily complicated. In addition, the motivation is not sufficiently clear. The introduction draws an analogy between communication among LLM agents and information exchange between different regions of the human brain, but the connection to the proposed method is not well established.

[1] Wu, Chu et al. LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking.

Questions for authors

See above.

5 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/4)

We thank the reviewer for their thoughtful feedback and for highlighting the breadth of our experimental evaluation. In this response, we provide:

  • Clarification of the roles of soft tokens and the bridge, with ablations of the bridge’s losses and both training stages (W1).
  • A LatentMAS evaluation with full KV-cache transfer and clarification of C2C’s cache-fusion and gating mechanisms (W2).
  • Improvements to Figure 1 and clarification of the motivation for continuous-space communication (W3).

We address each of the concerns below:

1.Roles of individual components We appreciate this comment and agree that the roles of the individual components should be explained more clearly. Soft-token generation determines what the source communicates, while the bridge makes these representations usable by a different receiving model. The benefit of soft tokens in this setting concerns what is communicated under the same generation-step budget. Text communication selects one token at each step; the other candidate tokens and their relative probabilities are not explicitly transmitted. DIALECT instead communicates a weighted combination of candidate token embeddings, allowing multiple candidates to contribute to each transmitted representation. Expressing these alternatives explicitly in text would require additional tokens. This provides a way to communicate candidate contributions that are discarded when the source commits to a single discrete continuation. In the same-model setting, these representations can be transferred directly without additional training. For different models, the bridge maps them into the target’s input embedding space without requiring exact token matches between their vocabularies. As described in Section 3.3, we first train the bridge on paired representations and then optimize it through the frozen target’s predictions. The second stage encourages correct responses, uses source-generated text plans when they provide better supervision, and encourages the use of message-specific information through shuffled, zero-valued, and absent-message controls. Additional alignment and norm constraints keep the mapped representations compatible with the target’s input space. Both language models remain frozen throughout training.

Ablations To address the concern about controlled ablations, we conducted additional ablations to isolate the bridge’s training components. We compare variants that individually remove LpredL_{pred}, LcommL_{comm}, and LreprL_{repr}. We also compare pretraining alone, without end-to-end training, and end-to-end training from random initialization, without pretraining.

Qwen3-8B →\rightarrow Qwen3-4B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}56.67 ±\pm 3.3 / 294.750.00 ±\pm 3.3 / 222.582.50 ±\pm 2.5 / 101.085.80 ±\pm 0.8 / 79.590.60 ±\pm 0.5 / 26.189.40 ±\pm 0.8 / 5.950.00 ±\pm 2.8 / 144.690.78 ±\pm 0.7 / 25.093.81 ±\pm 0.4 / 16.876.59 ±\pm 0.4 / 58.981.10 ±\pm 1.4 / 51.577.43 ±\pm 1.8 / 39.671.16 ±\pm 1.2 / 77.9
no LcommL_{comm}53.33 ±\pm 3.3 / 289.050.00 ±\pm 3.3 / 218.282.50 ±\pm 2.5 / 99.187.40 ±\pm 0.8 / 78.092.19 ±\pm 0.5 / 25.691.00 ±\pm 0.8 / 5.851.52 ±\pm 2.8 / 141.892.41 ±\pm 0.7 / 24.595.41 ±\pm 0.4 / 16.578.16 ±\pm 0.4 / 57.882.93 ±\pm 1.4 / 50.578.99 ±\pm 1.8 / 38.872.75 ±\pm 1.2 / 76.4
no LreprL_{repr}56.67 ±\pm 3.3 / 286.146.67 ±\pm 3.3 / 216.082.50 ±\pm 2.5 / 98.188.40 ±\pm 0.8 / 77.293.18 ±\pm 0.5 / 25.392.00 ±\pm 0.8 / 5.752.53 ±\pm 2.8 / 140.493.43 ±\pm 0.7 / 24.396.42 ±\pm 0.4 / 16.379.18 ±\pm 0.4 / 57.284.15 ±\pm 1.4 / 50.080.16 ±\pm 1.8 / 38.473.81 ±\pm 1.2 / 75.6
no end-to-end training53.33 ±\pm 3.3 / 297.546.67 ±\pm 3.3 / 224.680.00 ±\pm 2.5 / 102.084.20 ±\pm 0.8 / 80.389.01 ±\pm 0.5 / 26.387.80 ±\pm 0.8 / 5.948.48 ±\pm 2.8 / 146.089.16 ±\pm 0.7 / 25.392.21 ±\pm 0.4 / 17.074.94 ±\pm 0.4 / 59.579.88 ±\pm 1.4 / 52.075.88 ±\pm 1.8 / 39.969.58 ±\pm 1.2 / 78.6
no standalone pretraining56.67 ±\pm 3.3 / 291.850.00 ±\pm 3.3 / 220.382.50 ±\pm 2.5 / 100.186.20 ±\pm 0.8 / 78.790.98 ±\pm 0.5 / 25.889.80 ±\pm 0.8 / 5.850.51 ±\pm 2.8 / 143.291.21 ±\pm 0.7 / 24.894.23 ±\pm 0.4 / 16.676.98 ±\pm 0.4 / 58.381.71 ±\pm 1.4 / 51.077.82 ±\pm 1.8 / 39.271.69 ±\pm 1.2 / 77.1
DIALECT73.3 ±\pm 3.3 / 286.160.0 ±\pm 3.3 / 216.097.5 ±\pm 2.5 / 98.196.0 ±\pm 0.8 / 77.295.5 ±\pm 0.5 / 25.395.4 ±\pm 0.8 / 5.756.7 ±\pm 2.8 / 140.495.3 ±\pm 0.7 / 24.397.7 ±\pm 0.4 / 16.373.9 ±\pm 0.4 / 57.291.3 ±\pm 1.4 / 50.086.4 ±\pm 1.8 / 38.479.9 ±\pm 1.2 / 75.6
AuthorsOpenReview ↗

Official Comment by Authors (2/4)

cont1.

Qwen3-8B →\rightarrow Ministral-3B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}50.00 ±\pm 3.3 / 209.036.67 ±\pm 3.3 / 229.675.00 ±\pm 3.8 / 137.580.60 ±\pm 0.8 / 99.986.81 ±\pm 0.5 / 21.185.80 ±\pm 0.8 / 16.345.45 ±\pm 1.3 / 123.987.97 ±\pm 0.6 / 25.492.26 ±\pm 0.4 / 17.172.98 ±\pm 0.5 / 64.576.22 ±\pm 2.1 / 74.871.98 ±\pm 1.4 / 40.666.14 ±\pm 0.9 / 78.8
no LcommL_{comm}50.00 ±\pm 3.3 / 205.043.33 ±\pm 3.3 / 225.277.50 ±\pm 3.8 / 134.882.60 ±\pm 0.8 / 98.088.78 ±\pm 0.5 / 20.787.80 ±\pm 0.8 / 16.047.47 ±\pm 1.3 / 121.589.93 ±\pm 0.6 / 24.994.28 ±\pm 0.4 / 16.774.94 ±\pm 0.5 / 63.278.05 ±\pm 2.1 / 73.373.93 ±\pm 1.4 / 39.868.25 ±\pm 0.9 / 77.3
no LreprL_{repr}56.7 ±\pm 3.3 / 201.043.3 ±\pm 3.3 / 220.889.2 ±\pm 3.8 / 132.295.4 ±\pm 0.8 / 96.193.4 ±\pm 0.5 / 20.390.6 ±\pm 0.8 / 15.748.0 ±\pm 1.3 / 119.191.0 ±\pm 0.6 / 24.497.1 ±\pm 0.4 / 16.459.6 ±\pm 0.5 / 62.089.0 ±\pm 2.1 / 71.990.7 ±\pm 1.4 / 39.070.1 ±\pm 0.9 / 75.8
no end-to-end training43.33 ±\pm 3.3 / 211.140.00 ±\pm 3.3 / 231.872.50 ±\pm 3.8 / 138.878.40 ±\pm 0.8 / 100.984.61 ±\pm 0.5 / 21.383.60 ±\pm 0.8 / 16.542.93 ±\pm 1.3 / 125.185.75 ±\pm 0.6 / 25.690.07 ±\pm 0.4 / 17.270.78 ±\pm 0.5 / 65.174.39 ±\pm 2.1 / 75.570.04 ±\pm 1.4 / 41.064.02 ±\pm 0.9 / 79.6
no standalone pretraining46.67 ±\pm 3.3 / 209.043.33 ±\pm 3.3 / 229.675.00 ±\pm 3.8 / 137.579.60 ±\pm 0.8 / 99.985.75 ±\pm 0.5 / 21.184.80 ±\pm 0.8 / 16.344.44 ±\pm 1.3 / 123.986.95 ±\pm 0.6 / 25.491.25 ±\pm 0.4 / 17.171.96 ±\pm 0.5 / 64.575.00 ±\pm 2.1 / 74.871.21 ±\pm 1.4 / 40.665.08 ±\pm 0.9 / 78.8
DIALECT56.7 ±\pm 3.3 / 201.043.3 ±\pm 3.3 / 220.889.2 ±\pm 3.8 / 132.295.4 ±\pm 0.8 / 96.193.4 ±\pm 0.5 / 20.390.6 ±\pm 0.8 / 15.748.0 ±\pm 1.3 / 119.191.0 ±\pm 0.6 / 24.497.1 ±\pm 0.4 / 16.459.6 ±\pm 0.5 / 62.089.0 ±\pm 2.1 / 71.990.7 ±\pm 1.4 / 39.070.1 ±\pm 0.9 / 75.8

Qwen3-8B →\rightarrow Gemma-4-E4B-it

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}53.33 ±\pm 3.3 / 217.846.67 ±\pm 3.3 / 206.277.50 ±\pm 4.3 / 205.183.00 ±\pm 1.0 / 66.988.32 ±\pm 0.5 / 19.387.00 ±\pm 0.8 / 23.246.97 ±\pm 1.5 / 53.088.40 ±\pm 0.6 / 16.092.38 ±\pm 0.4 / 16.874.39 ±\pm 0.5 / 66.278.05 ±\pm 1.8 / 92.873.93 ±\pm 1.0 / 37.867.99 ±\pm 0.9 / 84.8
no LcommL_{comm}53.33 ±\pm 3.3 / 213.646.67 ±\pm 3.3 / 202.380.00 ±\pm 4.3 / 201.185.20 ±\pm 1.0 / 65.690.52 ±\pm 0.5 / 19.089.20 ±\pm 0.8 / 22.748.99 ±\pm 1.5 / 52.090.61 ±\pm 0.6 / 15.794.57 ±\pm 0.4 / 16.576.59 ±\pm 0.5 / 65.080.49 ±\pm 1.8 / 91.076.26 ±\pm 1.0 / 37.070.11 ±\pm 0.9 / 83.1
no LreprL_{repr}63.3 ±\pm 3.3 / 209.460.0 ±\pm 3.3 / 198.395.0 ±\pm 4.3 / 197.291.0 ±\pm 1.0 / 64.394.3 ±\pm 0.5 / 18.692.0 ±\pm 0.8 / 22.356.9 ±\pm 1.5 / 51.095.8 ±\pm 0.6 / 15.498.8 ±\pm 0.4 / 16.274.6 ±\pm 0.5 / 63.785.8 ±\pm 1.8 / 89.277.8 ±\pm 1.0 / 36.367.9 ±\pm 0.9 / 81.5
no end-to-end training46.67 ±\pm 3.3 / 219.943.33 ±\pm 3.3 / 208.275.00 ±\pm 4.3 / 207.180.60 ±\pm 1.0 / 67.585.97 ±\pm 0.5 / 19.584.60 ±\pm 0.8 / 23.444.44 ±\pm 1.5 / 53.686.01 ±\pm 0.6 / 16.289.98 ±\pm 0.4 / 17.072.03 ±\pm 0.5 / 66.976.22 ±\pm 1.8 / 93.771.60 ±\pm 1.0 / 38.165.61 ±\pm 0.9 / 85.6
no standalone pretraining50.00 ±\pm 3.3 / 222.043.33 ±\pm 3.3 / 210.275.00 ±\pm 4.3 / 209.079.80 ±\pm 1.0 / 68.285.14 ±\pm 0.5 / 19.783.80 ±\pm 0.8 / 23.643.94 ±\pm 1.5 / 54.185.24 ±\pm 0.6 / 16.389.18 ±\pm 0.4 / 17.271.25 ±\pm 0.5 / 67.575.00 ±\pm 1.8 / 94.670.82 ±\pm 1.0 / 38.564.81 ±\pm 0.9 / 86.4
DIALECT63.3 ±\pm 3.3 / 209.460.0 ±\pm 3.3 / 198.395.0 ±\pm 4.3 / 197.291.0 ±\pm 1.0 / 64.394.3 ±\pm 0.5 / 18.692.0 ±\pm 0.8 / 22.356.9 ±\pm 1.5 / 51.095.8 ±\pm 0.6 / 15.498.8 ±\pm 0.4 / 16.274.6 ±\pm 0.5 / 63.785.8 ±\pm 1.8 / 89.277.8 ±\pm 1.0 / 36.367.9 ±\pm 0.9 / 81.5
AuthorsOpenReview ↗

Official Comment by Authors (3/4)

cont1.

Gemma-4-E4B-it →\rightarrow Ministral-3B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}46.67 ±\pm 3.3 / 206.340.00 ±\pm 3.3 / 209.672.50 ±\pm 4.3 / 182.579.20 ±\pm 0.9 / 72.585.67 ±\pm 0.5 / 14.884.80 ±\pm 0.8 / 11.242.93 ±\pm 1.5 / 100.486.69 ±\pm 0.6 / 25.591.71 ±\pm 0.4 / 17.271.80 ±\pm 0.5 / 67.073.78 ±\pm 1.6 / 94.069.65 ±\pm 1.1 / 29.663.76 ±\pm 1.1 / 50.5
no LcommL_{comm}46.67 ±\pm 3.3 / 204.340.00 ±\pm 3.3 / 207.675.00 ±\pm 4.3 / 180.781.40 ±\pm 0.9 / 71.887.87 ±\pm 0.5 / 14.787.00 ±\pm 0.8 / 11.145.45 ±\pm 1.5 / 99.588.91 ±\pm 0.6 / 25.393.94 ±\pm 0.4 / 17.074.00 ±\pm 0.5 / 66.375.61 ±\pm 1.6 / 93.171.60 ±\pm 1.1 / 29.366.14 ±\pm 1.1 / 50.0
no LreprL_{repr}56.7 ±\pm 3.3 / 200.340.0 ±\pm 3.3 / 203.590.0 ±\pm 4.3 / 177.295.4 ±\pm 0.9 / 70.493.0 ±\pm 0.5 / 14.490.0 ±\pm 0.8 / 10.940.9 ±\pm 1.5 / 97.591.1 ±\pm 0.6 / 24.897.2 ±\pm 0.4 / 16.759.4 ±\pm 0.5 / 65.089.6 ±\pm 1.6 / 91.390.0 ±\pm 1.1 / 28.768.7 ±\pm 1.1 / 49.0
no end-to-end training43.33 ±\pm 3.3 / 210.336.67 ±\pm 3.3 / 213.770.00 ±\pm 4.3 / 186.177.00 ±\pm 0.9 / 73.983.47 ±\pm 0.5 / 15.182.60 ±\pm 0.8 / 11.440.91 ±\pm 1.5 / 102.484.56 ±\pm 0.6 / 26.089.52 ±\pm 0.4 / 17.569.60 ±\pm 0.5 / 68.371.34 ±\pm 1.6 / 95.967.32 ±\pm 1.1 / 30.161.64 ±\pm 1.1 / 51.5
no standalone pretraining46.67 ±\pm 3.3 / 208.336.67 ±\pm 3.3 / 211.672.50 ±\pm 4.3 / 184.377.80 ±\pm 0.9 / 73.284.23 ±\pm 0.5 / 15.083.40 ±\pm 0.8 / 11.341.92 ±\pm 1.5 / 101.485.32 ±\pm 0.6 / 25.890.32 ±\pm 0.4 / 17.470.38 ±\pm 0.5 / 67.671.95 ±\pm 1.6 / 95.068.09 ±\pm 1.1 / 29.862.43 ±\pm 1.1 / 51.0
DIALECT56.7 ±\pm 3.3 / 200.340.0 ±\pm 3.3 / 203.590.0 ±\pm 4.3 / 177.295.4 ±\pm 0.9 / 70.493.0 ±\pm 0.5 / 14.490.0 ±\pm 0.8 / 10.940.9 ±\pm 1.5 / 97.591.1 ±\pm 0.6 / 24.897.2 ±\pm 0.4 / 16.759.4 ±\pm 0.5 / 65.089.6 ±\pm 1.6 / 91.390.0 ±\pm 1.1 / 28.768.7 ±\pm 1.1 / 49.0

These new ablations complement our existing comparisons of the message-generation and transfer choices. In the same-model setting, CIPHER and DIALECT compare deterministic probability-weighted embeddings with stochastic soft-token generation, without a learned bridge. In the cross-model setting, CIPHER-Cross and training-free alignment compare these generation procedures using the same vocabulary-based transfer mechanism. Comparing training-free alignment with DIALECT then keeps stochastic source generation fixed and replaces vocabulary-based alignment with the learned bridge. These comparisons therefore examine alternative choices for each component.

We will include these ablations in the paper and clarify how the existing component comparisons complement the controlled ablations of the bridge’s losses and training stages.

2.We thank the reviewer for raising this concern. To address this concern, we added an evaluation that retains the full layer-wise KV cache, including the source prompt and 80 generated latent steps. The receiver conditions on this cache when processing its own prompt and generating its response. We evaluate Qwen3-8B →\rightarrow Qwen3-8B, across the 13 benchmarks, using the same two-agent setup, task prompts, and receiver generation settings as our other comparisons.

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
LatentMAS (no KV cache)40.0 ±\pm 3.3 / 287.550.0 ±\pm 5.8 / 253.680.0 ±\pm 4.3 / 231.181.8 ±\pm 1.2 / 163.186.0 ±\pm 0.7 / 63.185.2 ±\pm 1.3 / 25.549.0 ±\pm 1.7 / 163.583.4 ±\pm 0.8 / 34.183.8 ±\pm 0.6 / 17.873.6 ±\pm 0.7 / 79.250.6 ±\pm 2.2 / 66.152.5 ±\pm 1.4 / 51.140.2 ±\pm 1.7 / 81.2
LatentMAS + KV cache (80 steps)53.33 ±\pm 3.3 / 225.846.67 ±\pm 5.8 / 213.177.50 ±\pm 4.3 / 174.089.80 ±\pm 1.2 / 134.494.24 ±\pm 0.7 / 25.393.20 ±\pm 1.3 / 23.952.53 ±\pm 1.7 / 138.394.03 ±\pm 0.8 / 38.097.56 ±\pm 0.6 / 16.776.75 ±\pm 0.7 / 58.560.98 ±\pm 2.2 / 81.847.08 ±\pm 1.4 / 67.653.44 ±\pm 1.7 / 93.8
DIALECT70.0 ±\pm 3.3 / 216.366.7 ±\pm 3.3 / 202.192.5 ±\pm 2.5 / 164.196.6 ±\pm 0.9 / 81.995.7 ±\pm 0.5 / 22.895.8 ±\pm 0.9 / 19.760.6 ±\pm 1.3 / 125.896.4 ±\pm 0.7 / 33.098.5 ±\pm 0.5 / 15.981.4 ±\pm 0.5 / 50.091.3 ±\pm 1.4 / 47.692.6 ±\pm 1.9 / 39.081.2 ±\pm 1.4 / 76.7
Reviewer_WSYZOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies information exchange during model communication and introduces the DIALECT framework to facilitate this process. DIALECT can be applied in both same-model and cross-model settings and outperforms text transfer under the same generation step budget while achieving lower latency. These advantages are demonstrated across multiple benchmarks, ranging from knowledge QA to code generation.

Critical strengths
  1. Important research topic. In multi-agent systems, using agent swarms to solve complex problems may be a cost-efficient approach, and information exchange between models is important for helping users achieve lower costs and higher output quality.

  2. Broad experimental settings. The work evaluates DIALECT across a relatively broad range of models and benchmarks, making its results more reliable. In particular, DIALECT achieves lower latency than text transfer, which may make it suitable for practical applications.

Critical weaknesses
  1. As a research paper, this work needs to state its contributions more clearly. For example, key components of DIALECT, such as stochastic soft-token generation and learned projections for cross-model representations, have precedents in prior work. DIALECT combines these components into a framework, but its technical novelty is not evident. The paper also lacks controlled ablations of key design choices to establish their individual contributions to the reported gains. I think the authors should consider reorganizing the paper to highlight its unique contribution and conducting ablations of each component to show its contribution to the overall gains.

  2. The authors claim that DIALECT retains more information. DIALECT is primarily motivated by the possibility that text transfer may lose candidate information. Could the authors show cases in which information is lost during text transfer but retained by DIALECT?

  3. DIALECT underperforms full text communication on some benchmarks, but the reasons are not adequately analyzed. Could the authors provide a mechanistic analysis to explain why full text communication succeeds in certain cases while DIALECT fails?

Questions for authors
  1. In Section 4.2, the authors state that the bridge is trained for each model pair, task, and communication budget. This is a reasonable setup, but it is not sufficient to demonstrate practical reusability. Could the authors train a single bridge and evaluate whether it generalizes to other model pairs, task domains, or communication budgets? If successful, this could directly reduce the overhead of retraining bridges. If the bridge does not generalize, could the authors analyze the reasons for its failure?
7 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/6)

We thank the reviewer for their thoughtful feedback and for highlighting the importance of our work problem and the breadth of our experiments.

In this response, we provide:

  • Clarification of DIALECT’s contributions and ablations of the bridge’s losses and both training stages (W1).
  • An example comparing DIALECT with budget-matched text transfer (W2).
  • A failure analysis with interventions on the communicated information (W3).
  • Discussion of bridge reusability and its limitations across domains, model pairs, and communication budgets (Q1).

We address each of the concerns below:

W1. Contribution We appreciate this comment and agree that the contributions should be stated more clearly. We will reorganize the introduction and method discussion to distinguish our contributions from components adopted from prior work. As mentioned in lines 139–141, we build DIALECT on stochastic soft-token generation from Wu et al. (2026), and we acknowledge prior work on representation alignment in lines 142–156. Our contribution is to make these continuous thoughts usable for communication between agents built from the same or different LLMs, while keeping both language models frozen. In the same-model setting, DIALECT transfers continuous thoughts directly without additional training. For different models, our bridge maps the source representations into the target’s input embedding space without requiring exact token matches between their vocabularies. As described in Section 3.3, we first train the bridge on paired representations and then optimize it through the frozen target’s predictions. This second stage encourages correct responses, uses source-generated text plans when they provide better supervision, and encourages the target to use message-specific information through shuffled, zero-valued, and absent-message controls. Additional alignment and norm constraints keep the mapped representations compatible with the target’s input space.

Ablations To address the concern about controlled ablations, we conducted additional ablations to isolate the bridge’s training components. We compare variants that individually remove LpredL_{pred}, LcommL_{comm}, and LreprL_{repr}. We also compare pretraining alone, without end-to-end training, and end-to-end training from random initialization, without pretraining.

Qwen3-8B →\rightarrow Qwen3-4B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}56.67 ±\pm 3.3 / 294.750.00 ±\pm 3.3 / 222.582.50 ±\pm 2.5 / 101.085.80 ±\pm 0.8 / 79.590.60 ±\pm 0.5 / 26.189.40 ±\pm 0.8 / 5.950.00 ±\pm 2.8 / 144.690.78 ±\pm 0.7 / 25.093.81 ±\pm 0.4 / 16.876.59 ±\pm 0.4 / 58.981.10 ±\pm 1.4 / 51.577.43 ±\pm 1.8 / 39.671.16 ±\pm 1.2 / 77.9
no LcommL_{comm}53.33 ±\pm 3.3 / 289.050.00 ±\pm 3.3 / 218.282.50 ±\pm 2.5 / 99.187.40 ±\pm 0.8 / 78.092.19 ±\pm 0.5 / 25.691.00 ±\pm 0.8 / 5.851.52 ±\pm 2.8 / 141.892.41 ±\pm 0.7 / 24.595.41 ±\pm 0.4 / 16.578.16 ±\pm 0.4 / 57.882.93 ±\pm 1.4 / 50.578.99 ±\pm 1.8 / 38.872.75 ±\pm 1.2 / 76.4
no LreprL_{repr}56.67 ±\pm 3.3 / 286.146.67 ±\pm 3.3 / 216.082.50 ±\pm 2.5 / 98.188.40 ±\pm 0.8 / 77.293.18 ±\pm 0.5 / 25.392.00 ±\pm 0.8 / 5.752.53 ±\pm 2.8 / 140.493.43 ±\pm 0.7 / 24.396.42 ±\pm 0.4 / 16.379.18 ±\pm 0.4 / 57.284.15 ±\pm 1.4 / 50.080.16 ±\pm 1.8 / 38.473.81 ±\pm 1.2 / 75.6
no end-to-end training53.33 ±\pm 3.3 / 297.546.67 ±\pm 3.3 / 224.680.00 ±\pm 2.5 / 102.084.20 ±\pm 0.8 / 80.389.01 ±\pm 0.5 / 26.387.80 ±\pm 0.8 / 5.948.48 ±\pm 2.8 / 146.089.16 ±\pm 0.7 / 25.392.21 ±\pm 0.4 / 17.074.94 ±\pm 0.4 / 59.579.88 ±\pm 1.4 / 52.075.88 ±\pm 1.8 / 39.969.58 ±\pm 1.2 / 78.6
no standalone pretraining56.67 ±\pm 3.3 / 291.850.00 ±\pm 3.3 / 220.382.50 ±\pm 2.5 / 100.186.20 ±\pm 0.8 / 78.790.98 ±\pm 0.5 / 25.889.80 ±\pm 0.8 / 5.850.51 ±\pm 2.8 / 143.291.21 ±\pm 0.7 / 24.894.23 ±\pm 0.4 / 16.676.98 ±\pm 0.4 / 58.381.71 ±\pm 1.4 / 51.077.82 ±\pm 1.8 / 39.271.69 ±\pm 1.2 / 77.1
DIALECT73.3 ±\pm 3.3 / 286.160.0 ±\pm 3.3 / 216.097.5 ±\pm 2.5 / 98.196.0 ±\pm 0.8 / 77.295.5 ±\pm 0.5 / 25.395.4 ±\pm 0.8 / 5.756.7 ±\pm 2.8 / 140.495.3 ±\pm 0.7 / 24.397.7 ±\pm 0.4 / 16.373.9 ±\pm 0.4 / 57.291.3 ±\pm 1.4 / 50.086.4 ±\pm 1.8 / 38.479.9 ±\pm 1.2 / 75.6
AuthorsOpenReview ↗

Official Comment by Authors (2/6)

W1(cont).

Qwen3-8B →\rightarrow Ministral-3B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}50.00 ±\pm 3.3 / 209.036.67 ±\pm 3.3 / 229.675.00 ±\pm 3.8 / 137.580.60 ±\pm 0.8 / 99.986.81 ±\pm 0.5 / 21.185.80 ±\pm 0.8 / 16.345.45 ±\pm 1.3 / 123.987.97 ±\pm 0.6 / 25.492.26 ±\pm 0.4 / 17.172.98 ±\pm 0.5 / 64.576.22 ±\pm 2.1 / 74.871.98 ±\pm 1.4 / 40.666.14 ±\pm 0.9 / 78.8
no LcommL_{comm}50.00 ±\pm 3.3 / 205.043.33 ±\pm 3.3 / 225.277.50 ±\pm 3.8 / 134.882.60 ±\pm 0.8 / 98.088.78 ±\pm 0.5 / 20.787.80 ±\pm 0.8 / 16.047.47 ±\pm 1.3 / 121.589.93 ±\pm 0.6 / 24.994.28 ±\pm 0.4 / 16.774.94 ±\pm 0.5 / 63.278.05 ±\pm 2.1 / 73.373.93 ±\pm 1.4 / 39.868.25 ±\pm 0.9 / 77.3
no LreprL_{repr}56.7 ±\pm 3.3 / 201.043.3 ±\pm 3.3 / 220.889.2 ±\pm 3.8 / 132.295.4 ±\pm 0.8 / 96.193.4 ±\pm 0.5 / 20.390.6 ±\pm 0.8 / 15.748.0 ±\pm 1.3 / 119.191.0 ±\pm 0.6 / 24.497.1 ±\pm 0.4 / 16.459.6 ±\pm 0.5 / 62.089.0 ±\pm 2.1 / 71.990.7 ±\pm 1.4 / 39.070.1 ±\pm 0.9 / 75.8
no end-to-end training43.33 ±\pm 3.3 / 211.140.00 ±\pm 3.3 / 231.872.50 ±\pm 3.8 / 138.878.40 ±\pm 0.8 / 100.984.61 ±\pm 0.5 / 21.383.60 ±\pm 0.8 / 16.542.93 ±\pm 1.3 / 125.185.75 ±\pm 0.6 / 25.690.07 ±\pm 0.4 / 17.270.78 ±\pm 0.5 / 65.174.39 ±\pm 2.1 / 75.570.04 ±\pm 1.4 / 41.064.02 ±\pm 0.9 / 79.6
no standalone pretraining46.67 ±\pm 3.3 / 209.043.33 ±\pm 3.3 / 229.675.00 ±\pm 3.8 / 137.579.60 ±\pm 0.8 / 99.985.75 ±\pm 0.5 / 21.184.80 ±\pm 0.8 / 16.344.44 ±\pm 1.3 / 123.986.95 ±\pm 0.6 / 25.491.25 ±\pm 0.4 / 17.171.96 ±\pm 0.5 / 64.575.00 ±\pm 2.1 / 74.871.21 ±\pm 1.4 / 40.665.08 ±\pm 0.9 / 78.8
DIALECT56.7 ±\pm 3.3 / 201.043.3 ±\pm 3.3 / 220.889.2 ±\pm 3.8 / 132.295.4 ±\pm 0.8 / 96.193.4 ±\pm 0.5 / 20.390.6 ±\pm 0.8 / 15.748.0 ±\pm 1.3 / 119.191.0 ±\pm 0.6 / 24.497.1 ±\pm 0.4 / 16.459.6 ±\pm 0.5 / 62.089.0 ±\pm 2.1 / 71.990.7 ±\pm 1.4 / 39.070.1 ±\pm 0.9 / 75.8

Qwen3-8B →\rightarrow Gemma-4-E4B-it

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}53.33 ±\pm 3.3 / 217.846.67 ±\pm 3.3 / 206.277.50 ±\pm 4.3 / 205.183.00 ±\pm 1.0 / 66.988.32 ±\pm 0.5 / 19.387.00 ±\pm 0.8 / 23.246.97 ±\pm 1.5 / 53.088.40 ±\pm 0.6 / 16.092.38 ±\pm 0.4 / 16.874.39 ±\pm 0.5 / 66.278.05 ±\pm 1.8 / 92.873.93 ±\pm 1.0 / 37.867.99 ±\pm 0.9 / 84.8
no LcommL_{comm}53.33 ±\pm 3.3 / 213.646.67 ±\pm 3.3 / 202.380.00 ±\pm 4.3 / 201.185.20 ±\pm 1.0 / 65.690.52 ±\pm 0.5 / 19.089.20 ±\pm 0.8 / 22.748.99 ±\pm 1.5 / 52.090.61 ±\pm 0.6 / 15.794.57 ±\pm 0.4 / 16.576.59 ±\pm 0.5 / 65.080.49 ±\pm 1.8 / 91.076.26 ±\pm 1.0 / 37.070.11 ±\pm 0.9 / 83.1
no LreprL_{repr}63.3 ±\pm 3.3 / 209.460.0 ±\pm 3.3 / 198.395.0 ±\pm 4.3 / 197.291.0 ±\pm 1.0 / 64.394.3 ±\pm 0.5 / 18.692.0 ±\pm 0.8 / 22.356.9 ±\pm 1.5 / 51.095.8 ±\pm 0.6 / 15.498.8 ±\pm 0.4 / 16.274.6 ±\pm 0.5 / 63.785.8 ±\pm 1.8 / 89.277.8 ±\pm 1.0 / 36.367.9 ±\pm 0.9 / 81.5
no end-to-end training46.67 ±\pm 3.3 / 219.943.33 ±\pm 3.3 / 208.275.00 ±\pm 4.3 / 207.180.60 ±\pm 1.0 / 67.585.97 ±\pm 0.5 / 19.584.60 ±\pm 0.8 / 23.444.44 ±\pm 1.5 / 53.686.01 ±\pm 0.6 / 16.289.98 ±\pm 0.4 / 17.072.03 ±\pm 0.5 / 66.976.22 ±\pm 1.8 / 93.771.60 ±\pm 1.0 / 38.165.61 ±\pm 0.9 / 85.6
no standalone pretraining50.00 ±\pm 3.3 / 222.043.33 ±\pm 3.3 / 210.275.00 ±\pm 4.3 / 209.079.80 ±\pm 1.0 / 68.285.14 ±\pm 0.5 / 19.783.80 ±\pm 0.8 / 23.643.94 ±\pm 1.5 / 54.185.24 ±\pm 0.6 / 16.389.18 ±\pm 0.4 / 17.271.25 ±\pm 0.5 / 67.575.00 ±\pm 1.8 / 94.670.82 ±\pm 1.0 / 38.564.81 ±\pm 0.9 / 86.4
DIALECT63.3 ±\pm 3.3 / 209.460.0 ±\pm 3.3 / 198.395.0 ±\pm 4.3 / 197.291.0 ±\pm 1.0 / 64.394.3 ±\pm 0.5 / 18.692.0 ±\pm 0.8 / 22.356.9 ±\pm 1.5 / 51.095.8 ±\pm 0.6 / 15.498.8 ±\pm 0.4 / 16.274.6 ±\pm 0.5 / 63.785.8 ±\pm 1.8 / 89.277.8 ±\pm 1.0 / 36.367.9 ±\pm 0.9 / 81.5
AuthorsOpenReview ↗

Official Comment by Authors (3/6)

W1(cont).

Gemma-4-E4B-it →\rightarrow Ministral-3B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}46.67 ±\pm 3.3 / 206.340.00 ±\pm 3.3 / 209.672.50 ±\pm 4.3 / 182.579.20 ±\pm 0.9 / 72.585.67 ±\pm 0.5 / 14.884.80 ±\pm 0.8 / 11.242.93 ±\pm 1.5 / 100.486.69 ±\pm 0.6 / 25.591.71 ±\pm 0.4 / 17.271.80 ±\pm 0.5 / 67.073.78 ±\pm 1.6 / 94.069.65 ±\pm 1.1 / 29.663.76 ±\pm 1.1 / 50.5
no LcommL_{comm}46.67 ±\pm 3.3 / 204.340.00 ±\pm 3.3 / 207.675.00 ±\pm 4.3 / 180.781.40 ±\pm 0.9 / 71.887.87 ±\pm 0.5 / 14.787.00 ±\pm 0.8 / 11.145.45 ±\pm 1.5 / 99.588.91 ±\pm 0.6 / 25.393.94 ±\pm 0.4 / 17.074.00 ±\pm 0.5 / 66.375.61 ±\pm 1.6 / 93.171.60 ±\pm 1.1 / 29.366.14 ±\pm 1.1 / 50.0
no LreprL_{repr}56.7 ±\pm 3.3 / 200.340.0 ±\pm 3.3 / 203.590.0 ±\pm 4.3 / 177.295.4 ±\pm 0.9 / 70.493.0 ±\pm 0.5 / 14.490.0 ±\pm 0.8 / 10.940.9 ±\pm 1.5 / 97.591.1 ±\pm 0.6 / 24.897.2 ±\pm 0.4 / 16.759.4 ±\pm 0.5 / 65.089.6 ±\pm 1.6 / 91.390.0 ±\pm 1.1 / 28.768.7 ±\pm 1.1 / 49.0
no end-to-end training43.33 ±\pm 3.3 / 210.336.67 ±\pm 3.3 / 213.770.00 ±\pm 4.3 / 186.177.00 ±\pm 0.9 / 73.983.47 ±\pm 0.5 / 15.182.60 ±\pm 0.8 / 11.440.91 ±\pm 1.5 / 102.484.56 ±\pm 0.6 / 26.089.52 ±\pm 0.4 / 17.569.60 ±\pm 0.5 / 68.371.34 ±\pm 1.6 / 95.967.32 ±\pm 1.1 / 30.161.64 ±\pm 1.1 / 51.5
no standalone pretraining46.67 ±\pm 3.3 / 208.336.67 ±\pm 3.3 / 211.672.50 ±\pm 4.3 / 184.377.80 ±\pm 0.9 / 73.284.23 ±\pm 0.5 / 15.083.40 ±\pm 0.8 / 11.341.92 ±\pm 1.5 / 101.485.32 ±\pm 0.6 / 25.890.32 ±\pm 0.4 / 17.470.38 ±\pm 0.5 / 67.671.95 ±\pm 1.6 / 95.068.09 ±\pm 1.1 / 29.862.43 ±\pm 1.1 / 51.0
DIALECT56.7 ±\pm 3.3 / 200.340.0 ±\pm 3.3 / 203.590.0 ±\pm 4.3 / 177.295.4 ±\pm 0.9 / 70.493.0 ±\pm 0.5 / 14.490.0 ±\pm 0.8 / 10.940.9 ±\pm 1.5 / 97.591.1 ±\pm 0.6 / 24.897.2 ±\pm 0.4 / 16.759.4 ±\pm 0.5 / 65.089.6 ±\pm 1.6 / 91.390.0 ±\pm 1.1 / 28.768.7 ±\pm 1.1 / 49.0

These new ablations complement our existing comparisons of the message-generation and transfer choices. In the same-model setting, CIPHER and DIALECT compare deterministic probability-weighted embeddings with stochastic soft-token generation, without a learned bridge. In the cross-model setting, CIPHER-Cross and training-free alignment compare these generation procedures using the same vocabulary-based transfer mechanism. Comparing training-free alignment with DIALECT then keeps stochastic source generation fixed and replaces vocabulary-based alignment with the learned bridge. These comparisons therefore examine alternative choices for each component.

We will include these ablations in the paper and clarify how the existing component comparisons complement the controlled ablations of the bridge’s losses and training stages.

W2. We appreciate this comment and agree that concrete examples would make the motivation clearer. By retaining more information, we refer specifically to retaining contributions from multiple candidate continuations during communication. Text communication selects one token at each step; the other candidate tokens and their relative probabilities are not explicitly transmitted. Different candidate distributions can produce the same selected token, so the receiver cannot distinguish them from that token alone. DIALECT instead communicates a weighted combination of candidate token embeddings, allowing multiple candidates to contribute at each step under the same generation-step budget. Expressing these alternatives explicitly in text would require additional tokens.

Below is an example comparing DIALECT with the same-budget text transfer (Text 80). An example from GSM8K:

Question: Colby loves going to the movies and every month his parents give him 150 dollars to spend at the movies. Tickets for Fridays and Saturdays cost 10 dollars. Tickets for any other day cost 7 dollars. Popcorn costs 8 dollars and boxes of candy cost 2 dollars. It is the last day of the month and it's a Friday. He wants to make sure he gets a popcorn and box of candy that night. How many movies can he see if he already saw 5 movies on a Friday or Saturday, 8 movies on other days, had 2 tubs of popcorn, and four boxes of candy that month?

Reviewer_uBHHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DIALECT, a framework for continuous-space communication between LLM agents. Specifically, the agent generates continuous thoughts using stochastic mixtures of token embeddings. For heterogeneous models, a learned bridge translates these representations into the target model's embedding space. Experiments show improved accuracy over several latent communication baselines and reduced latency compared with full-text communication, while maintaining comparable performance.

Critical strengths
  • Important and practical: The paper addresses continuous communication between heterogeneous LLMs, where incompatible embedding spaces make direct information exchange difficult.
  • Extensive experiments. The evaluation covers many benchmarks and multiple model families and the reported accuracy–latency trade-offs look promising.
Critical weaknesses
  • Insufficient baselines: Comparison against some closed baselines are not comprehensive; Interlat, RecursiveMAS, StateBridge and XBridge are all mentioned in the paper but never compared. The paper does not clarify why they are not comparable.
  • Insufficient ablations: DIALECT combines stochastic soft-token sampling, two-stage bridge training, representation alignment, prediction supervision, and communication-specific losses. The paper does not clearly establish the individual contribution of each component.
  • Insufficient tuning on existing baselines: The authors studies the planner–solver setting, which is also covered by the original LatentMAS setting (including planer, critic, refiner and solver), but the reported accuracy performs considerably worse. Given that even the dataset considered in this work also significantly overlap with LatentMAS's original setting, it might be hard to infer the proposed method really outperform it by this large margin (at least under the homogeneous agent setting).
  • Insufficient analysis on the bridge: The bridge is trained from source-generated text plans. This makes it unclear whether the bridge learns to transfer source reasoning or mainly learns a task-specific soft prompt. Might need additional experiment to demystify this.
Questions for authors
  • Figure 2 is quite confusing, so where is other methods? Seems like all the curves and dots are the paper's proposed method.
  • Is there any efficiency analysis and comparison?
  • How does the paper's proposed bridge learning differ from existing work?
8 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/6)

We thank the reviewer for their thoughtful feedback and for recognizing the importance and extensive experimental evaluation of our work. In this response, we provide:

  • Additional baseline comparisons across 13 benchmarks (W1).
  • Ablations of the bridge’s losses and both training stages (W2).
  • A LatentMAS evaluation with full KV-cache transfer (W3).
  • Message-control results supporting input-specific communication (W4).
  • A comparison table clarifying how our bridge differs from related methods (Q3).

We address each of the concerns below:

W1. We appreciate the reviewer’s suggestion and agree that additional comparisons would strengthen the evaluation. . To address this concern, we have added StateBridge and RecursiveMAS to our two-agent planning-and-solving setting. We evaluate StateBridge in the same-model setting and RecursiveMAS across both same- and cross-model settings on the same 13 benchmarks. We also clarify why Interlat and XBridge are not included.

Qwen3-8B →\rightarrow Qwen3-8B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
RecursiveMAS R153.3 ±\pm 3.3 / 237.953.3 ±\pm 3.3 / 222.392.5 ±\pm 2.5 / 180.595.8 ±\pm 1.0 / 90.194.4 ±\pm 0.5 / 25.187.6 ±\pm 1.4 / 21.751.0 ±\pm 1.5 / 138.494.0 ±\pm 0.9 / 36.397.6 ±\pm 0.6 / 17.573.1 ±\pm 0.5 / 55.090.9 ±\pm 1.2 / 52.492.2 ±\pm 1.9 / 42.981.0 ±\pm 1.3 / 84.4
RecursiveMAS R350.0 ±\pm 3.3 / 324.546.7 ±\pm 3.3 / 303.187.5 ±\pm 2.5 / 246.192.4 ±\pm 1.4 / 122.995.0 ±\pm 0.5 / 34.292.4 ±\pm 1.2 / 29.557.1 ±\pm 1.5 / 188.794.5 ±\pm 0.9 / 49.598.4 ±\pm 0.5 / 23.976.5 ±\pm 0.5 / 75.090.9 ±\pm 1.2 / 71.492.2 ±\pm 1.9 / 58.581.0 ±\pm 1.3 / 115.1
StateBridge70.0 ±\pm 3.3 / 540.850.0 ±\pm 3.3 / 505.292.5 ±\pm 2.5 / 410.295.6 ±\pm 1.0 / 204.895.4 ±\pm 0.5 / 57.094.6 ±\pm 1.0 / 49.259.1 ±\pm 1.5 / 314.595.9 ±\pm 0.8 / 82.598.5 ±\pm 0.5 / 39.881.4 ±\pm 0.5 / 125.090.9 ±\pm 1.2 / 119.092.2 ±\pm 1.9 / 97.581.0 ±\pm 1.3 / 191.8
DIALECT70.0 ±\pm 3.3 / 216.366.7 ±\pm 3.3 / 202.192.5 ±\pm 2.5 / 164.196.6 ±\pm 0.9 / 81.995.7 ±\pm 0.5 / 22.895.8 ±\pm 0.9 / 19.760.6 ±\pm 1.3 / 125.896.4 ±\pm 0.7 / 33.098.5 ±\pm 0.5 / 15.981.4 ±\pm 0.5 / 50.091.3 ±\pm 1.4 / 47.692.6 ±\pm 1.9 / 39.081.2 ±\pm 1.4 / 76.7

Qwen3-8B →\rightarrow Qwen3-4B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
RecursiveMAS R146.7 ±\pm 3.3 / 340.336.7 ±\pm 3.3 / 231.480.0 ±\pm 7.5 / 111.192.4 ±\pm 1.0 / 84.692.6 ±\pm 0.6 / 28.788.0 ±\pm 1.2 / 6.655.1 ±\pm 3.0 / 154.892.4 ±\pm 0.9 / 29.096.8 ±\pm 0.5 / 18.369.9 ±\pm 0.4 / 62.990.9 ±\pm 1.2 / 55.085.6 ±\pm 1.9 / 44.076.5 ±\pm 1.3 / 84.8
RecursiveMAS R346.7 ±\pm 3.3 / 464.146.7 ±\pm 3.3 / 315.685.0 ±\pm 5.0 / 151.593.8 ±\pm 1.0 / 115.493.5 ±\pm 0.6 / 39.290.2 ±\pm 1.2 / 9.051.0 ±\pm 3.0 / 211.093.9 ±\pm 0.8 / 39.697.7 ±\pm 0.4 / 24.972.4 ±\pm 0.4 / 85.890.9 ±\pm 1.2 / 75.086.4 ±\pm 1.9 / 60.079.9 ±\pm 1.3 / 115.6
DIALECT73.3 ±\pm 3.3 / 286.160.0 ±\pm 3.3 / 216.097.5 ±\pm 2.5 / 98.196.0 ±\pm 0.8 / 77.295.5 ±\pm 0.5 / 25.395.4 ±\pm 0.8 / 5.756.7 ±\pm 2.8 / 140.495.3 ±\pm 0.7 / 24.397.7 ±\pm 0.4 / 16.373.9 ±\pm 0.4 / 57.291.3 ±\pm 1.4 / 50.086.4 ±\pm 1.8 / 38.479.9 ±\pm 1.2 / 75.6
AuthorsOpenReview ↗

Official Comment by Authors (2/6)

W1(cont).

Qwen3-8B →\rightarrow Ministral-3B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
RecursiveMAS R13.3 ±\pm 3.3 / 260.46.7 ±\pm 3.3 / 253.755.0 ±\pm 5.0 / 145.459.8 ±\pm 1.8 / 104.993.3 ±\pm 0.5 / 24.069.6 ±\pm 1.2 / 19.825.8 ±\pm 1.0 / 147.178.2 ±\pm 0.9 / 32.985.0 ±\pm 0.8 / 20.644.1 ±\pm 0.5 / 71.023.8 ±\pm 3.0 / 85.516.0 ±\pm 1.9 / 44.036.5 ±\pm 1.1 / 84.7
RecursiveMAS R330.0 ±\pm 3.3 / 355.010.0 ±\pm 3.3 / 345.960.0 ±\pm 5.0 / 198.370.8 ±\pm 1.8 / 143.193.4 ±\pm 0.5 / 32.774.2 ±\pm 1.2 / 27.030.8 ±\pm 1.0 / 200.582.1 ±\pm 0.8 / 44.890.6 ±\pm 0.7 / 28.050.7 ±\pm 0.5 / 96.848.2 ±\pm 3.0 / 116.690.7 ±\pm 1.6 / 60.069.8 ±\pm 0.8 / 115.5
DIALECT56.7 ±\pm 3.3 / 201.043.3 ±\pm 3.3 / 220.889.2 ±\pm 3.8 / 132.295.4 ±\pm 0.8 / 96.193.4 ±\pm 0.5 / 20.390.6 ±\pm 0.8 / 15.748.0 ±\pm 1.3 / 119.191.0 ±\pm 0.6 / 24.497.1 ±\pm 0.4 / 16.459.6 ±\pm 0.5 / 62.089.0 ±\pm 2.1 / 71.990.7 ±\pm 1.4 / 39.070.1 ±\pm 0.9 / 75.8

Qwen3-8B →\rightarrow Gemma-4-E4B-it

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
RecursiveMAS R120.0 ±\pm 3.3 / 253.820.0 ±\pm 3.3 / 222.962.5 ±\pm 10.0 / 216.671.2 ±\pm 1.6 / 72.887.6 ±\pm 0.7 / 22.092.0 ±\pm 0.8 / 23.254.0 ±\pm 1.5 / 56.294.4 ±\pm 0.7 / 22.098.2 ±\pm 0.5 / 17.874.6 ±\pm 0.5 / 78.879.9 ±\pm 1.8 / 122.177.4 ±\pm 1.2 / 41.867.7 ±\pm 0.8 / 93.5
RecursiveMAS R340.0 ±\pm 3.3 / 346.026.7 ±\pm 3.3 / 303.970.0 ±\pm 10.0 / 295.481.8 ±\pm 1.4 / 99.389.5 ±\pm 0.7 / 30.092.0 ±\pm 0.8 / 31.752.0 ±\pm 1.5 / 76.794.7 ±\pm 0.7 / 30.097.9 ±\pm 0.5 / 24.374.6 ±\pm 0.5 / 107.478.1 ±\pm 2.4 / 166.577.4 ±\pm 1.2 / 57.067.7 ±\pm 0.8 / 127.5
DIALECT63.3 ±\pm 3.3 / 209.460.0 ±\pm 3.3 / 198.395.0 ±\pm 4.3 / 197.291.0 ±\pm 1.0 / 64.394.3 ±\pm 0.5 / 18.692.0 ±\pm 0.8 / 22.356.9 ±\pm 1.5 / 51.095.8 ±\pm 0.6 / 15.498.8 ±\pm 0.4 / 16.274.6 ±\pm 0.5 / 63.785.8 ±\pm 1.8 / 89.277.8 ±\pm 1.0 / 36.367.9 ±\pm 0.9 / 81.5

Gemma-4-E4B-it →\rightarrow Ministral-3B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
RecursiveMAS R110.0 ±\pm 3.3 / 244.610.0 ±\pm 3.3 / 279.537.5 ±\pm 7.5 / 223.369.8 ±\pm 2.0 / 107.985.5 ±\pm 0.7 / 21.172.6 ±\pm 1.2 / 24.933.8 ±\pm 1.5 / 116.777.3 ±\pm 0.9 / 29.786.4 ±\pm 0.8 / 18.846.3 ±\pm 0.5 / 82.980.5 ±\pm 1.8 / 106.780.2 ±\pm 1.6 / 55.268.5 ±\pm 1.1 / 71.2
RecursiveMAS R33.3 ±\pm 3.3 / 333.66.7 ±\pm 3.3 / 381.147.5 ±\pm 7.5 / 304.568.8 ±\pm 2.0 / 147.184.8 ±\pm 0.7 / 28.872.6 ±\pm 1.2 / 33.931.3 ±\pm 1.5 / 159.175.5 ±\pm 0.9 / 40.585.6 ±\pm 0.8 / 25.759.4 ±\pm 0.5 / 113.178.1 ±\pm 2.4 / 145.576.7 ±\pm 1.6 / 75.368.5 ±\pm 1.1 / 97.1
DIALECT56.7 ±\pm 3.3 / 200.340.0 ±\pm 3.3 / 203.590.0 ±\pm 4.3 / 177.295.4 ±\pm 0.9 / 70.493.0 ±\pm 0.5 / 14.490.0 ±\pm 0.8 / 10.940.9 ±\pm 1.5 / 97.591.1 ±\pm 0.6 / 24.897.2 ±\pm 0.4 / 16.759.4 ±\pm 0.5 / 65.089.6 ±\pm 1.6 / 91.390.0 ±\pm 1.1 / 28.768.7 ±\pm 1.1 / 49.0

For Interlat, its original training procedure fine-tunes the target LLM together with the communication adapter, and its learned compression stage further trains the source LLM. Our evaluation keeps both LLMs frozen and restricts training to the communication modules. Including Interlat would therefore require either changing this evaluation setting or modifying its original training procedure. For XBridge, the original protocol transfers externally provided context rather than source-generated reasoning. The source encodes this context in a single forward pass, and the target accesses it through lexical anchors and a cross-attention bridge. In our planning-and-solving setting, both agents receive the same question, and the source generates reasoning to support the target. Applying XBridge would therefore require redefining its source input or introducing a plan-generation stage, changing the communication task and the computation included in its latency.

We will include these additional comparisons and baseline adaptation details in the paper and clarify why Interlat and XBridge are not included.

AuthorsOpenReview ↗

Official Comment by Authors (3/6)

W2. We appreciate this comment. In response, we conducted additional ablations to isolate the bridge’s training components. We compare variants that individually remove LpredL_{pred}, LcommL_{comm}, and LreprL_{repr}. We also compare pretraining alone, without end-to-end training, and end-to-end training from random initialization, without pretraining.

Qwen3-8B →\rightarrow Qwen3-4B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}56.67 ±\pm 3.3 / 294.750.00 ±\pm 3.3 / 222.582.50 ±\pm 2.5 / 101.085.80 ±\pm 0.8 / 79.590.60 ±\pm 0.5 / 26.189.40 ±\pm 0.8 / 5.950.00 ±\pm 2.8 / 144.690.78 ±\pm 0.7 / 25.093.81 ±\pm 0.4 / 16.876.59 ±\pm 0.4 / 58.981.10 ±\pm 1.4 / 51.577.43 ±\pm 1.8 / 39.671.16 ±\pm 1.2 / 77.9
no LcommL_{comm}53.33 ±\pm 3.3 / 289.050.00 ±\pm 3.3 / 218.282.50 ±\pm 2.5 / 99.187.40 ±\pm 0.8 / 78.092.19 ±\pm 0.5 / 25.691.00 ±\pm 0.8 / 5.851.52 ±\pm 2.8 / 141.892.41 ±\pm 0.7 / 24.595.41 ±\pm 0.4 / 16.578.16 ±\pm 0.4 / 57.882.93 ±\pm 1.4 / 50.578.99 ±\pm 1.8 / 38.872.75 ±\pm 1.2 / 76.4
no LreprL_{repr}56.67 ±\pm 3.3 / 286.146.67 ±\pm 3.3 / 216.082.50 ±\pm 2.5 / 98.188.40 ±\pm 0.8 / 77.293.18 ±\pm 0.5 / 25.392.00 ±\pm 0.8 / 5.752.53 ±\pm 2.8 / 140.493.43 ±\pm 0.7 / 24.396.42 ±\pm 0.4 / 16.379.18 ±\pm 0.4 / 57.284.15 ±\pm 1.4 / 50.080.16 ±\pm 1.8 / 38.473.81 ±\pm 1.2 / 75.6
no end-to-end training53.33 ±\pm 3.3 / 297.546.67 ±\pm 3.3 / 224.680.00 ±\pm 2.5 / 102.084.20 ±\pm 0.8 / 80.389.01 ±\pm 0.5 / 26.387.80 ±\pm 0.8 / 5.948.48 ±\pm 2.8 / 146.089.16 ±\pm 0.7 / 25.392.21 ±\pm 0.4 / 17.074.94 ±\pm 0.4 / 59.579.88 ±\pm 1.4 / 52.075.88 ±\pm 1.8 / 39.969.58 ±\pm 1.2 / 78.6
no standalone pretraining56.67 ±\pm 3.3 / 291.850.00 ±\pm 3.3 / 220.382.50 ±\pm 2.5 / 100.186.20 ±\pm 0.8 / 78.790.98 ±\pm 0.5 / 25.889.80 ±\pm 0.8 / 5.850.51 ±\pm 2.8 / 143.291.21 ±\pm 0.7 / 24.894.23 ±\pm 0.4 / 16.676.98 ±\pm 0.4 / 58.381.71 ±\pm 1.4 / 51.077.82 ±\pm 1.8 / 39.271.69 ±\pm 1.2 / 77.1
DIALECT73.3 ±\pm 3.3 / 286.160.0 ±\pm 3.3 / 216.097.5 ±\pm 2.5 / 98.196.0 ±\pm 0.8 / 77.295.5 ±\pm 0.5 / 25.395.4 ±\pm 0.8 / 5.756.7 ±\pm 2.8 / 140.495.3 ±\pm 0.7 / 24.397.7 ±\pm 0.4 / 16.373.9 ±\pm 0.4 / 57.291.3 ±\pm 1.4 / 50.086.4 ±\pm 1.8 / 38.479.9 ±\pm 1.2 / 75.6

Qwen3-8B →\rightarrow Ministral-3B

MethodAIME24AIME25AMC23MATH500GSM8KOpenBookQAGPQA-DiamondARC-CARC-EMedQAHumanEvalMBPPMBPP+
no LpredL_{pred}50.00 ±\pm 3.3 / 209.036.67 ±\pm 3.3 / 229.675.00 ±\pm 3.8 / 137.580.60 ±\pm 0.8 / 99.986.81 ±\pm 0.5 / 21.185.80 ±\pm 0.8 / 16.345.45 ±\pm 1.3 / 123.987.97 ±\pm 0.6 / 25.492.26 ±\pm 0.4 / 17.172.98 ±\pm 0.5 / 64.576.22 ±\pm 2.1 / 74.871.98 ±\pm 1.4 / 40.666.14 ±\pm 0.9 / 78.8
no LcommL_{comm}50.00 ±\pm 3.3 / 205.043.33 ±\pm 3.3 / 225.277.50 ±\pm 3.8 / 134.882.60 ±\pm 0.8 / 98.088.78 ±\pm 0.5 / 20.787.80 ±\pm 0.8 / 16.047.47 ±\pm 1.3 / 121.589.93 ±\pm 0.6 / 24.994.28 ±\pm 0.4 / 16.774.94 ±\pm 0.5 / 63.278.05 ±\pm 2.1 / 73.373.93 ±\pm 1.4 / 39.868.25 ±\pm 0.9 / 77.3
no LreprL_{repr}56.7 ±\pm 3.3 / 201.043.3 ±\pm 3.3 / 220.889.2 ±\pm 3.8 / 132.295.4 ±\pm 0.8 / 96.193.4 ±\pm 0.5 / 20.390.6 ±\pm 0.8 / 15.748.0 ±\pm 1.3 / 119.191.0 ±\pm 0.6 / 24.497.1 ±\pm 0.4 / 16.459.6 ±\pm 0.5 / 62.089.0 ±\pm 2.1 / 71.990.7 ±\pm 1.4 / 39.070.1 ±\pm 0.9 / 75.8
no end-to-end training43.33 ±\pm 3.3 / 211.140.00 ±\pm 3.3 / 231.872.50 ±\pm 3.8 / 138.878.40 ±\pm 0.8 / 100.984.61 ±\pm 0.5 / 21.383.60 ±\pm 0.8 / 16.542.93 ±\pm 1.3 / 125.185.75 ±\pm 0.6 / 25.690.07 ±\pm 0.4 / 17.270.78 ±\pm 0.5 / 65.174.39 ±\pm 2.1 / 75.570.04 ±\pm 1.4 / 41.064.02 ±\pm 0.9 / 79.6
no standalone pretraining46.67 ±\pm 3.3 / 209.043.33 ±\pm 3.3 / 229.675.00 ±\pm 3.8 / 137.579.60 ±\pm 0.8 / 99.985.75 ±\pm 0.5 / 21.184.80 ±\pm 0.8 / 16.344.44 ±\pm 1.3 / 123.986.95 ±\pm 0.6 / 25.491.25 ±\pm 0.4 / 17.171.96 ±\pm 0.5 / 64.575.00 ±\pm 2.1 / 74.871.21 ±\pm 1.4 / 40.665.08 ±\pm 0.9 / 78.8
DIALECT56.7 ±\pm 3.3 / 201.043.3 ±\pm 3.3 / 220.889.2 ±\pm 3.8 / 132.295.4 ±\pm 0.8 / 96.193.4 ±\pm 0.5 / 20.390.6 ±\pm 0.8 / 15.748.0 ±\pm 1.3 / 119.191.0 ±\pm 0.6 / 24.497.1 ±\pm 0.4 / 16.459.6 ±\pm 0.5 / 62.089.0 ±\pm 2.1 / 71.990.7 ±\pm 1.4 / 39.070.1 ±\pm 0.9 / 75.8

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.