- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes InfoLat, a latent communication framework for collaboration between heterogeneous LLMs. A message encoder and predictive decoder first convert reference text messages into compressed latent labels, represented as candidate vocabulary tokens and associated weights. The sender is then trained to generate these messages, while receivers are independently trained to use them for task completion. Vocabulary alignment enables communication across models with different vocabularies and embedding spaces, allowing a trained sender to remain fixed while new receivers are adapted. The authors evaluate the framework on mathematical reasoning benchmarks and ALFWorld, with ablations, message-shuffling experiments, and training-dynamics analyses examining its effectiveness and communication properties.
Critical strengths
-
I find the communication interface and decoupled training procedure practically useful. Representing messages through vocabulary tokens and weights, together with vocabulary alignment, provides a concrete way to connect different models’ embedding spaces. In particular, adapting additional receivers without retraining the sender offers a useful path toward extending collaboration to more model combinations.
-
I also appreciate the empirical results across mathematical reasoning and interactive environment tasks. In the evaluated model combinations, InfoLat achieves competitive task performance with shorter communication sequences. These results suggest that the approach is promising and worth further investigation.
Critical weaknesses
- My main reservation concerns the empirical validation of the benefits attributable to the proposed latent-label learning and compression mechanisms. Although the paper includes several useful ablations and analyses supporting the framework and individual components, I would appreciate more direct comparisons with simpler communication alternatives. For example, CIPHER also uses probability-weighted token embeddings, as discussed in the paper, but is not evaluated in compatible settings. The segment length directly determines the degree of compression, yet its effect is not examined. In addition, while the appendix explains the computational benefits expected from shorter sequences, these benefits are not supported by runtime measurements. These gaps do not negate the reported results, but leave me somewhat uncertain about the advantages of particular design choices and their performance–cost trade-offs.
Questions for authors
- The paper notes that CIPHER uses probability-weighted token embeddings and supports different models sharing a tokenizer. Could the authors explain why it was not included in the experimental comparison? A comparison in a homogeneous or shared-tokenizer setting would help clarify the additional benefits of InfoLat’s latent-label learning and compression mechanisms. If there are applicability or implementation constraints, clarification would also be helpful.
- What motivated the default segment length of 2? Could the authors provide results for segment lengths such as 1, 2, and 4, showing the trade-off between task performance and message length? I am particularly interested in the length-1 setting, which could help clarify the respective roles of the continuous representation and compression. The existing analysis of the interpolation width K does not directly address this question.
- I appreciate Appendix A.5’s analysis of the computational savings from fewer generation steps and receiver input positions, as well as its distinction between sequence length and transmitted bytes. Could the authors provide representative measurements of sender, receiver, and end-to-end inference time to show how these expected savings translate into practice? Reporting the actual message payload and distinguishing initial training costs from the cost of adapting an additional receiver would also help characterize the practical efficiency of the framework.
- The message-shuffling experiments and question–message compatibility analyses provide useful evidence that receivers use task-specific information. I would like to better understand how much a correctly matched message helps relative to receiving no message. Could the authors add no-message or fixed-message controls using the same trained receiver? For baselines that are relatively insensitive to message replacement, is there further evidence distinguishing insufficient information in the sender’s messages from the receiver’s failure to use that information?
8 comments
2. What motivates the default segment length of 2, and how does InfoLat perform with other lengths such as 1 and 4, especially length 1, which separates the effect of the continuous representation from that of compression? (W1, Q2)
Following the reviewer's suggestion, we add a sweep over the segment length ρ from 1 to 5. The experiment is run on Llama-3B→Qwen-1.5B; apart from ρ, latent label construction, sender training, receiver training and decoding settings are identical to the main experiments (3 seeds):
| ρ | GSM8K-aug | GSM-Hard | SVAMP | MultiArith | Avg |
|---|---|---|---|---|---|
| 1 | 68.03±0.12 | 18.45±0.31 | 74.67±0.33 | 97.78±0.00 | 64.73 |
| 2 (default) | 68.81±0.25 | 17.54±0.22 | 72.44±0.44 | 97.78±0.00 | 64.14 |
| 3 | 60.37±0.31 | 16.50±0.29 | 69.67±0.33 | 98.70±0.32 | 61.31 |
| 4 | 60.30±0.16 | 16.58±0.32 | 70.11±1.02 | 99.44±0.00 | 61.61 |
| 5 | 54.21±0.39 | 15.72±0.31 | 69.00±0.33 | 99.44±0.00 | 59.59 |
With ρ=1, each latent token corresponds to a single text token, so the message is not compressed at all, yet each latent token is still a continuous weighted combination of the embeddings of the top-K (K=10) vocabulary tokens. This setting therefore isolates the effect of the continuous latent representation itself. With ρ=1, InfoLat reaches an average accuracy of 64.73, 7.1 points higher than Text Communication (57.61), and outperforms Text Communication on all four datasets (GSM8K-aug 68.03 vs. 50.32, GSM-Hard 18.45 vs. 16.91, SVAMP 74.67 vs. 65.78, MultiArith 97.78 vs. 97.41). Since there is no compression here, this gain comes entirely from the latent message itself: compared with a text message that can carry only one discrete token per position, a latent token formed as a weighted combination of multiple vocabulary tokens and supervised by latent labels can carry richer and more useful information for the receiver.
We choose ρ=2 as the default because it offers the best balance between accuracy and message length. Compared with the uncompressed ρ=1, ρ=2 halves the message length (20.50 → 10.38) while lowering the average accuracy by only 0.6 points (64.73 → 64.14), and even slightly improves GSM8K-aug (68.03 → 68.81). A shorter message halves the number of autoregressive decoding steps of the sender, which directly reduces inference latency and compute. A larger ρ further shortens the message, at the cost of some accuracy, since each latent token has to summarize more text.
3. Can runtime measurements support the computational benefits discussed in the appendix, including sender, receiver and end-to-end inference times, the actual message payload, and the cost of initial training versus adapting an additional receiver? (W1, Q3)
(1) Inference time. We add stage-wise latency measurements on the full GSM8K-aug test set (1,319 questions), comparing InfoLat with Text Communication. The table reports per-question averages (in ms; p90 latency in parentheses, i.e., the time within which 90% of the questions are completed):
| Pair | Method | Sender steps | Sender prefill | Sender decode | Transfer | Receiver prefill | Receiver decode | End-to-end | Speedup | Payload (B) |
|---|---|---|---|---|---|---|---|---|---|---|
| Llama-3B→Llama-1B | Text | 28.77 | 20.5 | 519.6 | 0.71 | 12.3 | 192.6 | 745.7 (1058.1) | 1.00× | 29.8 |
| InfoLat | 13.90 | 20.0 | 234.2 | 0.19 | 12.0 | 175.1 | 441.4 (583.4) | 1.69× | 774.2 | |
| Llama-3B→Qwen-1.5B | Text | 28.77 | 20.6 | 517.2 | 0.91 | 20.6 | 410.5 | 969.8 (1318.7) | 1.00× | 29.8 |
| InfoLat | 13.94 | 20.3 | 236.4 | 0.20 | 20.5 | 386.9 | 664.3 (851.7) | 1.46× | 776.4 |
Both methods are measured under identical conditions: one A800-40GB GPU with batch size 1, the same sender and receiver with merged LoRA weights, a shared KV-cache decoding loop with identical sampling settings and per-question seeds, and 10 warm-up questions. We synchronize the GPU at every stage boundary, so the stage times sum to the end-to-end time.
InfoLat reduces the end-to-end latency to 59% and 68% of Text Communication (1.69× and 1.46× speedups). The savings come mainly from sender decoding (about 520 → 235 ms): since each latent token carries more information, the sender needs about 13.9 decoding steps instead of 28.8, while the time per step is almost unchanged (18.2 vs. 18.7 ms), showing that constructing the top-K weighted embedding adds negligible overhead (consistent with Appendix A.5). Prefill times are nearly identical, as they depend only on the question length.
(2) Message payload. Fewer decoding steps do not mean fewer transmitted bytes. Each latent token carries 10 vocabulary IDs (int32) and 10 weights (bf16), i.e., 60 bytes, so an InfoLat message takes about 774 bytes, compared with about 30 bytes for a UTF-8 text message. Although about 26 times larger, it is still below 1 KB (about 6 μs at 1 Gbps), and the measured transfer stage takes only 0.2 ms, less than the 0.7–0.9 ms for text, which requires detokenization and re-tokenization. Since model computation rather than bandwidth is the bottleneck, this trade-off is worthwhile. We will add this table to the revision and distinguish decoding steps from transmitted bytes in Appendix A.5.
(3) Training cost. The latent labels and the sender of InfoLat depend only on the sender, so they can be reused directly when a new receiver is added, and only the new receiver needs to be trained. Taking Llama-3B as the sender and adding Llama-1B-Instruct as a new receiver, InfoLat requires 1.35×10¹⁸ FLOPs, about 3.1% of the one-time cost of latent label construction and sender training.
We sincerely thank the reviewer for the careful assessment and for recognizing the value of our existing ablations and analyses. We group related weaknesses and questions and respond to each point below.
1. Why is CIPHER, which also uses probability-weighted token embeddings, not included, and how does InfoLat compare with it in a shared-tokenizer setting that clarifies the additional benefit of latent-label learning? (W1, Q1)
CIPHER was not included in the original submission because it feeds the embedding-weighted output distribution of the sender directly into the receiver, which requires the sender and receiver to share the same tokenizer and embedding space. It therefore cannot be applied to our cross-family settings (Llama-3B→Qwen-1.5B and Qwen-7B→Llama-1B), and we only selected baselines that cover all settings. We agree with the reviewer that, in a shared-tokenizer setting, CIPHER is an appropriate control for clarifying the role of latent-label learning. We therefore add a comparison with CIPHER on Llama-3B→Llama-1B, together with a new latent communication baseline, C2C (Cache-to-Cache), which transfers information between models through the KV cache rather than a token sequence:
For a fair comparison, all methods use the same sender and receiver backbones, the same test sets and the same decoding settings (temperature 0.6, top-p 0.95, 3 seeds). CIPHER is training-free, so we search its sender temperature on a 500-question GSM8K-aug validation set over {0.2, 0.4, 0.6, 0.8, 1.0, 1.5} and select the best value, 0.2 (accuracies 52.60 / 51.07 / 51.13 / 49.40 / 46.73 / 40.20, respectively). For C2C, we train the fuser on GSM8K-aug following its official configuration, with both the sender and the receiver frozen.
InfoLat reaches an average accuracy of 63.14, which is 9.2 and 9.4 points higher than CIPHER (53.91) and C2C (53.79). CIPHER uses the same type of message representation as InfoLat, namely probability-weighted combinations of vocabulary embeddings. The difference is that a CIPHER message is simply the sender's output distribution during normal decoding: it receives no latent-label supervision, and the receiver is not trained to read such messages. The improvement of InfoLat over CIPHER can therefore be attributed to latent-label learning and receiver training, not merely to the soft-token representation. C2C transmits the more information-rich KV cache, yet its accuracy is on par with CIPHER. This further suggests that the key to effective communication is not whether the transmitted representation is continuous, but whether the message is explicitly supervised to carry the reasoning content the receiver needs.