- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Neural Esperanto, a shared discrete-symbol communication interface that enables heterogeneous frozen LLMs to exchange and accumulate private information through model-specific Ear and Mouth adapters. Experiments demonstrate multi-stage information transfer and generalization to an unseen sender, achieving accuracy comparable to text-based communication on synthetic spatial-reasoning tasks.
Critical strengths
1, Shared communication interface: The framework enables communication across heterogeneous frozen LLMs without modifying their backbone parameters, potentially avoiding the need for pairwise translators.
-
Careful experimental validation: Symbol-count controls and message-replacement experiments provide convincing evidence that the models actually use the communicated information. The unseen-sender experiment also demonstrates promising cross-model transfer.
-
Useful adapter analysis: Controlled experiments show that the Ear adapter improves both incoming-message interpretation and autoregressive message generation.
Critical weaknesses
-
Unclear novelty and practical advantages: The shared symbols are derived from an English BPE tokenizer, and generated messages almost perfectly reproduce the original English encoding. A fixed text decoder also achieves comparable accuracy. Therefore, the advantages over simpler text-based communication or existing tokenizer-transfer approaches remain unclear, particularly in terms of efficiency and scalability.
-
Limited generalization: The evaluation focuses on simplified synthetic spatial-reasoning tasks. Despite a vocabulary of 32,768 symbols, the new-receiver experiments involve only approximately 39 distinct symbols. This raises concerns about generalization to more diverse vocabulary, complex relationships, and realistic tasks.
-
Limited evidence of multi-stage reasoning: The intermediate model primarily concatenates received facts with its own information, which can largely be reproduced without an LLM. Thus, the experiments demonstrate information forwarding rather than communication requiring intermediate reasoning or information transformation.
-
Insufficient comparison with existing approaches: Although the paper includes text and continuous-vector baselines, it lacks direct comparisons with representative latent communication methods under matched conditions. The reported communication-size advantage over continuous vectors also does not establish improved end-to-end efficiency.
Questions for authors
-
Advantages over simpler approaches: Given that generated messages closely match the English-derived reference code and a fixed text decoder achieves comparable accuracy, what specific benefits justify the learned communication interface over simpler fixed-code or tokenizer-transfer approaches?
-
Generalization: How would the framework handle substantially larger vocabularies, unseen symbols, or more complex and diverse information beyond the spatial-reasoning tasks?
-
Intermediate reasoning: Can the framework support multi-stage communication where intermediate models must perform nontrivial computations or transform information, rather than simply concatenate facts?
-
Efficiency and scalability: What are the end-to-end computational and communication costs compared with conventional text-based and existing latent communication methods, including adapter training and autoregressive message generation?