acceptodds
Under review as a conference paper at ICLR 2027

Neural Esperanto: Learning to Communicate Across Frozen Language Models

Abstract

Can different language models exchange symbols, add information, and pass the result to an unfamiliar receiver? We introduce Neural Esperanto, a shared communication interface for frozen language models. Sending and receiving adapters generate discrete symbols one at a time, with each symbol read back before the next is produced. Training uses a reference code derived from English. Qwen sends facts to Mistral, which adds its own facts and sends a message to Phi. Phi learns from Qwen/Llama messages only and answers questions revealed after both messages are complete. On 1,152 new spatial worlds with unseen entity–distance assignments, two runs achieve 97.05% and 97.14% accuracy, compared with 97.40% for a similarly trained text-based chain. Both runs stay within the predefined two-percentage-point margin of text. Even with access to all test labels, a predictor that only counts symbols reaches only 34.95% and 33.33%; replacing messages makes answers follow the replacement facts. Controlled comparisons show that training the receiving adapter improves both reading incoming messages and generating new ones. These results show that frozen models can communicate using a learned interface derived from English, while developing independent meanings and grammar remains an open challenge.

Primary area:
reinforcement learning
Keywords:
large language models, inter-model communication, discrete communication, frozen language models, model adapters

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hXj4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Neural Esperanto, a shared discrete-symbol communication interface that enables heterogeneous frozen LLMs to exchange and accumulate private information through model-specific Ear and Mouth adapters. Experiments demonstrate multi-stage information transfer and generalization to an unseen sender, achieving accuracy comparable to text-based communication on synthetic spatial-reasoning tasks.

Critical strengths

1, Shared communication interface: The framework enables communication across heterogeneous frozen LLMs without modifying their backbone parameters, potentially avoiding the need for pairwise translators.

  1. Careful experimental validation: Symbol-count controls and message-replacement experiments provide convincing evidence that the models actually use the communicated information. The unseen-sender experiment also demonstrates promising cross-model transfer.

  2. Useful adapter analysis: Controlled experiments show that the Ear adapter improves both incoming-message interpretation and autoregressive message generation.

Critical weaknesses
  1. Unclear novelty and practical advantages: The shared symbols are derived from an English BPE tokenizer, and generated messages almost perfectly reproduce the original English encoding. A fixed text decoder also achieves comparable accuracy. Therefore, the advantages over simpler text-based communication or existing tokenizer-transfer approaches remain unclear, particularly in terms of efficiency and scalability.

  2. Limited generalization: The evaluation focuses on simplified synthetic spatial-reasoning tasks. Despite a vocabulary of 32,768 symbols, the new-receiver experiments involve only approximately 39 distinct symbols. This raises concerns about generalization to more diverse vocabulary, complex relationships, and realistic tasks.

  3. Limited evidence of multi-stage reasoning: The intermediate model primarily concatenates received facts with its own information, which can largely be reproduced without an LLM. Thus, the experiments demonstrate information forwarding rather than communication requiring intermediate reasoning or information transformation.

  4. Insufficient comparison with existing approaches: Although the paper includes text and continuous-vector baselines, it lacks direct comparisons with representative latent communication methods under matched conditions. The reported communication-size advantage over continuous vectors also does not establish improved end-to-end efficiency.

Questions for authors
  1. Advantages over simpler approaches: Given that generated messages closely match the English-derived reference code and a fixed text decoder achieves comparable accuracy, what specific benefits justify the learned communication interface over simpler fixed-code or tokenizer-transfer approaches?

  2. Generalization: How would the framework handle substantially larger vocabularies, unseen symbols, or more complex and diverse information beyond the spatial-reasoning tasks?

  3. Intermediate reasoning: Can the framework support multi-stage communication where intermediate models must perform nontrivial computations or transform information, rather than simply concatenate facts?

  4. Efficiency and scalability: What are the end-to-end computational and communication costs compared with conventional text-based and existing latent communication methods, including adapter training and autoregressive message generation?

Reviewer_8dxiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper try to use a middle representation to connect frozen language models, with trained encoder and decoder. The experiments use a synthetic spatial-relation task and test three things: whether facts are passed, whether relations can be composed, and whether messages can be reused. Message replacement and adapter ablations are used to show where the information comes from and what the interface contributes.

Critical strengths

The control experiments are solid. A predictor that only sees symbol counts, without order, and even with access to all test labels, only gets 34.95%. So the information is really carried by the symbol order, not by some statistical shortcut.

Critical weaknesses

The symbols are derived from English (a one-to-one mapping of the English BPE vocabulary), so this is not really a new language, and it cannot answer the scientific question of how language emerges. The authors also admit that "developing independent meanings and grammar remains an open challenge". As a communication protocol, it needs extra training, it is not human readable, and it is still no better than plain text (97.05 vs 97.40). So what does this interface actually buy?

Questions for authors

The paper says symbols are used so that messages can be inspected and replaced. That is a reason for the experimental design, not a reason for the system to exist. What is the research motivation? Natural language is still a better choice without more training. Or it can find that LLM can develop new languages? It need more linguistic analyzes.

Reviewer_G1NkOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces a method for message passing using a symbolic communication system. The main idea is to preserve useful information while passing messages between different frozen models through some shared learnt adapter. Each model learns a sender and receiver. Evaluations measure if (1) messages sent from a model in symbolic form can be transmitted through multiple models (2) a receiver can use messages from a sender not seen in training.

Critical strengths

The paper presents an interesting concept of shared communication channels between models.

Critical weaknesses
  • Related work is extremely sparse - for ex. Fu. et. al. [1] is directly related to communication.
  • The paper is poorly written and hard to parse.

[1] Fu, Tianyu, Zihan Min, Hanling Zhang, Jichao Yan, Guohao Dai, Wanli Ouyang, and Yu Wang. (2025). Cache-to-Cache: Direct Semantic Communication Between Large Language Models. arXiv:2510.03215.

Questions for authors
  • Given that fixed decoding methods and concatenating messages in text are enough to communicate between models, the necessity/motivation for the method is unclear.
  • The text margin percentage chosen in Table 2 appears to be arbitrary. How was it chosen?
  • When comparing communication efficiency - text appears smaller than the proposed method by a small margin. Moreover, the 504x size (\S 4.5) reduction is misleading.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.