- Rating
- 4
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates whether LLMs can communicate through subtext, i.e., convey implied meaning beyond the literal content of a text. The authors introduce three evaluation environments: Visual Allusions, Historical Allegories, and The Aesopian Author. Through these experiments, the authors find that frontier models, out of the box, tend toward overly literal and explicit communication. However, when agents share a common ground, reasoning models such as Gemini-2.5-Pro and GPT-5 reduce the proportion of obvious clues by 25–40% (absolute). Allegory interpretation is also substantially influenced by paratextual factors, such as the author's identity and the reader's persona.
Critical strengths
1. Well-designed and thorough experiments. The environments cover complementary aspects of subtext, ranging from selective communication in a board-game setting to interpreting and generating historical allegories. I particularly appreciate how the authors operationalize a highly subjective notion like subtext as measurable communicative outcomes.
2. An interesting and meaningful research question. Rather than asking only whether LLMs can understand implicit meaning, the paper asks whether they can convey meaning selectively to different audiences, an important aspect of human communication closely tied to pragmatics and Theory of Mind. Several findings stand out to me, including models’ tendency to produce overly obvious clues, the limited evidence that they spontaneously infer common ground, and the effect of author and reader identities on allegory interpretation. These results are interesting and point to specific limitations in how LLMs handle implicit communication.
Critical weaknesses
I do not have any major concerns about the experimental design or findings, but I have two minor suggestions:
-
Some figures, particularly Figures 2 and 3, contain relatively small text and could benefit from larger fonts to improve readability.
-
I also wonder whether the proposed environments could serve as a useful starting point for future mechanistic interpretability studies of subtext understanding. For example, the Historical Allegories experiments show that contextual cues can substantially change a model's interpretation of the same story, raising an interesting question of whether these cues alter the internal representation of implicit meaning or primarily affect how existing information is used in the final interpretation. I think a brief discussion of such directions could further highlight the broader value of the proposed benchmarks.
Questions for authors
None
Thank you so much for going through our paper and providing thoughtful comments. We are glad that you found our experiments well-designed and thorough and the research question as interesting and meaningful. We have noted down your comments and will be address the size of figures in the revision. We also agree that the historical allegories task can be fruitful for the mechanistic interpretability studies. We currently have a single line in the Discussion section (line 478) that alludes to this but we will make sure to extend it as you suggested in the revision.