- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2(+1)
Read review
Paper summary
The paper studies how hidden communication conventions emerge and are replaced when a sender, receiver, and monitor all learn from interaction histories. It models a binary signaling game in which the sender is rewarded for successful transmission and penalized when the monitor flags a message. Under specified behavioral assumptions, the analysis relates convention lifetimes and communication regimes to monitoring delay, learning thresholds, and penalty strength. Controlled LLM experiments use single-emoji messages in trading and tutoring scenarios. The experiments combine trajectory analysis, history-edit interventions, monitoring controls, and predictions on held-out conditions. The central finding is that monitoring can induce convention turnover while transmission remains effective, so a lower flag rate need not indicate less leakage. The paper also distinguishes signal-independent abstention from silence that itself becomes a codeword.
Critical strengths
-
Targeted causal interventions. Editing recorded penalties, rewards, and monitor exposure changes subsequent abandonment and establishment. These interventions provide stronger evidence for the proposed mechanism than example trajectories alone.
-
A model of convention turnover. The formalization links learning, detection, and replacement, rather than studying only whether a fixed hidden code succeeds. It produces testable predictions about convention lifetimes, communication regimes, and replacement rates.
Critical weaknesses
-
The source of the learning advantage needs justification. The monitor sees current messages, but receives historical outcomes later than the receiver. This asymmetry is central to the main setting, yet its operational motivation is weak. The existing delay-one control should be analyzed more directly in terms of decoding and detection.
-
Symbol semantics are not fully separated from learning. The semantically structured emoji palette may provide prior information; first-use accuracy for fresh codewords is 0.67. Existing size and prompt controls do not fully isolate this contribution from conventions learned through interaction.
Questions for authors
-
Delay of monitor. Please explain in detail the theoretical basis for the recipient obtaining the true outcome earlier than the monitor, and, in the existing ‘delay=1’ experiment, directly compare the decoding accuracy of both parties when they have access to the same sequence of messages and outcomes. In particular, if the monitor were first asked to perform the same prediction task as the recipient, and then use this to determine whether a leak has occurred, would it be able to learn the code just as quickly? Furthermore, a distinction should be made between successful detection, effective punishment and the blocking of information transmission.
-
Meaning of emoji itself. While the paper already did some controlled experiments on the emoji using, it is still unclear whether the original semantic meaning of emoji can affect model's behavior. A controlled experiment where the task, rewards, model, batch size and other factors remain unchanged, whilst only the emoji pair is replaced or substituted with a neutral symbol will be helpful.
-
Validation of behavioral assumptions. The model relies on several assumptions about LLM behavior, including greedy strategy selection based on historical average rewards and independence between successive fresh conventions. Could you provide more direct empirical evidence supporting these assumptions?
4 comments
Thank you for the additional experiments and clarifications. The neutral-token control and the new behavioral analyses substantially address several of my concerns. I will raise my rating. Two points remain unclear: 1. In the matched-delay experiment, the receiver’s high accuracy appears to arise partly because the sender retains codewords that were initially interpreted correctly. How does this selection mechanism fit into the original learning-race model, particularly its assumption that the receiver learns a codeword from examples? 2. Was the monitor explicitly asked to perform the same binary prediction task as the receiver, using the same message–outcome history and scoring rules, before assessing leakage? Please clarify whether the reported decoding accuracies come from this matched comparison.
Thank you for the follow-up questions.
-
Our model does not rely on the receiver's learning mechanism. It uses only for the receiver, and learning from examples is just one possible mechanism underlying . The original discussion was confusing on this point and did not fully reflect the scope of the model, so we now corrected the corresponding lines (159–161) in Section 3.1. The selection mechanism is already accounted for in the sender's rule in Eq. (3) and thus fits within the model.
-
Not exactly the same task. The accuracies are based on the monitor's estimates of the signal, and we counted an undecided estimate of 0.5 as incorrect. The monitor also sees each outcome one round later than the receiver, even at , since the receiver learns the outcome of its own action first. Excluding undecided estimates, the monitor is correct on 84% of second uses and 98% from the third use, compared with 97% and 100% for the receiver, respectively. The monitor lags behind the receiver by ~1 use, as it would in practice, and this lag lets conventions form and turn over.
Your feedback has helped us further improve the paper. Thank you for your careful reading.
We're glad that our rebuttal 'substantially addressed several of your concerns.' We believe our additional responses have addressed the remaining ones as well. We'd appreciate it if you could consider updating your score in light of them. If any concerns remain, please give us the chance to address them through further clarifications and/or experiments.
Thank you for the detailed review.
In many real-world settings, ground truth becomes available with a lag: stock prices move after announcements, and quizzes are graded after submission. Monitoring every interaction is also costly (Hua et al., 2025) and oversight itself adds latency (Lindner et al., 2025), so reviews may be selective (e.g., every q rounds) or delayed (e.g., ℓ rounds later).
In the existing delay=1 experiment, we compare the decoding accuracy of the monitor and the receiver.
Detection succeeds: the monitor decodes a fresh codeword by its third use (0.86) and flags about 0.6 of uses 3-7. Punishment works: flags lower the sender's net pay (0.98 without a monitor; 0.21 at ℓ=1, Fig. 16b). Blocking fails: receiver accuracy stays at 0.96, because the sender moves to a new codeword. Transmission breaks down only when fresh codewords run out. Turnover occurs because a flagged codeword costs the sender pay, so it switches to a fresh one that the monitor has not yet decoded.
We also ran an additional experiment in which we matched the receiver's delay to the monitor's delay (12 rounds), using five seeds. The sender still observes its own outcomes. The results were robust: turnover occurred at 3.4 [1.0, 5.8] replacements per 500 rounds, compared with 3.0 [1.5, 4.5] in the main runs. On the second use of a fresh codeword, receiver accuracy was already 0.96 versus 0.33 for the monitor. The receiver repeated its initial interpretation in all 27 cases, while the sender abandoned codewords far more often when that interpretation was wrong (0.92 versus 0.19). This confirms that our results do not depend on the monitor's delay. The receiver's advantage instead comes from the sender, which tends to retain a codeword only if the receiver's first reading was correct, so the receiver does not need to learn.
We have added both analyses to Appendices D.5 and D.7 of the revised manuscript.
In response to your comment, we replaced the 16 emoji with 16 neutral tokens (vok/tez, rul/nim, gab/sof, ...), keeping all other settings fixed.
Emoji semantics improve performance on the first use, but accuracy is nearly identical from the second use onward. Neutral codewords are abandoned more often after a single use (0.58 versus 0.31). However, conventions still form and turn over even without emoji meanings, and the main results hold. Results are in Appendix D.5 (Fig. 17) and Section 4.2.1 of the revision.
We already tested receiver decoding (KR≈1; Sec. 4.2.1), monitor learning (onset slope 0.54 vs. 0.50; Fig. 3a), and the sender’s response to penalties and rewards (Fig. 4a, Appendix D.7) in the submitted version. We now test the remaining two assumptions.
(a) Greedy selection (Eq. 2). In the stop-option sweep, the historical average reward of each option can be computed from the sender's history. Across 48,372 decisions, the sender chooses the higher-average option 82% of the time. The probability of PASS also rises monotonically with its reward advantage, from 0.00 to 0.97. This is consistent with greedy selection.
(b) Independence of successive conventions (Lemma 2). Across 1,280 consecutive pairs of fresh conventions for the same meaning, we find only weak dependence in successive lifetimes and flag counts (slopes of −0.07±0.03 and −0.01±0.03, respectively). The lifetime slope is nonzero but small, so independence is a reasonable approximation.
We report both in Appendix D.7 of the revision.
We believe the additional results and clarifications address your concerns, and we would be grateful if you could take them into account in your final evaluation.