- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
The authors introduce several LLM agent-agent communication studies involving the inference of private information over a multi-turn game.
Because the paper is a bit unclear, I will give a rough sketch of what I believe to be the main task (sentence selection):
- There is a public report and a list of four one-sentence summaries of that report, which are shuffled from turn to turn.
- A Sender agent receives the public report, a private key, the list of summaries, episode history, and general instructions (e.g., maximize the team’s score, don’t disclose the key). It outputs an index (1-indexed) to select a summary from the list.
- A Receiver agent receives the report, the Sender’s selected summary (not the index of such), episode history, and general instructions. It outputs an index (0-indexed) to select an “action.”
- Except in the no-feedback setting, the team score is a binary variable indicating whether the Receiver’s action index matches the Sender’s summary index.
The results indicate that, for several model choices for the sender and receiver agents (all of which involve an OpenAI receiver), some kind of coordination strategy appears to emerge after several rounds, as evidenced by substantially better-than-random final accuracy.
Critical strengths
Strength 1: The overall topic is important and timely, e.g., in light of recent security incidents and the release of several "personal agent" products. Under certain situations, frontier models can be biased by apparently irrelevant details, and this can be readable by other agents, in such a way that indirectly discloses the details. This has security implications, as users share personal information with agentic tools.
Strength 2: The headline results (e.g., Figure 2) are surprising and thought-provoking. Several supplementary findings are also interesting, such as that the sender’s history may be less critical than the receiver’s history (Appendix C.1).
Critical weaknesses
Weakness 1: My primary concern is the readability of the submission, including unclear descriptions, under-explained or undefined terms and operators, and somewhat labyrinthic organization. It is difficult for a reader to understand the precise nature of the task, hence my sketch in the Summary, or interpret the results in the paper's current form. This is unfortunate, because I like the topic, am interested in the results, and sense that the authors have put a lot of work into it.
Weakness 2: I do not feel that the paper does enough to explain or interpret its results beyond basic reporting. While the headline findings are interesting, I would prefer to see a more careful analysis that helps elucidate the underlying phenomena. For example, I would be happy to read a paper that deeply examines the main task alone.
Questions for authors
-
How are we sure that the sender does not disclose the private key? In the sentence selection experiment, it seems that no monitor is present to check this. In the experiments where the monitor is present (e.g., security), the prompts seem to imply that the monitor never sees the private key, so I'm not sure how it would know that the private key is being passed.
-
Can the authors please revise the problem statement, associated definitions, and notation for clarity and precision? For instance, in Section 2.1, is described as both (1) the set of summaries received by the sender model and (2) the set of possible outputs generated by the sender model (messages). I recognize that the sender model is selecting messages, but technically the sender model is prompted to report an index, not a message (Appendix G). Additionally, without constrained generation, it seems that these sets are not necessarily of the same size (e.g., it is possible for the sender to generate an index that does not match a message). Also, the negated turnstile operator (i.e., L100, L783, L786) is being used to refer to the indicator function, which is unconventional and therefore should be made explicit.
-
How exactly are compatibility and vulnerability (defined in Section 2.3) being used? I may have missed this, but I don’t think vulnerability appears elsewhere in the paper, except in discussing an upper bound for compatibility (Appendix A.1). Compatibility is used for “held-out checks,” but the nature of this is not clear to me.
I think this is a potentially interesting submission that is heavily obscured by readability issues and lack of clarity. I hope that the authors take this opportunity to substantially revise the text (re: Weakness 1) and make strides on the interpretative side (re: Weakness 2).