- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper provides an analysis of the inferences that LLMs make in social interactions, through the lens of a game similar to Hanabi. In this setting agents can provide a hint to a game partner, where the hint follows a convention about which card is targeted and how the hint should be interpreted. The other agent then has to make an inference about the appropriate action. Probes are used to identify the extent to which the models seem to be representing the intent and target information and formulating the correct response. A total of 8 LLMs are evaluated in this setting via these measures, both when paired against the same LLM or against a different LLM. Further experiments explore how performance is affected by intervening with an appropriate activation vector or fine-tuning. The key results are that intent is recovered more accurately than target, and interventions via activation transfer and fine-tuning both improve performance.
Critical strengths
The main strength of this paper is its in-depth analysis of convention inferences in a controlled setting. In addition, the use of multiple converging methods -- not just probing but also activation transfer and fine-tuning -- provides deeper insight than would have been obtained with probing alone.
Critical weaknesses
The primary weakness of the paper is that it has relatively limited scope. While the use of the constrained domain makes it possible to carefully analyze the behavior of the model, I would want to see that similar results hold outside this limited setting.
Another significant limitation is that there are no reported statistical significance tests or error bars on most of the critical results (although some results are marked with an asterisk, which may denote statistical significance but I couldn't find an explanation in the text). All qualitative statements about the results (such as one model or method producing an improvement over another) need to be supported by statistical analysis. At present the key results depend entirely on reported differences in proportions and are difficult to evaluate. This applies to results in 5.2-5.7 and Tables 1-4, and Figures 4 and 5 both need error bars.
There is not a detailed discussion of the limitations of the work, which could give the impression that the results are broadly generalizable.
Questions for authors
For making the claim that models are worse at inferring target than intent, it seems like the relevant baselines should be included. For example, there may be a smaller set of possible targets than possible intents, so the task is simply easier. How do you correct for this?
3 comments
Question. "For making the claim that models are worse at inferring target than intent, it seems like the relevant baselines should be included. For example, there may be a smaller set of possible targets than possible intents, so the task is simply easier. How do you correct for this?"
The option sets have the same size. The target decision is the leftmost or the rightmost of the hinted cards, and the intent decision is PLAY or DISCARD, so both decisions are binary with chance 0.500. The convention labels used to evaluate knowing are also nearly balanced. At the final turn, their majority-class rates are 0.504 for target and 0.517 for intent in LLM games, and 0.500 and 0.531 on scripted boards (App. L.3, Table 21). The revision states the chance level in Sec. 5.3 and in the captions of Tables 1–2, and adds that equal chance does not make the two axes equally difficult.
The observation-feature classifier measures how readily each convention can be predicted from the recorded history without using LLM hidden states (revised prompt, new App. L.3). In LLM games at the final turn, its accuracy is 0.844 for target and 0.990 for intent, compared with 0.656 and 0.833 for the hidden-state probe. The target–intent difference therefore already appears in the observable events.
The paper's main claim does not depend on this difference. Under told, the true convention is given and no convention inference is needed (Qwen3-8B, Table 2, submitted prompt, 240 games per condition; intervals in App. J, Table 16). Target doing then rises from 0.515 to 0.821, while intent doing stays at 0.599, against 0.591 under raw. Intent, the axis the history reveals more readily, is the one the model does not consistently apply, so the rule-versus-action result on intent concerns application, not inference difficulty.
W3. "There is not a detailed discussion of the limitations of the work, which could give the impression that the results are broadly generalizable."
The limitations section (App. J in the submission, App. K now) was not referenced from the main text. The conclusion (Sec. 6) now points to it, and the revision extends it with two items. One states that knowing is recoverability by an external linear probe, which does not distinguish observable cues retained in the representation from a convention estimate formed by the LLM, and that the behavioral contrasts separately test how supplied conventions change decisions. The other states that the response-interface check covers only Qwen3-8B on recorded states, not complete games. App. K also states that the results concern controlled receiving decisions in a two-player environment with scripted senders.
These changes are in Sec. 5.3, the captions of Tables 1–2, Sec. 6, App. K, and the new App. L.3.
W2. "There are no reported statistical significance tests or error bars on most of the critical results (although some results are marked with an asterisk, which may denote statistical significance but I couldn't find an explanation in the text). ... This applies to results in 5.2-5.7 and Tables 1-4, and Figures 4 and 5 both need error bars."
The revision adds intervals for every result the review lists. Sec. 5.1 now defines the asterisk: it marks a paired difference whose board-level 95% bootstrap interval excludes zero (20,000 resamples). Pairs share the deal and convention assignment, or the recorded state, and the four games of a board are resampled together. With these intervals, every asterisk of the submission is unchanged. The table below gives where the statistics for each result are.
| Result | In the revision |
|---|---|
| Table 1, Sec. 5.2 | Intervals for every cell of seven of the eight models in App. J, Tables 14–15 (the hidden states of Qwen3-32B were not retained) |
| Table 2, Sec. 5.3 | Intervals for every cell in App. J, Table 16; asterisks on the score differences quoted in Sec. 5.3, with their intervals in App. J |
| Figure 4 | Error bars for doing and score |
| Figure 5, Sec. 5.4 | Single-cell interval half-widths in the caption (0.14–0.29 points); asterisks on the cross-play differences quoted in Sec. 5.4, with intervals for the 56-pairing means in App. J |
| Table 3, Sec. 5.5 | Asterisks on the mean aligned Δ in the table; intervals for every cell in App. J, Table 17 |
| Table 4, Sec. 5.6 | Asterisks in the table; intervals for every cell in App. J, Table 18 |
| Sec. 5.7 | Asterisk on the quoted adapter difference; score intervals in App. J, Table 19, and the adapter comparison in App. I |
Every comparison stated in Secs. 5.2–5.7 holds with these intervals. Recomputing Tables 1–4 and Figures 4–5 from the recorded runs reproduces every published value except five entries of Table 4, which now use the recomputed values. These differ from the submitted ones by at most 0.005, with no change in sign or conclusion (App. J). App. F now states the sources of its Gemma-3-12B row and of the Fig. 5 self-play cells.
We thank the reviewer for the careful reading and for finding the converging methods useful. Changes for this review have a green highlight in the revised PDF, on top of the other reviewer's text color where that reviewer raised the same point. This comment addresses W1, the next addresses the Question and W3, and the last addresses W2.
W1. "The primary weakness of the paper is that it has relatively limited scope. While the use of the constrained domain makes it possible to carefully analyze the behavior of the model, I would want to see that similar results hold outside this limited setting."
The central contrast is not specific to one model, and it holds under both single-token answers and free generation. In every LLM we evaluated, giving the true convention as a rule (
told) changes intent doing by −0.015 to +0.109, while replacing the rule with the current action (told + verb) adds a further +0.056 to +0.583 (App. F, Tables 10–11, 240 games per cell), andtold + verbhas the highest score. These are differences between point values without per-model intervals, so we read the pattern as directional.The contrast points the same way over pairs of different LLMs and under free generation. Over the 56 ordered cross-play pairings (App. J, board-level intervals), the mean score is 5.14 under
raw, 5.41 undertold, and 6.98 undertold + verb, so translating the true convention into the current action raises the cross-play score by +1.57* over the rule statement. On 480 recorded Qwen3-8B states (App. C.3), under free generation the rule raises intent doing by +0.040*, and translating it into the current action raises it by a further +0.279*.The controlled environment is what makes this checkable. The scripted sender fixes the meaning of every hint, so each decision is scored against a known convention (our response to LnLx, 3/3). Extending the test to other cooperative tasks, richer or learned conventions, generated hints, and more than two players is future work, and App. K states this scope.
These changes are in Sec. 6 (pointer to the limitations), App. K, and App. J (cross-play means).