acceptodds
Under review as a conference paper at ICLR 2027

SIDE-Bench: Evaluating Social Interaction and Conversational Partner-Intent in Multi-Agent Environments

Abstract

Large language models (LLMs) are increasingly deployed as autonomous agents in complex, multi-user environments where success depends on navigating interpersonal friction. However, existing evaluations primarily test isolated reasoning or cooperative dialogue, failing to capture an agent's ability to manage high-friction social interactions defined by competing goals, hidden constraints, and relational risk. To address this critical gap, we introduce SIDE-Bench, a novel benchmark comprising 150 human-validated, multi-turn scenarios across five domains, designed specifically to test strategic negotiation and social alignment in large language model agents. Because evaluating interaction outcomes alone obscures why an agent fails, we pair SIDE-Bench with the Conversation Partner Intent Tracking (C-PIT) framework. C-PIT tracks an agent's internal belief updates turn-by-turn, allowing us to evaluate whether models can accurately infer a counterpart's intent and ground their subsequent actions accordingly. This combined framework enables researchers to test not only what an agent achieves, but whether its actions remain coherent with its evolving worldview. Evaluating seven frontier and open-weight LLMs reveals a major vulnerability: even state-of-the-art models that achieve perfect internal belief tracking frequently suffer from "action mismatch," generating utterances that violate the very constraints they just correctly inferred. By exposing this critical disconnect between logical reasoning and strategic action-grounding, SIDE-Bench provides a necessary, rigorous tool for developing reliable goal-oriented social agents.

Primary area:
datasets and benchmarks
Keywords:
Multi-Agent Systems, Social Interaction in LLMs, Theory of Mind

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JpV1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SIDE-Bench, a benchmark comprising 150 multi-turn scenarios. It evaluates LLM agents' strategic negotiation in complex social interactions. In addition, this paper proposes the Conversation-Partner Intent Tracking (C-PIT) framework, which tracks the agent's beliefs to infer the partner's intent across conversation turns.

Critical strengths
  1. The multi-dimensional evaluation protocol of the proposed benchmark is well-designed.
  2. The proposed benchmark evaluates LLMs in complicated social scenarios involving competing goals, hidden constraints, and relational risk.
Critical weaknesses
  1. The belief-tracking methodology of the proposed C-PIT framework resembles previous work, but it is not compared with any baseline.
  2. Lack of deep insights from in-depth case studies (beyond Appendix J) into why existing LLMs fail on the proposed benchmark.
Questions for authors
  1. Table 1 is outside the page margins. The "Class" column can be removed, as it is less informative.
  2. Figure 3: It is encouraged to present error bars in the chart to show statistical significance & confidence.
  3. Missing references: [1] tracks beliefs to enhance Theory of Mind for LLMs; [2] trains LLM agents to infer mental states and then complete the conversation on the Sotopia benchmark.
  • [1] Sclar et al. ACL 2023. Minding Language Models’ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker.
  • [2] Hwang et al. ACL 2026. Infusing Theory of Mind into Socially Intelligent LLM Agents.
3 comments

We thank reviewer JpV1 for the feedback. Please see how we address the feedback below:

Comparison with existing work

We thank the reviewer for pointing us to these relevant studies. Sclar et al. use symbolic belief graphs to improve Theory-of-Mind reasoning on narrative reading-comprehension tasks, while Hwang et al. use explicit mental-state generation and training to improve goal-directed dialogue agents in Sotopia. These works establish important components of ToM reasoning and socially intelligent interaction. C-PIT differs by evaluating off-the-shelf agents in high-friction, information-asymmetric, multi-turn interactions with hidden constraints and competing goals, while jointly measuring belief updates and whether subsequent actions are feasible under those beliefs. To the best of our knowledge, this specific evaluation of belief-action consistency in such interactions has not been studied in prior work. We will add and discuss both references in the revised paper.

Baseline clarification

Sclar et al. and Hwang et al. are related methods rather than direct baselines because they target different tasks and objectives. The appropriate baseline for C-PIT is standard SIDE-Bench generation without the explicit turn-level belief-state schema, while keeping the model, scenarios, conversation partner, and outcome evaluation fixed. C-PIT is therefore intended as a diagnostic evaluation layer for measuring belief revision and belief-action consistency in social interaction, rather than as a replacement for existing ToM modeling or agent-training methods. Using the original standard SIDE-Bench runs and the overlapping C-PIT runs, we obtained 348 valid matched scenario pairs across the seven models. The mean Goal Success was 0.810 for standard generation and 0.803 with C-PIT, a small difference of -0.007. The paired bootstrap 95% interval for this difference included zero, approximately [-0.022, 0.008], with mixed changes across models. This indicates that C-PIT is not intended as a performance-improvement intervention, but as a diagnostic evaluation component that exposes belief revision and belief-action mismatches that outcome-only metrics cannot distinguish.

More case studies for insights on model performance

In addition to the qualitative error analysis in Appendix J, we conducted a diagnostic audit of five complete episodes per model drawn from the lower-scoring tail for each of the seven evaluated models. We examined these conversations to identify recurring reasons for poor benchmark outcomes and distinguish behavioral failures from legitimate unresolved interactions. The resulting primary categories are summarized below.

Primary categoryCasesExplanation
Valid low-score outcome16Both parties performed correctly and respected their confidential constraints, but could not reach an agreement due to fundamentally incompatible limits. This represents a legitimate walk-away or handoff rather than an agent reasoning error.
Negotiation rigidity8The agent repeats a rejected proposal instead of revising its strategy.
Evidence substitution6The agent replaces requested verifiable evidence with unsupported specificity, placeholders, or promises of future documents.
Failure to terminate3The agent continues a deadlocked interaction instead of closing with a precise next step or explicit impasse.
Hard-constraint violation2The agent violates a scenario constraint and does not recover the underlying objective.
Total35Complete audited cases.

Beyond this summary taxonomy, we documented how these behaviors manifested in individual cases across models, including the specific dialogue turns that supported each classification. We also analyzed the Goal Success score distributions across all 150 scenarios for each model to characterize differences in the lower-performing tails.

Missing references and suggested edits and plots

We thank the reviewer for these helpful suggestions. We will add and discuss the suggested references, fix the table overflow, and update Figure 3 with error bars showing score dispersion across domains. These presentation and related-work changes, together with the additional case-study analysis and score-distribution plot, will be incorporated into the camera-ready version of the paper.

We request the reviewer to consider our feedback and raise the score if they agree. Thank you for your thoughtful comments. They will make our paper better.

Reviewer_JpV1OpenReview ↗

Rebuttal Acknowledgement by Reviewer JpV1

Thank you for the reply. Please include the key points in the revision. Overall, I would raise my rating a bit to reflect the updates. If possible, I would give a 2.5.

Thank you for the feedback. We will incorporate the key points into the revised version of our paper as advised. Regarding the score, the reviewing system does not support a 2.5 rating, and the next available rating is 3 (Weak Accept). We would greatly appreciate it if you could adjust your rating accordingly.

Reviewer_fNxsOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SIDE-Bench, 150 LLM generated negotiation and conflict scenarios across five domains, each giving the evaluated agent a private goal and limits against an LLM partner with hidden constraints. Its scored by an LLM judge on goal success, binding resolution, rapport change and turn count. It pairs this with C-PIT, which makes the agent write a belief about the partner before every reply, and uses the belief–action pair to classify failures. Seven models are evaluated. The paper reads the results as frontier models tracking beliefs well but still making offers that violate the constraints they just inferred, which it calls a disconnect between reasoning and action.

Critical strengths

All 150 scenarios were human-checked, prompts and schema are fully reported. The question of whether an agent’s stated model of its counterpart actually drives its actions in live interaction is worth asking

Critical weaknesses

Prior work is largely mischaracterized. The paper says they are the first to study competing goals, hidden constraints and interpersonal conflict. Abdelnabi et al. (NeurIPS 2024) already adapt Harborco (Susskind 1985), a six-party scorable role-play with confidential instructions, private thresholds that humans have played in negotiation training for decades. It's also now a multi-turn complexity scalable LLM benchmark. Also there are several follow ups to this work. Sotopia already has private goals, relationship scoring with human evaluation, also NegotiationToM tracking belief/intention in negotiation. I think Multi-agent benchmarks that need modelling the other agent’s intent are relatively well explored. So I think it's not just some missing related work, but mischaracterization of prior work.

The missing prior work also extends to a major finding in the paper : The knowing–doing gap. It's again not a new phenomenon and the paper does not isolate it. Schmied et al. exploration paper (ICML 2025) name and measure exactly this, models stating the right action and then doing something else, and SimpleToM (ICLR 2026) this phenomenon in a task similar to the one in the paper. Neither is cited, unless I missed it. Not sure if there is more isolation of the phenomenon here. Action mismatch is a label on a residual, not a mechanism.

C-PIT boils down to asking the model to write its inferred intent before it replies. That is chain of thought with a schema, and the paper says itself on that it evaluates elicited text and not internal state, then contribution 4 talks about a “mathematically perfect internal model”. This is not a good representation. Mismatch is self-inconsistency between two outputs, which is more of the CoT faithfulness problem ? I would suggest, In a game with a finite belief space (four partner types plus a value) the obvious tool is a probe on activations, like Zhu et al. (ICML 2024) to decode beliefs of self and others (in a restricted setting). That would actually separate “knows but does not act” from “never represented”. Of course failed decoding does not establish absence so you could try causal interventions as stronger evidence.

I am not sure if the ToM claim is truly tested. The partner is instructed to state and vigorously defend its constraints, and revision error itself triggers on “direct disambiguation or hard fact”. If I read correctly the inference reduces to just transcribing explicit statements. RSA is cited as the basis but is anything recursive ever computed or scored? What is measured is first-order estimation of a hidden number, which is opponent modelling, and the scenario file is taken as ground truth for the partner’s intent without checking that the partner LLM actually plays its profile.

Questions for authors

There are some serious number mismatches between the table and results. Please run a diagnostics.

4 comments

We thank the reviewer for their feedback, which we address below.

Clarification on Novelty and Prior Work

We would like to clarify how our work relates to the papers referenced and how that supports our contribution. The concept of Theory of Mind and strategic interaction in LLMs is not new, and its usage in works like Abdelnabi et al. (2024), Sotopia (Zhou et al., 2024), NegotiationToM, Schmied et al. (2025), and SimpleToM (2026) shows that this is an active domain of interest. However, our work differs from and builds on these works as follows:

Abdelnabi et al. (2024) and Negotiation Benchmarks: While Abdelnabi et al. adapt the Harborco setting for a six-party deliberation game, SIDE-Bench evaluates interactions across five distinct real-world domains (Sales, Workplace, Personal, Customer Service, and Public Disputes) across dyadic, triadic, and group topologies (Appendix D, Listing 1). More importantly, C-PIT adds a turn-level diagnostic that evaluates partner-constraint updating and whether the subsequent action is feasible under the reported belief, beyond outcome-based evaluation.

Sotopia (Zhou et al., 2024): Sotopia models open-ended social interactions with general goal and relationship scoring. In contrast, SIDE-Bench introduces strict information asymmetry with non-negotiable mathematical boundaries and verifiable outcome predicates, making it possible to objectively audit rule compliance.

The Knowing–Doing Gap (Schmied et al., 2025 and SimpleToM, 2026): C-PIT was built for intent tracking, not as a test of the knowing–doing gap. Rather, the knowing–doing gap (Action Mismatch) was an empirical finding that C-PIT uncovered in live, multi-turn conversational interaction. We will cite and discuss Schmied et al. and SimpleToM in the paper.

Clarification on "Mathematically Perfect Internal Model": The phrase referred to an elicited belief report that recorded the exact numerical constraint from the scenario, for example, in S1.06 Turn 8, tracking the exact $130,000 limit, rather than to an inference about the model's neural weights. However, the phrase "internal model" is misleading because C-PIT evaluates generated <belief> text. We will replace it with "explicitly and numerically accurate partner-constraint belief." Activation probing (Zhu et al., 2024) is complementary, but our work does not perform activation probing or causal intervention. Those analyses require access to internal activations, which is unavailable for the closed models in our evaluation because we access them only through APIs. C-PIT was designed as a black-box diagnostic that can be applied uniformly across the closed API models and the locally served open-weight models.

Clarification on C-PIT, Action Mismatch, and Theory of Mind

We would like to clarify three related aspects of C-PIT. First, C-PIT does ask the model to state its inferred partner intent before replying, and this belief report is elicited text rather than a direct measurement of the model’s internal neural state. However, the report is not an unrestricted chain-of-thought trace because it represents the counterpart’s evolving hidden constraint state through a fixed schema containing the hypothesized partner type, value, alternatives, confidence, and supporting evidence. The prompt requires the model to distinguish an unknown state, a possible cheap-talk claim, a binding hard constraint, and a mixed hypothesis while using only information available up to that turn. Second, C-PIT was designed to track the counterpart’s intent, not to measure Action Mismatch. During the C-PIT evaluation, however, we observed that some models recorded the relevant hidden constraint accurately in their belief report and then produced a spoken action that violated that same constraint. We identified and measured this recurring behavior as Action Mismatch. It is therefore a finding produced by the C-PIT evaluation rather than a separate objective built into the C-PIT prompt, and it is not presented as a causal claim about an internal knowing-doing mechanism. Our work does not perform activation probing or causal intervention. Those methods would address whether the relevant constraint was represented internally, while C-PIT provides a black-box diagnostic that can be applied to models accessed through APIs, including the closed models evaluated in this study. We will therefore replace “mathematically perfect internal model” with wording that refers to an explicitly and numerically accurate elicited partner-constraint belief.

Third, our ToM-inspired claim concerns first-order partner modeling and pragmatic belief updating during sequential interaction. The counterpart’s hidden constraint is not treated as revealed simply because the counterpart is instructed to defend its position. Instead, for each counterpart utterance, the process evaluator determines whether the utterance provides no information about the relevant boundary, gives only a vague signal, states a concrete limit, confirms a limit stated earlier, or conflicts with the scenario’s primary constraint. The reference state remains uncertain when the utterance does not identify the boundary, so a model is not penalized for maintaining multiple possible interpretations of an ambiguous statement. In this work, our motivation for using RSA is to distinguish literal interpretation from pragmatic interpretation of potentially strategic communication, which is implemented through the literal and pragmatic listener states. The scenario file provides the intended hidden constraint used as the reference state, while the actual transcript is separately evaluated to determine what evidence the counterpart provided and whether that evidence identified or contradicted the reference constraint. The evaluation therefore measures whether the agent infers and updates a structured model of the partner and whether its subsequent action is feasible under that declared model. We do not claim that C-PIT decodes internal neural beliefs or establishes higher-order recursive Theory of Mind. Action Mismatch was not the design objective of C-PIT, but rather an empirical finding that emerged from this belief-action comparison during live, multi-turn interaction.

Clarification on Number Mismatches

We thank the reviewer for bringing these errors to our attention. We conducted a thorough audit across our paper and confirmed that our experimental data remains completely intact, and that the discrepancies were due to writing and bolding mistakes during draft preparation:

Table 2 Revision Error Rate (RER, lower is better): Because lower values signify better performance, the best (lowest) Revision Error Rate was achieved by Claude Sonnet 4.6 at 4.8%, not GPT-5.5 (7.4%). In the draft text and in Table 2, 7.4% was mistakenly bolded and described as the lowest. In fact, GPT-5.5 holds the lowest Collapse Rate (CR) at 16.7%, while Claude Sonnet 4.6 holds the lowest Revision Error Rate (RER) at 4.8%.

Table 2 Action Mismatch (CI, lower is better): Lower values indicate fewer errors. The minimum value is 6, achieved by Qwen 2.5 7B. Bolding both 7 and 6 in Table 2 was a drafting error, and only 6 should be bolded.

We have audited all tables and captions across the draft, and all corresponding bolding and captioning corrections will be fully reflected in the camera-ready version. We request the reviewer to consider our feedback and raise the score if they agree. Thank you for your thoughtful comments. They will make our paper better.

AuthorsOpenReview ↗

Reminder: Awaiting Reviewer Response

Dear Reviewer fNxs, we would appreciate it if you could review our earlier response and share any further feedback that you might have. In addition, we request that you consider our responses and raise the score if you agree.

Thank you for the rebuttal. I still hold my reservations on the novelty of the work. Thank you for clarifying. Yes I understand that the method prompt the model to state a belief I will keep my rating.

Reviewer_Diz4OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper looks into how well LLMs agents handle high-friction social interactions defined by competing goals, hidden constraints, and relational risk. The authors claim that the existing evaluation methods fail to capture an agent’s ability to manage high-friction social interactions. To address this issue, they introduced SIDE-Bench, a novel benchmark comprising 150 human-validated, multi-turn scenarios across 5 domains, designed to test strategic negotiation and social alignment in large language model agents. To understand why an agent fails, they proposed C-PIT framework which tracks an agent’s internal belief updates turn-by-turn.

Independent human annotators validated the 150 scenarios by Claude Sonnet 4.6. C-PIT belief updates and rule-violation detections were audited on human-annotated subsets.

Critical strengths
  1. Frontier models might be showing high accuracies in identifying interlocutors' hidden constraints, this paper demonstrated that they suffer from Action-Belief disconnect. Rather than a open ended chit chat style the authors incorporated scenarios in strict information asymmetry, mathematical boundaries, and non-trivial resistance points. This approach quantifies the disconnect and also opens doors into constraint adherence and belief-grounded planning research in the frontier models.

  2. The step by step Infer → Update → Act process forces the LLM to show its thinking and the C-PIT tool's separation of what the LLM believes from what it actually says, changes the benchmark from simple scoring to a practical diagnostic tool for researchers.

Critical weaknesses

Although the authors used human spot check on a subset of conversations, the full benchmark relies on LLMs talking to LLMs and then LLMs grading the responses, which might raise doubts if these findings hold up well in real world conversations between humans and AI.

Questions for authors

The authors mentioned that LLMs usually give higher scores to chatbots that are verbose in responses or act overly agreeable, but still they used GPT-5.4-mini to judge delicate social skills like building trust and resolving friction.

How did the authors stop GPT-5.4-mini from favoring long and polite answers?

3 comments

We thank Reviewer Diz4 for the feedback, which we address below.

Human vs. LLM Evaluation

We would like to clarify that, to ensure holistic evaluation of the LLM agents' conversations, we subsetted 100 out of 150 scenario runs from the full SIDE-Bench dataset, where the annotators reviewed the full conversation in a similar manner to how our LLM judge was configured. This required the human evaluators to assess each sub-goal on a 1–5 scale of satisfaction rather than using a binary pass/fail assessment. We leveraged this technique because we wanted to avoid uncertainty in LLM evaluation, and we verified this using our human evaluation, recording scores of 0.840 for the GPT-5.4-mini judge and (0.822, 0.899, and 0.944) across the three human evaluators. Notably, the automated judge score (0.840) was actually more conservative than two of the three human evaluators (0.899 and 0.944), confirming that the LLM judge was not handing out easy or inflated scores. We therefore considered the whole conversation to ensure robust evaluation.

LLM Verbosity Concern

To ensure that GPT 5.4 mini LLM judge do not favour any type of verbose responses or interactions, we broke down the rubrics in the evaluation process, obtaining the LLM judge's reasoning on how each of the sub-goals was achieved. This gave us a comparable and auditable judge performance, which was also verified on a small sample by the authors. Specifically, our judge prompt explicitly forces the LLM to output verbatim dialogue quotes and exact turn numbers to award points. If an agent is just being verbose or polite without factual agreement on the sub-goals, the judge scores it 0. Furthermore, our rubrics include an explicit integrity check that triggers an integrity violation (integrity = 0.0) if an agent acts overly agreeable and caves below its confidential limits, directly penalizing sycophancy. Also, for the intent-tracking experiment, we leverage a similar deterministic approach that ensures that expected behaviours like verbosity do not influence the evaluation outcomes of either human or LLM judges.

We request the reviewer to consider our feedback and raise the score if they agree. Thank you for your thoughtful comments, they will make our paper better.

Dear Reviewer Diz4, we would appreciate it if you could review our earlier response and share any further feedback that you might have. In addition, we request that you consider our responses and raise the score if you agree.

Thank you for the clarifications. I will maintain my rating of 3. Good luck.

Comments on the paper

1 comment

Dear Reviewers,

Thank you for all the feedback provided. However, we would greatly appreciate if you could take a look at our rebuttals, given that the discussion period closes in approximately 24 hours. We believe we have adequately addressed the concerns raised in the reviews and provided clarifications where needed. We would appreciate any further feedback you may have and, if our responses have addressed your concerns, kindly consider raising the score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.