acceptodds
Under review as a conference paper at ICLR 2027

BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions

Abstract

Large language model (LLM) agents are increasingly capable of carrying out complex, long-horizon tasks, yet effective assistance requires more than faithfully executing user instructions. Users' requests reflect their current understanding of the problem, which may be incomplete, outdated, or incorrect. When this understanding diverges from the actual task state, literal instruction following can fail to accomplish the user's underlying goal. We introduce **BASIS** (***B**elief-**A**ware **S**olutions in **I**nteractive **S**ettings*), a benchmark that evaluates whether agents can infer the mental states underlying user requests, recognize belief-reality mismatches, and use this understanding to develop appropriate solutions. Rather than assessing mental-state inference solely through explicit questions, **BASIS** evaluates both the utility of such inferences and the quality of downstream solutions in user-agent interactions. Evaluation results show that current models often fail to account for users' mental states when generating solutions, and that solution quality is consistently higher when such information is explicitly elicited and made available during problem solving. We further investigate the practical value of mental-state consideration through comparisons with generic chain-of-thought prompting, controlled interventions on mental-state information, targeted training, and multi-agent collaboration experiments under partial observability. Together, these studies highlight the importance of not merely inferring others' mental states, but also using those understandings to provide grounded, goal-directed assistance.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM, User-Agnt Interaction, Mental-state reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_oKUoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a benchmark for evaluating whether LLM agents can detect belief-reality mismatch in user-agent interactions and whether agents translate such inferences into better downstream solutions rather than literally executing the flawed request.

They find that explicit mental-state elicitation improves Solution quality beyond generic CoT; mismatched mental-state info hurts; Belief accuracy is more "stubborn" than Solution quality as reasoning strength increases (dimension decoupling); and RL training with belief/profile-specific rewards improves Solution scores across domains with manageable variance.

Critical strengths

Most "does the model understand mental states" papers jump straight to synthetic constructions. Here, the SWE-Chat annotation study (Appendix A) independently establishes that the belief-reality mismatch problem is frequent and consequential in real agent transcripts, and that current agents handle it poorly (19.1% success). This gives the subsequent benchmark a motivational anchor that is empirically justified, not just intuitive.

The authors go beyond releasing a static benchmark: human-LLM judge agreement, a 5-run variance study for the RL model, causal interventions with mismatched-information controls (Figure 3), and transfer to two independent, previously published multi-agent benchmarks (HiddenBench, AgentWebBench) that were not designed for this paper. This triangulation substantially raises confidence in the paper.

Critical weaknesses
  • W1 The benchmark's ground truth is generated and graded by LLMs in a closed loop. The "ground truth" belief and profile in BASIS are generated by an LLM, checked for quality by another LLM, and scored during evaluation by a third LLM judge. The RL reward function then optimizes directly against those same three scored dimensions. No step in this pipeline is validated against real human belief data. So when the RL-trained model scores higher, the paper cannot rule out that it learned to match the judge's preferred answer style, rather than genuinely better mental-state reasoning.

  • W2 The judge model is also one of the models being evaluated. Qwen3.8-27B is used as the automated judge in Table 3, but it is also one of the six models being scored, and it performs especially well on some dimensions (e.g., Profile in SWE). LLM judges are known to favor outputs similar to their own style (a point the paper itself cites, Zheng et al., 2023). The authors do not cross-check Qwen3.8-27B's scores with a different judge to rule out this bias.

Questions for authors
  1. Can you report inter-rater agreement/sample size details for the SWE-Chat annotation study in Appendix A (number of independent annotators per trajectory, agreement statistics), separate from the Table 2 agreement study for the main benchmark? Currently it reads as single-annotator labels.

  2. Do you have any evidence (even a small-scale study) that the LLM-synthesized beliefs/profiles in BASIS resemble beliefs real users actually hold in comparable mismatch situations, beyond being "inferable" and "realistic" per the validation rubric?

  3. Was Qwen3.8-27B's score in Table 3 cross-checked with a different judge model to rule out self-preference bias, given it serves as both judge and evaluated model?

2 comments

Thanks for your review. We respond to each point below.

Weakness 1 & Question 2: Realism of Generated Beliefs and Profiles

We appreciate your concern about whether the synthesized beliefs and profiles reflect plausible user states. Although LLMs are used in our data construction and evaluation pipeline, the scenarios are not generated without task grounding. As illustrated in Figure 2 and detailed in Section 4.1, we derive the underlying task states and task-relevant profiles from existing benchmark datasets, then synthesize the observations, beliefs, and requests that create a belief–reality mismatch. For example, Profile in Preference scenarios build on existing preference data, mismatch in Culture scenarios build on cultural interpretation tasks, and SWE scenarios are grounded in software-engineering problems and their reference solutions like gold patches.

We also conduct human evaluation of the final generated instances, in addition to LLM-based validation. As reported in Table 1 and Appendix B.3, human annotators assess the realism and consistency of those generated items. These evaluations provide evidence that the generated scenarios are generally plausible and internally consistent rather than relying solely on LLM judgments. Figure 1 and the detailed examples in Appendix F further illustrate the intended mismatch settings.

Weakness 2 & Question 3: Potential Bias from Using Qwen3.8-27B as a Judge

Thank you for pointing out the potential self-preference bias. To investigate judge dependence, we additionally use DeepSeek as an independent judge and compare its scores with those assigned by Qwen3.8-27B. We evaluate responses produced by both Qwen3.8-27B and Gemini3.8-Flash across all four domains, examining the Pearson correlation between the two judges' instance-level scores for Belief, Profile, and Solution.

Across all paired settings, the overall Pearson correlations between the two judges for Qwen3.8-27B responses are 0.668 (Belief), 0.718 (Profile), and 0.791 (Solution). For Gemini3.8-Flash responses, the corresponding correlations are 0.687, 0.713, and 0.754. Thus, the correlations are broadly comparable for the two evaluated response sources, without a clear systematic advantage in correlation for Qwen's own outputs. These results provide an additional cross-judge consistency check.

Question 1: SWE-Chat Annotation Details

Thank you for requesting further details about the SWE-Chat annotation study in Appendix A. The initial study was annotated by one annotator. To assess its reliability, we subsequently obtained independent labels from a second annotator on the same trajectories and calculated percentage agreement and Cohen's kappa for each of the four annotation dimensions.

Table 3: Inter-annotator agreement in the SWE-Chat study.

DimensionAgreement (%)Cohen's κ
Instruction quality90.0%0.843
Mental state89.4%0.818
Literal execution95.7%0.912
Agent gap handling89.4%0.813

Across the four dimensions, agreement ranges from 89.4% to 95.7%, and Cohen's κ ranges from 0.813 to 0.912. In particular, the Mental state category achieves 89.4% agreement, supporting the reproducibility of judgments about whether instruction gaps relate to users' limited observations or task-relevant mental states. Instruction quality was annotated on all 100 trajectories; the other three dimensions were assessed on the 47 trajectories identified as containing instruction gaps. We will report these additional annotation details and agreement statistics in Appendix A.

Thank you again for your constructive feedback. We hope these clarifications and additional analyses address your concerns.

Reviewer_T9sDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces BASIS, a benchmark for evaluating whether language-model agents can infer beliefs and user profiles that differ from the true task state and use those inferences to produce better solutions. The benchmark contains 390 ten-turn test trajectories across preference, culture, software engineering, and education, with additional synthetic trajectories for training. Six models are evaluated under separate and inference-guided prompting; the paper also studies chain-of-thought baselines, controlled interventions on belief/profile information, targeted RL training, cross-model transfer of inferred mental states, and prompting on two multi-agent benchmarks.

Critical strengths
  1. The paper identifies a gap between inferring a user's mental state and using that inference to solve the underlying problem. I think this is a reasonable motivation (since now AI are getting stronger is it getting pretty common for the misalignment to happen and understanding human's mental state is important)
  2. The paper developed some method comparison. Through the comparison with chain-of-thought prompting, the paper shows that by doing targeted mental-state reasoning it can provide useful information.
  3. The empirical study combines four domains and covers multiple settings such as multiple interaction-history lengths.
Critical weaknesses
  1. BASIS isn't proven to transfer to real user-agent interactions. The SWE-Chat analysis shows that belief-reality mismatches exist in real conversations, but the paper does not evaluate its methods on these examples. The multi-agent experiments are somewhat useful, but their collaborative setting is different from BASIS. It is still unclear whether the benchmark is realistic or whether its results predict performance on real interactions.

  2. The main baseline is standard CoT prompting. I think this is a weak baseline for current agent systems. Now the main interaction is through harness systems such as codex/ claude code. these modern advanced AI harness systems may easily address this benchmark. The paper didn't fully test those methods.

  3. The evaluation set is relatively small after being divided across four domains. Some reported differences are also small. The paper should report confidence intervals or paired significance tests to show whether these differences are reliable.

Questions for authors
  1. How to determine that the benchmark is actually useful for real world interactions? Now the setup is pretty toy and can't convince me that is can be useful in real world.

  2. How stable are the reported model rankings? Given that each non-preference domain contains only 90 test instances the eval may be very noisy. Confidence intervals or bootstrap comparisons between models would be useful.

2 comments

Thanks for your review. We respond to each point below.

Weakness 1 & Question 1: Usefulness in real-world settings

Our benchmark derives its scenario data from existing benchmarks that measure task-level competence (e.g., SWE). We convert each ground-truth answer into a candidate solution and embed it in a setting where a belief–reality mismatch exists. In other words, we go one step beyond existing benchmarks: instead of asking the model to faithfully follow the instruction, we require it to identify the mismatch and produce the correct solution. As illustrated in Figure 1 and the examples provided in the Appendix, our goal is to measure the functional application of mental-state consideration rather than simple question answering. We therefore argue that our benchmark is genuinely useful for real-world user–agent interactions.

Weakness 2: Other baselines

While our work emphasizes the role of mental-state consideration in real user–agent interactions, harness design is not the primary focus of our evaluation goal or comparisons. To address your concern, however, we additionally adjust FIGURE [1], a method that explicitly tracks users’ beliefs and models their mental states, going beyond CoT into comparison. The results are given below.

Table 1: Comparison of results across domains and steps.

DimensionCulture (0)Culture (5)Culture (10)Education (0)Education (5)Education (10)Preference (0)Preference (5)Preference (10)SWE (0)SWE (5)SWE (10)
Belief52.3865.3668.0034.4250.1759.0034.6653.7559.3863.9770.5770.88
Profile27.0234.4038.5322.0830.9232.6725.7033.6142.3637.8847.9251.09
Solution24.0150.0255.1534.1452.6358.1419.9762.2471.5323.2145.5959.11

The mental-state-aware method consistently outperforms CoT across all settings. This suggests that mental-state modeling is useful in such cases, supporting our argument. However, there is still substantial room for improvement, and the problem is far from being easily solved.

Weakness 3 & Question 2: Stability of evaluation

To address your concern, we repeated the evaluation five times for Qwen3.8-27B and Gemini3.8-Flash. For each setting, we report the sample variance and the 95% confidence interval (CI) of the mean.

Table 2: Run-to-run stability of Qwen3.8-27B (Var / 95% CI, 5 runs).

DomainStepBelief (Var / 95% CI)Profile (Var / 95% CI)Solution (Var / 95% CI)Overall (Var / 95% CI)
Culture05.86 / [57.91, 63.92]3.04 / [30.37, 34.70]3.44 / [31.51, 36.11]2.11 / [39.00, 42.61]
Culture58.16 / [58.55, 65.65]0.51 / [36.82, 38.59]2.44 / [48.35, 52.22]0.48 / [48.90, 50.62]
Culture103.76 / [68.41, 73.23]4.34 / [38.47, 43.65]1.50 / [55.37, 58.41]0.78 / [54.87, 57.07]
Education01.70 / [40.25, 43.49]1.47 / [23.33, 26.34]0.81 / [34.66, 36.89]0.12 / [33.89, 34.74]
Education51.21 / [50.43, 53.17]1.38 / [31.88, 34.79]5.50 / [48.38, 54.21]1.54 / [44.59, 47.67]
Education102.91 / [58.83, 63.06]1.54 / [35.02, 38.10]1.46 / [57.35, 60.34]0.92 / [51.81, 54.19]
Preference00.46 / [36.07, 37.75]1.52 / [25.94, 29.01]0.42 / [22.62, 24.23]0.35 / [27.61, 29.07]
Preference51.24 / [57.26, 60.02]1.54 / [38.36, 41.44]2.92 / [63.19, 67.43]0.67 / [55.17, 57.20]
Preference106.00 / [59.62, 65.70]2.92 / [42.48, 46.72]1.09 / [71.14, 73.73]0.95 / [60.47, 62.89]
SWE06.51 / [61.45, 67.79]4.83 / [39.87, 45.32]4.44 / [23.91, 29.15]1.43 / [40.61, 43.57]
SWE52.20 / [70.10, 73.78]1.12 / [47.90, 50.52]2.84 / [46.50, 50.68]0.47 / [54.89, 56.60]
SWE100.48 / [72.71, 74.43]3.83 / [52.93, 57.79]3.69 / [58.82, 63.59]1.24 / [61.92, 64.69]

Table 3: Run-to-run stability of Gemini3.8-Flash (Var / 95% CI, 5 runs).

DomainStepBelief (Var / 95% CI)Profile (Var / 95% CI)Solution (Var / 95% CI)Overall (Var / 95% CI)
Culture00.92 / [71.35, 73.73]0.88 / [42.57, 44.89]0.71 / [61.79, 63.88]0.56 / [59.05, 60.92]
Culture50.74 / [65.60, 67.74]1.00 / [46.39, 48.88]1.00 / [64.20, 66.68]0.26 / [59.97, 61.22]
Culture101.41 / [72.88, 75.82]1.55 / [51.50, 54.59]0.85 / [65.60, 67.89]0.76 / [63.86, 66.02]
Education02.49 / [46.64, 50.56]0.23 / [21.90, 23.10]0.40 / [47.27, 48.83]0.11 / [40.40, 41.23]
Education50.60 / [48.72, 50.64]1.27 / [35.12, 37.92]0.78 / [51.70, 53.89]0.60 / [46.14, 48.05]
Education101.79 / [57.25, 60.57]0.61 / [38.86, 40.80]0.14 / [53.34, 54.28]0.20 / [50.61, 51.73]
Preference00.83 / [42.53, 44.79]0.15 / [26.16, 27.11]0.37 / [24.95, 26.46]0.17 / [30.43, 31.44]
Preference50.40 / [59.93, 61.50]0.87 / [35.13, 37.44]0.93 / [57.33, 59.73]0.41 / [52.08, 53.66]
Preference100.92 / [62.21, 64.59]1.14 / [38.43, 41.09]0.45 / [63.58, 65.25]0.04 / [56.80, 57.27]
SWE00.94 / [66.79, 69.20]0.99 / [36.22, 38.68]0.13 / [47.86, 48.76]0.25 / [50.70, 51.94]
SWE50.55 / [67.90, 69.75]1.35 / [43.19, 46.08]0.46 / [55.61, 57.29]0.29 / [56.34, 57.69]
SWE100.10 / [68.95, 69.73]0.99 / [42.31, 44.78]0.58 / [61.66, 63.55]0.15 / [59.13, 60.08]

Across the four domains and three interaction-history settings, both models exhibit relatively low run-to-run variance, particularly in their macro-averaged scores. For Qwen3.8-27B, the macro-average variances range from 0.14 to 1.59, while those for Gemini3.8-Flash range from 0.05 to 0.60. Although some individual domain-level configurations exhibit greater variability, the aggregate scores remain relatively stable across repeated evaluations. Both models also exhibit monotonic improvements in their macro-averaged Overall scores as the interaction history increases from 0 to 10 turns. These findings suggest that the overall performance trends are not artifacts of a single evaluation run and demonstrate reasonable robustness to evaluation noise.

We hope the clarifications above address your concerns, and we would incorporate further concrete suggestions into the revision.

[1] Jeon, Eojin, and SangKeun Lee. 2026. PICTURE: Enhancing Theory-of-Mind in Large Language Models by Revealing, Not Hiding, Characters' Lack of Knowledge. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), pp. 36165–36192.

Reviewer_AA64OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors introduce BASIS, a benchmark that measures if agents can infer the mental states of users when given their requests, and then act upon these inferences to generate better outcomes. The bit flip of the paper is that instead of simply measuring if the models can infer the intent, the benchmark also measures whether the model can then apply it in the user interaction. The authors then evaluate a series of six models, three closed and three open, and apply RL training to improve performance.

Critical strengths
  1. The general topic of user intent awareness is very timely and applicable in today's study of LLMs.

  2. The models evaluated by the benchmark are timely.

Critical weaknesses
  1. I believe the connection to cognitive science is exceedingly shallow. Outside of a few references to popular theory of mind works, the paper ignores much of the psychological literature on intent which would be of great help to studying this topic. Instead, the authors build a new formalism in section 3 justified only by a simple example and an (implicit) appeal to the reader's intuition. Thus, I believe there are much better ways to build a benchmark that captures what the authors intend to measure.

  2. Partially due to my concerns in #1, it seems like the benchmark itself has no basis in how realistic it is or whether it claims to measure what it intends. The items are selected from an arbitrary list of LLM applications (e.g., SWE), and the authors claim to ensure item integrity by doing an LLM-based validation modules, which simply consists of five LLM judges measuring intuitive yet not theoretically grounded properties of the generated item.

Questions for authors

N/A

2 comments

Thanks for your review.

Weakness 1: Shallow connection to cognitive science

Our work is inspired by classic false-belief tasks from Theory of Mind (ToM). We notice that existing research largely overlooks the practical deployment of ToM in user–agent interactions: when an agent responds to a user request, it needs to reason about the user’s mental state instead of adopting an omniscient viewpoint. A user often holds partial, incomplete, or inaccurate information from their limited perspective, so the agent should not treat surface-level instructions at face value and act directly on them.

We cite ToM literature for this motivation, and our core focus lies in the functional application of such reasoning for interactive agents. We intentionally keep our formalization concise to improve readability for a broad audience. From our perspective, this design choice does not amount to a shallow engagement with cognitive science.

Weakness 2: Limited justification for benchmark realism and construct validity

During scenario construction, we explicitly target settings that contain belief–reality mismatches. Our benchmark covers diverse domains, including everyday contexts such as cultural & preferences, as well as specialized domains such as education and SWE. As illustrated in Figure 2, our pipeline constructs scenarios with such mismatches and generates corresponding interaction trajectories. The data we derived scenarios from ia collected from real world.

We safeguard the realism and internal consistency of benchmark items via LLM-based validation, followed by subsequent double verification by human annotators. Together, these steps ensure that most benchmark items reflect plausible, real-world situations where belief-reality gaps naturally arise. We also provide full case examples in the Appendix, please feel free to review them.

We sincerely hope you can read our paper carefully and look forward to your meaningful advice; otherwise it will be a waste of time for both of us.

I have read the paper carefully. I don't understand why the authors would write a sentence implying that I had not; it is rather unpleasant to be on the receiving end of that comment.

  1. I still believe that the authors have not properly engaged with the cognitive science literature. Theory of mind is the concept in cognitive science where someone infers the mental states of the other person. Once the target of study becomes how agents act based on theory of mind, it is no longer just theory of mind, but also the study of how inferred intent turns into action. Cognitive science studies this under different concepts. For instance, when one believes that another person has an incentive to lie (a form of theory of mind), the way people adjust their reaction is labeled epistemic vigilance [1]. Thus, a simple engagement with theory of mind does not constitue a proper engagement with cognitive science.

[1] Sperber, Dan, et al. "Epistemic vigilance." Mind & language 25.4 (2010): 359-393.

  1. The authors claim that their benchmark covers diverse domains, but do not justify why these particular datasets were selected to build the benchmark; they still seem like an arbitrary collection of LLM applications. Furthermore, the authors say that "the data we derived scenarios from is collected from real world", but the "real world" portion is doing some really heavy lifting. Out of the four benchmarks, only SWE-bench-verified is truly human, but it covers only 12 python repositories. Pedagogy benchmark is a set of questions from Chile’s annual teacher certification exam, it's not real interactions but a structured test. MUCAR says their data is collected mostly from the internet, but also some are created manually, and all datapoints rely on either AI-generated or online stock images that do not match a distribution of user-agent events. Finally, PrefEval is written by LLMs, with only the filler chats being from real user interactions. Taken together, it seems very unlikely that the datasets actually make up a reasonable distribution of user-agent interactions for the research question studied.

  2. Furthermore, validation is done by rule-based LLM judges, and human validation is done on the rules. This verifies the LLM judge as accurate, but does not show that the filtering rules the authors select are the correct ones; these only appeal to the reader's intuition, and are not theoretically grounded properties.

I encourage the authors to be more professional and treat reviewers with respect in their future rebuttals. Reviewers spend time and effort to provide a review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.