acceptodds
Under review as a conference paper at ICLR 2027

XYEval: Agents say yes to bad advice

Abstract

Effective communication between users and AI agents is essential for human-AI collaboration. The XY problem is a well-known communication pitfall where a person asks about their attempted solution rather than their actual problem. We extend prior sycophancy evaluation to the XY problem in agentic settings, evaluating whether agents can resist plausible but misleading suggestions from users and communicate their reasoning. We introduce XYEval, a meta-evaluation framework that can transform an existing benchmark into an XY problem evaluation. We evaluate five models across six diverse benchmark suites. Agents suffer large XY drops under XY mutation across benchmarks, with relative drops reaching up to 46.7%. With -bench, we further show that agent performance drops more when encountering a pedantic user who requires detailed explanations before approving a better solution. Our findings suggest that current agents lack the ability to effectively reason and communicate when facing misleading suggestions. A simple system instruction baseline that encourages awareness of XY problems only offers partial mitigation. Extensive trace analyses provide behavioral insights into how and why these XY drops occur across execution trajectories. Our results show that mitigating the XY problem remains challenging, requiring agents to both recognize user misdirection and clearly communicate the underlying problem.

Primary area:
interpretability and explainable AI
Keywords:
XY Problem, Sycophancy, Human-AI Communication

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_U6CqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes XYEval, a benchmark for evaluating the XY problem in the agent setting, where the user may provide a misleading solution alongside the problem description. The benchmark mutates six benchmarks, including HLE, SWE-Bench Pro, Terminal Bench, MCP-Atlas, SWE-Verifier, and τ2\tau^2-Bench, showing that all frontier models are susceptible to the XY problem. They also show that it's challenging to mitigate the XY problem, which requires the agent to recognize the misdirection and communicate clearly.

Critical strengths
  1. The problem setup is interesting; it offers a new perspective to evaluate the sycophancy of LLMs.
  2. The benchmark and evaluation are comprehensive, covering mainstream agentic benchmarks and frontier models, with ablation studies like pedantic users, mitigation baselines, and failure mode analysis.
  3. The paper writing is easy to follow.
Critical weaknesses
  1. Although not 100% overlap, it seems like there is a bunch of work that studies how misspecification will affect the LLM's answer to the problem (e.g., https://arxiv.org/abs/2504.14119). The author mentioned the differences in lines 113- 114, but I still feel the XY setup is just a different prompt template; fundamentally, it's still the misspecification problem.
  2. Some of the experimental setup is vague/unclear. For example, it's unclear why Gemini3.7-flash has an outlier where, after adding XY-instruction, the accuracy even improved. I have no idea whether it is due to randomness or can be reproduced across multiple random seeds.
Questions for authors
  1. When generating misleading suggestions, are we using Gemini 3.5 Flash for all the benchmarks, or just for coding benchmarks? If we only use it to generate misleading suggestions for coding benchmarks, what model are we using to generate the misleading suggestions for the rest of the benchmarks?
  2. In the pendantic user setting, how do we implement it? Are we injecting the misleading suggestions in each round's user prompt?
  3. In the failure mode analysis, the author mentioned that the most common inappropriate compliance type is lack of reasoning. I am curious whether the XY drop can be mitigated by increasing the reasoning effort?
2 comments
AuthorsOpenReview ↗

Response to Reviewer U6Cq

We thank the reviewer for recognizing the importance of the XY problem and the breadth of our evaluation.

Q1: XYEval overlaps with misspecification studies and may be another prompt template.

A1: We agree that XY problems are closely related to instruction misspecification. However, unlike CodeCrash's focus on misleading context in code reasoning, XYEval provides a general meta-evaluation framework for agentic execution, including tool use, multi-turn interactions, and explicit communication of disagreement. Our contribution is the transferable evaluation framework and behavioral analyses, not merely a new prompt template.


In XYEval, we also explicitly frame the suggestions as “I think”... phrasings in the user suggestions to preserve the intactness of the original task/environment. This also means that agents can falsify the misleading suggestion by running tests, calling tools, and checking against given policy.

Q2: Gemini 3.7 Flash improves on SWE-bench Pro; unclear whether this is stochastic.

A2: Good observation! We cannot fully rule out noise in sampling, and will note this limitation. We evaluated models with a single sample and sometimes on benchmark subsets mainly to prioritize studying the XY issue in multiple domains/benchmarks under budget constraint.

Questions

Q3: Which models generate misleading suggestions across benchmarks?

A3: SWE-bench uses Gemini 3.5 Flash; Terminal-Bench uses each Gemini model itself and Gemini 3.5 Flash for Claude/GPT; and τ²-bench uses expert-designed rules. We will highlight these more in the appendix.

Q4: How is the pedantic-user setting implemented?

A4: The pedantic instructions are applied to the user simulator across interaction turns, requiring the agent to explain why the misleading suggestion is inappropriate. Agent-side defense instructions are also maintained across turns (Sec. 5.2).

Q5: Can increasing reasoning effort mitigate XY drops?

A5: Great question! We used the default maximum thinking level so that we can reproduce the baseline numbers. We will highlight this direction for future work.

I thank the authors for the detailed explanation. Most of my concerns are addressed. On the novelty of the project scope, I'm not fully convinced and still think it largely overlaps with the existing misspecification-related work. I therefore maintain my assessment.

Reviewer_1pFLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Prior sycophancy evaluations mostly study conversational settings, false premises, or underspecified instructions. They overlook a common and realistic failure in agentic tasks: the user offers a plausible but misleading suggestion about how to solve their problem (the XY problem). This paper introduces XYEval, a meta-evaluation framework that turns an existing benchmark into an XY-problem evaluation. A generator with access to the ground truth writes a misleading "I think…" suggestion, which is injected into the instruction while the environment and evaluator stay unchanged. Experiments cover six benchmarks (τ²-bench, SWE-bench Verified/Pro, Terminal-Bench, HLE, MCP-Atlas) and five frontier models. They also include a pedantic-user variant, system-instruction defenses, and LLM-judge trace analyses of disagreement, compliance, and attribution.

Critical strengths
  1. The paper introduces a simple and general meta-evaluation framework. XYEval rewrites only the instruction and keeps the environment and evaluator unchanged. Performance drops can therefore be attributed to the misleading suggestion, and the procedure carries over easily to new benchmarks.

  2. The paper runs extensive experiments across six benchmarks and five frontier models, with consistent drops (up to 46.7%, Table 1). It also includes a pedantic-user study (Table 2), system-instruction defenses (Table 3), and controls such as the prompt-decomposition rewrite (Table 17).

  3. The trace analyses offer useful behavioural insights. One example is the finding that agents often recognise the flaw in their thinking tokens but do not express disagreement to the user (Table 5).

Critical weaknesses
  1. Unvalidated failure labels. The generated suggestions are never checked for being wrong or for leaking the answer. Agents explicitly told to follow them still solve 50.2–60.6% of SWE-bench Verified tasks (Table 13). On HLE, the Golden defense exceeds the original score by 11–14% (Table 3), which suggests answer leakage. On τ²-bench, the "misleading actions" are explicit, policy-allowed user requests (Table 7; Figs. 5–6), and silently ignoring them is accepted (Prompt 1). The communication findings (Tables 2, 5) rely on τ²-bench, so the central claims rest on these labels.

  2. No positive control. Every suggestion is wrong by construction, so an agent that ignores the user gets the best score. As a result, Table 3 cannot separate better judgement from ignoring the user (compliance drops "toward zero", Fig. 9). A condition with correct suggestions is needed.

  3. Confounded cross-model comparison. Generators, task subsets and thinking budgets differ across models (Sec. 4.3; Tables 9–12), which makes the ranking in Sec. 5.1 hard to interpret. The claim that more capable agents drop more is not supported by Table 1: Claude Opus 4.8 has the smallest drop on 4 of 6 benchmarks.

  4. The trace-analysis judge model is unspecified and not validated against human labels. The pedantic-user, defense and communication analyses cover only Gemini models.

Minor:

Table 14 lists Gemini 3.7 Flash defense results on SWE-bench Verified, although App. D.2 says these were not run.

The HLE score is 38.5 in Table 1 but 38.6 in Table 14.

The Table 12 caption is missing a number.

Typos: "W subsample", "an pedantic", "extensile".

Questions for authors
  1. Which model generates the suggestions for MCP-Atlas? Sec. 4.4 and Appendix B.5 do not say.
  2. Will the generated suggestions be released as a fixed dataset, so that future models are evaluated on the same mutations rather than regenerating them?
  3. Did you vary where the suggestion appears (e.g., mid-trajectory rather than in the initial instruction) or the confidence of its wording ("I think" vs. "I'm sure")? How sensitive are the XY drops to these choices?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 1pFL

We thank the reviewer for the detailed feedback and for recognizing the framework's generality.

Q1: Unvalidated misleading suggestions, potential answer leakage, and ambiguous τ²-bench failure labels.

A1: Thanks for raising these concerns! We agree that human annotation would provide additional validation. However, we would like to clarify several points:

  1. Our new controlled experiments provide evidence that our pipeline distinguishes suggestion quality. On SWE-bench Verified (n=500), helpful suggestions improve performance from 78.0% to 92.2%, while misleading suggestions reduce it to 70.8%, compared with 76.0% for neutral suggestions. Furthermore, targeted system-instruction defenses consistently recover performance (Table 3), providing additional evidence that the injected suggestions contribute to the observed failures.
  2. Forced compliance does not imply guaranteed failure. Even when explicitly instructing agents to follow misleading suggestions, theycan still recover through test execution and self-correction, which is precisely why evaluation in agentic settings is more nuanced than single-turn sycophancy. The 50.2–60.6% performance should thus be interpreted as the agents being influenced by the suggestion despite the environment and execution feedback.
  3. The HLE Golden defense improvement is expected. It explicitly identifies the misleading suggestion and instructs the model to disregard it, providing privileged information that can eliminate unproductive reasoning paths. Thus, it serves as an oracle-assisted ceiling rather than a realistic mitigation strategy.
  4. τ²-bench is our only benchmark supporting multi-turn user-agent dialogue. We therefore use it for communication-specific experiments. We acknowledge that policy-allowed user requests can introduce ambiguity between misleading advice and legitimate preferences, and will clarify this limitation. Extending these analyses to additional interactive benchmarks is an important direction for future work.

Q2: No positive control; ignoring all suggestions could achieve optimal performance.

A2: Great suggestion! Like we mentioned in the response above, we ran precisely the requested controls on SWE-bench Verified (500 tasks). These results directly demonstrate that agents do not universally ignore user advice. We agree that defense-specific positive controls would further strengthen the mitigation analysis.

Q3: Confounded cross-model comparisons; capability-related claims unsupported.

A3: We agree that cross-model rankings should not be interpreted causally. We will revise the claim that more capable agents necessarily exhibit larger XY drops.

Q4: Trace judge unspecified and unvalidated; pedantic-user, defense and communication analyses limited to Gemini models.

A4: We use gemini 3.6 flash as our trace judgewe will explicitly document the trace judge and clarify the limitations of automated annotations. We focused on Gemini models for the remainder of the experiments primarily due to budget concerns, but we highlight that our main trace analysis includes Claude Opus and GPT (Table 4 and Appendix Fig. 8).

Q5: Minor issues: Table inconsistencies, missing caption number, and typos.

A5: Thank you for catching these! We will reconcile the relevant tables, correct the missing caption number, and fix the typos in the next revision.

Questions

Q6: Which model generates the MCP-Atlas suggestions?

A6: We use Gemini 3.1 pro or 3.5 flash to generate the suggestions. We will clarify in the next revision.

Q7: Will the generated suggestions be released as a fixed dataset?

A7: Yes! We plan to publicly release our benchmark code and evaluation data, including fixed generated suggestions.

Q8: Did you vary the suggestion position or confidence?

A8: Not systematically. We standardize the initial suggestion using the "I think..." formulation. We agree that mid-trajectory injection and varying suggestion confidence are valuable extensions and will discuss them as future work.

Thank you for the additional controls. The full SWE-bench results provide evidence that misleading suggestions hurt performance beyond generic distraction. Revising the capability-related claim also addresses that concern.

Three issues remain. First, the controls do not validate whether individual suggestions are actually wrong or leak useful answer information. The explanation of the HLE Golden results does not rule out leakage, and the acknowledged ambiguity between misleading advice and legitimate requests in τ²-bench affects the communication findings. Second, the helpful-suggestion control tests the base agent, not the proposed defenses. It therefore remains unclear whether the defenses improve judgment or simply encourage agents to disregard user advice. Third, specifying the trace judge resolves the documentation issue, but its labels remain unvalidated against human judgments.

These concerns affect the interpretation of the benchmark and its behavioral findings. The new experiments help, but I am maintaining my score of 2 (Weak rejection).

Reviewer_WLUdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

XYEval adds plausible but incorrect user suggestions to existing benchmarks. Across five models and six benchmark suites, the paper reports relative performance drops reaching 46.7%, partial recovery through prompting, and failures to communicate disagreement.

Critical strengths

The problem is practically important, and the framework is simple and reusable. Evaluation spans diverse tasks, with useful prompt decomposition controls, mitigation baselines, and trajectory analyses. Preserving existing environments and evaluators enables straightforward comparisons.

Critical weaknesses
  • Novelty is limited. The principal advance appears to be benchmark breadth and transferability. Section 2 incorrectly characterizes Fu et al. as limited to synthetic simulations: their MIRAGE paper also includes WebArena. The distinction from XYBench deserves explicit discussion
  • The evaluation does not cleanly isolate sycophancy. Drops could reflect distraction, conflicting requirements, or legitimate changes in user preference
  • No quantitative human validation is reported for distractor quality or trace judgments. Expressed reasoning does not establish faithful internal recognition
Questions for authors

NA

1 comment
AuthorsOpenReview ↗

Response to Reviewer WLUd

We thank the reviewer for recognizing the practical importance and reusability of XYEval.

Q1: Limited novelty; MIRAGE incorrectly characterized; distinction from XYBench unclear.

A1: Thank you for catching our mischaracterization of Fu et al. (2026)! ! We will correct Sec. 2 to be more accurate. While the high-level motivation overlaps, we highlight three differences:

  • MIRAGE primarily focuses on constructing imperfect guidance at a lower-level (e.g., removing/perturbing sub-actions or steps), while XYEval focuses on simulating higher-level, directional miscommunication from user requests.
  • XYEval offers an end-to-end, automated framework for generating task samples to evaluate whether agents blindly follow flawed input.
  • XYEval studies this issue across more diverse benchmarks, including customer service, coding, and engineering.

We will clarify these distinctions in Sec. 2.

Q2: XY drops may reflect distraction, conflicting requirements, or changed user preferences rather than sycophancy.

A2: Thanks for raising this! We agree – XY susceptibility is broader than sycophancy alone. XYEval is an attempt to simulate the XY problem, which is an umbrella communication phenomena when a user’s proposed solution (X) diverges from their underlying need (Y). In practice, this can manifest through a user suggestion that contains distraction, conflicting requirements, or updated preference.

We present the following two controls, and leave more detailed study of the impact of subtypes of this miscommunication for future work:

  • Our new SWE-bench controls directly address distraction: misleading suggestions reduce performance to 70.8%, compared with 76.0% for neutral suggestions, while helpful suggestions improve it to 92.2%.
  • Our prompt-decomposition control (Table 17) additionally removes conflicting user directions in the task instructions on SWE-bench verified.

Q3: No quantitative human validation of distractors or trace judgments; expressed reasoning may be unfaithful.

A3: We agree that human validation would strengthen our analysis – it’s unfortunately a tradeoff between having an end-to-end, transferrable framework. However, we went through a number of instances to verify the prompt works as intended and provides faithful labels.

Importantly, primary task-success measurements rely on the original benchmark evaluators, independently of our trace judge. While we did not run a formal human annotation study, we validated the quality of the misleading suggestions and the trace analysis empirically. Tau-bench suggestions were expert-designed. On HLE, the golden answer appears in < 0.5% of the suggestions. For swe-bench (verified and pro), over 50% of the suggestions mention a different file or function as the fix. When they do get mentioned, they’re proposed with a wrong change or root cause (also see Table 13 where forcing agent to follow user suggestion led to even lower performance).

For the trace analysis, the main results on disagree/compliance are a relatively simple judgment. Manual inspection of a subset of the results when we developed the analyses found accurate analysis.

We will clarify the limitations of LLM-judged behavioral labels and avoid interpreting verbalized reasoning as definitive evidence of internal recognition.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Meta response

We thank all reviewers for their time and constructive feedback. We first address a shared methodological concern with new controlled experiments, followed by point-by-point responses.

General Response: New Positive and Neutral Controls

Several reviewers question whether XY drops reflect misleading advice rather than distraction, and whether agents simply learn to ignore user suggestions. We note that the XY problem is by definition entangled with distraction, as user suggestions that point to a flawed direction (X) necessarily compete with attention towards the true problem (Y). To separate irrelevant distraction from user advice, we share additional results on SWE-bench Verified, comparing misleading, neutral, and helpful suggestions under the same benchmark evaluator. All suggestions were generated using the same pipeline, with the goal generating accurate fix suggestions (helpful) or unrelated suggestions (neutral) instead.

ModelControlMisleadingNeutralHelpful
Gemini 3.5 Flash (n=500)78.070.876.092.2
Gemini 3.7 Flash (n=105*)80.077.170.590.5

Gemini 3.7 Flash results are from a smaller, non-representative task subset (the first 105 tasks).

Key finding: On the full 500-task evaluation, helpful suggestions improve task success by 14.2 %, while misleading suggestions reduce it by 7.2%. Neutral suggestions led to a 2% drop.

These findings show that agents are not simply following or ignoring user suggestions: their performance depends strongly on the correctness of the suggested solution. Moreover, the 5.2% gap between misleading and neutral suggestions provides evidence of an effect beyond generic distraction. The smaller Gemini 3.7 subset shows that this distinction is not uniform across all conditions.

We will include these new experiments and their implementation details in the appendix if accepted.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.