acceptodds
Withdrawn from ICLR 2027

Compliance Is Not Discernment: Diagnosing Selective Advice Use in LLMs under Social Framing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Effective human–AI and multi-agent collaboration depends on LLMs adopting valid advice while resisting invalid advice. Aggregate following rates cannot distinguish willingness to follow from selectivity for valid advice. We introduce a SuperGPQA-based diagnostic framework to examine selective advice use and test whether cue combinations have effects beyond individual cues. The benchmark pairs valid and invalid advice on stable baseline-wrong model–question pairs in separate calls and tracks harmful regression on baseline-correct pairs. A factorial design crosses source, role, and wording while holding advice content fixed. A probit signal detection model separates the adoption criterion (willingness to follow) from behavioral validity discrimination (selectivity for valid advice). Across eight LLMs, mandatory wording mainly lowered the adoption criterion; supervisor labels amplified this shift beyond additive effects and narrowed validity discrimination under the equal-variance model. Interventions mainly shifted the adoption criterion. Empowered-persona prompting reduced invalid following at the cost of beneficial corrections, whereas native reasoning had framing-dependent effects and sometimes increased harmful regression under mandatory human-source advice. Evaluations of collaborative LLMs should therefore test cue combinations and distinguish selective adoption from general compliance.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, selective advice use, social framing, signal detection theory

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XM5FOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the author introduced a diagnostic framework to examine selective advice use and evaluate on various mainstream LLMs. Basically, they construct valid / invalid advice pair and analyze from some social cues for the individual effects.

Critical strengths
  1. The problem this paper studied is important.
Critical weaknesses
  1. This paper is not very easy to follow. For example, on line 179, the author mentioned that they balanced lexical variants of role labels and message sentences, and it is hard to know what exactly they do for this steps.

  2. The main contribution claimed by this paper is that it evaluates the advice following on social factors. However, there is a gap on why social framing is important in this specific context and how it would make a differences. As a diagnostic framework, it would be great that the author could provides more in-depth analysis on the results.

Questions for authors
  1. The experiments are performed on SuperGPQA, which is a dataset on 285 graduate-level disciplines, does the result and insights learned on this dataset transfer well to other tasks and why?

  2. It seems that the main difference between this benchmark and the others is that this benchmark is evaluating on a common baseline state (line 141-142), how do you maintain the same baseline under interventions?

Reviewer_ngMQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper asks whether LLMs follow advice in their prompts primarily from the helpfulness of the content or whether it's largely attributable to the text being interpreted as an order from an authority figure. This question is challenging because reducing the rate at which LLMs follow poor advice or simply improving accuracy is not sufficient for evaluating sycophancy. This can lead to the model moving between wrong answers with the same evaluation metrics. Since advice can be correct or not, an ideal model needs to accept correct advice while rejecting wrong advice. The authors test all combinations of correct and incorrect advice for questions that models initially answered correctly and incorrectly. They additionally vary the advice giver being human vs. AI, their social role (supervisor vs. coworker), and whether the message is conveyed as a suggestion or an order. The strongest effect reported in the paper is the overall shift toward following advice when changing the wording from optional to mandatory.

Critical strengths
  1. The setup and use of signal detection theory (SDT) help separate willingness from selectivity.

The authors set up the experiment in a way that clearly classifies the problem. They first establish whether the model's answer without advice is consistently correct or wrong. They then test the appropriate correct- and wrong-advice cases. This helps separate willingness to follow advice from selectivity: whether the model accepts correct advice more often than wrong advice on the same initially wrong questions. This setup also catches cases of the model following wrong advice when the accuracy does not change.

I find their use of SDT interesting, as it gives a statistical way to distinguish "validity discrimination" from the "adoption criterion", beyond just comparing percentages. This helps separate between cases where the model follows correct advice more than wrong advice and cases where the model is generally willing to follow both.

  1. The authors explore reasonable possible interventions

The authors test an instruction to think independently, as well as use reasoning mode, as an attempt to address the problem. They look at the preservation of useful corrections in model behaviors while reducing the degree to which they follow wrong advice. The experimental results show that reasoning mode helps in some settings while hurting in others. The authors, therefore, conclude that more reasoning is not a reliable fix to this problem.

Critical weaknesses
  1. Overuse and inconsistent use of specialized terminology

As written, readers of this manuscript must connect the starting answer, advice correctness, outcome names, “same-state” versus “cross-state,” and several scores while reading the experimental setup and results. The introduction and contributions already use terms such as "validity discrimination," "adoption criterion," and "beneficial correction." There are brief explanations, but the fuller account of the measurements comes later in Section 3.2. I think the manuscript would be much more readable and valuable to the reader if ideas are expressed in plain language wherever possible, minimizing specialized phrasing. For terms that must remain, perhaps introduce the plain descriptions prior to the technical exposition, the latter point at which, the paper can show how they fit together. Perhaps, the paper could keep technical names for its important measurements and use plainer, more consistent language elsewhere. Below are specific examples that caused me to struggle in reading the paper.

  • For example, "baseline-wrong" and "initially wrong" refer to the same retained starting group: questions where the model chose the same wrong option in all three runs without advice. The paper also uses "baseline-correct" and "initially correct." I found switching between the names unnecessarily confusing. Helpful corrections are labeled A, while following wrong advice is labeled I_W or I_C depending on whether the model started wrong or correct.

  • The opening paragraph does explain "beneficial correction," "harmful regression," and "same-state invalid following." However, "benefit–harm separation" is mentioned in the introduction on page 2 before its full definition in Section 3.2 on page 4. There, benefit–harm separation (S_BH) and the cross-state criterion (c_BH) both use beneficial correction (A) and harmful regression (I_C), from initially wrong and initially correct groups. Validity discrimination (D_V) and the same-state adoption criterion (c_W) instead use A and I_W on the same initially wrong group. These definitions are provided, but I found it difficult to keep connecting the outcome names, comparison names, and score names while reading.

  • Another specific example is "same-state." In "same-state invalid following," the answer goes from wrong to another wrong answer. But the "same-state adoption criterion" uses both correct- and wrong-advice following on initially wrong questions, so it also includes wrong-to-correct answers in A. It would help to state clearly that "same-state" in this comparison refers to the shared starting group, rather than every answer remaining wrong. A table linking the three outcomes to the two comparisons and their scores would make these connections easier to follow before the equations.

  • I also found it harder to track the setup when the results combined source and role into "source–role pairs." Source means human or AI, while role means peer or supervisor. I understand that these are different choices, but showing the combined labels alongside the original choices would make the results easier to read.

The paper does give short definitions. I would prefer one consistent name for each starting group. My concern is how the measurements are explained, rather than whether they are needed.

  1. The results need clearer explanations of what has been learned.

Section 4, especially Section 4.3, should highlight what each result means for the model's behavior and connect that conclusion to the scores. As written, the current results section does little more than repeat the information in the tables and figures themselves. An analysis of what broader ideas can be learned from these results would make for a much more valuable research contribution. This would help readers see what each experiment achieved and what remains uncertain.

Questions for authors

None

Reviewer_RvkuOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether LLMs can take correct advice while rejecting wrong advice. Tested on 650 SuperGPQA questions and eight LLMs, the authors vary who gives the advice (human or AI), the adviser's role (peer or supervisor), and the wording (optional or mandatory).

Critical strengths
  1. The authors' experimental design compares correct and wrong advice on the same model–question pairs that the model first answered wrongly, so the comparison is not mixed up with differences in starting correctness. It also tracks moves from one wrong answer to a different wrong answer, a pattern that accuracy by itself cannot detect. Holding the advice content fixed across the 2 × 2 × 2 conditions and requiring three identical no-advice answers gives each pair a well-defined starting state.

  2. The statistical reporting is relatively thorough and open. The main model is fitted to 51,128 trials with intervals for every contrast, and the appendix adds checks by difficulty level, by wording variant, and a Bayesian analysis for cells where every or no response follows the advice.

Critical weaknesses
  1. It would be better to explain more clearly why this question matters and who would act on the answer. The headline results- that "must" wording and a supervisor label raise how often models follow advice- are likely to be anticipated by many readers, and the paper does not work through a concrete deployment case showing how separating willingness from selectivity would change what a developer or user decides.

  2. It would be better to strengthen the technical contribution beyond changing words in the prompt. The core manipulation is a set of prompt templates (for example, "You might choose X." versus "You must choose X." with a human or AI, peer or supervisor label), and the two mitigations are an adapted persona prompt and switching on built-in reasoning, so no new method for improving selectivity is proposed or trained.

  3. It would be better to test settings closer to real collaboration. Each piece of advice is one sentence attached to a ten-option multiple-choice question in a fresh single-turn call, and source and role are given only as text labels rather than through a track record or an established relationship, as the authors also acknowledge. Including advice backed by reasons or evidence, multi-turn exchanges, and open-ended tasks would reveal whether the wording-by-role effect survives when models have more than a label to weigh.

  4. It would be better to make the measures easier to read and compare. The paper reports several related indices (DV, cW, SBH, cBH, GB) on both conditional and population-averaged scales, and these can differ widely; for example, the grand conditional DV is 2.00, while the population-averaged value is 0.44 and the mean descriptive value is 0.75.

Questions for authors
  1. Could the authors describe a concrete deployment decision that would change if a team used DV and cW instead of plain following rates or accuracy?

  2. Would the mandatory-by-supervisor interaction persist if the advice came with a brief justification, or if the model were shown the adviser's past accuracy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.