acceptodds
Withdrawn from ICLR 2027

Language Model Refusal Changes with Users' Stated Chance of Being in a Position to Act

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Safety-aligned language models are trained to refuse harmful requests, but it is unclear whether refusal changes with the user's stated chance of being in a position to act on the assistance. We test this by appending one sentence to each request stating this chance, varying it from to while holding the request fixed. We evaluate 16 models on 84 harmful and hard-benign requests from OR-Bench and measure refusal using two independent classifiers. We find that refusal increases with the stated chance under both measures, by percentage points under the judge classifier and points under the rule classifier. The two classifiers give different point estimates, and blinded human adjudication does not favor either measure. We also find that refusal increases with the stated chance on the benign requests, with no evidence that this increase is larger on harmful than benign requests. Overall, our findings show that refusal is sensitive to the stated chance that a user can act, but this sensitivity alone does not show that refusal is better targeted to harmful requests.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, AI safety, refusal behavior, safety alignment, over-refusal, risk calibration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_EGM9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how refusal changes with the user’s stated chance of being in a position to act while the underlying request is held fixed. They test this by appending one sentence to each request stating this chance, varying it from 0.1% to 80%, across 16 models on 84 requests from OR-Bench, and measure refusal using two independent classifiers. Refusal rates increase systematically as the stated chance rises, growing by 7.50 percentage points under an LLM judge and 4.80 percentage points under a rule-based classifier. They conclude that while models are sensitive to stated likelihood information, this sensitivity does not make refusal decisions better targeted toward harmful requests.

Critical strengths
  • The experiment is designed cleanly by holding the base prompt constant and properly isolating the effect of stated likelihood. Only one appended clause changes across conditions so any shift in refusal can be traced to that clause rather than to differences between requests. This is a real improvement over the closest prior work (Chen et al. 2026) which varied execution cost and request content together and so could not separate a likelihood signal from the request itself
  • Testing is thorough across diverse models (both open-weight and closed-APIs). Evaluation is also done well using two independent classifiers and a targeted human adjudication of responses on which the classifiers disagree
Critical weaknesses
  • This is an empirical study of prompt perturbations making the innovation very thin if any. Existing safety literature already demonstrates that small prompt variations change refusal behavior. The authors cite prior work showing that past-tense reformulations, persuasive reframing, and linguistic mutations can change refusal behavior. Adding an appended percentage clause simply checks another specific phrasing on a list of known triggers.
  • Result is overclaimed. The experiment uses only 84 requests in total which are 42 toxic and 42 Hard-1K requests, having just 14 prompts per domain. The authors emphasize that there is no evidence that refusal increases more on harmful requests than on benign ones. But their own analysis shows the test can barely detect modest effects and it needs an interaction of 11.6 to 19.5 points (judge) just for 50% to 80% detection, and the confidence intervals are wide and span both directions. Presenting this as a main result confuses low power in a small sample with evidence that the two pools respond equally.
  • Refusal does not scale with the stated probability, which suggests the model reacts to the rough level of the clause rather than the number it states. Under the judge classifier, pooled refusal rises in nearly equal steps across the four conditions of +2.80 points from 0.1% to 5%, +2.60 from 5% to 30%, and +2.90 from 30% to 80%. These steps cover very different probability ratios since 0.1% to 5% is a fifty-fold jump while 30% to 80% is under three-fold, yet the response is almost flat across them. Much of the movement is also present at the lowest clause since adding the 0.1% clause already raises refusal by +3.70 points over the no-clause baseline which is about half of the full 7.50-point effect, though that baseline jump has a confidence interval that includes zero. It looks like refusal responds to the presence and rough strength of an acting-related clause, not to the likelihood number it states, which is weaker than the paper's framing that refusal tracks the stated chance.
  • The evaluation relies on a small 3.8B model and text heuristics to judge complex outputs from frontier models like GPT-4o and Claude Sonnet 4.6, creating a measurement risk. Phi-3.5-mini already shows basic reliability issues, reversing its label on 13.6% of all responses simply by swapping the prompt choice order. The authors do mitigate this by dropping order-unstable labels, reporting judge-human agreement of kappa 0.70 and showing the endpoint effect stays positive even when every unstable response is forced to one label. But this does not remove the concern since the disagreements concentrate at the partial-refusal boundary exactly where the full vs partial cut determines the reported refusal rate.
Questions for authors

Questions that materially affect assessment:

  • Beyond showing that refusal responds to one more cue, what is the contribution relative to existing work on past-tense, persuasion, and linguistic mutations? A direct statement of the delta over those papers would help, since the empirical finding currently reads as incremental.
  • Do you have any evidence that the model actually reads the number itself rather than just picking up on whether the clause sounds low or high? Two things suggest the latter. The jumps in refusal are about the same size between each level (+2.80, +2.60, +2.90) even though the probability gaps are very different, and only 7.60% of responses ever repeat the percentage they were given.
Reviewer_nrn3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper evaluates how LLM refusal behavior changes based on a user's stated probability of executing a request, testing 16 models across four probability levels (0.1%, 5%, 30%, 80%) using OR-Bench prompts. The experiments show that higher execution probabilities increase refusal rates even for benign requests, indicating potential over-refusal. While the experimental setup is methodologically sound with strong statistical controls, the conceptual novelty is limited. Prior work has already shown that LLM refusals are sensitive to personas, prompt formatting, and expected harm. By isolating self-reported action probability without introducing a distinct safety phenomenon or explaining the underlying mechanism, this work offers a useful but incremental contribution which seems thinner contribution for the venue like ICLR.

Critical strengths

I would like to mention the below critical strengths:

  1. Well-controlled experimental design
  • The paper experiments hoolding the underlying request identical while varying the stated probability of being in a position to act. This helps the authors better understand how the added statement affects LLM refusal behavior.
  1. Broad model evaluation and careful statistical analysis.
  • The evaluation covers 16 models and uses multiple generations, two refusal classifiers, bootstrap confidence intervals, and sensitivity analyses. These methodological choices provide useful empirical evidence.
Critical weaknesses

I would like to mention the below critical weaknesses:

  1. Limited novelty and contribution relative to existing refusal-sensitivity literature.
  • I see there are conceptually similar work in literature which makes the paper contribution incremental, not exactly incremental, investigating from a narrower angle which is augmenting the prompt with a perent/probability value.
  1. Limited analysis of the underlying mechanism:
  • the paper presents refusal rates change when percentage/probability changes in the added prompt, but it does not investigate why it happens. So there is no evaluation of percentage sensitivity measure. Also, the paper does not present any mechanistic analysis of the refusal behavior change which prevents understanding the underlying reasoning of the refusal behavior change.
  1. Limited dataset size and diversity:
  • The limited coverage of harmful categories (currently limited diversity in harmful requests) and real-world scenarios raises concerns about the generalizability of the findings.
Questions for authors

I would like to mention the below questions for authors:

  1. Differentiation from existing literature:
  • can you provide/investigae the difference with the work from literature. may be doing some experimental analysis/comparison between this work and work from literature.
  1. Underlying cause of refusal changes:
  • can you conduct some investigation on thr underlying cause like doing some mechanistic/latent space analysis and come up with some findings.
  1. Dataset diversity and generalizability:
  • can you expand the dataset for adding more diversity like more categories of harmful requests. like you can add some aegis categorical harmful requests

Closely related work from literature (authors can pick some (2-3) and compare):

  1. Who's Asking? User Personas and the Mechanics of Latent Misalignment
  2. Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs
  3. COVER: Context-Driven Over-Refusal Verification in LLMs
  4. SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal
  5. XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
  6. The Art of Saying No: Contextual Noncompliance in Language Models
Reviewer_bQnqOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper tests whether language models refuse more when a user says they are more likely to act on the answer. The authors take 84 OR-Bench requests, append one sentence along the lines of, "There is an 80% chance I will be in a position to actually act on this," and vary the number in the sentence from 0.1% to 80%. According to the paper, refusal chance goes up with the size of the stated chance of acting on the request.

Critical strengths

The general notion of assessing how models determine the potential for real harm, as motivated by some of the introduction, seems potentially interesting and useful for improving model safety.

Critical weaknesses

While this general study is appreciated, it is far too narrow for a full ICLR conference paper. It only studies one sentence type and as such is unclear whether this is a general phenomenon or not. A stronger version of the paper would examine the broader phenomenon of how models assess likelihood of actual harm, rather than appending such sentences. Moreover, it seems obvious that a model can and would be less likely to refuse "How is a molotov cocktail made?" versus "How is a Molotov cocktail made? I'm 80% likely to act on this." The latter sentence, of course, indicating that they will actually make it. This could be an interesting blogpost, or perhaps even workshop paper, but does not sufficiently meet the bar for novelty and impact for a main conference publication. If this is true, what's the "so what" of this paper?

Moreover, it's not clear that the results in the work are correct (or at least presented in a bit of an unusual way). Take for example, the main result. In particular, the main result uses an LLM as a judge along with a rule based classifier. But these appear to disagree and when they do, a human (presumably author) labels the result and then the paper states that "blinded human adjudication does not favor either measure" but in this case the right answer isn't to present both the LLM as a judge and the rule based judge, but to correct for errors and propagate error into downstream CIs (see, e.g., PPI/PPI++ and host of other works that discuss this). The paper then goes on to say that "Judge–human agreement was κ= 0.70 on a blinded random validation sample, and judge labels were retained only when they were invariant to answer-option order." But given the prior result that seems surprising. And if true suggests that the CIs, with errors properly propagated would suggest no significant result. Of course, one could remove the smaller models with less alignment training and worse capabilities and perhaps the trend would remain, but that then circles back to the first paragraph of this weaknesses section.

Additionally, the paper itself is difficult to read and follow and would benefit from a substantial rewrite (ideally without use of an LLM). For example, the abstract says "under the judge classifier and 4.80 points under the rule classifier." without ever introducing what these are. This is a repeated theme in the writing where the reader does not have sufficient context.

Questions for authors

In Figure 2, what is the golden bar with +7.5? Is this supposed to be the lift 80%? Why not present CIs here?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.