acceptodds
Under review as a conference paper at ICLR 2027

Nudgeability: Measuring How Confidence Signals Steer Tool Delegation

Abstract

Reasoning language models that can call tools must decide during inference whether to answer unaided or delegate. Any proposal for such self-reflection mechanism has to answer three questions: *where does the reflective signal come from* (verbal reports, output distributions, hidden states, a separate predictor), *how it is presented to the model* (numerical prediction, confidence token, prompt injection), and *whether it changes the model's subsequent action*. We isolate and study the third question. We intervene on otherwise identical reasoning trajectories by inserting, at a fixed point after the same prompt and reasoning prefix, a single first-person sentence expressing either confidence or doubt. The model then continues reasoning and chooses whether to answer directly or call a tool. Comparing these counterfactual continuations measures the causal effect of the reflective signal on delegation while holding the preceding reasoning trajectory fixed. We define this change in behavioral response as Nudgeability. It is a property of large language models, measured along two dimensions: *sensitivity*, how strongly confidence and doubt change delegation rates, and *targeting*, whether delegation increases for problems the model cannot solve unaided and decreases for those it can. Across nine small-to-medium open-weight reasoning models from three families (Qwen, Gemma, and GLM) and two tasks, models are consistently sensitive to the intervention: doubt increases delegation and confidence decreases it, with a median confidence-to-doubt swing of 20.6 percentage points. The larger provider-served models exhibit swings of 53 to 70 percentage points. This responsiveness, however, is poorly targeted. Although a median 42% of induced behavioral flips are well-targeted, this represents only a +2 percentage-point median lift over a random-selection baseline. Thus confidence language provides a strong control surface for delegation, but current models seem to use such signals only weakly in accordance with their actual unaided competence. Nudgeability gives us a simple post-training-free way to evaluate both, sensitivity and targeting, as endogenous self-reflection mechanisms mature.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, reasoning models, tool use, tool delegation, confidence signals, uncertainty, self-reflection, metacognition, selective delegation, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_roVyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Nudgeability, a framework for measuring how confidence and doubt signals affect tool delegation in reasoning models. The main finding is that such signals strongly change delegation behavior, but the induced changes are only weakly aligned with whether the model actually needs help.

Critical strengths
  1. The focused question is unusual.
  2. The motivation is clear.
  3. It provides empirical evidence.
Critical weaknesses
  1. The practical significance is not sufficiently demonstrated. The paper motivates Nudgeability through realistic tool-routing scenarios, but the experiment only studies an externally injected confidence sentence followed by a SELF/TOOL decision. The tool is never actually executed, so the study does not show whether delegation improves final accuracy, utility, latency, or cost. The confidence signal is also artificial rather than generated by an actual uncertainty estimator or self-reflection mechanism.
  2. The paper provides limited mechanistic insight. The experiments establish that confidence language changes delegation behavior, but do not explain why this happens, why sensitivity and targeting are weakly related, or why different models show substantially different behavior. The contribution is therefore primarily behavioral rather than mechanistic.
  3. Several main findings are intuitive or largely descriptive. It is unsurprising that explicitly expressing doubt increases tool use while expressing confidence decreases it. The more interesting observation is that high sensitivity does not imply competence-aware targeting, but the paper mainly documents this phenomenon without providing a deeper explanation for why it arises.
  4. Generalization to realistic agent settings remains unclear. The experiments cover only two QA tasks with search- or retrieval-style tools and a specific natural-language intervention at a particular reasoning boundary. It is unclear whether the findings extend to coding, planning, multi-step tool use, long-horizon agents, or realistic environments involving heterogeneous tool costs and failures.
  5. Evidence from frontier systems is still limited. Although the paper includes two larger provider-served models, most of the evaluation focuses on smaller open-weight systems. Broader evaluation on contemporary frontier models would be needed to establish that the sensitivity–targeting pattern is general.
Questions for authors
  1. What pratical value will the paper provide?
  2. Do the authors have any machanistic insight beyond empirical observation?
  3. Do the findings generalize beyond QA and search settings?
  4. Can the authors evaluate a broader set of frontier reasoning models?
3 comments

Thank you for the review and for the suggestions on practical evaluation, generalization and frontier coverage. We respond to each concern.

Practical significance: the tool is never executed; no accuracy, latency or cost evidence.

Not executing the tool is deliberate. It isolates the delegation decision from the quality of any particular tool. The paper also includes a supporting condition in which the scoped tool is replaced by a free oracle that the prompt describes as always correct (Section 3, Appendix I.8). If the tool is correct when called, the decision maps directly onto outcomes. A well-targeted switch to TOOL on a failing problem converts an error into a correct answer. A well-targeted switch to SELF on a solvable problem saves a call at no accuracy cost.

The new oracle analysis turns this into concrete guidance for practitioners (General Response §1–3):

  • Ceiling. A perfect estimator delivered through the reasoning trace would raise the median lift to +40.0 pp (open-weight) and +47.4 pp (provider).
  • Required estimator quality. An estimator with AUROC 0.8–0.9 yields a median lift of +18 to +26 pp for the open-weight models (Figure 3a).
  • Bottleneck. Even a perfect estimator corrects a median 21.5% of baseline-wrong decisions for open-weight models, 72.7% for provider models, and only 4–7% for GLM-Z1.

This tells a system designer which models' trace channels are worth connecting an estimator to, and how good that estimator must be.

The signal is artificial, not from a real estimator.

This is also intentional. The exogenous signal is what lets us measure the model's response separately from estimator quality. The new estimator-quality analysis then connects the measurement to real estimators: lift is computed exactly from an estimator's hit and false-alarm rates, so any estimator with a known operating point can be evaluated. Nor is the framework tied to natural-language signals. A hidden-state correctness predictor such as Gnosis (Ghasemabadi & Niu, 2025), or a learned confidence token, can be evaluated on exactly the same footing (General Response §4). As a first check, three simple label-free signals read from the trace (AUROC about 0.6) realise a median lift of +4.9 to +7.7 pp against +2.0 pp, matching the lift the analysis predicts from their hit and false-alarm rates (Appendix G).

Limited mechanistic insight; findings are intuitive or descriptive.

Our contribution is a measurement framework, and we agree that explaining why models respond as they do is the important next step. We thank the reviewer for pushing on it. The new analysis does add one structural insight that the descriptive results did not show: the model's own selectivity, not the channel, is what fails. The same paired continuations become well targeted when the signal is, and the remaining loss is attributable to sensitivity. Explaining that sensitivity is listed as future work: through internal representations, matched checkpoints before and after RL, graded signal intensity, and independently annotated analysis of how the continuations refer to the inserted sentence.

[continued]

(Continued from the previous comment.)

Generalization beyond QA and search.

We agree; our tasks are multi-hop QA (MuSiQue) and StrategyQA with a search-style tool, and the revision states this scope explicitly. The measurement itself is task-agnostic, since it needs only a binary delegation decision and a competence label. We thank the reviewer and list code-execution and calculator tools as next settings.

More frontier models.

We thank the reviewer for this suggestion, and we would have liked to include more frontier systems. The constraint is the intervention itself, not cost. Nudgeability requires taking the model's own reasoning prefix, splicing a sentence into it at a fixed point, and having the model continue from the edited trace. The closed frontier APIs do not permit this. As noted in the introduction (Section 1), OpenAI, Google (Gemini) and Anthropic (Claude) do not return the full raw reasoning trace, returning summaries or encrypted content instead. Any reasoning that is passed back is encrypted or cryptographically signed and validated, so an edited trace cannot be submitted for continuation. Open-weight frontier models served by third-party providers are the only remaining route, and even there, access is uneven. Before running the provider cohort, we probed each candidate model and provider for byte-exact delivery of the edited prefix, continuation without restarting the reasoning, and deterministic replay. Many providers do not offer raw completions, or they silently prepend tokens to the supplied prefix. GLM-5.2 restarted its reasoning instead of continuing the edited trace, and Kimi-K3 passed the structural checks but not determinism. DeepSeek-V4-Pro (CoreWeave) and MiniMax-M3 (Together) were the large models whose providers passed all checks. That is why the provider cohort is small.

The two provider models nonetheless show that the pattern holds at a larger scale: high sensitivity (53–70 pp swings), near-random targeting, and the largest oracle gains (+47.4 pp median lift with a perfect estimator). The revision states the access constraint explicitly in the limitations (Section 5) and documents the route checks in Appendix I.2. Should closed providers expose editable reasoning in the future, the same measurement applies without change.

Thank you for the detailed response and additional analyses. I appreciate the new estimator-quality experiments and the clarification of the framework's scope. However, my main concerns remain. The reported improvements are largely based on simulated delegation decisions, without demonstrating actual downstream benefits. The findings also remain primarily descriptive, with limited mechanistic insight and unclear generalization beyond QA settings. Therefore, I will maintain my original rating.

Reviewer_om7qOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper examines how model self-reflection (specifically in its reasoning trace) impacts its decision to use tools when solving a problem. Specifically, it focuses on signals of confidence; the authors hypothesize that models may choose to delegate to tools more often when they are less confident, and that natural language insertions like “I do not feel confident in my answer” might increase tool use (even when the model can solve the problem on its own). They find a pattern of results largely in accordance with their hypothesis: inserting confidence signals into the reasoning trace before the termination of thinking can greatly sway models towards or away from using tools, largely independently of the models’ actual capacity for solving the problem unaided. Different models display different steering profiles; not all models are equally controllable through their reasoning traces. Overall, this work serves as an interesting starting point for investigating the role of self-reflective language in reasoning traces on model behavior.

Critical strengths
  1. The main result is interesting: the fact that an inserted natural language confidence signal can sway models’ choices to delegate to a tool regardless of the models’ actual ability to solve the problem unaided indicates an interesting dissociation between content received through the reasoning trace and the models’ own self assessment or internal trajectories. I think this is a strong basis for future work, especially on how first-person language impacts model behavior.
  2. The authors test a broad range of models, and the methods seem sound. Different models display different nudgeability profiles, and the authors provide a helpful taxonomy / description of those profiles.
Critical weaknesses
  1. The framing is a bit confusing. The introduction begins with the problem of self-reflection as it relates to tool use (i.e., how does a model assess whether or not it needs to call a tool?). It then goes into some detail about three kinds of questions related to self-reflection signals (which seems unnecessary, since only the third question is relevant). This separation of questions related to self-reflection in the introduction is later claimed as a contribution (contribution #1), which seems like a bit of an overclaim (since it’s more like framing than a contribution in its own right; the abstract itself says the paper only focuses on the third type of question). The introduction then seems to shift into answering a different, broader question not necessarily related to self-reflection, confidence, or tool use (i.e., can we use natural language interventions on reasoning traces as a control surface for models?). As a result, it is unclear which of these is meant to be the central question of the work, and the framing as a whole is a bit disorienting.

  2. Relatedly, the actual contributions and scope of the work are quite narrow, and under either framing, the current results do not seem sufficient. If the broader question about natural language interventions is the central question of the work, I would expect a much broader scope than just testing confidence signals on tool use (i.e., more kinds of signals and more kinds of behavioral outputs). If the central question is about self-reflection and confidence as it relates to tool use, I would expect more depth than is provided by the current work: for example, do models naturally use confidence language in their reasoning, and is this natural use correlated with tool use? Does the position of the inserted nudge in context matter? How does processing confidence language change models’ internal representations? Can other sorts of natural language insertions (i.e., not explicit confidence language) adjust the models’ confidence and/or resulting tool use? Why do different models display different nudgeability profiles? Overall, while the main result is interesting and the direction is valuable, it does not seem like enough on its own to constitute a full main track paper.

Questions for authors

Clarification requests:

  1. Can the authors clarify the intended scope of the work? i.e. is it more targeted towards understanding self-reflection mechanisms as they relate to tool use, or is it more about using inserted natural language signals to control models?
  2. How does nudgeability differ from generic reasoning trace interventions?

Suggestions for future work:

  1. Did the authors consider testing other kinds of natural language signals beyond confidence?
  2. Did the authors perform any reasoning trace analysis on baseline models to discover if they use confidence language spontaneously?
1 comment

Thank you for a detailed and constructive review, and for the many suggestions for future work. The framing criticism is well taken, and we have revised the paper accordingly.

Framing and scope.

We agree that presenting the "three questions" decomposition as contribution #1 was objectionable, and we have removed it. To answer the scope question directly: the paper is about whether a self-reflective signal steers tool delegation in reasoning models, and whether it steers it on the right problems. Natural-language insertion is the setting in which we instantiate this measurement; it is not the subject of the paper. Sensitivity and targeting are defined independently of the signal's representation. The same measurement applies to:

  • learned confidence tokens (Chuang et al., 2025);
  • correctness predictors read from hidden states, such as Gnosis (Ghasemabadi & Niu, 2025);
  • semantic-entropy probes (Kossen et al., 2024);
  • activation-level feedback.

We chose plain English because it is the one interface shared by all models we evaluate and requires no fine-tuning or activation access. The revision states this as a property of the framework (General Response §4).

How this differs from generic reasoning-trace interventions

(e.g. Thinking Intervention, prefill steering, When2Tool)

  1. We hold a fixed reasoning prefix and splice paired arms at the same think-close boundary, so every comparison is within-problem and within-prefix.
  2. The outcome is a single discrete agentic decision (answer vs. delegate).
  3. We score where the induced changes land against each problem's unaided competence, relative to a matched random reference.

Prior intervention work reports whether behaviour changes, not whether the change is targeted. The new oracle analysis builds on this: it shows how much a perfect or realistic estimator would gain through the same channel (General Response §1–2).

Answers to the specific questions:

  • Why do model profiles differ?

The clearest difference is in sensitivity, and the oracle analysis shows it is the dimension that matters most. With a perfect estimator, the share of baseline-wrong decisions corrected is:

  • 72.7% (median) for provider models;
  • 21.5% (median) for open-weight models;
  • 4–7% for GLM-Z1.

Size within our range (4B–32B) does not track these differences. Family and training recipe are confounded in our model set, so we do not claim a cause. We thank the reviewer for the question and list a controlled comparison (matched checkpoints, before and after RL) as future work. We have explored fitting probabilistic models which model the obstinacy of the models as a parameter and are actively exploring this avenue.

  • Internal representations.

We agree this is the natural next step, and the framework is built to accommodate it. A hidden-state correctness predictor (e.g. Gnosis, or the When2Tool probe) can serve as the signal source. Its output can then be fed back as a sentence, a token or an activation edit, and the result scored with the same S, WT and L. The revision describes this experiment in the future-work section.

  • Position of the nudge and other insertions.

We chose the think-close boundary after a fixed prefix so that the reasoning preceding the intervention is identical across arms. We thank the reviewer for the suggestion. Varying the position and the intensity of the sentence is listed as future work. The closest existing variant is the sealed-continuation control (Appendix I.7), which changes what follows the sentence, not where it sits.

  • Other signal types.

Beyond doubt and confidence, the paper includes a neutral sentence and a cut-only control, an expert-attributed version of the sentence (Appendix I.10, Table 10) and a sealed-continuation condition (Appendix I.7). These separate the effect of content from the effect of interrupting the trace. As noted above, confidence tokens and hidden-state signals fit the same measurement directly.

  • Natural / spontaneous confidence language.

This can be studied from our stored, unmodified traces without new generations, and we thank the reviewer for the suggestion. We ran a first analysis (Appendix G). We counted generic hedging phrases (a fixed list of 25, such as “not sure”, “maybe” and “perhaps”) in each model's own reasoning prefix. On MuSiQue, the count is strongly associated with the model's own tool call (median AUROC 0.83) and only weakly with the failure of its unaided answer (0.65). On StrategyQA, both are near chance (0.51 and 0.55). The models' spontaneous uncertainty language thus tracks their delegation more than their competence. This is a first look with an unvalidated phrase list. A fuller analysis, and an analysis of how the continuations refer to the inserted sentence, are listed as future work (Section 5).

Thank you again; the review has substantially improved the paper's presentation.

Reviewer_XK6NOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper operationalizes ‘nudgeability’ as a prompt-based intervention for influencing when an agent delegates actions to an external tool vs. when it solves the problem itself unaided and links this to the model’s internal confidence in its own ability. Concretely, the authors splice a single first-person sentence expressing confidence or doubt (“I feel confident in my answer” / “I do not feel confident in my answer”) into a model’s reasoning trace just before it commits to an action, and measure how much the choice to call a tool changes. They find that prompt–based interventions inspiring or reducing confidence can induce a model to either be more or less delegating in its behavior.

Critical strengths

The paper does seem to honestly try and engage with a wide range of model families, sizes, and sample sizes.

Critical weaknesses

The finding that a simple prompt injection changes behavior is not necessarily novel - the correlation between confidence and delegation is likely well-established at this point and is observable even to naive users through simple coding IDE’s with agents. Ie. the claim that “The consistency and magnitude of the Nudgeability scores across model families, datasets, and sizes establish confidence language inside reasoning as a potential control surface for the model’s subsequent action” is not surprising? Moreover, the results section doesn’t handle representation of uncertainty in a rigorous enough way for me to say that anything about uncertainty quantification is handled here either - this stays at the level of saying that prompting with confidence-leaning tokens inspires less delegation, which is fundamentally unsurprising.

My main concern is with how the targeting result is interpreted. The inserted sentence is exogenous and carries no information about whether the model will actually get the problem right. “Instead, the induced action changes are only weakly concentrated on problems where the model’s unaided behavior indicates that help is needed. A model can therefore act on the meaning of “I do not feel confident”, delegating more, with little regard for whether it would actually fail the current problem.” This is highlighted as a central contribution/finding, and while I understand in theory why separation can be useful, in practice this seems to make the whole method not trustable and usable.

Questions for authors

None

1 comment

Thank you for the review. We address the three points in turn.

"The inserted sentence is exogenous, so the method is not trustable or usable."

We would like to clarify the design. The paper does not claim the sentence reflects the model's internal confidence. It is exogenous on purpose, so that we can measure how the model responds to a signal separately from how good any particular estimator is. In deployment, the sentence (or a confidence token, or a hidden-state readout) would come from a real estimator.

The weak targeting we report (median lift +2 pp) therefore describes what the model itself does with a signal that ignores competence. It does not say that a signal delivered this way is useless. Our new oracle analysis (General Response §1–2) makes this concrete. If the same sentences were supplied by a perfect estimator, the median lift would be +40.0 pp for open-weight models and +47.4 pp for provider models, with CIs excluding 0 in 21/22 experiments. An estimator with AUROC 0.8–0.9 would give a median lift of +18 to +26 pp for the open-weight models (+15 to +24 pp for the provider-served ones). What limits the benefit is how often the model responds at all, which our sensitivity measure quantifies. As a first end-to-end check, three simple label-free signals read from the trace (AUROC about 0.6) realise a median lift of +4.9 to +7.7 pp, as the analysis predicts for estimators of that quality (Appendix G).

Seen this way, the method is directly usable. Given a model and an estimator's hit and false-alarm rates, it predicts the delegation lift before deployment.

"The finding is not novel; it is unsurprising that injected text changes behaviour."

We agree that the direction of the effect is expected, and the revised introduction says so. Two things are not obvious a priori:

  • its size: a median swing of 20.6 pp for open-weight models and 53–70 pp for provider models from a single sentence;
  • the fact that the induced flips are barely better targeted than random selection, while a perfect estimator would make the same channel well targeted.

The second point separates the model's own selectivity from the potential of the channel. As far as we know, no prior work measures this.

"Uncertainty quantification is not handled rigorously."

We agree that we do not propose an uncertainty estimator, and beyond the three simple signals of Appendix G we do not evaluate one. The revision states this explicitly (Section 5). Our object is the downstream response to a supplied signal, together with the new analysis of how estimator quality translates into delegation lift. For the uncertainty of our own estimates, we report problem-level bootstrap CIs for every quantity.

Thank you also for the suggestions. We hope the oracle analysis addresses the soundness concern.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers: revised paper and new signal-source analysis (1/2)

We thank all three reviewers for their careful reading and for the many concrete suggestions for future work: varying the insertion position and other kinds of insertion, examining internal representations, studying spontaneous confidence language, adding tasks beyond QA, and evaluating more frontier models. We have incorporated these into an expanded future-work section and address each one in the individual responses.

Two concerns recur across the reviews:

(i) the inserted signal is exogenous, so it is unclear what the measurement implies for a real uncertainty estimator (XK6N, roVy); and (ii) the framing, in particular the "three questions" contribution and the role of natural-language injection, is confusing (om7q).

We address both with a new analysis on the existing data (no new generations) and a revised framing.

1. The oracle setting: what if the signal came from a perfect estimator? _(new: Section 2 "Quality of signal sources", Section 4, Appendix F)

Our design deliberately assigns doubt and confidence regardless of competence. This lets us measure how the model responds to the signal, separately from the quality of any estimator. Because the arms are paired on the same reasoning prefix, the stored continuations also let us evaluate the opposite extreme: a perfect estimator that shows doubt exactly when the unaided answer fails and confidence exactly when it succeeds. The random-selection reference is unchanged.

Competence-blind signalPerfect estimator (new analysis)
Open-weight (18 experiments): median lift+2.0 pp+40.0 pp (CI excludes 0 in 17/18)
Open-weight: median WT42.0%91.6%
Provider-served (4): median lift+0.6 pp+47.4 pp (CI excludes 0 in 4/4)
Provider-served: median WT29.6%98.2%

2. How good must a real estimator be? (new: Section 4, Figure 3a)

Real estimators sit between these two extremes. For an estimator that shows doubt on a fraction α\alpha of failing problems (its hit rate) and a fraction β\beta of solvable ones (its false-alarm rate), lift can be computed exactly from the paired data. The published setting is the zero-information point α=β=12\alpha = \beta = \tfrac{1}{2}, and the perfect estimator is α=1\alpha = 1, β=0\beta = 0. For an estimator with AUROC 0.8 (equal-error operating point), the median lift is +17.9 pp (open-weight) and +15.3 pp (provider-served); at AUROC 0.9 it is +26.1 and +23.7 pp (Figure 3a). This assumes the estimator's errors are independent of the model's response. A practitioner can therefore read off, for a given model, how much delegation targeting a candidate estimator would buy.

We also ran a first check with real signals (new: Appendix G). Three simple, label-free signals choose the sentence for each problem, and the outcome is read from the stored arm: the length of the model's own reasoning prefix, a count of hedging phrases in it, and an external judge's rating of the confidence expressed in the trace. These are weak estimators (median AUROC 0.57–0.61 for the open-weight models). They raise the median lift from +2.0 pp to between +4.9 and +7.7 pp, and from +0.6 pp to +7.6 and +9.3 pp for the provider-served models. The lift predicted from each signal's hit and false-alarm rates agrees with the realised lift (correlation 0.87–0.91 across experiments), which supports the independence assumption. Most of the +40 pp remains for better confidence sources to deliver.

[continued...]

AuthorsOpenReview ↗

General response to all reviewers: revised paper and new signal-source analysis (2/2)

(Continued from the previous comment.)

3. What limits the benefit is sensitivity. Even the perfect estimator corrects only a median 21.5% of baseline-wrong decisions for open-weight models. Here, baseline-wrong decisions are failures answered unaided plus successes delegated. The figure is 72.7% for provider models, and only 4–7% for GLM-Z1. This is because most problems do not flip under either sentence, and the share corrected tracks sensitivity closely (Figure 3b). Separating sensitivity from targeting is what reveals this. We treat the oracle numbers as an upper bound, since the oracle uses the same single-run competence label that scores WT.

4. Natural-language injection is the setting we chose, not a limitation of the framework (Sections 1 and 5). Nudgeability is defined by two behavioural quantities: sensitivity (S) and targeting (WT and lift L against a matched random reference). Both are measured on paired, fixed-prefix continuations. Neither depends on how the signal is represented. We chose a plain English sentence as the instantiation because it is the one interface shared by every model we evaluate. It needs no learned confidence token, activation access or fine-tuning, and it can be inserted at a controlled point in an existing trajectory (Section 1). The same measurement applies unchanged to:

  • learned confidence tokens (e.g. Chuang et al., 2025);
  • correctness predictors read from hidden states, such as Gnosis (Ghasemabadi & Niu, 2025), whose output can be supplied through any of these interfaces;
  • semantic-entropy probes (Kossen et al., 2024);
  • continuous or activation-level feedback.

For any of these, the same pair of questions applies: does the representation move delegation, and does it move it on the right problems? The revision makes this explicit and moves it from a hedge to a stated property of the framework.

5. Framing. We have removed the "three questions" decomposition as a stated contribution and kept it as motivation only. The contributions are now (a) the Nudgeability framework: a paired, fixed-prefix measurement of sensitivity and targeting against a matched random reference, applicable to any signal representation; (b) its application to natural-language confidence across 22 experiments; and (c) the oracle and estimator-quality analysis above.

During the revisions, we also discovered a closely related work When2Tool (Sun et al., 2026), which steers the same decision with a probe-selected sentence but evaluates only the combined system (Appendix A). All new and changed text in the revised PDF is set in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.