acceptodds
Under review as a conference paper at ICLR 2027

When Confidence Is Not Enough: Action-Contrastive Self-Models under Capability Drift

Abstract

A calibrated agent may estimate its probability of success accurately and still choose the wrong recovery. Retrieval, verification, replanning, escalation, and abstention repair different failures, so scalar confidence removes the direction needed for action choice. We formulate self-knowledge as an action-contrast surface: the change in terminal value induced by each available recovery branch. SCOPE factors this surface through learned operational response modes, aligns their action–diagnostic signatures after a model, tool, access, or context change, and routes by predicted net value. Under conditional response homogeneity, is sufficient for one-step recovery choice. In a multi-regime linearization, sharing modes across regimes changes the stylized parameter count from to , while signature separation gives a condition for transporting persistent modes across versions. We evaluate these claims with real local-model and tool executions: every legal branch is run from the same reset pre-action state, terminal outcomes are scored by task-native evaluators, and only one logged branch outcome is exposed to the learner. Across 6 open-weight checkpoints and six transition families, action conditioning raises mean normalized utility from to , generic rank-matched sharing reaches , and SCOPE reaches . SCOPE further reduces held-out intervention-gain error by – relative to the rank-matched control. Direct-AV catches up with abundant feedback and achieves higher utility when new response mechanisms or noisy diagnostics break the shared geometry.

Primary area:
reinforcement learning
Keywords:
Language Agents, Self-Knowledge, Action-Contrast Surface, Failure-Aware Decision Making, Adaptive Routing, Value of Information

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_Y9NBOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a framework for choosing the right type of action to proceed when an agent faces an uncertain decision. It argues that an uncertainty score is insufficient to make the correct decision in practical scenarios, and follow-up actions among escalation, retry, or search should be properly distinguished. Toward this objective, the paper develops a mathematical basis for calculating the probability / value of each type of action under a scenario, and implements SCOPE to enable fine-grained control over these decisions.

Critical strengths
  • The paper provides rigorous mathematical reasoning to ground a quantitatively the scoring of each action type, which resolves possible ambiguity of how success / importance of each action type might be defined.

  • The paper tackles a practical problem: the agent must decide the right next action type before spending irrelevant compute on it, and it presents a framework with concrete implementations.

Critical weaknesses
  • The paper's introduction is hard to read without sufficient context. The context is not hard to grasp -- it is just a decision problem when the agent faces a blocker in the trajectory. However, the paper makes the introduction so dry that even this point is buried in technical-sounding text.
  • The method section is not well motivated. The equations are not connected properly to the introduction, nor to the proposal of the framework SCOPE, since not most of the equations even appear again in SCOPE.
  • The utility of the framework is limited, since it is not compared with a prompting baseline which just asks the model to be aware of the alternative choices and think more carefully.
  • No code is supplemented to verify the concrete implementation or the reproducibility.
Questions for authors
  • Can you give the actual code or pseudocode that implements every component of SCOPE?
  • Can you give at least two case studies that include actual trajectories from the agent?
  • Can you report the baseline performance with just careful prompting the agent to be thoughtful?
Reviewer_dHaWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The problem that this paper attempts to address is that eliciting an agent's confidence in success does not tell it which actions lead to recovery, especially as capabilities change. The proposed solution is to learn latent response modes determining how much different types of recovery actions help, then uses this to route towards the best actions. Minor improvements on constructed baselines are shown.

Critical strengths

None

Critical weaknesses

This paper appears to be almost entirely written by AI, as flagged by state of the art detection tools like Pangram. The level of AI detected is not consistent with the AI Use Statement claiming it was only used for edits, and suggests that entire sections were written by AI.

Concretely, the issue with using AI for writing is that the explanations of motivation, impact, and results in the paper are consistently unclear. Additionally, there are basic issues like using variables in the abstract without defining them, numeric discrepancies throughout, and major differences between the setup described in the main body and in the appendix.

In the main body, the theory is elementary, for example Lemma 1 is a subtraction. In the experiments, most of the claimed improvement comes from applying existing methods that are skipped in the baselines created.

Questions for authors

None

Reviewer_c9ffOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents SCOPE a method to quickly adapt agents when their underlying tools or models might change which might influence how specific actions to recover might succeed. To do this adaptation, it learns a soft-belief encoder over failure modes along with regime-conditioned response matrices that store how well each recovery action works for each mode. At the crux, when a model which has failure modes and their recovery actions identified and learned has its tools/checkpoints change, SCOPE fist matches the old failure modes to the new ones, marking others as unknown. It then learns how often each mode occurs and how well recovery actions work using the outcomes of the executed trajectories. This is used to revise the belief-encoder and the regime-conditioned response matrices allowing agents to adapt to tool-change etc faster

The authors conduct an empirical evaluation with on 6 open-weight models and six task types and confidence and action based baselines. Most of the gain appears to come from confidence-based routing (scalar)

Critical strengths

The problem tackled by the paper where model checkpoints and tool definitions can change are of great interesting to the AI community. The paper offers an approach to adapt to such circumstances without relearning from scratch.

Critical weaknesses

I found the paper writing very hard to follow. There is a lot of technical jargon in short sentences. The key claim of capability change is not represented at all. Fig. 1 and Fig 2 show two different tasks, not capabilities changing. I am not sure how that relates since the recovery action is different and has not drifted.I could only find a tiny section in 6.4 related to verifier-swap etc. but it is buried deep in the paper and the runnign examples are not exercising or showcasing the right pitch.

The empirical results are not strong. Most of the gain appears come from simply predicting each actions value. Direct-AV seems to add 0.037 here while SCOPE only adds about .006 over the low-rank baseline. Also the gains are not clear. They seem small relative to the noise in the reported std. deviation.

Questions for authors

Please check my weaknesses. Happy to revise and reconsider if i have misunderstood the work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.