acceptodds
Under review as a conference paper at ICLR 2027

Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents

Abstract

Reliable confidence estimation is increasingly central to the trustworthy deployment of language models: a calibrated estimate of the probability that an output is correct decides what to ship, what to escalate, and what to retry. Yet existing methods estimate confidence in a new answer from the current inference alone, leaving the model's record of past successes and failures unused. We propose XConf (eXperiential Confidence): estimating confidence from the model's accumulated experience. XConf maintains a memory of graded past episodes, each recording the task, the stated confidence, the outcome, and a lesson drawn from the feedback. Given a new task, XConf first Recalls similar episodes to estimate the model's historical success rate and then Reflects on the retrieved outcomes and lessons to identify relevant failure patterns and revise its confidence in the current answer. Our estimator is format-general, requiring no logit access or weight updates, and costs only one answer generation. Across nine benchmarks spanning reasoning, coding, multimodal QA, and interactive agents, and four models from three families, XConf outperforms or matches ten-sample self-consistency in discrimination (AUROC) on 23 of 24 comparisons, with much lower calibration error (ECE), at a tenth of the generation cost. Used for selective prediction, abstaining on the 10% least-confident episodes raises the delivered success rate by up to 8.7 points on agent tasks. We therefore see experiential confidence estimation as a new paradigm for future general-purpose confidence estimation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
confidence estimation, agents, reasoning, memory

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_rT1TOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a new method (XCONF) that leverages past experience to correct the confidence of the output. The method mainly contains two parts (RECALL and REFLECT). RECALL selects up to k past similar episodes from a task episode bank and measures the success rate. REFLECT takes the selected episodes as context and uses an LLM prompt to identify relevant failures and revise the confidence. The paper gives comprehensive experimental evaluation on 3 family models and 9 benchmarks.

Critical strengths
  • The problem formulation and the defination of the episode are interesting.
  • The paper shows diverse tasks and models in the experimental section.
  • The ablation results are well-designed and strong; I like the part that tests the retrieval quality of RECALL.
Critical weaknesses
  • The system design is simple, and it heavily depends on the bank of episodes, and each episode requires a label of success. In practice, the bank seems to be difficult to build from scratch or extend to a large scale that covers long-tail tasks.

  • The design choice of combining the success rate in the retrieved episode and the reflection is not well justified; especially, the output of REFLECT is not well justified. Why is this reflection a reasonable design? For example, the recall is a sample of k past "similar" episodes. If the candidate pool is large and sucess/failr half half, if it happens to retrieve "all success" or "all failure", it will change the confidence(e) dramatically.

  • I noticed that the bank size is not reported for Table 2 and Table 3, and I am confused by the score reported in Figure 3; it seems that the max number reported in Figure 3 does not match Table 2. For example, the AppWorld result in Figure 3 is never larger than 0.9, but in Table 3, the result contains 0.91. Could you clarify the setting?

Questions for authors
  • It seems that the bank depends on the model, since the episode contains the model's self-reflection and confidence states. And labeling the output is not easy. So this makes me think a similar setting: it seems that the SC@10 result is very close to XCONF, but XCONF needs to build a large bank. It seems that only label 10 SC outputs and using it to replace the RECALL is a reasonable baseline and good for cold start. Have you tried this?
  • I notice that Figure 3 does not show a monotonic pattern with bank size increases. This links to my concerns that the larger number of past episodes will affect the RECALL and therefore change the results. Do you have an expectation of what would happen with a large number for each group? For example, in the setting that is given any input e, the number of similar candidates is not smaller than m.
4 comments

Thank you for the review, and for the positive words on the problem formulation and the retrieval-quality ablation. Since the ICLR committee encourages brevity, we keep this reply short; please let us know if anything remains unclear.

W1: regarding the dependence on labels and building the bank

Thank you for this great question, which is also mentioned by other reviewers. Here we explain in detail:

  • Every calibrated estimator consumes outcome labels. Platt, isotonic, conformal, and trained verifiers all fit on graded outcomes, and Section 4 gives each of them exactly the outcomes our bank contains, so the comparison is matched on supervision. The label-free estimators in our comparison, verbalized confidence, self-consistency, semantic entropy, and P(True), are the ones we match or beat on both metrics.
  • The labels need not be gold. An independent LLM judge that never sees the answer key, with only 91% agreement with gold, costs almost nothing in calibration quality; what fails is the model labeling itself (as we mentioned in Section C). The requirement is independence from the model's own judgment, not perfect accuracy.
  • A system has graded episodes before it ships. In real deployment, any system that was evaluated before deployment already has some graded episodes: development and test sets, staging runs, and acceptance checks (Section 3.2). The bank can start from those, with no separate labeling effort, and grows with whatever graded feedback deployment later provides (unit tests and CI for code, task assertions for agents).
  • Cold start is cheap. A bank of 100–250 episodes already works, and about twenty minority-class episodes are enough for the key to learn what failure looks like (Section I). On MMLU-Pro and BBEH, Gemini 3.5 Flash and Claude pass ten-sample self-consistency within the first hundred episodes (Figure 11).
  • Long tail. This is indeed an important case, and what happens is a graceful fallback to a conventional calibrator. The retrieval key weights the stated confidence most heavily (Section E), so where the bank is sparse around a task the fifty neighbours are only weakly similar in content but still share the current stated confidence, and the hit rate becomes the model's historical success rate at that confidence level (Section 3.3). That is exactly what a post-hoc calibrator of verbalized confidence computes (the Platt and histogram baselines of Section A), so in the tail XConf degrades to those baselines rather than to a wild estimate, and recovers its margin as episodes in that region accumulate (Figure 3).

W2: regarding the Recall / Reflect combination and "all-success" retrieval

Here we explain why reflect is needed and clarify a misunderstanding of recall retrieval:

  • Why Reflect. Two reasons:
    1. The record contains more than a hit rate. Each retrieved episode carries a lesson, and the model can recognise whether the current attempt shows the same failure pattern. Three results show this is real: showing the same cards instead of their average gains .04–.06 AUROC on AppWorld and SWE-bench (Figure 8); the revision is in the right direction, pulling wrong episodes down by −.18 and correct ones by only −.06 (Figure 9); and the same second call without the record does not help (Table 19).
    2. It names the failure pattern. Reflect states the recurring failure mode behind the estimate in words, which gives the confidence an explanation that Recall, a count, cannot provide.
  • Recall does not sample from the pool. It reads the fifty nearest neighbours, so the estimate is the model's success rate on the local neighbourhood of this task at this stated confidence, not a draw from a global 50/50 pool. A neighbourhood that is all successes is informative precisely because the global rate is 50/50; it is still a fifty-episode estimate, which is why its robustness is checked empirically in Section I rather than assumed.
  • On the combination, we agree with you and Reviewers ftAN and Rz5j that the equal-weight blend is a default rather than a claim; Recall alone, Reflect alone, and the blend are three operating points, and we will present them as such (please see our reply to Reviewer ftAN).

W3: regarding bank size and the Figure 3 vs Table 3 mismatch

Bank sizes are in Table 25, and the Figure 3 mismatch is a caption error on our side. Under five-fold rotation the bank behind each estimate is 80% of the per-column count in Table 25; we will add this to the captions of Tables 2 and 3. Figure 3 is a Recall-only replay with no Reflect calls, which is why its endpoints match Table 20's Recall-alone row (AppWorld: .810 for Claude, .888 for Gemini 3.5 Flash) rather than Table 3's blend (.911). The caption should have said so, and we will fix it.

Q1: regarding labeling ten SC outputs as a cold-start baseline

Thank you for the suggestion; if we understand it correctly, it needs a label the estimator cannot have. You propose sampling ten SC outputs for the current task, labeling them, and using their success rate in place of Recall. Labeling them against ground truth requires the current task's answer, after which there is nothing left to estimate. Labeling them by majority vote makes the estimate SC@10 itself, which is in every table and which the self-labeling result of Section C shows cannot catch the confidently-wrong cases. And SC@10 is not close: 21 wins, 2 ties, and 1 loss against it with 3–8× lower ECE, and it does not apply to the agent domains. For cold start, Figure 11 is the relevant comparison: a bank of a hundred graded episodes already passes SC@10 where verbalized confidence is weak.

Q2: regarding non-monotonic Figure 3 and large banks

The curves need not be monotone, and a larger bank tightens the neighbourhood rather than destabilising the estimate. Each point in Figure 3 is a finite-sample estimate from fifty neighbours, so a wobble of a point or two at small banks is expected, and the trend over the full range is upward on every panel. In the regime you describe, where every input has at least m similar candidates, new episodes keep replacing neighbours with closer ones, so the neighbourhood becomes more homogeneous and the hit rate tracks the model's local success rate on that kind of task more closely; what remains is the sampling variance of a fifty-episode count, which does not grow with the bank. The evidence for this stability is empirical rather than theoretical: the k and σ ablation of Section I, and the flattening of the dense panels (MMLU-Pro at twelve thousand episodes), while BBEH and the agent domains are still climbing because they are not yet dense.

We hope this clears up your concerns; please let us know if anything remains unclear!

Thank you for the detailed answer. I find the explaination on cold-start and RECALL conflict. As you pointed out, the recall retrieves the local neighbourhood with k=50k=50. And it indicates that the size of the pool should be massive: suppose you have n tasks and each task has a neighbourhood much larger than 50 (in order to get high-quality 50 retrieval), the pool should be >> n*50. Consider the 100-200 episodes cold start setting: for most of task XCONF, retrieve 0 or few episode match the task (if task number > 200). So the RECALL part doesn't work. Another concern is the question I pointed out in Figure 3; I understand that monotonicity is not necessary, but given the small size from 500 to 1000 episodes in the science world, the increased episodes do decrease the AUROC a lot. I suspect this failure is very related to the RECALL design.

Thank you for following up on both points. We went back to the data for each, and the second one turned out to be an artifact of how we built Figure 3, which we should have caught earlier.

regarding cold start and the size of the neighbourhood

Recall does not need fifty episodes of the same task; at a hundred episodes it is mostly a calibrator of the stated confidence, and the gain from retrieval appears as coverage grows.

  • What a neighbour has to share. The retrieval key weights the stated confidence most heavily (Section E), so a useful neighbour is a similar task met with similar confidence, and neighbourhoods of different tasks overlap. The n × 50 count assumes disjoint, same-task neighbourhoods.
  • What a small bank gives (Table F1). We compare Recall with a calibrator that uses the stated confidence only, fit on the same bank. With 100 random episodes, about three per task type on ScienceWorld, Recall is already slightly ahead on ScienceWorld and by .05 AUROC on AppWorld, and level on SWE-bench; the margin widens as the bank grows.
  • The case you describe, no matching episode at all. In the stored-order bank of 500 ScienceWorld episodes (see below), 89% of test queries belong to a task type with no episode in the bank. On those queries Recall scores .894 AUROC against .895 for the confidence-only calibrator. This is the fallback of Section 3.3: with nothing similar to retrieve, the estimate becomes the success rate at this stated confidence.

So you are right that retrieval contributes little at 100–200 episodes on some domains, and "already works" in our first reply overstated it. We will reword Section I to say that a small bank gives a calibrator and retrieval adds to it as episodes accumulate.

Table F1. Recall / confidence-only calibrator on the same bank, AUROC, random banks of B episodes (10 seeds), mean over the four models.

BScienceWorldAppWorldSWE-bench Verified
100.898 / .887.739 / .688.754 / .745
250.916 / .899.775 / .690.769 / .750
full pool.962 / .900.811 / .688.783 / .745

regarding the drop from 500 to 1,000 episodes on ScienceWorld

The drop comes from the order in which Figure 3 adds episodes, and our earlier explanation of it as sampling noise was incomplete.

  • What happened. Figure 3 grows the bank in stored order, and ScienceWorld is stored grouped by task type. The first 500 episodes hold only the first few of thirty task types; the next 500 add whole new types with a very different success rate (for Claude Sonnet 4.6, 12% in episodes 250–500 against 54% in 500–1,000), while the test queries span all thirty. All four models share this prefix, which is why their curves dip together.
  • With random banks the curve rises at every step (Table F2). Drawing the same number of episodes at random, over ten seeds, AUROC increases monotonically for each of the four models, with a standard deviation of at most .017 at 100 episodes and .006 from 500 on.
  • Where your suspicion is right. A bank that covers few task types is worse than a random bank of the same size, and the reason is retrieval: queries from an uncovered type receive neighbours from other types. At the 1,000-episode point of the stored-order curve these queries sit about .03 below the confidence-only calibrator, and the gap closes once their types enter the bank. Coverage of task types, more than the raw episode count, is what a bank needs.

We will redraw Figure 3 with random subsets and a seed band, state the sampling in the caption, and keep the stored-order curve in the appendix as the low-coverage case.

Table F2. Recall AUROC on ScienceWorld as the bank grows, mean over the four models: stored order (Figure 3) vs random subsets (10 seeds).

B1002505001,0002,0004,000full pool
Stored order.882.891.903.884.903.940.962
Random.898.916.930.943.952.958.962

Thank you for pressing on this figure; the paper is more accurate for it.

Reviewer_Rz5jOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes XConf, which extends large-model confidence estimation from “relying only on the current reasoning trajectory” to “leveraging the model’s historical experience bank of successes and failures on similar tasks.” The method combines two modules: RECALL retrieves similar historical samples in a feature space through supervised reweighting and computes the historical win rate at the same initial confidence level; REFLECT places retrieved historical error cases and reflection cards into the context, guiding the model to identify common failure patterns and revise its confidence. The authors conduct extensive validation on 4 large models and 9 benchmarks covering reasoning, code, multimodal, and interactive Agent tasks, demonstrating that it significantly outperforms single-pass verbalized confidence (Verbalized) and 10-sample self-consistency sampling (SC@10) in discriminative power (AUROC) and calibration error (ECE).

Critical strengths
  1. High practical value in “non-votable” scenarios such as long code and multi-step Agents: Traditional sampling-consistency methods (such as SC@10) struggle to perform equivalence voting on complex programs or Agent trajectories spanning dozens of steps, and the interaction cost of rerunning 10 times is unacceptable in engineering practice. XConf’s retrieval keys rely only on the task description, reflection text, and initial confidence, completely avoiding direct comparison of output answer strings, thereby achieving significant discriminative and selective-prediction gains at very low additional cost in silent-failure scenarios such as LiveCodeBench, AppWorld, and SWE-bench.

  2. Solid and highly convincing controlled ablation analysis: The authors not only supplement in Appendix A fair comparisons with post-hoc calibrators that consume the same labels (Platt, Isotonic, kNN-Conformal) and a trained discriminator (HTC), but also use difficulty-stratified tests (Appendix F) and temporal replay (Appendix H) to strongly demonstrate that the experience bank captures “the specific model’s own failure patterns” rather than merely a question-difficulty prior or data leakage.

Critical weaknesses
  1. The paradigm’s generality is limited by the strong assumptions of “same-distribution data” and “external independent ground truth”: XConf’s excellent performance in the main table is built on 5-fold cross-validation, i.e., the experience bank and test questions are highly same-distributed, and there is an accurate external grader (such as unit tests, exact matching, or a verifier with standard answers) to provide true labels yy and fit the supervised retrieval probe (Logistic Probe). However, Appendix C confirms that once external ground truth is unavailable and model self-evaluation labels (SC-majority) are used instead, the system fails completely on incorrect cases due to confirmation bias; meanwhile, Appendix I (Figure 12, Table 24) shows that when a completely cross-domain external experience bank (all-foreign) is used, AUROC on LiveCodeBench and BBEH drops sharply by 0.12 to 0.24. This indicates that the method is more suitable for domain-specific systems with automated feedback loops rather than as an unconditionally applicable general paradigm.

  2. The contributions of RECALL and REFLECT are severely imbalanced, and the latter degrades calibration (ECE) in most scenarios: Although the main text presents statistical retrieval (RECALL) and contextual reflection (REFLECT) as co-equal core mechanisms, Appendix D (Table 20) reveals that on all 6 reasoning/code/multimodal benchmarks and ScienceWorld, purely statistical RECALL alone achieves almost identical AUROC to full XConf (and even better on ScienceWorld), while achieving significantly lower calibration error (ECE) than full XConf on almost all tasks. Introducing REFLECT not only adds an extra LLM call cost and causes cliff-like degradation on models smaller than 27B (Appendix J), but also generally increases ECE; its positive discriminative gain is actually reflected only in two heterogeneous Agent tasks, AppWorld and SWE-bench.

Questions for authors
  1. From Table 20, RECALL alone (essentially a supervised local kernel smoother combining task features and initial confidence) requires no second LLM call, and its ECE is significantly better than the final model fused with REFLECT on almost all datasets (for example, on SuperGPQA and SWE-bench, the ECE after fusion worsens by 2 to 3 times). Since REFLECT only improves AUROC on AppWorld and SWE-bench, why not directly adopt RECALL in other scenarios, or apply post-hoc calibration to the output of REFLECT to eliminate the overconfidence bias it reintroduces?

  2. Appendix C mentions that an “independent LLM Judge that does not look at standard answers (91.1% accuracy)” can replace ground truth to build the experience bank. However, in complex tasks such as AppWorld or SWE-bench where “failures are silent and trajectories appear normal,” even the LLM Judge baseline in Table 3 itself performs poorly (AUROC only 0.63–0.83). In production environments without unit tests, how can one obtain an independent grader with up to 91% accuracy to prevent the experience bank from being contaminated by incorrect labels?

  3. Appendix I proves that borrowing experience banks across models causes obvious performance loss (Self-other asymmetry); this means that in actual deployment, once the underlying model version is upgraded or the Agent prompt/toolchain is iterated, the old experience bank becomes “data from a different model.” It is suggested that future work explore experience forgetting or dynamic weight decay mechanisms during continuous system iteration.

2 comments

Thank you for the detailed and accurate summary, and for the positive assessment of the controlled ablations. Since the ICLR committee encourages brevity, we keep this reply short; please let us know if anything remains unclear.

W1: regarding the same-distribution and external-ground-truth assumptions

Thank you for raising these two key points; we take them in turn.

On external ground truth.

  • Every calibrated estimator consumes outcome labels. We want to highlight that Platt, isotonic, conformal, and trained verifiers all fit on graded outcomes, and Section 4 gives each of them exactly the outcomes our bank contains, so the comparison is matched on supervision. The label-free estimators in our comparison, verbalized confidence, self-consistency, semantic entropy, and P(True), are the ones we match or beat on both metrics.
  • The labels need not be gold. As shown in Section C, an independent LLM judge that never sees the answer key, with only 91% agreement with gold, costs almost nothing in calibration quality; what fails is the model labeling itself. The requirement is independence from the model's own judgment, not perfect accuracy. "Fails completely" overstates that result: the self-labeled bank loses .14–.24 only where SC-majority is itself a bad labeler, and −.02 where SC agrees with gold at .86 or better.
  • A system has graded episodes before it ships. We consider in real deployment, a system that was evaluated before deployment already has some tested data with ground truth: development and test sets, staging runs, and acceptance checks (Section 3.2). The bank starts from those, with little separate labeling effort, and grows with whatever graded feedback deployment later provides (unit tests and CI for code, task assertions for agents).

On same-distribution experience. The paper reports the clean case and the honest boundary:

  • Same kind of task, different corpus. A bank built from R2E-Gym calibrates SWE-bench Verified as well as SWE-bench's own bank (.720–.734 vs .726; Section 6), so transfer within a task family costs nothing.
  • Across task families. The median cost is .08 AUROC. The .12–.24 you cite is the all-foreign case on BBEH and code, where the bank contains no native experience at all, and a few hundred native episodes close it (Section I).
  • Not i.i.d., not curated. Time-ordered replay (Section H, −.006 to −.020) and uncurated merging (Table 24, ≤ .01) show that the bank needs neither i.i.d. sampling nor curation.

That said, we will state both limitations explicitly in the limitations section, noting which are specific to our method and which are shared by every calibrated confidence estimator.

W2 / Q1: regarding Recall vs Reflect, and post-hoc calibration of Reflect

We agree, and we will present Recall, Reflect, and the blend as three operating points rather than one recipe. Reviewer ftAN raised the same point. Each setting has its own strength: Recall alone is the best-calibrated and cheapest point, with no second call; Reflect alone is the only reading that names the failure pattern behind the estimate, an interpretability Recall cannot offer; the blend buys discrimination where failure is silent (AppWorld up to +.023, SWE-bench up to +.039 over the better component) at an ECE still a fraction of the verbalized and LLM-judge baselines'. The main tables use the blend only so that one recipe is reported everywhere.

  • Why not Recall alone elsewhere? That is a legitimate choice and we will say so. The sweep below shows that Recall alone (w = 1) is the ECE optimum and the blend (w = .5) the AUROC optimum on both lines, and nothing in between dominates; Section 3.4 and Section D will present Table 20 as the guide to that choice.
  • Post-hoc calibration of Reflect? Not needed: where calibration matters most, Recall alone already provides it, and Reflect's role is ranking and naming the failure pattern. On the size cliff (Section J), what degrades below 27B is the verbal reading; Recall is size-invariant, and the blend still beats verbalized confidence on all 24 reasoning cells at every size.

Table R2 (mean over the cells of each line). w weights Recall in w·Recall + (1−w)·Reflect.

w=0 (Reflect).25.5 (shipped).751 (Recall)
Reasoning, AUROC ↑.806.824.829.828.822
Reasoning, ECE ↓.081.059.039.026.023
Agents, AUROC ↑.780.851.872.872.852
Agents, ECE ↓.157.117.091.062.034

Q2: regarding obtaining an accurate grader in production

The bank does not have to be built in production, and the two numbers you compare measure different things.

  • The bank starts before deployment, from the graded data noted under W1; the R2E-Gym → SWE-bench result above is this workflow in miniature: a bank built on a graded public corpus calibrates the deployment task of the same kind at no loss.
  • Table 3's judge AUROC (.63–.83) is how well a judge ranks rollouts; the .911 in Section C is how often its binary verdict agrees with gold. A mediocre ranker can be a sufficient labeler, and what the bank needs is the latter.
  • Where the grader is the system's own feedback (tests, CI, environment scoring, user acceptance), the only requirement is independence from the model's own judgment; Section C shows the bank tolerates label noise and Section H that it tolerates delay.

Q3: regarding experience forgetting after model upgrades

Agreed, and thank you for the suggestion. Section O already notes that a self-improving actor is non-stationary, so older episodes gradually become another model's and calibration memory may need to forget in step with capability growth. We will cite your framing (model upgrades, prompt and toolchain iteration) there and list experience decay as the open question it is.

Thank you again for a review that engaged with the appendices as closely as the main text; we hope the clarifications above are useful and would be glad to discuss further.

Reviewer_dTAgOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes the notion of eXperiential Confidence (XCONF), where confidence estimates are made based on retrieved records of past successes and failures stored in an experience bank.

The approach specifically looks for episodes on which in the past a similar confidence was stated, and asks how often it was right.

Critical strengths

The idea is simple, well-motivated, and appears to work well in the considered setting.

Critical weaknesses

The paper reads a lot like AI-generated.

In practice we often lack sufficient knowledge of whether a result was right.

While the approach may be novel in this setting, it is not clear that the approach is particularly exciting. Memory-based learning is a well-established concept and in many settings it is straightforward to leverage this to also provide confidence scores.

Questions for authors

Wouldn't it be better for the model to establish its confidence itself by reasoning directly over the memory bank rather than using an ad hoc approach where recall is separate from reasoning?

1 comment

Thank you for your time to review our work. Since the ICLR committee encourages brevity, we keep this reply short; please let us know if anything remains unclear!

W1: regarding the writing

The idea, the hypotheses, and the experimental design are entirely our own, and the paper was written with substantial human effort. We did use an LLM to polish prose we had drafted and to help write code, and we disclosed exactly this in the official submission form ("AI Assistance: Yes, to aid or polish writing. Yes, for research ideation or execution. Details are described in the paper.") and in the AI Use Statement. We would be glad to clarify any passage you found unclear.

W2: regarding not knowing whether a result was right

We would like to point out that the need for outcome labels is a property of calibration, not only of our method, and the bank is unusually tolerant of how those labels are obtained. Here are the breakdowns:

  • Every calibrated estimator consumes outcome labels. Platt, isotonic, conformal, and trained verifiers all fit on graded outcomes, and Section 4 gives each of them exactly the outcomes our bank contains, so the comparison is matched on supervision. The label-free estimators in our comparison, verbalized confidence, self-consistency, semantic entropy, and P(True), are the ones we beat on both metrics.
  • The labels need not be gold. As we show in Section C, an independent LLM judge that never sees the answer key, with only 91% agreement with gold, costs almost nothing, and the one thing that fails is the model labeling itself. The requirement is independence from the model's own judgment, not accuracy.
  • A system has graded episodes before it ships. We consider in real agent deployment, any system that was evaluated before deployment already has them: development and test sets, staging runs, and acceptance checks (Section 3.2). The bank can start from those, with no separate labeling effort, and grows with whatever graded feedback deployment later provides (unit tests and CI for code, task assertions for agents).

W3: regarding novelty over memory-based learning

Memory-based learning is indeed well established; using it for confidence estimation is not. Table 1 lays this out: every existing estimator family reads the current inference alone, and none consults the model's own graded history at inference time. To our knowledge, we are the first to build a memory of graded episodes, retrieve from it, and apply it to confidence estimation, and the first to show this working on code and agent trajectories, where sampling-based estimators do not apply.

Q: regarding reasoning directly over the memory bank

That variant is in the paper, and the comparison is why we keep the two readings.

  • Reflect alone is the model reasoning over the retrieved record (Table 20). It ranks well but its ECE is 3–10× worse than Recall alone on nearly every cell, because a verbally restated number stays scale-miscalibrated even after reading the record. The statistical reading is what calibrates.
  • Reflect degrades with model size; Recall does not (Section J). Below 27B the model can no longer read its own record usefully, while the counted hit rate is size-invariant.
  • Retrieval is needed regardless. The banks hold up to 12k episodes; the model cannot read them whole, so something must select what it reads. We do not see the two stages as ad hoc: Recall selects and counts, Reflect reads and names the pattern, and Figure 8 shows that neither reduces to the other. A fully agentic variant in which the model searches the bank itself is a natural extension, and we will add it to the open questions in Section O.

We hope this addresses your concerns about what is new here and where the labels come from.We are happy to discuss during the remaining period.

Reviewer_ftANOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes XConf, a black-box confidence estimator that uses a model's graded historical episodes to assess the correctness of its current output. Recall estimates a confidence-conditioned hit rate over similar past episodes retrieved using a correctness-supervised rescaling of the feature space, while Reflect asks the model to revise its confidence after reading the retrieved outcomes and lessons. The final estimate averages these two scores equally. The method requires no actor finetuning or token probabilities and uses one answer generation followed by a short Reflect call. Evaluation spans nine benchmarks covering reasoning, coding, multimodal QA, and agents, using four models. The paper reports matching or exceeding ten-sample self-consistency in AUROC on 23 of 24 non-agent comparisons, alongside selective prediction improvements and analyses of experience-bank size, transfer, label quality, and component contributions.

Critical strengths

S1. Broad practical applicability with meaningful improvements in challenging settings.

The same framework applies across substantially different output formats, including programs and agent trajectories, without requiring repeated task execution. The results are particularly compelling where introspection is unreliable: on AppWorld/Qwen3.5-397B, XConf achieves 0.837 AUROC, compared with 0.583 for Verbalized (on rollout) and 0.665 for the LLM judge. The selective prediction experiments also connect confidence estimation to practical decisions: on AppWorld/Gemini 3.5 Flash, withholding the lowest-confidence 10% raises accuracy on the retained episodes from 80.5% to 89.2%. These results provide a substantive case for the usefulness of graded historical experience.

S2. Substantial analysis beyond the headline comparisons.

The paper provides extensive analyses of the Recall key, the individual contributions of Recall and Reflect, bank-size scaling, cross-dataset and cross-model transfer, and the role of outcome labels. In particular, Table 19 compares "Second ask, no record" with "Reflect (reads the record)," providing evidence that the improvement comes from the retrieved record rather than simply asking the model to estimate confidence again. Appendix C further shows that an independent but weak judge preserves much of the benefit, whereas SC-majority pseudo-labels can record the model’s confident failures as successes and inflate confidence on consensus-wrong items. Together, these analyses strengthen the empirical evidence for the value of graded past experience and clarify the conditions under which it is useful.

Critical weaknesses

W1. Insufficient justification for the equal-weight combination of Recall and Reflect.

The final estimator uses the fixed equal-weight combination, Confidence(e)=12(Recall(e)+Reflect(e))\mathrm{Confidence}(e)=\frac{1}{2}(\mathrm{Recall}(e)+\mathrm{Reflect}(e)), motivated by prior work on forecast combination (Section 3.4). However, this motivation alone does not establish that equal weighting is appropriate for calibrated confidence estimation. Even if Recall and Reflect were individually calibrated, their arithmetic mean would not necessarily satisfy E[y∣Confidence(e)]=Confidence(e)\mathbb{E}[y\mid\mathrm{Confidence}(e)]=\mathrm{Confidence}(e). My concern is therefore the empirical justification for this fixed combination, rather than the absence of a theoretical guarantee.

Indeed, Table 20 shows that the final blend has higher ECE than Recall alone in 31 of 36 settings, including all twelve agent settings. Although combining Recall and Reflect improves AUROC over Recall alone in some settings (e.g., AppWorld/Qwen3.5-397B: 0.738 to 0.837), ECE simultaneously increases from 0.062 to 0.116. This indicates a trade-off between discrimination and calibration that is not sufficiently addressed by the fixed combination. Moreover, Table 6 does not report Brier scores for Recall alone, leaving the comparison incomplete. Evaluating a few alternative mixing weights and providing paired Brier score and AURC comparisons between Recall and the final blend would help establish whether the equal weighting and the additional Reflect call are justified.

W2. The contribution of Recall beyond supervised correctness prediction is not sufficiently isolated.

Appendix E defines the feature vector F(e)=[ϕ(x);ϕ(r);v;log⁡∣ρ∣;log⁡∣r∣]F(e)=[\phi(x);\phi(r);v;\log|\rho|;\log|r|]. Each embedding block is reduced to thirty PCA dimensions, and the resulting feature vector is standardized to F~(e)\widetilde{F}(e) before fitting a logistic probe to predict the outcome yy. The fitted coefficients are used to construct the retrieval key, κ(e)∝∣w∣⊙F~(e)\kappa(e)\propto |w|\odot\widetilde{F}(e), normalized to unit length for cosine retrieval, while the probe’s own prediction is discarded. However, the paper does not report the direct predicted probabilities of this same logistic probe as a baseline.

This comparison matters because the same episode features and outcome labels already support a supervised correctness predictor. The retrieval-key ablations examine how the choice of neighbors affects performance, but they do not establish whether estimating confidence from retrieved outcomes improves on using the fitted probe’s predictions directly. Likewise, the other label-consuming methods do not provide this comparison using the same features and supervision. Comparing the probe’s held-out predictions with Recall under identical folds, preprocessing, and outcome labels would more clearly establish the additional value of retrieval. Then including XConf would show whether the complete method also improves on this matched baseline.

Questions for authors

Q1. Comparison with the logistic probe.

How do the held-out predicted probabilities of the logistic probe fitted in Appendix E compare with Recall and XConf under the same feature preprocessing, outcome labels, and folds? Reporting AUROC, ECE, and Brier scores would help establish the additional value of retrieving graded past episodes.

Q2. Justification of the equal-weight blend.

Could you provide paired Brier-score and AURC comparisons between Recall and the final blend, and evaluate a few alternative mixing weights? Given the higher ECE of the blend in most settings in Table 20, please clarify how equal weighting balances discrimination, calibration, and selective prediction. Any selection of mixing weights should use data separate from the held-out evaluation folds.

Q3. Interpretation of the question-level difficulty analysis.

Appendix F defines difficulty as the fraction of the other three model columns that solved an item, and computes AUROC within strata sharing the same d^\hat d. However, questions within a stratum may still differ in underlying difficulty. Could you clarify whether the conclusion is limited to controlling for this measured difficulty score, or provide evidence that the remaining gains are not attributable to residual question-level difficulty within the strata?

Q4. Sensitivity of Reflect to the number of retrieved cards.

Appendix M specifies eight in-context cards for Reflect, whereas Appendix G refers to “twenty concrete precedents.” Could you clarify the number of cards used for the main results and for Figure 8? Beyond the neighborhood-size analysis in Appendix I, how sensitive are Reflect and the final blend to the number of cards actually shown in the prompt? A small comparison on representative reasoning and agent settings, reporting AUROC, ECE, and the cost of the Reflect call, would help assess this design choice.

7 comments

Thank you for the careful reading, and in particular for pointing out the probe comparison. Since the ICLR committee encourages brevity, we keep this reply short; please let us know if anything remains unclear.

W1: regarding the equal-weight blend and its higher ECE than Recall alone

We agree, and we will reframe our claim. Recall, Reflect and blend have different strengths:

  • Recall alone is the best-calibrated and cheapest operating point: no second call, ECE .01–.06 on every cell, and within .02 AUROC of the blend on the reasoning line.
  • Reflect alone is the only reading that names the failure pattern behind the estimate, an interpretability that Recall cannot offer.
  • The blend is the better operating point where discrimination is hardest, the silent-failure agent domains, where it beats both components (AppWorld up to +.023 AUROC, SWE-bench up to +.039) at an ECE still a fraction of the verbalized and LLM-judge baselines' (on the cell you cite, .116 against .397 and .368).

The main tables use the blend only so that one fixed recipe is reported everywhere. We will reframe the three settings in Section 3.4 and Section D accordingly: which reading to use depends on whether calibration, discrimination, interpretability, or cost matters most, and Table 20 is the guide. We also agree that the calibration of an averaged score is an empirical matter rather than a guaranteed one, and will drop the forecast-combination motivation. We hope this reframing sounds right to you.

Q2: regarding alternative mixing weights and paired Brier / AURC

The comparison you asked for is Table R1 below. It sweeps the mixing weight with w chosen for each evaluation fold on the other four folds only, and its w = 1 and w = .5 columns are the paired Recall-vs-blend comparison on Brier and AURC. Three facts: w = .5 is at or near the top for AUROC and AURC on both lines (on agents, .75 ties it and the nested choice is within .003); Recall alone has the best mean ECE; and Brier is nearly unchanged between Recall and the blend on the reasoning line (.131 vs .131) while favouring Recall on agents (.128 vs .136). On agents the picture is therefore clear: the blend improves ranking and selective prediction and pays for it in ECE and Brier. No weight dominates, which is why we now present them as operating points rather than as a recipe.

Table R1. Mixing weight w in w·Recall + (1−w)·Reflect, mean over the cells of each line. "Nested" chooses w for each evaluation fold on the other four folds only (min Brier).

w=0 (Reflect).25.5 (shipped).751 (Recall)nested
Reasoning, AUROC ↑.806.824.829.828.822.827
Reasoning, ECE ↓.081.059.039.026.023.024
Reasoning, Brier ↓.144.136.131.129.131.130
Reasoning, AURC ↓.114.105.103.104.106.104
Agents, AUROC ↑.780.851.872.872.852.875
Agents, ECE ↓.157.117.091.062.034.050
Agents, Brier ↓.200.161.136.125.128.122
Agents, AURC ↓.319.272.263.265.282.264

W2 / Q1: regarding the logistic probe as a baseline

We computed the probe's held-out prediction under identical folds, preprocessing, and labels on all 36 cells, and Table R2 compares it with Recall under the bank composition of Table 24, where the other five datasets' banks are merged into the target's with no curation.

  • On a single-domain bank the two statistical readings are close, as two readers of the same supervision should be. In practice, however, a bank is a merged collection of different tasks, and on a merged bank they separate: the probe loses .05 AUROC on average over the 24 reasoning cells (up to .26 on LiveCodeBench) because the foreign episodes pull its global boundary, while Recall loses .01, because foreign episodes are rarely among the fifty nearest. Retrieval is what lets experience be pooled without curation, which Section 6 reports for Recall and which a parametric predictor on the same bank cannot do.
  • Retrieval is also what gives Reflect something to read. A probe outputs a number; retrieval returns concrete past episodes with their lessons, and Section G shows that on AppWorld and SWE-bench compressing them to a scalar costs .04–.06 AUROC against reading the cards.
  • The complete method is ahead of the probe on eight of nine datasets, with the largest margins where failure is silent (AppWorld +.023 AUROC, SWE-bench +.025, up to +.07 on AppWorld / Qwen3.5); the exception is ScienceWorld (−.008), where Section D already notes that the statistical reading alone is the better operating point.

We will add the probe as a row, with AUROC, ECE, and Brier, to every table of Section A, and Table R2 to Section I.

Table R2. Recall vs probe, AUROC, mean over the four models, when the bank is the target's own (single-domain) or the target's plus the other five datasets' full banks (merged, the naive-all policy of Table 24). Same folds; key and probe refit on whichever bank is used.

DatasetSingle-domain: Recall / probeMerged: Recall / probeProbe loss from merging
MMLU-Pro.828 / .829.825 / .805−.024
SuperGPQA.737 / .732.723 / .694−.038
BBEH.822 / .819.822 / .786−.033
OlympiadBench.888 / .892.874 / .868−.024
LiveCodeBench.863 / .867.845 / .685−.182
MMMU-Pro.794 / .790.775 / .777−.013
Mean.822 / .822.811 / .769−.052

Q3: regarding residual difficulty within strata

The stratified test rules out the measured difficulty score, nothing more. You are right that items within a stratum can still differ, and we will say so in Section F. Two further results hold the question fixed rather than a difficulty score, so any question-level property, measured or not, is identical across the comparison:

  • Cross-model transfer (Section I, Figure 12 right). On the very same questions, retrieving from another model's bank instead of the model's own loses .03–.06 AUROC on five of six datasets. What is lost cannot be question difficulty, since the donor bank holds the same questions.
  • Key ablation (Table 21). Removing the stated confidence from the key, so that retrieval conditions on the task alone, costs .08 AUROC on reasoning and .12 on agents and leaves scores huddled at the base rate, which is what a pure difficulty predictor looks like.

Q4: regarding the number of cards shown to Reflect

Thank you for pointing out this. We confirm it is eight cards, in every run including Figure 8. This is the default in the released scripts and the number stated in Sections L an M; "twenty concrete precedents" in Section G is a typo. The card count affects only the verbal reading: Recall reads k = 50 neighbours regardless, and Section I shows a wide plateau for k from 5 to 200. We will add a card-count sweep to the revision.


We are grateful for a review that pointed us to two controls we should have reported; both will be in the revision, and we would welcome any further question on them.

Thank you for the detailed response and additional analyses. Table R1 and the proposed reframing of Recall, Reflect, and the blend as different operating points substantially address my concern about the equal-weight combination. I also appreciate the clarifications regarding the difficulty-control analysis and the number of in-context cards.

My main remaining concern is the matched logistic probe comparison. Table R2 provides useful evidence that Recall is more robust than the probe when banks are merged, but the two perform nearly identically on single-domain banks (mean AUROC: 0.822 vs. 0.822). Moreover, the reported comparison currently includes only AUROC for the six non-agent datasets.

Could you share the promised ECE and Brier scores for the logistic probe, along with the matched results on the three agent benchmarks, particularly AppWorld and SWE-bench Verified? I checked the currently available PDF, but these additional results do not appear to have been incorporated yet. These results would help clarify the additional contribution of retrieval under the original evaluation setting, beyond its demonstrated robustness to merged banks.

I appreciate the additional experiments and will take these results into account when reassessing my recommendation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion and Revision

We sincerely thank the AC and all reviewers for their time and the constructive discussion. Here is a brief summary.

First of all, we would like to highlight that our core contribution remains unchanged:

  • Confidence from experience. Inspired by how people judge their own accuracy, XConf estimates confidence from an experience bank of the model's past episodes.
    • It matches or exceeds ten-sample self-consistency on 23 of 24 comparisons at a tenth of the cost.
    • Reviewers found the idea "simple, well-motivated" (dTAg), the evaluation broad (ftAN, Rz5j, rT1T), and the ablations "well-designed and strong" (rT1T; also ftAN, Rz5j).
  • Advantages no prior method combines (Table 1).
    • XConf is black-box, training-free, single-generation, and improves as experience accumulates.
    • It is not limited to a particular task type and applies to agent tasks.
    • Like any calibration method, it needs a set of graded episodes, a limitation shared by all such methods.

We hope this work draws the community's attention to a new and scalable paradigm for confidence estimation.

Meanwhile, the main concerns and our responses:

  • Building the bank (dTAg, Rz5j, rT1T).
    • The labels need not be gold: an independent LLM judge works almost as well (Section C).
    • A bank can start from the evaluation data a system already has before deployment.
  • Combining Recall and Reflect (ftAN, Rz5j, rT1T).
    • We now present Recall, Reflect, and the blend as three operating points, with a mixing-weight sweep (Table R1).
    • Reviewer ftAN found this to "substantially address" the concern.
  • Gain over a matched probe (ftAN).
    • The probe is itself fit on our experience bank, so it is another way of reading the same experience and supports the core contribution.
    • Retrieval adds robustness: when experience from different tasks is pooled, merging costs Recall .011 AUROC and the probe .053 (Table R4).
    • The full method shows a significant mean gain on AppWorld and SWE-bench Verified (Table R5).
    • Reviewer ftAN raised their rating to 3 (Weak acceptance).

All new results (Tables R1–R5, F1–F2) are in the revised PDF, with changes marked in purple; most are collected in Appendix P.

Thank you again. The discussion has made the paper clearer and more accurate, and we are grateful for it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.