acceptodds
Under review as a conference paper at ICLR 2027

Evaluating Persistent Calibration under Evolving Model Knowledge

Abstract

As AI systems move from static repositories to agents that are capable of continual adaptation and learning, maintaining their trustworthiness means equipping the models backing them with the ability to produce confidence estimates that dynamically reflect their changing skills and knowledge. We introduce the problem of _persistent calibration_, which requires a confidence estimator to faithfully reflect the knowledge contained in a model as that knowledge changes, without recurring supervision. We operationalize this by examining persistent calibration across checkpoints of open models, asking whether confidence estimators trained on earlier checkpoints can generalize to later ones. Specifically, we aim to shed light on whether confidence is dependent on knowledge, a question with implications for the reliability of confidence estimates. To measure this relationship, we define and evaluate calibration on _knowledge contrast sets_: subsets containing questions that one checkpoint answers correctly and another checkpoint answers incorrectly, reflecting a change in knowledge. We show that both inference-time and fine-tuning methods fall short on contrast-set calibration compared to oracle methods trained on future checkpoints, even for methods that are well-calibrated on the full dataset. We provide evidence for the hypothesis that persistent calibration is challenging because there is a vast space of possible confidence functions that are well-calibrated on a given checkpoint, out of which only some rely on meta-knowledge features that would generalize to other checkpoints. Towards improving contrast-set calibration, we show that multi-checkpoint training helps, suggesting an avenue for identifying confidence features that remain robust across changing knowledge.

Primary area:
foundation or frontier models, including LLMs
Keywords:
confidence calibration, calibration across time, LLM uncertainty

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cFbsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether a confidence estimator for an LLM keeps tracking the model's knowledge as that knowledge changes, without new supervision. The authors call this persistent calibration. They study it with pre-training checkpoints of Olmo 3 (7B and 32B) and Marin 8B. Estimators are trained on earlier checkpoints, or are training-free (self-consistency), and are evaluated on later ones.

The main evaluation uses knowledge contrast sets, meaning questions that one checkpoint answers correctly and another answers incorrectly. This is meant to remove the confound that correctness is highly correlated across checkpoints. Confidences on a contrast set are renormalized per question (Eq. 1), and the main metric is a class-balanced pairwise score Δ0b\Delta_0^b. The methods are self-consistency (SC), LoRA verbalized-confidence adapters (optionally trained on several checkpoints), an "oracle" adapter trained on the evaluation checkpoints, a trivial end-correct baseline, and an external verifier (GCM). Surrogate and copy ablations control for knowledge already present in an earlier checkpoint.

The headline claims are that SC and adapters trained on earlier checkpoints fall short of the oracle on contrast sets even when full-set AUC is similar, that multi-checkpoint training helps, and that many confidence functions fit one checkpoint equally well while only some generalize.

Critical strengths
  1. The question is timely and well posed. Confidence estimators are usually built and tested on frozen snapshots, and model updates make that assumption fragile.
  2. The contrast-set idea is a useful diagnostic. The copy ablation shows crisply that full-set AUC is inflated by correlation across checkpoints (0.83 to 0.85 on TriviaQA while contrast AUC is 0.5).
  3. Several design choices are careful, including the class-balanced Δ0b\Delta_0^b, per-checkpoint ECE and a dependence-aware paired bootstrap. The end-correct, surrogate and copy controls are the right ones.
  4. Open checkpoints make the setup reproducible in principle, and the paper reports negative domain-transfer results and full appendix tables.
Critical weaknesses
  1. The main target is never formally defined. Persistent calibration is described only informally, and the main tables report ranking metrics (AUC and Δ0b\Delta_0^b) while ECE appears only in the appendix. On full-set ECE, SC with an undefined "Post-hoc" step beats the oracle in all six settings (Table 11). The title and the claims speak of calibration, but the headline results are about ranking, and the ECE results point the other way.

  2. Contrast sets do not isolate the model's own knowledge. On a contrast question the two checkpoints give different answers, so any estimator that can judge the answer can score well. The paper's own results show this. The GCM, which sees only the question and answer, matches the oracle on Olmo 3 7B. On Marin Jeopardy, an older checkpoint scoring the new answers beats the oracle (Δ0b\Delta_0^b 0.515 and 0.504 against 0.459). The claim that contrast sets are "designed to require meta-knowledge" does not follow. It is also unclear whether the GCM's TriviaQA training data overlaps the evaluation split.

  3. Non-oracle training fails the paper's own test. The paper says that beating the surrogate and copy ablations is necessary for a nontrivial demonstration. In Table 12, non-oracle training never significantly beats its surrogate on Δ0b\Delta_0^b in any of the six settings, and SC does so in only 2 of 6. The text frames this as mixed results, but the test is failed for non-oracle training in every setting, and Appendix C is worded more positively than Table 12 supports.

  4. The headline contrast AUC mostly reflects that later checkpoints are right more often. For the end-correct baseline, contrast AUC equals the share of improvements exactly, and my recomputation matches Table 1 to three decimals in all six settings. The abstract example (SC 0.760 against oracle 0.825) should be read against a trivial 0.735. The abstract and Table 1 lead with the unbalanced metric, and gains on it can reflect this prior. On Marin TriviaQA, multi-checkpoint training raises unbalanced Δ0\Delta_0 from 0.213 to 0.460 but Δ0b\Delta_0^b only from 0.221 to 0.227. Solving from the end-correct share (0.783) and these two metrics gives roughly +0.64 on improvements and -0.19 on regressions. This is approximate, since the tables aggregate over checkpoint pairs. It suggests the gain comes from improvements, and the paper does not show that it detects forgetting.

  5. The multi-checkpoint claims go further than the numbers. Table 4 says that both the training data and the checkpoint views contribute, but single-checkpoint data scores 0.373 against 0.379 for the full method (not significantly worse), and on Jeopardy it scores 0.517 against 0.506. Non-oracle gains from multi-checkpoint training are absent on Olmo 7B and Marin TriviaQA. The clear gains are in the oracle setting, which cannot be deployed.

  6. The normalization rests on questionable assumptions. Eq. 1 assumes the two checkpoints' correctness is independent, but different answers to a single-answer question make correctness mutually exclusive. It also needs joint calibration, not only marginal calibration. The robustness check in Appendix B.1 forces an antisymmetric map and is fit on the evaluation labels, so it cannot detect this alternative. The effect is on contrast AUC, Brier score, ECE and Δ\Delta, though not on Δ0b\Delta_0^b.

  7. Contrast sets may be dominated by unstable flips, and there is no noise floor. Regressions are 22 to 37% of contrast items, and between Olmo 7B at 90% and 100%, 538 questions flip while accuracy moves from 64.8 to 65.6. Correctness comes from one deterministic beam search under one fixed prompt, judged by an LLM. The paper does not report contrast sets between two prompts at the same checkpoint, or results stratified by SC margin or flip persistence.

  8. The setting is narrow and the explanation is untested. "Evolving knowledge" means consecutive pre-training checkpoints of one run, with English factoid QA. There is no fine-tuning on new facts, no knowledge editing and no post-training. The one domain shift (BioASQ) collapses all methods. The "vast hypothesis space" explanation rests on full-set AUC with no copy reference per cell, random labels with an unexplained below-chance AUC (0.310) and hidden-state variance that is not normalized for activation scale.

  9. The baselines, statistics and reproducibility have gaps. There is no P(True), probe, semantic-entropy or label-free adaptation baseline. There is no per-seed standard deviation, Section 4 uses one seed, and Marin conclusions rest on two checkpoint pairs that both involve phoenix. The judge is a closed model with no prompt given, the audit covers 100 TriviaQA items and leaves out incomplete references, and the LoRA loss and code or data release are not stated.

Questions for authors
  1. Can you report contrast-set results where every checkpoint scores the same fixed answer, so that answer plausibility is held constant? Can you also stratify results by whether the estimator's own backbone answers correctly (the earlier checkpoint for the surrogate, Qwen3-8B for the GCM), and state what deduplication was applied between the GCM's training data and the evaluation split?
  2. Since non-oracle training never significantly beats its surrogate on Δ0b\Delta_0^b, is there evidence that the adapters track the new checkpoint's knowledge beyond the earlier one? Why is Appendix C phrased positively?
  3. What do contrast sets look like between two prompts at the same checkpoint, and how many flips persist at later checkpoints? Can you stratify results by SC margin?
  4. Can you report a class-balanced contrast AUC and include the end-correct row in every table? Can you also report Δ0\Delta_0 separately for improvements and regressions per checkpoint pair, and the AUC of logit(plater)−logit(pearlier)\mathrm{logit}(p_{later})-\mathrm{logit}(p_{earlier}) for separating improvements from regressions? That AUC is invariant to a constant shift, so it would show whether the estimators do more than apply a checkpoint-level offset.
  5. What is "Post-hoc" in Tables 11 and 16, and why does it beat the oracle on full-set ECE but not appear in the main text?
  6. Can you report contrast results under the mutual-exclusivity normalization p1/(p1+p2)p_1/(p_1+p_2) and without the antisymmetry constraint?
  7. Why is the Fig. 2 AUC below chance (0.310) with random labels, and what is the copy-ablation AUC for each cell?
  8. What are the LoRA loss, the confidence prompt and the judge prompt, and will code, contrast sets and adapters be released?
7 comments
AuthorsOpenReview ↗

Response to reviewer cFbs (Part 1)

Thank you for describing our research question as “timely and well-posed”, our contrast sets as a “useful diagnostic”, and our design choices as “careful”. We address your comments below. We have also uploaded a revised version of our paper, which we reference in our responses, with revisions in blue. Note that the revised manuscript contains different table, etc. numbers from the original manuscript.

W1: Evaluation metrics

The main targets are the evaluation metrics given in Section 2. We have now clarified this in the introduction: “Persistent calibration requires both full-set and contrast-set calibration, which we concretely evaluate using the metrics defined in Section 2.” Based on your suggestion, we have expanded the main tables to include the metrics BS and ECE for the contrast set and full set. In our response to Q4, we describe the other metrics we added based on your suggestions. In our response to Q5, we discuss post-hoc calibration and the discrepancy between full-set ECE and other metrics.

AuthorsOpenReview ↗

Response to reviewer cFbs (Part 2)

W2: Whether contrast sets require meta-knowledge

Our practical objective is to obtain informative confidence estimates from a lightweight extension of a continually updated generator model, without recurring supervision for confidence estimation or reliance on an external verifier whose capabilities must keep pace with it. The generator’s evolving representations are a natural source of features for this purpose. We use persistent calibration to describe the desired behavioral property, rather than a particular introspective mechanism. Knowledge contrast sets test whether confidence distinguishes changes in answer correctness that unchanged question-level scores cannot capture. However, because our estimators observe the candidate answer, success can also arise through answer verification using existing or newly acquired capabilities. Our evaluation therefore does not uniquely identify meta-knowledge: it tests behavior consistent with knowledge tracking, while the surrogate ablations assess whether access to the updated generator provides benefits beyond verification by an earlier checkpoint. Nonetheless, we believe that meta-knowledge is a promising direction to pursue for persistent calibration. Furthermore, we can require meta-knowledge with a simple modification to our setup: require the confidence estimator to predict, from the question alone, whether the generator will predict the correct answer. In this setup, even an external oracle that knows every correct answer cannot predict the generator's correctness without knowledge of the generator's knowledge. This is a cleaner test of introspection, though not the practical setup we adopt in this paper, given that confidence estimation in practice is improved by conditioning on the candidate answer (Orgad et al., 2025; Ni et al., 2025). We have added this discussion to Section 2.

We acknowledge that Marin 8B on Jeopardy is the one case out of six in which SC or non-oracle training surpasses the oracle on \Delta^b_0. We emphasize that the surrogate ablations do not consistently match oracle performance, and that the surrogate’s performance often doesn’t extend to other metrics such as full-set and contrast-set ECE (contrast-set ECE 0.182 or 0.154 vs. 0.123; full-set ECE 0.126 or 0.048 vs. 0.027). We describe one intuition for why the surrogate ablations achieve higher contrast-set AUC than one might expect, given that they use an earlier checkpoint that often did not predict the correct answer. Although different checkpoints are generally not independent with regard to their predictions, it may be instructive to approximate them as such, especially for questions where some of them made different predictions. Since the checkpoints are similarly performant (Table 7), ensembling their predictions by taking the majority vote can be expected to achieve a higher accuracy than any individual checkpoint (Huang et al., 2017; Garipov et al., 2018). This would suggest that, for a question on which two evaluation checkpoints produce differing predictions, a third checkpoint (i.e. the surrogate checkpoint) can distinguish the correct answer from the incorrect answer with nontrivial probability, leading to a nontrivial contrast-set performance. Intuitively, different checkpoints' incorrect predictions tend to differ, so even if the surrogate checkpoint predicted an incorrect answer, it is likely to consider the correct answer more plausible than an incorrect answer that is different from its own. We have added this discussion to Appendix B.4. However, we emphasize that it is far from a reliable occurrence for the SC or non-oracle training surrogate to surpass oracle training. Furthermore, although the above explains why surrogate ablations can often achieve strong discrimination between the correct answer and the non-self-generated incorrect answer, it does not suggest that they would be calibrated (in the sense of ECE/BS), and indeed, they are highly miscalibrated (as measured by ECE/BS) compared to oracle training or even to the non-surrogate versions of SC or non-oracle training. Overall, our results position the surrogate ablations as a diagnostic tool rather than a reliable confidence estimator.

We enforce a train-test split between the GCM’s training data and our evaluation data. We have added this clarification to Appendix A.4.

AuthorsOpenReview ↗

Response to reviewer cFbs (Part 3)

W3, Q2: Wording of non-oracle vs. surrogate results

We agree that non-oracle training fails to consistently outperform its surrogate ablation. The mixed results refer to contrast-set AUC, where non-oracle training outperforms its surrogate ablation in 3/6 cases in Table 16. Nonetheless, we acknowledge that this does not hold across metrics, as you point out with \Delta^b_0. We have updated the wording in Section 3.2, Table 2, Table 16, and Appendix C to reflect this.

W4: Breakdown of \Delta^b_0 into improvement and regression

We agree that contrast-set AUC reflects the end-correct prior, but we believe that this does not affect the validity of the revealed trends. For example, in the contrast-set AUC example you give with [end-correct baseline 0.735, SC 0.760, oracle 0.825], the main finding that the oracle outperforms SC still holds, regardless of the moderate performance of the end-correct baseline.

Based on your suggestion, we have added the breakdown of \Delta^b_0 into \Delta_0 for improvement and regression; see our response to Q4. We agree that an improvement on \Delta^b_0 does not guarantee an improvement on both the improvement and regression components. However, we argue that such a guarantee is not actually desirable, and that \Delta^b_0 captures the faithfulness (independent of begin/end-correct priors) that we want to capture. Consider the end-correct baseline, which scores \Delta_0 = 1 on the improvement set but \Delta_0 = -1 on the regression set, and the begin-correct baseline, which scores \Delta_0 = 1 on the regression set but \Delta_0 = -1 on the improvement set. Consider also a confidence estimator that scores \Delta_0 = 0.8 on both the improvement and regression sets. Clearly, this confidence estimator is more faithful to a checkpoint’s knowledge than the begin/end-correct baselines, and yet it does not improve over both baselines on both the improvement-set \Delta_0 and regression-set \Delta_0, because each one can be independently exploited without any faithfulness. On the other hand, \Delta^b_0 captures faithfulness by not rewarding a prior favoring either the earlier or the later checkpoint (unless \Delta^b_0 is currently below the trivial 0). We have added this discussion to Appendix A.2.

W5: Multi-checkpoint training ablations, significance of results

We agree that the multi-checkpoint views are most responsible for the gains of multi-checkpoint training, as suggested by the strong performance of the single-checkpoint data ablation. The reason we do not state that multi-checkpoint views are the sole contributor is that the single-checkpoint data ablation consistently has worse calibration than multi-checkpoint training (Table 23, e.g. contrast-set ECE of 0.096 vs. 0.061 on TriviaQA, and 0.087 vs. 0.057 on Jeopardy), suggesting that the training data also plays a role. We have updated the caption of Table 23 to highlight this.

Our contributions in this paper are introducing the problem setting of persistent calibration, establishing contrast-set calibration as a distinct and necessary criterion, and exploring the viability of explicitly training a confidence estimator to work on multiple checkpoints. We do not claim to solve persistent calibration. Nonetheless, our results suggest an avenue for improving persistent calibration with multi-checkpoint training. Notably, in the oracle setting, which has respective-checkpoint training as a strong baseline that trains a dedicated confidence estimator for each checkpoint and does not require any transfer, multi-checkpoint training still does better. Furthermore, we would like to highlight that also in the non-oracle setting, multi-checkpoint training always at least matches single-checkpoint training, and sometimes delivers drastic improvements: when measuring contrast-set AUC in 6 cases (3 models * 2 datasets; Tables 17-18 extend main Table 3), multi-checkpoint training is statistically significantly worse in 0 cases, statistically significantly better in 5 cases, and 2 of those cases are drastic (0.806 vs. 0.642 TriviaQA, 0.759 vs. 0.562 Jeopardy; Marin 8B).

Reviewer_R7SKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

As language models evolve through continual pretraining or training on new datasets, an important question arises: How can reliable confidence calibration be maintained as model knowledge changes? Traditional calibration methods evaluate the agreement between predicted confidence and empirical accuracy, primarily focusing on fixed models. This paper investigates whether confidence reflects evolving model knowledge by constructing knowledge contrast sets, where two checkpoints of the same model produce different correctness outcomes. These sets are used to evaluate whether confidence estimators can capture changes in model knowledge.

Critical strengths

A clear and important evaluation criterion. Contrast sets isolate whether confidence reflects a specific checkpoint’s knowledge rather than question-level heuristics. The copy ablation (Table 2) makes the point sharply: a method with no access to the new checkpoint reaches full-set AUC above 0.8 but contrast-set AUC of exactly 0.5, so full-set calibration cannot distinguish faithfulness from inter-checkpoint correlation.

Critical weaknesses
  1. Limited technical contribution. The paper’s new method, multi-checkpoint training, is an empirical observation that pooling checkpoints in LoRA training helps (Table 3) rather than a method motivated by the analysis. The gains are modest in the non-oracle setting (contrast AUC .748 vs .750 on Olmo 7B; .688 vs .673 on 32B), and the paper does not propose a way to identify the meta-knowledge features it argues are needed. As an evaluation paper, the contribution rests on the contrast-set criterion; the AC should weigh whether that is sufficient for ICLR.

  2. Contrast sets may not isolate knowledge change. Correctness is the top beam from deterministic decoding, so a flip between checkpoints can arise from a near-tie between two candidate answers rather than from acquiring or losing a fact. The paper does not measure how many contrast-set items are near-ties, or whether results hold when correctness is defined by probability mass on the correct answer. Relatedly, the contrast-set normalization assumes p1,p2p_1, p_2 are already calibrated, which is part of what is under test.

Questions for authors

Clarifications (could change my assessment)

  1. What fraction of contrast-set questions are near-ties between the top two candidates at the flipping checkpoint? Do the conclusions hold if correctness is defined by the probability mass on the reference answer? The normalization in Sec. 2 assumes calibrated p1,p2p_1, p_2. How sensitive are the AUC and Δ results to this assumption beyond the fitted-normalizer check in App. B.1?
  2. What is the “Post-hoc” row in Table 11? If it is temperature or Platt scaling of self-consistency, please define it in the main text and discuss whether post-hoc recalibration on an earlier checkpoint transfers.
  3. Can the main tables report ECE and Brier alongside AUC, since the paper’s claim is about calibration rather than discrimination?

Suggestions (future work)

  1. Extend to post-training checkpoints (base → SFT → RL) of the same model, where knowledge and verbalization both change.
  2. The relationship between confidence and knowledge change may be useful for training-data membership inference; this could be an interesting direction.
5 comments
AuthorsOpenReview ↗

Response to reviewer R7SK (Part 1)

Thank you for describing our introduced problem setting of persistent calibration as an “important question”, and our knowledge contrast sets as a “clear and important evaluation criterion”. We address your comments below. We have also uploaded a revised version of our paper, which we reference in our responses, with revisions in blue.

W1: Technical contribution

We agree that the introduced method, multi-checkpoint training, is technically straightforward, but would like to emphasize that our work is not intended as a methods paper. Instead, our core contributions are in introducing the persistent calibration setting and studying the performance of reasonable methods under this novel setting. Nonetheless, we believe that our empirical findings on the benefits of multi-checkpoint training are novel and unexpected a priori. Notably, even in the oracle setting, which has respective-checkpoint training as a strong baseline that trains a dedicated confidence estimator for each checkpoint and does not require any transfer, multi-checkpoint training still does better. Our ablations further reveal the benefit of multi-checkpoint training to come mainly from leveraging different checkpoints’ representations rather than more data. Thus, we believe that these findings are significant in suggesting an avenue for improving persistent calibration.

We agree that multi-checkpoint training has an intuitive motivation, but we believe it is also motivated by the analysis in Section 4. Figure 3 finds that high AUC on a given checkpoint does not predict AUC on another checkpoint, demonstrating the existence of checkpoint-specific features that don’t generalize to other checkpoints. These checkpoint-specific features motivate multi-checkpoint training – we penalize a confidence estimator for using checkpoint-specific features that don't work on other checkpoints. We have added this discussion to Section 4.

Finally, we would like to highlight that also in the non-oracle setting, multi-checkpoint training always at least matches single-checkpoint training, and sometimes delivers drastic improvements: when measuring contrast-set AUC in 6 cases (3 models * 2 datasets; Tables 17-18 extend main Table 3), multi-checkpoint training is statistically significantly worse in 0 cases, statistically significantly better in 5 cases, and 2 of those cases are drastic (0.806 vs. 0.642 TriviaQA, 0.759 vs. 0.562 Jeopardy; Marin 8B).

W2, Q1: Near-tie flips, normalization assumptions

Thank you for the suggestion to analyze the effect of filtering near-tie flips from the contrast set. We have added results with the contrast set restricted to instances with a P(correct) margin between checkpoints of at least 0.2, where P(correct) is the total generation probability on answers judged correct. In Tables 8-9, we have added statistics for contrast sets with the P(correct) margin alongside the standard contrast sets. We generally observe a 10-20% decrease in size compared to the standard contrast sets. Table 15 (with P(correct) margin) mirrors Tables 11-14 (without P(correct) margin) in the trend that oracle training outperforms non-oracle training. We note that the P(correct) margin gives SC an advantage, since SC and P(correct) are both based on generation probabilities, unlike the trained verbalized methods. Tables 19-20 (with P(correct) margin) mirror Tables 17-18 (without P(correct) margin) in the trend that multi-checkpoint and multi-answer training help.

The normalization to condition on contrast set membership intentionally assumes that p1,p2p_1, p_2 are globally calibrated. The purpose of the normalization is not that of post-hoc calibration (e.g. temperature scaling) but rather only to correct for the selection bias of the contrast set. For example, suppose that on each question in a dataset, two checkpoints each have an independent 80% chance of predicting the correct answer. The ideal confidence estimates are thus 80% on every predicted answer, and the expected accuracy is 80%. However, in the contrast set, each checkpoint has an accuracy of 50%, so the ideal global confidences of 80% appear miscalibrated when viewed within the contrast set, which is why we must normalize (p1,p2)(p_1, p_2) = (0.8, 0.8) into (0.5, 0.5). On the other hand, if we have this setup except that checkpoint 1 predicts 70% confidence, this would be globally miscalibrated (since it should be 80%), and we would want our contrast-set-conditioned confidence to reflect this miscalibration instead of covering it up. Thus, we want to normalize (0.7, 0.8) into something like (0.37, 0.63) instead of (0.5, 0.5). We have added this discussion to Appendix A.1. Nonetheless, in response to your comments about the sensitivity of our results to the normalization function, Appendix B.2 now describes 4 normalization functions (2 heuristics and 2 fitted) and finds that they all lead to the same trends.

AuthorsOpenReview ↗

Response to reviewer R7SK (Part 2)

Q2: Post-hoc calibration

Based on your helpful suggestion, we define the post-hoc calibration in Section 3.1.1: isotonic regression fitted on the train set and the training checkpoints. We added the post-hoc results to Table 1 and discuss the results in Section 3.2: Even when SC is augmented with post-hoc calibration to match or surpass oracle training on saturating full-set ECE near 0, SC + post-hoc does not improve over SC on contrast-set metrics. This strengthens the story that full-set calibration does not reliably predict contrast-set calibration, establishing the latter as a distinct criterion for persistent calibration.

Q3: Additional metrics

Based on your helpful suggestion, we have added contrast-set and full-set BS and ECE to the main tables. This helps emphasize that our findings hold across different evaluation metrics, such as the result above about SC + post-hoc.

Suggestions for future work

Thank you for identifying promising extensions of our research question on the relationship between confidence and knowledge as these change over time. The connection to training-data membership inference is indeed one we hadn’t considered. We look forward to exploring these directions in our future work.

AuthorsOpenReview ↗

Follow-up on rebuttal

Dear Reviewer R7SK,

There are only 3 days left in the discussion period. In our response, we addressed your questions on our paper’s contribution, near-tie flips, normalization assumptions, and post-hoc calibration. We would be grateful if you could consider revisiting your score in light of our responses. If you have other unresolved questions, please let us know – we would be happy to engage in the days that remain.

Reviewer_Tg3HOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces the concept of persistent calibration to evaluate whether confidence estimators can faithfully track the evolving knowledge of large language models during continual learning. To disentangle genuine meta-knowledge from static heuristics, the authors propose evaluating calibration on knowledge contrast sets, which consist of questions where model correctness changes across different training checkpoints. Empirical findings reveal that while existing confidence estimation methods achieve high calibration on full datasets, they suffer significant performance drops on these contrast sets, indicating a failure to dynamically reflect true knowledge updates. To address this limitation, the paper proposes a multi-checkpoint training method that prevents overfitting to checkpoint-specific features and encourages the acquisition of generalized meta-knowledge, thereby improving persistent calibration.

Critical strengths

S1. The paper introduces persistent calibration and knowledge contrast sets as a targeted evaluation framework for assessing whether confidence estimators track changes in a model's knowledge across checkpoints. By evaluating confidence on instances where correctness differs between checkpoints, the proposed framework exposes limitations of conventional calibration metrics that may otherwise obscure a failure to reflect evolving knowledge.

S2. The paper provides an empirical investigation of multi-checkpoint training as a practical approach to improving persistent calibration. The ablation studies suggest that jointly training on multiple checkpoints can improve contrast-set calibration by leveraging representations from different stages of training, with consistent gains over checkpoint-specific training in the oracle setting.

Critical weaknesses

W1. Although the paper provides empirical evidence of asymmetric transfer between earlier and later checkpoints, the proposed explanation based on neural network plasticity and the evolution of confidence-related features remains largely speculative. The hidden-state variance analysis and random-label experiments demonstrate heterogeneity in the space of confidence estimators, but do not establish the mechanism underlying the observed transfer asymmetry. Similarly, while multi-checkpoint training improves calibration in several settings, it remains unclear why this approach promotes more generalizable confidence features or under what conditions these improvements should be expected.

W2. The domain-transfer results raise concerns about the generalizability of the learned confidence signals. The advantage of oracle training over non-oracle methods does not consistently persist when transferring from TriviaQA or Jeopardy to BioASQ. However, the evaluation is primarily based on pre-training checkpoints and factual QA benchmarks, which capture knowledge changes along a pre-training trajectory but do not directly test continual learning under explicitly shifting data distributions. Consequently, it remains unclear whether the proposed evaluation framework and observed transfer behaviors generalize to more realistic settings involving sequential domain adaptation or changing knowledge distributions.

Questions for authors

None.

3 comments
AuthorsOpenReview ↗

Response to reviewer Tg3H

Thank you for highlighting that our proposed framework “exposes limitations of conventional calibration metrics” and highlighting our novel finding that multi-checkpoint training “can improve contrast-set calibration by leveraging representations from different stages of training”. We address your comments below. We have also uploaded a revised version of our paper, which we reference in our responses, with revisions in blue.

W1: Mechanistic explainability

We appreciate you acknowledging our steps towards the goal of mechanistic understanding: our hidden state variance analysis showing greater heterogeneity in later checkpoints, and our ablations showing that the gains of multi-checkpoint training are mainly due to leveraging different checkpoints' representations rather than more data. We would like to clarify that our contributions are introducing the problem setting of persistent calibration, establishing contrast-set calibration as a distinct and necessary criterion, and exploring the viability of explicitly training a confidence estimator to work on multiple checkpoints. We argue that a complete mechanistic account of transfer asymmetry and multi-checkpoint training is beyond the scope of this work but acknowledge that it would be a promising direction for future work and have added a discussion of these limitations to the conclusion. We report the phenomena of transfer asymmetry and random-label results to show that calibration on one checkpoint does not imply calibration on another checkpoint due to checkpoint-specific features, which positions persistent calibration as a nontrivial capability that must be explicitly targeted and evaluated. These checkpoint-specific features are also precisely what motivate multi-checkpoint training – we penalize a confidence estimator for using checkpoint-specific features that don't work on other checkpoints.

W2: Generalization to distribution shifts

We agree that ultimately we would like for our models to be persistently calibrated across distribution shift. It is a well-documented phenomenon that trained confidence estimators struggle with domain transfer even within a single checkpoint (Sky et al., 2024; Srey et al., 2026; Ying et al., 2026), as our BioASQ transfer experiments corroborate. Thus, a finding of poor calibration in a setup involving domain transfer could simply be explained by this known phenomenon and would not be novel. Our key finding in this paper is that even in the in-distribution setting, persistent calibration is still difficult to achieve. It is clear that this difficulty would only be exacerbated in the OOD setting, and the gap between non-oracle and oracle training (i.e. trained on the pre-drift and post-drift distribution, respectively) would further widen. We have added this discussion to the “Domain Transfer” paragraph in Section 3.2.

Based on your helpful suggestion to expand the scope of our evaluation beyond factual QA and pre-training checkpoints, we have added new experiments on MATH with mid-training checkpoints from Olmo 3 7B. We use linear probes (as opposed to LoRA used for factual QA) to prevent overexpressivity, i.e. so the confidence estimator does not improve its calibration by simply improving its MATH problem solving skills and checking its answer against the generated answer. We omit SC due to prohibitive inference costs. See Appendix B.3 for experimental details, data statistics, and results. Results:

MethodContrast Δ₀ᵇContrast Δ₀Contrast ΔᵇContrast ΔContrast AUCContrast BSContrast ECEFull AUCFull BSFull ECE
End-correct baseline0.0000.2220.0000.2220.6110.3890.3890.5250.4750.475
Non-oracle
Multi-ckpt0.2540.3010.1510.1770.7080.2210.0760.7750.2400.204
Single-ckpt0.2170.2790.1360.1710.6950.2280.0790.7430.2450.191
Oracle
Multi-ckpt0.3290.3890.2020.2350.7580.2010.0390.8200.1740.031
Respective-ckpt0.3140.3460.1930.2100.7330.2120.0660.8120.1770.028

We observe the same trends of non-oracle training having headroom from oracle training, and multi-checkpoint training helping in both the oracle and non-oracle settings. This shows that our findings hold across training stages (pre-training and mid-training) and domains (factual QA and math reasoning).

References:

Sky et al., 2024. Do androids know they're only dreaming of electric sheep?

Srey et al., 2026. From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models.

Ying et al., 2026. The Truthfulness Spectrum Hypothesis.

AuthorsOpenReview ↗

Follow-up on rebuttal

Dear Reviewer Tg3H,

There are only 3 days left in the discussion period. In our response, we addressed your questions on generalization to distribution shifts. We would be grateful if you could consider revisiting your score in light of our responses. If you have other unresolved questions, please let us know – we would be happy to engage in the days that remain.

Today is the last day of discussion, so we wanted to give you a gentle reminder to see if our response addressed your comments. Thank you for your time.

Comments on the paper

5 comments

Thank you for the detailed response and the revision. Several of my concerns are resolved. Post-hoc calibration is now defined, Brier score and ECE are in the main tables, the alternative normalizations are reported, and the prompt-only contrast sets and the P(correct) margin filter address the instability concern. The prompts and code are now available. I also appreciate the clear statements that the evaluation does not uniquely identify meta-knowledge and that non-oracle training does not consistently beat its surrogate.

Two points remain, and they determine my rating.

  1. Detecting regressions. The response supports multi-checkpoint training with unbalanced contrast AUC, for example 0.806 vs. 0.642 for Marin 8B on TriviaQA. In the same setting, Δ0b\Delta_0^b is 0.227 vs. 0.221, and the end-correct baseline already reaches 0.783. So the AUC gain mostly reflects the prior that later checkpoints are more often right. Could you quote Δ0\Delta_0 on the regression set alone for SC, non-oracle training and non-oracle multi-checkpoint training, for each model and dataset, from the new appendix tables? Values clearly above 0 would show that the methods detect forgetting, not only improvements. You also report that multi-checkpoint training is significantly better in 5 of 6 cases on contrast AUC. How many of those hold on Δ0b\Delta_0^b?

  2. Headline metric. Do the abstract, the introduction and Table 1 now lead with the balanced metrics? On contrast AUC the end-correct baseline already scores 0.63–0.78, so unbalanced numbers are hard to interpret without it.

A smaller point: for MATH you use linear probes so that the estimator cannot improve simply by checking the answer itself. That is the answer-verification route from my W2, and it also applies to the LoRA adapters on factoid QA. One sentence on this in the limitations would help.

I am keeping my rating at 2 for now and will revisit it once I have seen these numbers.

Thank you for reviewing our rebuttal and acknowledging that many of your concerns have been resolved. We address your remaining points below.

1. Detecting regressions

You are right in that the contrast-set AUC metric reflects the end-correct prior. When we instead use the balanced metric Δ0b\Delta^b_0 to compare non-oracle multi-checkpoint and single-checkpoint training (Tables 17-18), multi-checkpoint training is significantly worse in 0 out of 6 cases, and significantly better in 4 out of 6 cases. Using the balanced contrast-set AUCb\text{AUC}^b, it’s significantly worse in 0 out of 6 cases, and significantly better in 5 out of 6 cases.

As you suggest, below we report regression-Δ0\Delta_0 (i.e. the subscript of Δ0b\Delta^b_0 in Tables 11, 17-18) for SC and non-oracle single/multi-checkpoint training.

ModelDatasetSCNon-oracle single-ckptNon-oracle multi-ckpt
Olmo 3 7BTriviaQA0.1870.3100.365
Olmo 3 7BJeopardy0.2410.4340.493
Marin 8BTriviaQA-0.1250.237-0.184
Marin 8BJeopardy0.2320.5900.405
Olmo 3 32BTriviaQA0.1990.2770.344
Olmo 3 32BJeopardy0.1970.2320.432

Multi-checkpoint training does not consistently improve over single-checkpoint training on regression-Δ0\Delta_0 even when it improves on the aggregate balanced metrics (Δ0b\Delta^b_0 and AUCb\text{AUC}^b). We have added this discussion to Section 3.2: “Although Δ0b\Delta^b_0 is class-balanced and does not reward the end-correct prior, Appendix B finds that increasing Δ0b\Delta^b_0 does not always increase both its components, improvement-Δ0\Delta_0 and regression-Δ0\Delta_0. Regression-Δ0\Delta_0 often lags behind improvement-Δ0\Delta_0, suggesting that detecting forgetting may be more challenging.”

2. Headline metric

We agree that the unbalanced metrics are harder to interpret since they reflect the end-correct prior. Following your suggestion, we have now updated the abstract, introduction, and results discussion (Section 3.2, paragraph “Confidence estimators struggle on contrast sets”) to lead with the balanced Δ0b\Delta^b_0 instead of contrast-set AUC. The tables also list Δ0b\Delta^b_0 as the first metric.

3. Learning answer verification

We agree that improving answer verification is one way that the LoRA adapters on factoid QA could be improving calibration. We have added this discussion to the limitations paragraph in the conclusion: “We have not determined how much of the LoRA adapters' calibration is due to gaining factual knowledge rather than meta-knowledge, although we expect that our deduplication and train-test split of factual recall (i.e. non-reasoning) questions mitigate this.”

Today is the last day of discussion, so we wanted to give you a gentle reminder to see if our response addressed your comments. Thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.