acceptodds
Under review as a conference paper at ICLR 2027

Calibrating Fine-Tuned Models via Selective Confidence Reuse

Abstract

Fine-tuning can improve a model's predictions without making its confidence a reliable guide to whether those predictions are correct. Calibration based only on the final confidence score cannot distinguish equally confident predictions with different probabilities of being correct, even when the model before fine-tuning retains information that separates them. We recast calibration as selective inheritance: preserve reliable information from the model before fine-tuning (the predecessor) where confidence remains stable, and repair confidence estimates elsewhere. Our method, A-PAIR, aligns predecessor scores into a reference and uses labeled source data and unlabeled target inputs to audit where this reference can be retained and where correction is needed. Our theory shows that predecessor information can recover distinctions inaccessible to score-only calibration, and that selective inheritance can reduce finite-sample estimation error when repair is localized. On public benchmarks under controlled distribution shifts, A-PAIR improves the calibration of fine-tuned language models while keeping their predictions accurate.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
calibration, fine-tuning, multicalibration, distribution shift

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dffTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies confidence calibration for answers with the help of predecessor model, labeled source data, and unlabeled target inputs. The main motivation is that scores from predecessor model may retain useful distinctions. The proposed method, A-PAIR, constructs a predecessor-derived confidence anchor, audits candidate stable regions, preserves anchor confidence where supported, and applies importance-weighted multi-calibration updates elsewhere. The method is evaluated using Qwen2.5-1.5B and its instruction-tuned version on MMLU-Pro and SciQ under controlled distribution shifts, comparing against raw confidence, CPCS, and DACA.

Critical strengths
  • The paper clearly distinguishes score-only information limits, observable audit guarantees, and the transfer assumptions required for target calibration.

  • The evaluation separates construction, auditing, and testing, demonstrates complementary predecessor information.

Critical weaknesses
  • Missing ablations leave the benefit of selective inheritance unclear. Without all-repair, anchor-only, and simple pair-aware recalibration baselines, the experiments do not isolate the contribution of selective inheritance. On MMLU-Pro, repair and TCE evaluation share subject indicators, so the improvement may reflect subgroup correction. The mixed ECE results and weaker SciQ calibration further limit evidence for a consistent benefit.
Questions for authors
  1. Can you report results for all-repair, anchor-only, and a simple importance-weighted pair-aware recalibrator on both benchmarks? Comparisons using the same source-label budget and paired features would help distinguish selective inheritance from subgroup repair and additional information access.

  2. With an all-zero initializer, the reference Brier loss equals the source correctness rate, potentially making the screen permissive in a high-accuracy setting such as SciQ. How often does the screen reject candidate iterates, and why use this reference rather than the fine-tuned confidence or anchor?

1 comment

Thank you for the rebuttal. The additional evaluations have addressed most of my original concerns. In particular, the label-budget experiments provide useful evidence that selective inheritance can improve calibration when local supervision is scarce. Overall, I find the rebuttal satisfactory and will maintain my original score of Weak Acceptance.

Reviewer_47Y9OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies confidence calibration of a fine-tuned model in a setting where its pre-fine-tuning model remains available, while only labeled source data and unlabeled, covariate-shifted target inputs can be used for calibration. The authors argue that fine-tuning can collapse samples with different correctness rates onto the same confidence; as a result, no recalibration of the fine-tuned score can separate them, whereas the predecessor’s score still can. To address this, they propose A-PAIR, which compares, for each unlabeled input, the log-odds of the two models’ confidence in the selected answer, after rescaling the predecessor’s log-odds to the fine-tuned model’s range. Where the two are close, and an audit on source labels finds the predecessor reliable, A-PAIR keeps the rescaled predecessor confidence. Elsewhere, it re-estimates confidence from source labels with multicalibration-style updates, reweighting the source data to match the target distribution. They show that when the predecessor is correct outside a small unknown region, a variant of this procedure attains excess Brier risk scaling with the size of that region rather than the whole state space, and that its holdout-based selection carries a finite-sample guarantee on an observable audit criterion. With Qwen2.5-1.5B and its instruction-tuned version, A-PAIR achieves the lowest group-wise calibration error under subject-mixture shifts on MMLU-Pro, whereas importance-weighted temperature scaling is better on SciQ.

Critical strengths
  1. The motivation is well presented. The paper explains early and concretely why score-only recalibration is not enough after fine-tuning: the example of two groups with correctness rates 0.2 and 0.8 sharing the same fine-tuned logit (Sec. 1) makes the problem easy to grasp, and Prop. 2.1 states it precisely.
  2. The theoretical analysis is rigorous and broad. The paper theoretically explains the information limit of score-only correction (Prop. 2.1), the statistical benefit of selective inheritance (Thm. 5.1), a finite-sample selection guarantee (Thm. 5.2), and the non-identifiability of target calibration without a transfer condition (Props. 5.3–5.4). Assumptions are stated explicitly, and the key proofs are correct.
Critical weaknesses
  1. Limited technical contribution. A-PAIR combines known components: the predecessor as a calibration reference (DACA, BaseCal), multicalibration boosting for repair, and importance weighting for covariate shift. Selective inheritance, the only novel design, is not well established by either theory or experiment. Most of the gain over the raw score comes from a single temperature (CPCS: TCE 0.2334 → 0.0271), and A-PAIR improves on the label-free DACA by only 0.003.
  2. Incomplete baselines and unfair comparison. Label-free baselines that directly use the pre-trained model, such as BaseCal (Tan et al., arXiv:2601.03042), are neither cited nor compared. Meanwhile, A-PAIR uses labeled source data while DACA needs no labels, yet it improves on DACA by only 0.003 TCE on MMLU-Pro, so it is unclear whether the gain comes from the method’s design or simply from access to labels.
  3. The main experiments are too narrow to support generality. All results use a single model pair, Qwen2.5-1.5B and its Instruct version, which is both small and from an earlier model generation. It is unclear whether the findings carry over to larger models, other model families, or other post-training methods.
  4. The ablation study is insufficient. The paper reports no ablation of any component, so it is unclear which part of A-PAIR drives the results.
Questions for authors
  1. Do the results hold for larger and more recent models? Results on at least one larger model and a newer model family, e.g., Qwen-3.5, would help establish generality.
  2. Could the authors provide a complete ablation study showing that each component of A-PAIR is necessary? In particular, please isolate the effect of inheritance (versus the all-repair candidate θrep\theta_{\mathrm{rep}}), the anchor map (identity vs. affine, and how often each is selected), the coverage-first selection rule, the density-ratio weighting, and the subgroup features used for repair.
  3. Is the improvement over DACA statistically significant? Please report paired per-run differences and a paired test, since all methods share the same splits and the runs are not independent.
Reviewer_mW6ROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors recast calibration as selective inheritance: preserve the reliable confidence information of the pre-fine-tuning model (the predecessor) where it remains stable under fine-tuning, and repair confidence estimates only where it does not. They propose A-PAIR, which aligns predecessor scores into a common reference and uses labeled source data together with unlabeled target inputs to audit where this reference can be retained and where correction is needed. The theory shows that predecessor information can recover distinctions that score-only calibration cannot access, and that selective inheritance can reduce finite-sample estimation error when repair is localized. On public benchmarks under controlled distribution shifts, A-PAIR improves the calibration of fine-tuned language models while preserving their predictive accuracy.

Critical strengths

The inherit or repair framing is meaningful, which explains why information from the predecessor model can help when recalibrating the final score alone is insufficient. The empirical diagnostic supports this motivation. The theory is clear about what can and cannot be guaranteed without target labels, and the separation of construction, audit, and evaluation data is a strength. I also appreciate the negative SciQ result and the discussion of when simpler score-only recalibration is enough.

Critical weaknesses

The evaluation is quite limited, with one small model pair, two multiple-choice datasets, and only synthetic shifts. More importantly, the paper lacks a direct calibrator using both models’ scores and an all-repair ablation. Without these comparisons, it is hard to tell whether selective inheritance adds value beyond simply using predecessor features. The gains are small and inconsistent, and the method loses to CPCS on several metrics for SciQ. The headline metric also closely matches the subgroup and score-bin structure that the method optimizes, making the comparisons with temperature-scaling baselines less convincing. Finally, the theory does not directly cover the implemented algorithm, and the many hyperparameters and design choices seem difficult to justify given the modest gains.

Questions for authors
  1. How does A-PAIR compare with 1. a weighted logistic or isotonic calibrator on (z_ft, z_pre), 2. multicalibration on the same subgroup class without inheritance, and 3. A-PAIR's all-repair candidate?
  2. Do the results hold for larger models, other model families, other kinds of fine-tuning, and natural (non-synthetic) distribution shifts?
  3. Do the conclusions hold under evaluation classes that differ from the repair and audit classes (e.g., held-out subgroup definitions)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.