acceptodds
Under review as a conference paper at ICLR 2027

Evidence-Aware Robust Ensembling for Calibrated Uncertainty in Fine-Tuned Foundation Models

Abstract

As foundation models are increasingly deployed in high-stakes domains, their tendency to generate incorrect or low-quality content has become a major reliability concern. This risk is often amplified after fine-tuning for downstream tasks, where domain shift and limited task-specific data can degrade calibration and output quality. A common approach to uncertainty estimation is to sample multiple candidate responses through probabilistic decoding and use their disagreement as an uncertainty signal. From a Bayesian perspective, aggregating multiple generations can be interpreted as approximating marginalization over a posterior distribution over model parameters, whereas selecting a single maximum-probability output corresponds to a point estimate. However, fine-tuning on small or heterogeneous datasets can induce complex, flat, and potentially multimodal posteriors over model parameters, for which decoding-based sampling provides only limited and locally biased coverage, leading to miscalibrated uncertainty estimates. To address this limitation, we propose a diversity-inducing ensemble framework guided by Distributionally Robust Optimization (DRO) and evidential theory. It promotes complementary hypotheses that better cover disparate modes of the fine-tuned posterior, while inducing structured perturbations in the loss landscape to approximate Bayesian marginalization. Meanwhile, evidential modeling enables a principled decomposition of predictive uncertainty. We establish theoretical guarantees linking coverage of complementary posterior modes to improved uncertainty calibration. Experiments on benchmark datasets and large vision-language models show that our method consistently improves calibration and more accurately estimates content quality (correctness of generated outputs through confidence-based filtering), thereby enhancing the reliability of fine-tuned foundation models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Uncertainty quantification, model calibration, ensemble learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_MREpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method for improving the uncertainty calibration of large language models (LLMs) by increasing ensemble diversity through evidentially motivated sample weighting of varying strengths. The method trains multiple LoRA adapters on a shared pretrained backbone. Token logits are converted into evidential belief and vacuity scores, which are then used to determine sample weights of different strengths for each ensemble member.

At inference time, responses are sampled from the ensemble and grouped according to semantic equivalence. The resulting answer frequencies are used to determine both the final prediction and its associated confidence. The paper reports improvements in calibration and downstream task performance compared with established baselines.

Critical strengths
  1. Important practical problem. Uncertainty estimation after fine-tuning foundation models is an important practical problem for safe deployment. While prior work has focused on efficient ensemble architectures or stochastic decoding from a single model, this work proposes a plausible approach for improving calibration by explicitly increasing output diversity.

  2. Improvement over a strong baseline. The proposed method improves over a vanilla ensemble, which is a challenging and relevant baseline for uncertainty quantification.

Critical weaknesses
  1. Incremental novelty of the proposed method. The proposed approach appears closely related to existing techniques. In particular, given its similarity to focal loss, the authors should include a focal-loss ensemble baseline using the same hyperparameter-selection protocol as the proposed method. Furthermore, the idea of increasing ensemble diversity through different weightings of training samples is conceptually similar to well-established and simpler techniques such as bagging or training on random subsets of the data. Direct comparisons with these approaches would help clarify the added value of the proposed method. Additionally, the hyperparameters used in the individual experiments, as well as the procedure used to select them, are not described clearly enough. In particular, the choice of (\gamma) appears to have a substantial effect on performance according to Table 4. It therefore remains unclear how sensitive the proposed method is to hyperparameter choices across datasets and models, and whether these hyperparameters were selected using a held-out validation set.

  2. Missing directly relevant ensemble literature. The paper does not sufficiently discuss closely related work on LoRA-based ensembles. In particular, the proposed construction is related to prior LoRA ensemble methods [1], which also aim to increase ensemble diversity and can exhibit particularly strong diversity for smaller LoRA ranks relative to fully fine-tuned ensembles. A clearer discussion and empirical comparison would help position the contribution.

Reference [1] Mühlematter et al. LoRA-Ensemble: Efficient Uncertainty Modelling for Self-Attention Networks. TMLR, 2026

Questions for authors
  1. Is the same set of gamma hyperparameters used across all datasets and model sizes? How were these values selected or tuned?

  2. How does the proposed method compare with bagging and focal-loss ensembles under equal hyperparameter-tuning budgets?

  3. How does the ensemble diversity of the proposed method compare with that of competing approaches, and how strongly does diversity correlate with downstream task performance and calibration? For example, the authors could consider disagreement scores, Jensen–Shannon divergence between predictive distributions, or differences between ensemble members' learned parameters or weights.

  4. How does the method compare with a vanilla ensemble when using a larger number of ensemble members, such as 6 or 8, if computationally feasible?

Reviewer_HtqqOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper aims to calibrate the uncertainty of LLMs. Specifically, it combines the EDL together with DRO to obtain a ensemble of models to approximate the marginalization over the model's parameter space, following the classic Bayesian framework. The paper conducts theoretical analysis on loss lower bound and the ensemble's coverage in the parameter space. Empirically, the paper also validates the proposed approach on several MCQA and open-ended QA datasets.

Critical strengths
  1. EDL has a known limitation of only using a single point estimate in the model's parameter space. The paper aims to tackle this important open challenge for the domain.
  2. The paper compares a large set of inference-time baselines.
Critical weaknesses
  1. The paper is very trivial and incremental. The proposed method is more accurately characterized as a heuristic, evidentially inspired reweighting of autoregressive supervised fine-tuning losses, rather than a principled integration of classical EDL or Bayesian posterior inference. Although the method derives uncertainty scores from a ReLU transformation of model logits, these logits are optimized primarily through weighted SFT rather than a Dirichlet-aware evidential objective. So, the evidence masses are not independently established as calibrated uncertainty estimates. In addition, the closed-form weights do not solve the stated DRO inner maximization, and the deterministic ensemble obtained by varying the weighting exponent is not shown to approximate samples from a Bayesian posterior. The theoretical analysis appears to rely on identities and optimization assumptions that do not fully correspond to the implemented objective.

  2. The authors only cite the early EDL literature e.g. Sensoy et al. 2018 (and I am pretty sure that they don't understand this paper at all). The more recent EDL literature such as R-EDL, I-EDL, F-EDL etc. are omitted in the related work and comparison experiments.

  3. From the perspective of presentation, the paper's Methodology section contains a lot of contents from existing literature, making readers hard to understand which part is the original contribution of this paper. In addition, the wording in Introduction is so cumbersome and lacks focus on key claims or concepts. Furthermore, the terminology used in the paper does not adhere to conventions in Bayesian literature. For instance, "Standard multi-output methods", I guess they want to refer to sampling methods.

Questions for authors
Reviewer_bb8uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a LoRA ensemble for uncertainty calibration after fine-tuning. Each member uses a different exponent to reweight SFT losses based on evidential vacuity and incorrect belief. Predictions and confidence are obtained from semantically grouped responses, with adaptive sampling to reduce generation cost. The analysis connects the method to DRO, focal loss, and parameter separation. Experiments report improved ECE and AUROC over vanilla ensembles and decoding-based baselines.

Critical strengths
  • Practical relevance. The method addresses post-fine-tuning calibration using LoRA adapters sharing a frozen backbone (Section 3.2).

  • Consistent empirical gains. Table 1 reports lower ECE and higher AUROC than vanilla ensembles across the evaluated model–dataset combinations. Table 3 supports the contribution of evidential weighting.

  • Useful ablations and efficiency analysis. Appendices F.1–F.3 examine design choices, limited-data settings, and additional models. Table 2 demonstrates reduced generation counts with some calibration degradation.

Critical weaknesses
  • The theoretical justification does not fully cover the proposed weighting. Section 3.2 identifies Eq. 6 as a DRO surrogate, but its claimed parameterization of the uncertainty radius lacks a specified divergence and a derived γ\gamma-to-η\eta relationship. Lemma 3.1 uses adjusted vacuity, whereas Eq. 6 uses unadjusted, separately max-aggregated quantities. Consequently, applying the focal-loss argument to the actual objective requires further justification. Appendix D.2 does not supply this connection, and the lower bound in Theorem 3.2 alone does not establish improved calibration.

  • The posterior-mode interpretation exceeds what Lemma 3.3 establishes. Appendix D.3, Eq. 27, omits derivatives of the logit-dependent weights; this requires clarification of whether weights are detached or fixed. The proof also drops the Taylor remainder before stating the final distance bound, leaving its applicability to γ=50,100\gamma=50,100 unclear. Additionally, Eq. 5 permits u+binc>1u+b^{inc}>1, outside the range assumed in Appendix D.3. Even a valid parameter-distance bound would not establish distinct posterior modes. The qualitative examples in Appendix F.7 do not replace a quantitative comparison of between-member predictive diversity.

  • The experiments do not sufficiently isolate the benefit over related re-weighting methods. Table 4 shows substantial sensitivity to the exponent set, but Appendix E does not specify the validation protocol used to select it. Appendix F.1.6 does compare focal-loss variants, but only in a two-member, exponent-2 setting on ScienceQA minitest; it also reports comparable performance for the minimum-probability variant. A comparison with similarly tuned focal-loss ensembles and bagging, under matched training and inference budgets, is needed to establish the added value of the proposed weighting.

Questions for authors
  • Can you derive the claimed relationship between γ\gamma and the DRO radius, and show how Lemma 3.1/Theorem 3.2 apply to Eq. 6? Alternatively, can you explicitly limit these claims to motivation rather than guarantees for the proposed objective?

  • Are the weights detached during training, and how are u+binc>1u+b^{inc}>1 and the Taylor remainder handled in Lemma 3.3? Can you support the diversity claim with a quantitative comparison against vanilla ensembles?

  • How were the exponent sets selected, and were they shared across datasets and models? Can you provide a representative comparison with focal-loss and bagging ensembles using matched tuning, training, and inference budgets?

Reviewer_Q14aOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper propose DPO motivated diversity driven

Critical strengths
  1. Interesting method.
Critical weaknesses
  1. Some design are not grounded, e.g. check the questions below.

  2. It seems the LoRA m choose adds additional tuning probelm.

  3. Missing some baselines, e.g., could we do {1, 1, 1}? The benefits comes from duplication/ensemble and the \lambda_m selection, which should be distinguished.

Questions for authors
  1. Why use the ReLU to process the logit (any theoretical motivation?

  2. Can we directly use \mathcal{J}_m(\theta)=\frac{1}{n} \sum_i \ell_i(\theta)^{p_m}, \quad p_m \in{1,2,5} to do the emsemble.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.