- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper considers the problem of quantifying predictive uncertainty in predicting the outcome y given the input object x. More specifically, the authors consider the case where there is a true hidden well-specified intended version of the input, referring to it as z*, and argue that the uncertainty in the predictive outcome given this z* consists of aleatoric and epistemic components. Consequently, the uncertainty of the predictive outcome given x has three components: aleatoric, epistemic (the same as when z* is available, plus an additional term that exists due to under-specification of the input, which we refer to as the specification term, and we propose a method to quantify the three terms. This is particularly relevant for applications where the input can be clarified.
Critical strengths
The problem of uncertainty quantification is becoming more and more important for reliable deployment of ML models, especially the widely used LLMs. The problem of uncertainty quantification given under-specified input is very interesting and quite relevant in this context. I also believe nowadays it is important to acknowledge when a paper is not (or at least does not seem to be) LLM-generated.
Critical weaknesses
- I still have a hard time digesting how the intent is rigorously defined. How can one simply define z*? 147- suggest it is the fully specified input. What does fully-specified mean? Also, from whose perspective? A user? Maybe user A has a different idea of a well-specification than user B. Is it user-independent? Then it should be defined. If it is not well-defined, how can the specification uncertainty be even quantified?
- The method is mainly motivated by the context of Large Language Models, where the user providing the input x can be requested to clarify their intent by providing additional information. However, it is not clear what this entire approach means for the case of images or audio, for which the authors provided experiments. What does the intent mean in the context of the image or audio? How can a user clarify them?
- The LLM/VLM experiments are either focused on specification or the aleatoric uncertainty! This is particularly important, as the authors claim on lines 74-76 that the other baselines approximate epistemic uncertainty based on a single model, which ignores the uncertainty about the model parameters.
- The third experiments with the audio are very underrepresented and can't be understood easily.
- What is BALD? Why should it be considered a well-known acronym in the field of UQ?
- An important yet missing related work is [1] (see section 4.2, for instance).
[1] Gruber et al., Sources of Uncertainty in Supervised Machine Learning – A Statisticians’ View
Questions for authors
- line 73: However, the outcome can be random even for a fully clarified input, so the remaining uncertainty is not necessarily epistemic.==> can authors provide an example of a fully clarified input for which the outcome is random?
- how about the discrepancy between and . What kind of uncertainty is that?
3 comments
The LLM/VLM experiments are either focused on specification or the aleatoric uncertainty! This is particularly important, as the authors claim on lines 74-76 that the other baselines approximate epistemic uncertainty based on a single model, which ignores the uncertainty about the model parameters.
Thank you for pointing to it. First, one important note. One of the main benefits of the decomposition we introduce is that the resulting measures of epistemic uncertainty are canonical, and they are already well-studied prior work [1]. For this reason, in our empirical evaluation we were mostly interested in those new measures (finegrained parts of aleatoric uncertainty), that are novel in a sense of interpretation we provide. Second, those other decompositions we mentioned (in a sense we discussed in Section 3) conflate irreducible uncertainty with epistemic one. What they refer to as “epistemic uncertainty” is “residual aleatoric” in our paper, and can be computed without ensembles, using a single model. Third, we report experiments on OOD detection (a typical proxy for epistemic uncertainty evaluation) in classification setting - see Appendix M, showing that the epistemic component of SEA is empirically valid.
[1] Malinin, A., & Gales, M. (2020). Uncertainty estimation in autoregressive structured prediction. arXiv preprint arXiv:2002.07650.
The third experiments with the audio are very underrepresented and can't be understood easily. We thank the reviewer for feedback and have clarified the presentation of the audio experiments in the final version of the paper - see Section 4.3 and Appendix P.
What is BALD? Why should it be considered a well-known acronym in the field of UQ?
We thank the reviewer for pointing this out and apologize for the use of jargon. By BALD (Bayesian Active Learning by Disagreement) [2], we refer to a classical acquisition function in active learning, derived from information-theoretic considerations. The acquisition function is the conditional mutual information between model parameters and target labels and is ubiquitously used as a measure of epistemic uncertainty in the literature. The way it is introduced also allows recovery of total and aleatoric uncertainty (and epistemic, which is defined by BALD itself). Strictly speaking, only the epistemic component is denoted as BALD in the original paper, used as an acquisition function in the active learning setup. We agree that referring to the whole decomposition as such is not particularly precise. We have made this distinction and convention explicit in the updated version. Thanks for bringing this to our attention.
[2] Houlsby, N., Huszár, F., Ghahramani, Z., & Lengyel, M. (2011). Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745.
An important yet missing related work is [1] (see section 4.2, for instance).
Thanks for bringing this paper to our attention, as we believe it is a great counterpart to our work. We agree with Gruber et al. that what we call “specification uncertainty”, and they refer to as “data-model uncertainty”, primarily affects the aleatoric component. They assume a very similar data model, and make the same assumption on conditional independence between and given . Our main difference here, is that they define aleatoric uncertainty as the variance of the distribution, and obtain a decomposition of it into expectation of variance and variance of expectation. We believe that this particular form is, in fact, a specific case of SEA decomposition, if one would assume MSE loss (proper, but not strictly proper s.r.), instead of log-loss, as we did in our experiments.
We also agree on the estimability of epistemic uncertainty with the first-order models. They claim, and we agree, that proper estimation of epistemic uncertainty requires a posterior distribution over model parameters, hence our focus on ensembling in experiments. It is important to stress that ensembles are not an essential part of SEA, but rather a particular choice of a second-order approximation, and other approximations exist.
We have therefore added the reference to the updated version of the paper, and a proper discussion on another special case of SEA.
line 73: However, the outcome can be random even for a fully clarified input, so the remaining uncertainty is not necessarily epistemic.==> can authors provide an example of a fully clarified input for which the outcome is random?
Yes, sure!
One narrow type of such questions concerns properties of the world that are, in principle, unpredictable. For example, the question "Will this radioactive atom decay within the next hour?" cannot be answered deterministically because of the inherent randomness of quantum mechanics.
A broader and more practical type of question is one whose answer is not determined by the information available at prediction time, even though the question itself is fully specified. For example, "Will it rain in London on December 1, 2026?" is unambiguous, but its answer is random given the information available today. Adding more details to the question does not change this. Weather is known to be unpredictable more than about two weeks ahead, since small errors in the measured initial state of the atmosphere grow exponentially over time [3]. Similarly, the outcome is random for questions with several valid answers (e.g., "Suggest a name for a new coffee shop"). In our decomposition, this randomness corresponds to the residual aleatoric term. Neither clarification nor model improvement can reduce it, so treating it as epistemic would suggest refining a predictor that may already be perfect.
[3] Lorenz, E. N. (2017). Deterministic nonperiodic flow. In Universality in Chaos, 2nd edition (pp. 367-378). Routledge.
how about the discrepancy between and . What kind of uncertainty is that?
Thank you for the interesting question!
As we define in line 234, our predictor is , where the expectation is taken under . Hence, any discrepancy between and results in a discrepancy between and , which is taken into account in the epistemic term of Eq. 3. We want to emphasize that this agrees with the canonical definition that the lack of knowledge about the ground-truth process is epistemic uncertainty. In contrast, in the single-model decompositions we discussed (Hou et al., 2024, Walha et al., 2026), the excess risk is not part of the decomposition (Eq. 6), so their terms do not capture this discrepancy.
We also use as a plug-in estimate of when computing the components in Eq. 4. For this part, the discrepancy is not taken into account and should instead be considered an estimation error. It affects all three estimates, particularly those of the specification and residual aleatoric terms. The coarser the plug-in estimate, the less accurate they are.
We consider x to be a (possibly lossy) representation of z∗ The true value of z∗ is defined by the data-generating process, which relates x to y through z∗. For LLMs, this process is the user, so z∗ is indeed a user-specific concept, the complete query that the user has in mind when writing x. For images and audio, z∗ is the complete image or recording. It might be that z∗=x, it might not, and the data-generating process is the only source of it. This is exactly why we do not assume that we can recover z∗ without requesting additional information, i.e., by asking for clarification. Instead, we consider an approximation of p(z∣x) by π(z∣x), and the behavior of the model under this approximation drives the decision whether we should ask for clarification. Importantly, specification uncertainty does not depend on a particular value of z∗. It depends on the distribution p(z∣x) of possible values, given x. This is natural, since different users may write the same x with different intents in mind, and p(z∣x) represents this variability. Hence, specification uncertainty is well-defined even though the intent is user-specific. Regarding the term "fully specified", in Section 3 we indeed describe the intent as a fully specified input to give an intuition about z∗. However, our derivations do not require full specification. Formally, we only require the conditional independence p(y∣x,z)=p(y∣z), i.e., once z is known, x provides no additional information about y.
In all our specification-related experiments, we are primarily interested in a setup, where x is the transformation of z∗ which carries less information about the label y. From this point of view, our audio and image experiments are the continuation of what we did for texts. In the text-to-text domain, we model x by either removing parts of the context related to the question being asked (z∗) or by using the existing ambiguous questions (AmbigQA dataset). We extend this to the multimodal setup. We still use an LLM to answer the questions, but the context is either an image or an audio recording. In the former case, we model lack of information by occluding part of the image with a mask, and in the latter case we mask part of the recording with noise. One could come up with other cases where an input of these kinds can be considered underspecified (in the sense, that it might be possible to improve upon it): a blurry image, an image that crops out the important object, a low-res image with crucial details which are impossible to distinguish. In this case an LLM might ask the user to retake the picture, take a close-up image etc.