acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty Estimation via Bayesian Decision Layer with Wasserstein Geometric Matching

Abstract

Reliable uncertainty estimation should distinguish predictive confidence from the degree of compatibility between an input and the data distributions associated with different classes. Existing approaches typically characterize uncertainty through predictive probabilities, posterior variability, or distances to class structures, but do not jointly model these aspects. Probabilistic methods do not necessarily quantify compatibility between input- and class-level distributions, whereas distance-aware methods commonly operate on point representations. To address these challenges, we introduce the Wasserstein Generalized Bayesian Decision Layer (W-GBDL), which derives uncertainty from the compatibility between an input-conditioned distribution and learnable class reference distributions. W-GBDL represents each input and each class with variational distributions and measures their compatibility using the Wasserstein distance. This geometry provides a unified basis for training and minimum-risk prediction, while the entropy of the resulting class probabilities captures predictive ambiguity. Experiments across image and language tasks show improved uncertainty estimation and calibration while maintaining competitive classification performance.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Uncertainty Estimation, Bayesian Last Layer, Wasserstein Distance, Calibration

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_BoY9OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes W-GBDL, a Bayesian last-layer method for uncertainty quantification that extends the Bayesian non-negative decision layer (BNDL). On top of a deterministic backbone, each input is mapped to a posterior over non-negative latent factors, and each class is represented by a learnable reference distribution. Both use Gamma priors and diagonal Weibull variational posteriors. The input posterior is compared to each class reference with the 2-Wasserstein distance, which has a closed form for diagonal Weibulls, and the closest class is predicted. The distances define Dirichlet concentrations and the uncertainty score is the normalized entropy of the Dirichlet mean. Training combines a Gumbel-Softmax-relaxed cross-entropy, KL terms toward the Gamma priors, and an EDL-style Dirichlet regularizer. Input-side and class-side dispersion are interpreted as aleatoric- and epistemic-related uncertainty. Experiments with ResNet-50 on CIFAR-10/100 cover accuracy, calibration, misclassification and OOD detection, and noisy inputs. Experiments with LLaMA3-8B fine-tuned on six multiple-choice QA benchmarks cover calibration. Ablations cover the posterior components, the KL terms, the distribution family and the temperature.

Critical strengths

Papers main strength are:

  • Consistent empirical gains at single-pass cost, across two modalities. OOD detection: on CIFAR-10/ResNet-50, W-GBDL has the best mean OOD AUPR among single-model methods. Its run-to-run variance is also much lower. It matches a 5-member deep ensemble (85.92) at roughly single-forward-pass inference cost, and combining it with an ensemble improves further, so it complements ensembling rather than replacing it.

  • A tractable closed-form construction. Two pieces make every training term analytic: the Wasserstein distance between diagonal Weibull distributions in closed form (Eq. 11, via 1-D quantile matching), and the analytic Weibull–Gamma KL. No Monte Carlo transport estimation or posterior sampling is needed. That makes distribution-to-distribution matching cheap enough.

This is what allows the approach to scale. It is a modest but solid technical contribution that others could reuse for distance-based uncertainty layers.

The method is a drop-in last layer that improves uncertainty estimates without sampling or multiple models, on both a vision backbone and an LLM.

Critical weaknesses

The paper argues that reliable uncertainty must distinguish predictive confidence from input–class compatibility. However, the score U_W (Eq. 15) is exactly the normalized entropy of softmax, which does not change when all distances d_ic shift by the same constant. An input uniformly far from every class reference receives the same uncertainty as one uniformly close to them, so absolute compatibility is discarded and only relative class preference remains. Similarly, tr(Σ_θ,i) cancels from U_W, and Eq. 14 is a generic identity valid for any coupling, in which dispersion enters only as a mismatch in spread. The aleatoric/epistemic interpretation in Sec. 3.4 is therefore unsupported. The paper’s conceptual contribution rests on a property the score provably lacks. The observed OOD gains are thus unexplained by the stated mechanism.

Questions for authors

1. Why Gamma priors and non-negative latents? Gamma priors and non-negative latents seem to be inherited from BNDL rather than required by W-GBDL. The likelihood is Dirichlet–Categorical, so conjugacy plays no role. W2W_2 with squared-Euclidean cost does not need positive support, and the features are already non-negative via Softplus. Could the authors explain why positivity of θ\theta and ϕ\phi is essential to the method?

2. Why not other distribution families? Several families give both closed-form W2W_2 (for diagonal factorizations) and analytic KL. Examples are diagonal Gaussians, and log-normals: with the comonotone coupling, the W2W_2 cross term is exp⁡(μ1+μ2+(σ1+σ2)2/2)\exp\big(\mu_1+\mu_2+(\sigma_1+\sigma_2)^2/2\big). Log-normals are also positive and skewed. Have the authors tried such alternatives? If Gamma–Weibull is specifically needed, could you justify why exactly ?

3. Is the Weibull only there for closed-form inference? Sec. 3.2 justifies the Weibull by its similarity to the Gamma and by the analytic Weibull–Gamma KL. Both arguments apply equally to a Gamma posterior, whose KL to a Gamma prior is also analytic and which is reparameterizable via implicit gradients. Is the Weibull chosen mainly because its closed-form quantile function gives the analytic W2W_2 in Eq. (11) ? If so, please state this explicitly.

4. Is the aleatoric/epistemic interpretation justified? Sec. 3.4 links input-side dispersion to aleatoric uncertainty and class-reference dispersion to epistemic uncertainty. Could the authors justify this formally? Several observations argue against it:

  • Eq. (14) holds for any coupling of any two distributions. In it, dispersion enters only as a mismatch in spread, so a wide input distribution matched to an equally wide reference adds no cost.
  • tr(Σθ,i)\mathrm{tr}(\Sigma_{\theta,i}) is the same for every class and cancels in UWU_W.
  • The class-side variance is global, so it cannot grow for inputs far from the training data, unlike typical epistemic uncertainty.

5. Why is the Gumbel-Softmax relaxation needed? Gumbel-Softmax is meant for backpropagating through samples of an unobserved discrete variable. In W-GBDL the label yy is observed, and the exact likelihood (Eq. (6)) is already closed-form and differentiable in all parameters. Could the authors explain why the relaxation in Eqs. (10)–(12) is needed? As written, the relaxation s a biased, noisy surrogate of the exact cross-entropy.

6 comments

We sincerely appreciate the valuable suggestions you have provided for our paper. Your feedback has played a crucial role, offering constructive and detailed suggestions for enhancing the overall framework of the paper.

Q1 & Q2: Why Gamma priors and non-negative latents? & Why not other distribution families?

Our choice follows a joint modeling and computational consideration: the latent representation should describe positive activation intensities, and the resulting distributions should support efficient regularization and distributional matching. As discussed in our responses to Reviewer D4UX W1/W3, these considerations motivate the Gamma–Weibull specification.

Non-negative latent representation. We interpret each coordinate of θi\theta_i and ϕc\phi_c as the strength of a latent feature, with values near zero corresponding to weak or inactive factors. Modeling both sides on the positive domain gives the input and class reference a consistent activation-based interpretation. Softplus and positive-support distributions serve complementary purposes: Softplus constrains the deterministic features or parameters, while the distribution family constrains the support of the random latent activations. A positive Gaussian mean, for example, does not prevent negative latent realizations. Thus, positive-support distributions are needed to preserve this interpretation throughout the probabilistic representation.

Joint choice of the prior and variational family. The Gamma prior directly regularizes positive activation intensities. In the reported experiments, Gamma(1,1)\mathrm{Gamma}(1,1) is an exponential prior whose density decreases with activation magnitude, favoring smaller intensities while permitting larger activations when supported by the supervised objective. The Weibull family provides flexible positive distributions, including densities concentrated near zero and skewed unimodal shapes. Combined with the Gamma prior, it gives an analytic KL term; its explicit quantile function additionally yields the closed-form Wasserstein cost in Eq. (11). This combination allows both posterior regularization and distributional matching to be evaluated analytically.

Differences from alternative families. A diagonal Gaussian offers analytic computation but does not preserve the positive activation support. A log-normal preserves positivity and skewness, but its density approaches zero at the origin, whereas Weibull distributions can represent substantial near-zero density through their shape parameter. This provides a distinct inductive bias for representing weak or inactive factors alongside strong activations.

We tested these alternatives in the distribution-family ablation reported in the newly added Table 9 in the Appendix, which further extends the scope of our ablation study, as discussed in our response to Q2 from Reviewer o6an. The Gamma–Weibull configuration achieves 86.24% OOD AUPR, compared with 80.58% for the Gaussian configuration and 83.14% for the log-normal configuration. These results provide empirical support for the proposed combination. Although the non-negative latent formulation builds on BNDL, W-GBDL uses the resulting distributions on both sides of an explicit Wasserstein comparison. Our choice therefore connects the activation-based representation with tractable distribution-to-distribution matching, which is the central design of our decision layer.

Q3: Is the Weibull only there for closed-form inference?

The main computational reason for choosing Weibull is its explicit quantile function, which gives the analytic Wasserstein cost in Eq. (11). We agree that similarity to Gamma and an analytic KL alone do not distinguish this choice from a Gamma posterior.

For a Weibull variable, F−1(u)=λ[−log⁡(1−u)]1/k.F^{-1}(u)=\lambda[-\log(1-u)]^{1/k}. Substituting this into the one-dimensional quantile formula and setting t=−log⁡(1−u)t=-\log(1-u) gives ∫01Fθ−1(u)Fϕ−1(u) du=λθλϕΓ ⁣(1+1kθ+1kϕ).\int_0^1F_\theta^{-1}(u)F_\phi^{-1}(u)\,du=\lambda_\theta\lambda_\phi\Gamma\!\left(1+\frac{1}{k_\theta}+\frac{1}{k_\phi}\right). Together with the second moments, this yields the closed-form transport cost for arbitrary positive Weibull shape and scale parameters.

A Gamma posterior also has an analytic Gamma--Gamma KL; therefore, KL tractability alone does not justify choosing Weibull. The remaining difficulty is the transport cross term. For Gj=Gamma⁡(aj,bj)G_j=\operatorname{Gamma}(a_j,b_j) in rate parameterization, the quantile is FGj−1(u)=P−1(aj,u)/bjF_{G_j}^{-1}(u)=\mathcal P^{-1}(a_j,u)/b_j, where P\mathcal P is the regularized lower incomplete Gamma function. The transport cross term therefore involves 1b1b2∫01P−1(a1,u)P−1(a2,u) du.\frac{1}{b_1b_2}\int_0^1\mathcal P^{-1}(a_1,u)\mathcal P^{-1}(a_2,u)\,du. For general unequal shapes, this does not simplify through the Gamma-function identity used for Weibull; direct quantile evaluation generally requires numerical inversion and quadrature. Implicit reparameterization enables differentiation through Gamma samples but does not itself provide an analytic transport cost. Special cases remain tractable: if a1=a2=aa_1=a_2=a, then W22(G1,G2)=a(a+1)(b1−1−b2−1)2.W_2^2(G_1,G_2)=a(a+1)(b_1^{-1}-b_2^{-1})^2.

Thus, Weibull provides positive latent variables together with both analytic Wasserstein matching and analytic KL regularization against the Gamma prior. We have added a concise explanation of this choice immediately after the closed-form Weibull derivation in Appendix B.1.

Q4: Is the aleatoric/epistemic interpretation justified?

The concerns raised here are consistent with the qualification in Reviewer o6an Q3, but we further clarify their mathematical implications. Equation (14) is valid for any coupling and should not be interpreted as an identifiable decomposition of uncertainty. A broad input distribution does not necessarily incur an additional spread-mismatch cost when the matched class-reference distribution has a comparable spread, although a mismatch in their means may remain.

For a fixed input, the explicit local covariance term is shared across all classes. Since the class probabilities are obtained by normalizing the class-wise transport costs, this class-independent contribution cancels from the predictive entropy. Any remaining effect of the local dispersion can only arise indirectly through the class-dependent optimal-coupling term.

The class-reference covariance is learned globally and shared across inputs. It can affect the compatibility cost for a class, but it cannot increase as a function of how far a particular test input lies from the training data. We therefore do not interpret it as an input-dependent epistemic uncertainty estimate.

These observations further clarify the distinct modeling roles of the two variational factors. The local factor is input-conditioned, whereas the class-reference factor is learned globally and shared across inputs. Their dispersions therefore influence the transport cost through different pathways, as discussed in Reviewer o6an Q3. We will make this distinction clearer in Sec. 3.4.

Q5: Why is the Gumbel-Softmax relaxation needed?

The Dirichlet–Categorical hierarchy and Gumbel–Softmax serve connected roles in our stochastic decision formulation. Following Eq. (6), we analytically marginalize the Dirichlet variable, obtaining the categorical label distribution Categorical⁡(ρˉi).\operatorname{Categorical}(\bar{\boldsymbol\rho}_i). Gumbel–Softmax then provides a continuous stochastic relaxation of this marginalized categorical decision.

During training, we draw one independent Gumbel vector per input and optimization step to estimate the expectation in Eq. (12). This objective trains the decision layer over randomly perturbed soft class assignments, with gradients propagated through their continuous relaxation. Thus, the purpose of Gumbel–Softmax is to introduce stochasticity into decision training, rather than merely to make the predictive probabilities differentiable. At test time, predictions and predictive entropy are computed directly from ρˉic=softmax⁡(−di/τ)c\bar{\rho}_{ic}=\operatorname{softmax}(-d_i/\tau)_c, without Dirichlet or Gumbel sampling.

The exact observed-label cross-entropy, −log⁡ρˉi,yi-\log\bar{\rho}_{i,y_i}, provides a direct categorical alternative. We compare this alternative with our full hierarchical stochastic formulation in the table provided in our response to Reviewer o6an's W1, removing the Dirichlet KL and Gumbel–Softmax training step together while retaining the Wasserstein matching and local/global latent regularization. The full formulation improves mean accuracy from 92.70% to 93.11%, reduces ECE from 1.58% to 1.31%, and lowers NLL from 0.25 to 0.23. These results support the joint benefit of distributional regularization and stochastic decision training. We will clarify that Eq. (12) defines this stochastic training surrogate, while Eq. (6) supplies the analytically marginalized predictive probabilities.

Reviewer_D4UXOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies uncertainty estimation simultaneously with classification. Specifically, the paper proposes Wasserstein Generalized Bayesian Decision Layer (W-GBDL), which is a probabilistic framework that treats inputs and class references as non-negative distributions and uses Wasserstein geometry for uncertainty estimation and classification. The underlying distributions for the proposed method is Gamma distribution and Weibull distribution. These give an analytic Wasserstein discrepancy, and hence sample and class-side uncertainty can be estimated.

Critical strengths

The paper gives their method with substantial information and derivation, which makes the paper more readable. The experimental results show its potential in uncertainty estimation.

Critical weaknesses

While I enjoyed reading the paper, I have several things to note here:

  1. I could not find a clear motivation or challenges that the authors want to solve. Even in the abstract, it is hard to get the main challenge that the authors consider. I could not understand what to solve by using the method. Moreover, there are many derivation and settings without providing reasons. For example, why non-negative latent variables are used? Why Gamma distribution is used for prior? So, all the derivation and the proposed method are not convincing to me.
  2. This mean-field approximation implies all variables are independent, right? Isn’t it too strong assumption? If they are independent, what is the point to measure OT distance between those?
  3. Near Eq (7), it seems that the use of Weibull distribution is just its connection to Gamma family, and analytic expression. But then, why Gaussian is not used? Guassian has easier analytic expression. It seems to me that setting distributions Gaussian make the work much easier. Are there any reason or comparision results?
  4. Dataset scale is a bit small, so it is hard to see if the method is generalizable.
Questions for authors

I have asked several questions in Weaknesses section above. Please, find the questions.

3 comments

W1: I could not find a clear motivation or challenges that the authors want to solve. I could not understand what to solve by using the method. Moreover, there are many derivation and settings without providing reasons.

Thank you for your insightful question, which gives us the opportunity to further clarify our motivation.

Motivation and the role of distributional matching. Our objective is to make distributional information on both the input and class-reference sides explicitly available to the classification decision. A conventional linear softmax head computes class scores from a point representation. Likewise, comparing only distribution means cannot distinguish representations with identical means but different spreads or shapes. W-GBDL instead represents each input and each class reference as a distribution and compares them through Wasserstein matching, allowing these distributional differences to influence class preferences.

Figure 7 in Appendix E provides a conceptual illustration of how matching information can be incorporated into a decision rule. By introducing an auxiliary unknown option, it visualizes the total probability assigned to the known classes. The localized high-score regions observed for the OT-based construction illustrate a possible use of distance-based compatibility information. This visualization concerns the extended decision rule, rather than the predictive entropy used in our main experiments.

In the main W-GBDL formulation, the negative Wasserstein costs are normalized over the known classes, and their predictive entropy summarizes the ambiguity among these distributional matches. The matching costs encode distributional discrepancies, while the final entropy reflects relative class preferences. Consequently, this score does not guarantee high uncertainty for an input far from every class reference.

Our additional ablations in table 9 directly assess the proposed matching mechanism: they retain the Weibull representations on both sides and replace Wasserstein matching with comparisons of distribution means or parameters. The improved OOD-ranking results support the usefulness of Wasserstein matching in the evaluated setting. Separately, the experiments in table 8 show that adding the tested within-class distance penalties did not improve ranking over the original objective.

Why non-negative latent factors and Gamma priors. We use non-negative latent factors because we interpret each latent coordinate as an activation intensity or additive evidence component. Positive and negative contributions cannot cancel within a coordinate, and the resulting representation has a more direct component-wise interpretation. The Gamma prior is a compatible prior for such variables because it has positive support and can represent skewed activation intensities with substantial probability near zero. For shape parameters below one, the density places even stronger mass near zero and induces a stronger shrinkage effect. In our experiments, we use the a=b=1a=b=1 setting, which corresponds to an exponential prior; thus, we present sparsity as a modeling bias supported by empirical analysis rather than as a formal guarantee.

To examine the compactness of the learned representation, we apply the same top-k dimension-retention protocol to the ResNet-50 representation and the W-GBDL representation in Figure 5. W-GBDL maintains substantially higher classification accuracy when only a small number of dimensions are retained, whereas the accuracy of the standard ResNet representation decreases more sharply. This result suggests that W-GBDL concentrates task-relevant information into fewer latent factors.

We further use Grad-CAM visualizations to examine whether the learned representation is spatially aligned with the task-relevant object in Figure 6. In the examples shown in the revised manuscript, W-GBDL produces more concentrated activation maps on the main object than the standard ResNet-50 baseline.

W2: This mean-field approximation implies all variables are independent, right? Isn’t it too strong assumption?

We thank the reviewer for raising this point. The mean-field approximation assumes independence between local and global variables under the variational distribution; it does not assert independence in the true posterior. We adopt it for tractable inference and acknowledge that it restricts the posterior dependencies our approximation can represent.

Why retain the mean-field approximation? Consider a structured alternative,qs(Θ,Φ)=qs(Φ)∏iqs(θi∣xi,Φ).q_{\mathrm{s}}(\Theta,\Phi)=q_{\mathrm{s}}(\Phi)\prod_i q_{\mathrm{s}}(\theta_i\mid x_i,\Phi). Obtaining the input-side marginal for distributional matching then requires$q_{\mathrm{s}}(\theta_i\mid x_i)=\int q_{\mathrm{s}}(\theta_i\mid x_i,\Phi)q_{\mathrm{s}}(\Phi),d\Phi.$ Even when each conditional distribution is Weibull, this marginal is generally a mixture that is no longer a single Weibull distribution. The analytic Wasserstein expression in Eq. (11) therefore no longer applies directly, and evaluating the original marginal-matching objective would generally require numerical marginalization and transport approximation.

Nor can we generally replace the marginal distance by an average of conditional distances. Writing Pi,Φ=qs(θi∣xi,Φ)P_{i,\Phi}=q_{\mathrm{s}}(\theta_i\mid x_i,\Phi), Pi=∫Pi,Φqs(Φ) dΦP_i=\int P_{i,\Phi}q_{\mathrm{s}}(\Phi)\,d\Phi, and Qc=qs(ϕc)Q_c=q_{\mathrm{s}}(\phi_c), we have W22(Pi,Qc)≤Eqs(Φ)[W22(Pi,Φ,Qc)],W_2^2(P_i,Q_c)\leq\mathbb E_{q_{\mathrm{s}}(\Phi)}[W_2^2(P_{i,\Phi},Q_c)], with equality not guaranteed. Averaging conditional costs would thus generally change the matching objective. Retaining dependence also introduces expectations over Φ\Phi into the local KL terms, which may require sampling even when the conditional KL is analytic. Our mean-field approximation directly parameterizes the input marginals and avoids these additional integrations. Structured alternatives with tractable marginals may be possible; we do not claim that mean-field is the only feasible approach.

Why is OT meaningful under this approximation? The variational joint distribution and the transport coupling serve different purposes. OT compares two given marginals through πic⋆∈arg⁡min⁡π∈Π(q(θi∣xi),q(ϕc))Eπ∥θi−ϕc∥22.\pi^\star_{ic}\in\arg\min_{\pi\in\Pi(q(\theta_i\mid x_i),q(\phi_c))}\mathbb E_\pi\|\theta_i-\phi_c\|_2^2. This auxiliary coupling is not constrained to equal the independent product of the variational factors and is not interpreted as their inferred joint posterior. Moreover, the factors are optimized jointly through the classification objective, so variational independence does not imply independent parameter optimization. We have added the marginalization argument, transport inequality, KL derivation, and expressiveness limitation to Appendix B.3.

W3: Near Eq (7), it seems that the use of Weibull distribution is just its connection to Gamma family, and analytic expression. But then, why Gaussian is not used?

The motivation for using non-negative latent factors and Gamma priors is discussed in detail in our response to W1. Here we focus on why we use a Weibull variational posterior rather than a Gaussian one.

A diagonal Gaussian is a tractable alternative and can also yield an analytic Wasserstein distance. However, it does not preserve the positive-support, skewed, and near-zero-concentrated structure motivated in W1. In particular, Gaussian variables can take negative values and allow positive and negative activations to cancel, which is inconsistent with our interpretation of the latent variables as activation intensities.

The Weibull posterior is therefore selected to remain consistent with the Gamma-based positive latent representation. In addition, its closed-form quantile function gives the analytic Wasserstein cost in Eq. (11), while its KL divergence to the Gamma prior is given in Eq. (7).To examine this choice empirically, we evaluated Gaussian and log-normal alternatives while retaining Wasserstein matching, as reported in Reviewer o6an Q2. The Gaussian and log-normal variants achieve OOD AUPRs of 80.58% and 83.14%, respectively, compared with 86.24% for the Weibull configuration. These results support the Gamma–Weibull configuration in the evaluated setting, while demonstrating that other distribution families are compatible with the framework. We have also added these ablation results to Table 9 in Appendix D.

W4: Dataset scale is a bit small, so it is hard to see if the method is generalizable.

We thank the reviewer for raising this concern. To evaluate our method on a larger and more challenging image classification benchmark, we conducted additional experiments on Tiny-ImageNet. This dataset contains 200 classes, with 100,000 training images and 10,000 validation images at a resolution of 64×6464\times64. Compared with CIFAR-10 and CIFAR-100, it provides more training examples and a larger label space, allowing us to assess performance under increased classification complexity. In this experiment, we report top-5 classification accuracy, which measures the proportion of examples whose ground-truth label appears among the five highest-probability predictions. We additionally report ECE and NLL to evaluate predictive calibration and probability quality. As shown in the table below, W-GBDL achieves a top-5 accuracy of 78.51%, an ECE of 11.28%, and an NLL of 2.02, obtaining the competitve values among the compared methods. These results provide additional empirical support for the applicability of W-GBDL beyond the CIFAR benchmarks. We will include these results and the corresponding experimental details in the revised manuscript.

MethodsACCECENLL
MC Dropout77.37±0.6577.37\pm0.6522.25±0.4322.25\pm0.432.50±0.092.50\pm0.09
BM78.32±0.0378.32\pm0.0322.25±0.3822.25\pm0.382.56±0.032.56\pm0.03
Re-EDL71.32±0.0671.32\pm0.0614.53±0.3514.53\pm0.352.05±0.002.05\pm0.00
VBLL78.10±0.2778.10\pm0.2710.46±0.28\textbf{10.46}\pm\textbf{0.28}2.54±0.012.54\pm0.01
BNDL77.54±0.2177.54\pm0.2126.88±0.4126.88\pm0.412.96±0.012.96\pm0.01
W-GBDL78.73±0.24\textbf{78.73}\pm\textbf{0.24}11.28±0.1311.28\pm0.132.02±0.02\textbf{2.02}\pm\textbf{0.02}
Reviewer_WY29OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes W-GBDL, which represents each input and class reference using diagonal Weibull distributions. Classification is performed using closed-form squared Wasserstein distances between these distributions. The training objective combines a supervised distance-based loss with Gamma-prior and Dirichlet regularization, while normalized predictive entropy serves as the uncertainty score. The authors evaluate the method on image classification, OOD detection, noise robustness, and calibration, as well as multiple-choice question answering with LLMs.

Critical strengths
  • The distributional matching is mathematically well defined and computationally tractable. Appendix B.1 explicitly states the coordinate-independence assumption and derives Eq. (11) using one-dimensional quantile matching. This allows the Wasserstein distance to be computed analytically without Monte Carlo estimation.

  • The experiments evaluate several aspects of uncertainty estimation beyond classification accuracy. Tables 1–4 cover OOD detection, calibration, and robustness to input noise, while the LLM experiments report both ECE and NLL. The appendix also includes inference-cost comparisons. These results provide useful evidence of the method's effectiveness and computational efficiency across the evaluated settings.

Critical weaknesses
  • The proposed uncertainty score does not account for absolute incompatibility with the learned class distributions. Eq. (15) computes the entropy of softmax(−d/τ)\mathrm{softmax}(-d/\tau), which is unchanged if the same constant is added to all class distances. More importantly, consider one-dimensional Weibull distributions with shape parameter 1, class scale parameters 1 and 2, and an input scale parameter RR. As RR increases, the Wasserstein distances to both classes grow without bound, but the predictive entropy approaches zero. The model can therefore assign low uncertainty to an input that is increasingly distant from every class reference. This limits the interpretation of the score as a measure of distributional compatibility, although it does not invalidate the reported empirical improvements.

  • The ablation studies do not fully isolate the contribution of Wasserstein geometry. Table 5 compares deterministic DUQ with models using one-sided or two-sided distributional representations, while Table 6 examines latent KL regularization and Table 8 compares prior–posterior families. However, none of these comparisons keeps the distributional representations and regularization fixed while replacing Wasserstein distance with an alternative matching measure. As a result, the experiments support the effectiveness of the overall framework but do not clearly establish how much of the improvement comes specifically from optimal transport.

Questions for authors
  • Does the actual OOD uncertainty score use any information about absolute Wasserstein distances or total Dirichlet concentration beyond the predictive entropy in Eq. (15)? If entropy is the only score, how does the method handle inputs that are far from all class references, as illustrated by the equal-shape Weibull example? This clarification would help determine the scope of the claimed distributional compatibility mechanism.

  • Could the authors clarify which loss terms, inference networks, and temperature settings are held fixed across the Table 5 ablations? Are there any existing experiments that compare Wasserstein distance with alternative matching measures using the same distributions and regularization? Please also provide the Gamma-prior parameters, regularization coefficients, Gumbel temperature and sampling count, and ID/OOD sampling proportions. These details would help assess the source of the improvements and the reproducibility of the reported results.

2 comments

We are grateful to the reviewers for their meticulous review and thoughtful comments. Below are our responses to the specific questions raised by the reviewers.

Q1: Does the actual OOD uncertainty score use any information about absolute Wasserstein distances or total Dirichlet concentration beyond the predictive entropy in Eq. (15)? If entropy is the only score, how does the method handle inputs that are far from all class references, as illustrated by the equal-shape Weibull example? This clarification would help determine the scope of the claimed distributional compatibility mechanism.

As we shared in our response to Reviewer o6an’s Q1, Our additional experiments suggest that absolute Wasserstein distances are less effective than relative class ambiguity for OOD ranking in the evaluated setting. To investigate this, we added an anchor loss that penalizes the distance between each training sample and its ground-truth class reference, and evaluated an energy score computed from the unnormalized class-wise distances. Across the tested anchor strengths and evaluation temperatures, the energy scores did not outperform the predictive entropy of the original model. Moreover, enforcing stronger within-class proximity generally degraded entropy-based OOD ranking. We have included this analysis and the corresponding results in Appendix C.

These findings suggest that, under our discriminative training objective, the learned distances are more useful for comparing competing class matches than for measuring absolute proximity to the training distribution. A possible explanation is that difficult or OOD inputs are often reflected in increased competition among class references, while their absolute distances are not consistently informative. This is an empirical interpretation of the observed results, rather than a guarantee for all OOD inputs.

Accordingly, our reported OOD score uses predictive entropy alone; neither absolute distances nor total Dirichlet concentration is included as an additional score. We appreciate the reviewer’s observation, which helped us clarify that Wasserstein matching supplies the distributional costs underlying class competition, while the resulting entropy measures relative predictive ambiguity. Appendix C discusses both the empirical comparison and this limitation.

Q2: Could the authors clarify which loss terms, inference networks, and temperature settings are held fixed across the Table 5 ablations? Are there any existing experiments that compare Wasserstein distance with alternative matching measures using the same distributions and regularization?

We further supplement that ablation here by isolating the effect of the matching geometry and the distribution The matching-geometry ablations use the objective in Eq.13, with the Wasserstein classification loss and all three KL terms retained. The Gamma priors use shape and rate aθ=bθ=aϕ=bϕ=1a_\theta=b_\theta=a_\phi=b_\phi=1, the variational regularization coefficients are βθ=βΦ=βρ=10−6\beta_\theta=\beta_\Phi=\beta_\rho=10^{-6}, the distance-to-probability temperature is τ=1\tau=1, and the Gumbel--Softmax temperature is τg=1\tau_{\mathrm g}=1.

For the distribution-family ablations, the Gaussian variant uses standard normal priors on both the local and global latent factors. The LogNormal variant instead uses standard normal priors in log-space, corresponding to LogNormal priors in the original positive latent space. All reported means and standard deviations use five independent seeds (1000--1004).

Gumbel noise is used only during training. Specifically, we draw one independent Gumbel vector per input and optimization step to estimate the expectation in the Wasserstein classification loss. At test time, no Gumbel variable is sampled: predictions use the deterministic marginal probabilities ρˉic=softmax(−dic/τ)\bar\rho_{ic}=\mathrm{softmax}(-d_{ic}/\tau), and the uncertainty score is their normalized entropy.

DUQ uses its original method-specific decision layer adapted to a ResNet-50 backbone. The other ablations use the ResNet-50 backbone and W-GBDL inference-network architecture, with the distributional representation or matching measure changed according to the ablation setting. We therefore distinguish the comparison with DUQ from the additional comparisons between matching measures within the W-GBDL architecture.

Specifically, as reported in our response to Reviewer o6an Q2, the additional parameter-Euclidean and mean-Euclidean baselines retain Weibull distributions on both sides of the match and replace Wasserstein matching with Euclidean matching of the distribution parameters or means. These comparisons more directly address the contribution of the matching measure.

Each OOD evaluation uses 10,000 CIFAR-10 test samples as ID data. The corresponding OOD sample counts and ID:OOD ratios are listed below. We use the ID-positive AUPR convention described in Appendix A.1 and report the sample proportions explicitly because AUPR depends on class prevalence.

OOD datasetsID samplesOOD samplesRatio
CIFAR-10010,00010,0001:1
GTSRB10,00012,6301:1.263
Food10110,00025,2501:2.525
Places36510,00036,5001:3.650
Reviewer_o6anOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes W-GBDL, a probabilistic classification layer that represents inputs and class references as Weibull distributions and measures their compatibility using Wasserstein distance. The resulting distances determine class probabilities, whose entropy is used for uncertainty estimation. Experiments on image classification and LLM fine-tuning report improvements in OOD detection, calibration, and classification accuracy over several uncertainty estimation baselines.

Critical strengths
  1. W-GBDL achieves strong results on OOD detection and calibration across image and language tasks, while maintaining competitive classification accuracy.
  2. The closed-form Wasserstein distance between diagonal Weibull distributions enables distribution-to-distribution matching without expensive numerical OT computation, with relatively low inference overhead.
Critical weaknesses
  1. The paper argues that Wasserstein-based distributional matching provides more reliable uncertainty estimates, but the connection is not sufficiently established. In particular, the final uncertainty score is simply the entropy of class probabilities obtained by applying softmax to negative Wasserstein distances. Moreover, the interpretation of local and global variational distributions as aleatoric and epistemic-related uncertainty is not adequately justified. These distributions are learned through a discriminative objective, and their dispersion does not necessarily correspond to the claimed uncertainty sources. The reported calibration and OOD detection results do not directly validate this interpretation.

The method combines several existing techniques, including Weibull variational distributions, Wasserstein matching, a Dirichlet–Categorical hierarchy, and Gumbel–Softmax. However, the necessity of these components is not clearly explained. In particular, the Dirichlet distribution appears unnecessary for obtaining the final predictive probabilities, which are equivalent to a softmax over negative Wasserstein distances. Similarly, it is unclear why Gumbel–Softmax is needed when the class probabilities are already differentiable.

  1. The introduction contains a questionable categorization of uncertainty estimation methods, particularly grouping deep ensembles and MC Dropout as deterministic approaches while discussing Bayesian neural networks separately. This mixes different methodological distinctions and does not accurately reflect the relationship between MC Dropout and approximate Bayesian inference. BNDL is used as a key reference method without sufficiently explaining its formulation or how W-GBDL differs from it. Given the apparent similarities in their local/global latent-variable structures, the paper should make the methodological differences and incremental contributions explicit. The generalized Bayesian formulation and its connection to the actual training objective are also not clearly developed. Overall, these presentation issues make the motivation, novelty, and theoretical interpretation harder to assess.
Questions for authors
  1. How does W-GBDL prevent overconfident predictions when an input is far from all class-reference distributions, given that its uncertainty is computed from normalized relative distances?

  2. Does Wasserstein matching outperform simpler alternatives, such as Euclidean distance between distribution means or parameter-matched probabilistic prototype classifiers, under the same architecture and regularization?

  3. What evidence supports interpreting the local and global variational distributions as aleatoric- and epistemic-related uncertainty, rather than simply learned latent representations?

5 comments

We thank the reviewer for their careful review and thoughtful comments. We provide responses to the reviewer’s specific questions below.

W1 & Q3: What evidence supports interpreting the local and global variational distributions as aleatoric- and epistemic-related uncertainty, rather than simply learned latent representations?

Our intention is not to identify the local and global variational distributions with aleatoric and epistemic uncertainty, respectively, or to claim an additive decomposition of predictive uncertainty. Equation (14) decomposes the Wasserstein matching cost; its covariance and optimal-coupling terms show that the effects of the two distributions are intertwined. We describe the local factor as aleatoric-related because it is conditioned on the individual input, and the global class-reference factor as epistemic-related because it is learned from the training data and shared across inputs. These are interpretations of their modeling roles, not a claim that their dispersions uniquely measure the two uncertainty types. The corruption and training-size results are consistent with the resulting predictive uncertainty responding to both settings, but do not isolate either factor's contribution. We will clarify this distinction and avoid wording that implies a separately validated decomposition.

W2:

In introduction, we did not intend to claim that deep ensembles or MC Dropout perform deterministic inference. Our intended meaning was that both methods are built on deterministic base networks: deep ensembles aggregate predictions from independently trained networks, whereas MC Dropout obtains uncertainty from different dropout-mask realizations of the underlying network.

We therefore revise “deterministic methods” to “methods built on deterministic base networks” and clarify that the stochasticity used for uncertainty estimation comes from model aggregation or dropout-mask sampling. We agree that MC Dropout has an approximate variational Bayesian interpretation, while its implementation remains distinct from methods that explicitly model a posterior distribution over network parameters [1].

Relationship to BNDL. Figure 1 illustrates the conceptual distinction between BNDL and W-GBDL. W-GBDL builds on the local/global latent-variable structure used in BNDL; we do not claim this structure itself as a new contribution. Our contribution concerns how the resulting distributions enter the decision rule. Specifically, W-GBDL directly compares the input-conditioned variational distribution q(θi∣xiq(\theta_i\mid x_i) with each class-reference distribution q(ϕc)q(\phi_c) through their squared Wasserstein distance. Within the chosen variational family, this comparison incorporates distributional shape and scale beyond differences in the means. The resulting transport costs determine both the supervised classification loss and the minimum-cost prediction rule. Thus, the distinction lies in the distribution-to-distribution matching mechanism, rather than simply in introducing stochastic local and global factors.

Connection to the training objective. Our formulation uses a distribution-dependent supervised risk together with prior regularization. The analytic Wasserstein costs in Eq. (11) determine the categorical probabilities; Eq. (12) defines the supervised risk using a Gumbel–Softmax stochastic surrogate. Equation (13) combines this risk with KL penalties that regularize the local and global variational factors toward their Gamma priors, together with an additional Dirichlet KL regularizer. We therefore use “generalized variational objective” to describe this loss-based, prior-regularized optimization over the variational distributions.

Q1: How does W-GBDL prevent overconfident predictions when an input is far from all class-reference distributions, given that its uncertainty is computed from normalized relative distances?

Our paper does not claim that this entropy alone is an absolute-distance detector or that it guarantees high uncertainty for every input that is far from all class references. The intended claim is narrower: Wasserstein matching introduces distributional compatibility costs into the decision layer, and the induced predictive entropy summarizes the relative ambiguity of those class-wise matches.

We also tested absolute-distance-based scores, but their scale was poorly calibrated and unstable across inputs in our representation, so we do not use them as the primary uncertainty score. From the perspective of the learned representation, W-GBDL learns relative class competition rather than forcing every sample to be close to a single class-reference distribution. To examine this effect, we add an anchor loss,

=\lambda_{anchor}\frac{1}{B} \sum_{i=1}^{B}d_{i,y_i},$$ which penalizes the distance between each sample and the reference distribution of its ground-truth class. We further evaluate an energy score based on the absolute class-wise distances [2]: $$E(x;T_E) =-T_E\log\sum_{c=1}^{C} \exp\!\left(\frac{-d_c(x)}{T_E}\right),$$ where $T_E$ is the energy temperature used during evaluation. The results of adding the anchor loss and evaluating the corresponding energy score for OOD detection on CIFAR-100 are shown below. All models are trained on CIFAR-10. Values in parentheses denote the temperature used for the corresponding score. When\$\lambda_{\mathrm{anchor}}=0\$, the energy score does not effectively distinguish OOD inputs from ID inputs in this setting. | $\lambda_{\mathrm{anchor}}$ | Entropy AUPR ($\tau$) | Energy AUPR ($T_E$) | | :-------------------------: | :-------------------: | :-----------------: | | 0.00 | 90.05(2) | -- | | 1e-3 | 87.28 (3) | 77.52 (0.01) | | 1e-2 | 80.45 (2) | 81.76 (0.7) | | 0.02 | 78.61 (2) | 82.96 (0.7) | | 0.05 | 79.58 (3) | 83.51 (0.7) | | 0.1 | 77.89 (5) | 80.31 (0.5) | These results indicate two effects. First, the model does not benefit from replacing relative class competition with an objective that directly compresses samples toward their class references. Second, enforcing this absolute-distance constraint can reduce the quality of uncertainty estimation and lead to stronger overconfidence. We conjecture that, during feature learning, most OOD or difficult samples are expressed primarily through increased competition among class references rather than through a uniform displacement from all reference distributions. We will clarify this modeling scope in the revised manuscript. Thus, W-GBDL is intended to quantify relative class ambiguity induced by distributional matching; it does not guarantee high uncertainty for inputs that are uniformly far from all class references.

Q2: Does Wasserstein matching outperform simpler alternatives, such as Euclidean distance between distribution means or parameter-matched probabilistic prototype classifiers, under the same architecture and regularization?

We have conducted additional ablations using two Euclidean alternatives: parameter-Euclidean, which compares the Weibull parameter vectors, and mean-Euclidean, which compares the means of the two Weibull distributions. Both alternatives retain Weibull distributions on the input and class-reference sides and use the ResNet-50 backbone with the W-GBDL inference networks. The temperature used to convert matching costs into class probabilities is fixed at τ=1\tau=1.

As shown in the table below, W-GBDL achieves an OOD AUPR of 86.24%, compared with 83.43% for parameter-Euclidean and 83.15% for mean-Euclidean, corresponding to improvements of 2.81 and 3.09 percentage points, respectively. Classification accuracy remains comparable, although parameter-Euclidean achieves the highest accuracy in this comparison. These results support the benefit of Wasserstein matching for OOD ranking in the evaluated setting, beyond representing both inputs and class references as distributions.

We also evaluated Gaussian and log-normal alternatives with Wasserstein matching. Their OOD AUPRs are 80.58% and 83.14%, respectively, compared with 86.24% for the Weibull configuration. These comparisons provide empirical support for our choice of Weibull distributions, while also showing that the framework can accommodate other distribution families.

Methodsθ\thetaΦ\PhiGeometryCls AccOOD Detect
parameter-EuclideanWeibullWeibulll2l_293.04±0.2393.04\pm0.2383.43±0.3783.43\pm0.37
mean-EuclideanWeibullWeibulll2l_293.09±0.2693.09\pm0.2683.15±0.4183.15\pm0.41
Gaussian-W2W_2GaussianGaussianW2W_292.70±0.1792.70\pm0.1780.58±0.4980.58\pm0.49
LogNormal-W2W_2LogNormalLogNormalW2W_292.92±0.1492.92\pm0.1483.14±0.3883.14\pm0.38
W-GBDLWeibullWeibullW2W_293.11±0.2193.11\pm0.2186.24±0.5286.24\pm0.52

We also synchronized this result to Table 9 of the paper, serving as a supplementary example for the ablation experiment in Table 5.

Reference:

[1] Gal, Yarin, and Zoubin Ghahramani. "Dropout as a bayesian approximation: Representing model uncertainty in deep learning." international conference on machine learning. PMLR, 2016.

[2] Liu, Weitang, et al. "Energy-based out-of-distribution detection." Advances in neural information processing systems 33 (2020): 21464-21475.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.