- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work compares sharpness aware minimizer (SAM) to bayesian modeling methods to compare their accuracy and uncertainty quantification on classification and object detection tasks. The authors argued that bayesian methods that are global and perform full-parameter weight-space sampling are ill-posed because they either hit dimensionality dilution or functional disruption, and showed empirically that SAM and Adaptive SAM (ASAM) models are comparable to bayesian models on calibration and OOD detection.
Critical strengths
- This paper does a good job motivating single-basin method. The authors presented their experiment protocol and results clearly.
- The extensive experiment results are included in the appendix is appreciated.
Critical weaknesses
The novelty of this paper needs clarification. It is known that current bayesian methods can't sufficiently disentangle epistemic from aleatoric uncertainty (Mucsányi et al. 2024), which leads to suboptimal performance in OOD detection. It is also known that SAM and its variants can do well on calibration and OOD detection (e.g. Tan et al. 2026).
- Tan et al. 2026, TOWARDS UNDERSTANDING THE CALIBRATION BENEFITS OF SHARPNESS-AWARE MINIMIZATION, ICLR
- Mucsányi et al. 2024, Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks, NeurIPS
Questions for authors
-
The stated weight-space variance dilemma argues that bayesian methods are impractical because the injected noise is either too small to matter or too big that it will disrupt the learned parameters. However, is SAM and ASAM more theoretically principled for uncertainty tasks?
-
From the presented experiment results, SAM and ASAM are often still outperformed by deep ensembles, especially when looking at the full table across OOD shifts. Arguably one benefit of using SAM instead of deep ensembles would be the computation saving, I am curious to see how SAM would compare to other types of ensembles that are more computationally efficient? e.g. TreeNet (Lee et al. 2015, Kellner et al. 2024) and Asymmetric Duos (Zhou et al. 2025).
- Kellner et al. 2024. "Uncertainty quantification by direct propagation of shallow ensembles." Machine Learning: Science and Technology
- Lee et al. 2015, "Why m heads are better than one: Training a diverse ensemble of deep networks."
- Zhou er al 2025. "Asymmetric duos: Sidekicks improve uncertainty." NeurIPS