- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors introduce a method to estimate state-action higher-order aleatoric uncertainty from the variance of SGD iterates around an EMA anchor, with the goal of improving upon AID (Yu et al., 2026)'s global scaling, which dampens early-training overestimation.
Critical strengths
- The approach is relatively clear and straightforward.
Critical weaknesses
- The authors are eager to present their approach as the first heuristic-free method (the term is used 13 times in the paper), yet it merely shifts the heuristic of prior work to other parts of the pipeline: e.g., is a heuristic, and , , and scale the surrogate's level, just as Amini et al.'s does.
- Discrepancy between claim and theory. Theorem 1 shows that the surrogate converges to an SGD noise floor, while the target cross-data variance vanishes; i.e., DEAR does not estimate the absolute magnitude of higher-order uncertainty.
- The implemented method is not covered by the theory. The residual between EMA targets used in the method is not covered by the theory.
- The claim of state-of-the-art performance is unjustified. The authors report only three seeds and omit many baselines, such as TQC (Kuznetsov et al., 2020), REDQ (Chen et al., 2021), and SimBaV2 (Lee et al., 2025).
(Kuznetsov et al., 2020), Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics
(Chen et al., 2021), Randomized Ensembled Double Q-Learning: Learning Fast Without a Model
(Lee et al., 2025), Hyperspherical Normalization for Scalable Deep Reinforcement Learning
Questions for authors
- (36) shows that the surrogate converges to a value , while the target vanishes. Why is DEAR considered an accurate estimate of the absolute magnitude?
- How does DEAR's dependence on , , and differ from its dependence on , which the paper considers heuristic?
- How does DEAR differ from prior work such as Mandt et al. (2017) and SWAG (Maddox et al., 2019), which already estimate uncertainty from SGD iterates?
(Mandt et al., 2017), Stochastic Gradient Descent as Approximate Bayesian Inference
(Maddox et al., 2019), A Simple Baseline for Bayesian Uncertainty in Deep Learning