- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the OOD detection performance degradation of EMA models in the late stages of training. It proposes ReDiv, which mitigates this degradation by adjusting the late-stage training process without requiring additional OOD data. Extensive experiments demonstrate that the proposed method achieves performance improvements across multiple datasets and post-hoc detection methods.
Critical strengths
- OOD detection is an important task for reliable model deployment.
- Investigating the late-stage OOD performance degradation of EMA models from the perspective of training dynamics is interesting.
- The proposed method requires no additional OOD data, relying solely on ID data to perform diagnosis and training intervention.
Critical weaknesses
-
Missing essential ablations: This paper lacks the ablations using SAM + EMA from scratch, as well as switching to SAM in the late stage without Rewind. Consequently, the distinct roles of Rewind and delayed SAM remains unclear.
-
Lack of ID performance evaluation: This paper does not report ID classification accuracy, making it impossible to judge whether the OOD performance improvement comes at the expense of sacrificing original ID performance.
-
Lack of cross-architecture evaluation: All experiments are conducted solely using ResNet-18, leaving it unverified whether the observed degradation phenomena and method benefits can generalize to other architectures.
-
Insufficient related work discussion and contribution positioning: This paper lacks adequate discussion on late-stage SAM [1], SAM for OOD detection [2], and weight averaging[3]. Furthermore, it fails to sufficiently clarify the distinction between this work and existing studies on EMA dynamics, making the contribution unclear.
-
Lack of computational efficiency analysis: This paper omits the computational efficiency analysis compared to standard training.
-
Missing some post-hoc methods: This paper lacks comparisons with several post-hoc methods, such as [4], and [5].
[1] Zhanpeng Zhou, Mingze Wang, Yuchen Mao, Bingrui Li, and Junchi Yan. Sharpness-aware minimization efficiently selects flatter minima late in training. In International Conference on Learning Representations, pages 20949–20980, 2025.
[2] Chi Zhang, Wei Wang, Yao Zhao, Nicu Sebe, and Yue Song. Sharpness-aware fine-tuning for ood detection. IEEE Transactions on Image Processing, 2026.
[3] Daniel Morales-Brotons, Thijs Vogels, and Hadrien Hendrikx. Exponential moving average of weights in deep learning: Dynamics and benefits. Transactions on Machine Learning Research, 2024.
[4] Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distribution detection. In The Eleventh International Conference on Learning Representations, 2023.
[5] Yue Song, Nicu Sebe, and Wei Wang. Rankfeat: Rank-1 feature removal for out-of-distribution detection. Advances in Neural Information Processing Systems, 35:17885–17898, 2022.
Questions for authors
Please see the Weaknesses.