- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This article argues that in some low-cost scenarios such as hyperparameter sorting and architecture search, the optimization cost of ADAM can be computationally expensive. Therefore, the author proposed EvE, a method that combines differential evolution with gradient-based optimization. The author conducted experiments on three real network training tasks and claimed that the proposed method can reduce wall-clock time while maintaining reasonably competitive performance, although at some cost to final model quality.
Critical strengths
- The paper is well-written and well-structured.
- The motivation behind the proposed method is reasonable and the results are competitive.
Critical weaknesses
- The novelty of the author's core approach is rather limited. There have been some studies[1-3] on hybrid evolutionary algorithms and gradient optimizers. However, the author failed to clearly delineate the core contributions of the proposed method from the existing works.
[1] Xue Y, Tong Y, Neri F. An ensemble of differential evolution and Adam for training feed-forward neural networks[J]. Information Sciences, 2022, 608: 453-471.
[2] Yu Q, Liang X, Liu J, et al. Adaptive Niching-Based Gradient-Accelerated Differential Evolution for High-Dimensional Nonconvex Optimization[J]. IEEE Transactions on Neural Networks and Learning Systems, 2026.
[3] Neural DE: An Evolutionary Method Based on Differential Evolution Suitable for Neural Network Training.
-
Among the comparison methods chosen by the author, there is a lack of methods that combine with evolutionary algorithms and gradient optimizers.
-
The author did not conduct detailed ablation experiments to verify the role of each component, which makes it difficult to convince people that each component contributes to the proposed method.
Questions for authors
See weaknesses above.
3 comments
The authors have adequately addressed most of my concerns. Therefore, I have raised my score.
Dear Reviewer,
Thank you for your reply and for raising your score. Could you tell us which concerns remain? We would be glad to address them before the discussion ends on November 18.
Best regards, The Authors
Dear Reviewer,
Thank you for your careful review, and for pointing us to these three related works. Your comments have helped us strengthen the paper.
We have added Appendix I ("Relation to Hybrid Evolutionary–Gradient Methods"), which starts at the section of that name after Appendix H. In brief, it contains:
On running these three methods as baselines. We looked for reference implementations and could not locate any for EDEAdam or AdaptiveGDE in their manuscripts. For Neural DE, the authors' code is part of their general-purpose library OPTIMUS, which we did not integrate. Re-implementing these methods from their descriptions would risk comparing against a method that differs from the original, which is why we did not do so.
If you happen to have implementations of any of these three methods, we would be glad to run them under our equal-cost budget and add the comparison as another subsection of Appendix I. We would also be grateful for any pointer to code we may have missed.
The reference implementation of EvE is available at https://anonymous.4open.science/r/eve-2D59/. It is a standard
torch.optim.Optimizerand should be very easy to run; the README gives a short usage example.We hope the new appendix resolves your concerns. If it does, we would be grateful if you would consider revising your score. We are happy to answer any further questions.
Thank you again for your time and effort.
Best regards, The Authors