- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Overall, this paper mainly investigates the problem of parameter selection in GAE calculations within PPO methods. Briefly, the authors observe that previous studies have largely discussed parameter selection from the perspective of the bias-variance trade-off, without directly characterizing its impact on policy updates. This paper addresses this problems and proposes GA$^2$E; extensive experiments appear to validate/support the proposed method.
Critical strengths
I believe the problem this paper propose to address is significant (and might be interested by RL community); understanding and improving GAE within PPO is indeed a common challenge in the practical application of reinforcement learning. Furthermore, the paper presents a substantial number of experiments and results.
Critical weaknesses
This paper appears to contain numerous errors, including incorrect, fabricated, or incomplete citations, which severely undermine its credibility.
For instance, the authors of many cited works are listed simply as "Unknown"—examples include "Unknown. A closer look at deep policy gradients. In International Conference on Learning Representations, 2020." and "Unknown. Fast proximal policy optimization. In International Conference on Intelligent Computing, Lecture Notes in Networks and Systems." These appear to be AI-generated or improperly used citations.
Questions for authors
I recommended that the author should carefully check the citations in the manuscript before submission to avoid similar errors (see "Critical Weaknesses"), or alternatively, provide an explanation regarding this issues.