- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
As far as I understand, the paper tries to study the gradient-based optimization algorithms from the perspective of how to mix previous gradient informations. The paper designs an algorithm in Algorithm 1 and claims that it is a "safeguarded conditional SURE aggregate over spectral memories". The paper provides guarantee on some expected aggregated distance the author defined as "". The bounds are in terms of a term which they defined also in terms of the distance metric. It seems that the experiment sections provides experimental results on a certain setting that is hard to decode from the writing of the main text, which claims to demonstrate the benefit in terms of the "mean cumulative latent descent loss".
Critical strengths
The paper seems to be studying an important question of how to mix the history in gradient-based optimization algorithms. There is a provided algorithm that, according to the author, seems to provide interesting theoretical and empirical result.
Critical weaknesses
The writing of the paper raises significant concern regarding its clarify and soundness. To start, many notations are not defined. To name a few:
- at the beginning of Section 2.
- in Eq. (3)
- in Eq. (4), possibly due to the lacking of a parathesis.
- is defined as "Let contain the past" with no clear context of what is "the past"
Terms are used without explicit explanation or definition:
- "conditional SURE aggregate" in the intro, with no definition or citation
- "candidates", "latent signals" in the intro
- "prior" and "temperature" before Eq. (6).
Statement of Algorithm 1 is unclear:
- "update every raw memory" need specification
- "include the predictable safety action" need specification
Theorem statements are not clear: in Theorem 2, it requires "preceding assumptions" but unclear what are the "preceding assumptions".
Experimental setting is not clearly explained. In Section 5, the first sentence is "The fixed final plan comprises 272 synthetic streams in 16 cells.", but it is unclear from the context what is "final plan", "stream", or "cells" Proofs have many unjustified points, to name a few:
- Lemma B.1 "Its expected inverse radius is no larger than the centered value."
- Proof of Lemma 1 "the internal symmetric, internal skew, and external coefficients are. All are bounded by 1/smin."
The above issues are not just clarity issues. They are critical to make the paper's contribution and soundness possible to be evaluated by a human reader. From the current status of the paper, I unfortunately do not believe that the paper is possible to be evaluated in terms of the soundness or the quality. Therefore, simply replying to the example issues above is not sufficient. I recommend:
- Re-writing the intro to clearly state the problem being tackled. Make sure terms are sufficiently explained.
- Create a detailed mathematical setup and make sure that every notation is clearly defined.
- Make formal statement of the assumptions.
- Clearly write out the algorithm, without using overly vague or general statements.
- Check proofs for insufficiently justified statements.
Questions for authors
I don't think clarifying questions is sufficient to affect my assessment at this point.