- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies prefix-sum variational inequalities. The authors propose CSVRG, which extends the work of (Mavrothalassitis et al., 2024) in the minimization setting to the VI setting. Further, they also provide an analysis for an extragradient inspired variant CSVRE. Some numerical experiments showcase the performance of the proposed methods.
Critical strengths
To the best of my knowledge, the proofs contain no major errors and the obtained results under the stated assumptions make sense.
Critical weaknesses
The main (critical) weakness is that the assumptions are too restrictive: compact domain as well as uniformly bounded gradients. In this setting, we could just use standard stochastic methods periodically since the distance between solutions of the prefix-sum problems goes down as shown in Lemma C.1. Doing this eliminates the (linear) dependence on , which is called SGD-sparse in (Mavrothalassitis et al., 2024) . This also makes the connection to continual learning more difficult to interpret. In particular, if after some point, the solutions do not change much, then learning has effectively stopped.
Minor comments:
- In the abstract line 15, should be . Similar in Definitions 3.1-3.2 on line 172
- Line 459: Nemirovski (2004) and Korpelevich (1976) are not valid citations for SGDA
- Line 588: The book of Nesterov is single-author, now the author of this citation reads Nesterov et al.
- Expectations should be handled more rigorously. For example, in Lemma 5.2, it is now unclear over what the expectation in the right-hand side is (since is arbitrary). It would be useful to define the appropriate filtrations.
- The setup of the experiments (especially for the standard methods SGD, SVRG, etc..) is not entirely clear to me. How are these adapted to the prefix-sum problem. Could you elaborate on this? Currently, L-SVRE achieves already quite good results at 0 IO calls in Figure 2. In general, I do not agree with “clear and consistent superiority” on larger datasets. It would be beneficial to include an SGD-sparse like method also.
- The step on line 841 requires a sufficiently large iteration number. In particular, .
Questions for authors
See weaknesses.