- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper presents a framework for the unrolling of optimization methods for solving min-max and max-min problems using majorization-majorization. The authors claim that their approach allows to swap between min-max and max-min formulations of the MM problem without loss of accuracy, which allows the practitioner to pick whichever is easier computationally. They provide a guarantee of objective decrease. The paper includes experiments which show a significant improvement in solution accuracy on several problem instances.
Critical strengths
- The idea of using majorization-minorization in the unrolling of optimization methods for min-max and max-min problems is novel and well-motivated.
- The numerical experiments are realistic, and the proposed method shows a significant gain in performance against the baselines. The code to reproduce the experiments is provided in the supplementary material and appears sound, with the baseline (competing) methods implemented faithfully.
Critical weaknesses
-
The overall readability of the paper requires significant improvement; as it is, the paper is difficult to parse due to an abundance of flair and imprecise wording; refer to Questions for Authors for main comments.
-
Some of the mathematical derivations in the proofs are questionable, and sometimes contradict the statement of the theorems. It is possible that the theoretical results are not wrong, but are still hard to reproduce due to poor readability of the paper (see point above). Again, refer to Questions for Authors.
Questions for authors
Mathematical results
The core theorem of the paper is Theorem 1, which is necessary for the other theoretical results of the paper. As stated, It assumes either $ \mathcal{X} $ is compact ("hypothesis (i)"), or $ \mathcal{Z} $ compact and $ g_s(\cdot, \mathbf{z} \mid \mathbf{x}^t) $ has bounded sublevel sets for some $ \mathbf{z} \in \mathcal{Z} $ ("hypothesis (ii)"). The proof is presented in Appendix C.2. However:
- The section of the proof corresponding to hypothesis (i) requires both $ \mathcal{X} $ and $ \mathcal{Z} $ to be compact. Could the authors clarify this?
- In the section of the proof corresponding to hypothesis (ii), the last step in the chain of derivations
is unclear. Could the authors explain how is justified (and, relatedly, which set does $ \mathbf{x}' $ live in here)?
- Related to 1: the problem statement in Section 2 does not mention a requirement on compactness of either of the two variable sets, but it is required in Theorem 1.
Readability
As stated in Weaknesses, the readability of the paper needs significant improvement. The following are the major points which, in my opinion, need to be addressed the most to this end, in order as they come up in the paper:
- The statement "No single objective has a saddle point for its minimizer" is broad and poorly worded. Given that the paper repeatedly refers to "eliminating the saddle point" as one of its core contributions, the statement should be replaced with a more precise one.
- In my opinion, the introduction and contributions sections could, and should, be condensed significantly without losing meaning.
- The restriction of the class of functions $ f $ on line 165 reads as a restriction of the entire class of functions for which the method described in the paper is applicable, and not a particular application. Similar issues of conflating the general problem statement and the worst-case-loss instance appear throughout the work. The paper would benefit from a more precise separation of the two.
Related works
The authors compare their proposed method to the Learned Primal-Dual of Adler & Öktem (2018). Although they do not employ majorization-minorization, the methods superficially appear similar. In my opinion, the paper would benefit from a more detailed discussion of Learned Primal-Dual. Particularly, I would be interested to see how the RUN-D network differs from Unrolled GDA in the setup of the logistic loss experiment, if first-order gradient method was chosen instead of Newton, and, futhermore, if closed-form solutions or specialized solvers are unavailable in neither "window".
Reviewer's Conclusion
The contribution that the paper claims - relaxation-free unrolling of min-max/max-min optimization using majorization-minorization - is non-trivial and would warrant publication. The experimental results show clear advantage over state-of-the-art on realistic instances. However, given the points outlined above, I am hesitant whether the method's theoretical guarantees indeed hold up; moreover, the presentation of needs significant work. I would be willing to increase my score if:
- The presentation is vastly improved.
- The authors address the questions regarding the mathematical results and the positioning of the paper.