- Rating
- 2(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper show that learning transmission parameters (p,q) in Optimized Restricted Additive Schwartz algorithm can improve the convergence. ORAS is an important linear solver where preconditioning is used in each subdomains, and the control parameters between the domains are critical for optimizing the main (joined) task. The authors show that learning those parameters outperforms the minimax-tuned approach.
Critical strengths
Algorithm unrolling is a strong tool to combine the strength of iterative algorithms with learnable units. The author showed that even a simple learnable unit replacing a sub-optimimal parameter is a worthy cause. In practice they show improved results on a known and important algorithms widely used.
Critical weaknesses
Usually there is a price to pay when w learn very few parameters (2 here) in poor generalization. A small number of experiments was done in a similar setup of modified Helmholtz Equation on a square. This approach might not generalize well to other equations on one hand, or might be irrelevant at all for a general linear equation seen for the first time.
Questions for authors
Did you consider changing p and q during runtime and not keeping them fixed (non-stationary) ? Do you have additional results to share? Did you try to learn p,q from one domain and transfer to another? (show generalization)
3 comments
We noticed that the score was lowered from 3 to 2. We do not see a comment on this change and would appreciate knowing what concerns remain before the discussion period ends.
Usually there is a price to pay when w learn very few parameters (2 here) in poor generalization.
We also added an ablation on the MLP architecture (Appendix H.5). Note that UORAS does not learn two fixed parameters: an MLP predicts the two correction coefficients from the problem features and the current residual, so they adapt to each problem and iteration. The ablation includes a baseline that learns two fixed coefficients, and it shows that this baseline generalizes poorly to larger grids, whereas MLP with 2 hidden layers consistently improves upon OO2.
We wanted to follow up on our previous comment, as the discussion period ends soon. We would be grateful for feedback on our responses and the additional experiments, in particular regarding the concerns that led to the change in score.
"Did you consider changing p and q during runtime and not keeping them fixed (non-stationary)" --- Yes, this is our main variant. UORAS predicts new corrections to p and q at every iteration from the current residual. Fig.1b shows how they evolve.
"(show generalization)" Because UORAS only corrects the analytical OO2 parameters, the small number of learned parameters helps generalization. The same model transfers to grids that are more than 10 times larger than the training grids and also across different numbers of subdomains J.
“Other equations.” We added the indefinite Helmholtz equation (Section 6.2 in the revised uploaded manuscript), where UORAS reduces the iterations of OO2 by 40–60%, and diffusion with discontinuous coefficients (Appendix H.2). UORAS is not a black-box solver for linear systems: like OO2, it requires analytical optimized parameters, which are available for many PDEs of practical interest.