- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes POP, a prior-fitted first-order optimizer that learns to adapt step sizes for continuous optimization. The authors formulate the optimization process as a sequential decision-making problem and train a Transformer-based policy using PPO on synthetic functions sampled from a compositional prior. The learned policy predicts coordinate-wise learning rates based on past optimization steps while following the negative gradient direction. POP is trained on two-dimensional synthetic functions and tested on 43 benchmark functions, higher-dimensional problems, and real-world optimization tasks to examine its generalization to unseen problems.
Critical strengths
-
The paper addresses an important challenge in learning to optimize: the generalization of learned optimizers to unseen optimization tasks. Training the optimizer exclusively on synthetic functions, without relying on downstream-specific training data, provides an interesting perspective on learning transferable optimization strategies.
-
The experimental evaluation provides encouraging evidence of zero-shot generalization, particularly from two-dimensional training problems to 43 benchmark functions and selected real-world optimization tasks. The inclusion of comparisons with conventional optimizers and ablation studies on the synthetic prior further supports the empirical findings.
Critical weaknesses
-
The novelty of the method is limited. Learning optimization strategies or adaptive learning rates from past optimization steps has already been studied (e.g., Wichrowska et al., 2017; Shu et al., 2020). Although POP only learns positive step sizes for each coordinate, it is not clear how this design improves upon existing methods.
-
The experiments mainly focus on low-dimensional problems. The model is trained only on 2D functions, and the higher-dimensional tests only go up to 32D. It would be useful to see how POP performs on larger problems, especially when there are strong interactions between variables.
Questions for authors
-
What are the main differences between POP and existing learned optimizers (e.g., Wichrowska et al., 2017; Shu et al., 2020)? What are the specific benefits of the proposed design?
-
Why is the comparison between POP and Joint POP limited to 2D problems? Would the same conclusion hold for higher-dimensional, non-separable functions?
-
Why are the higher-dimensional experiments limited to 32D? How would POP perform on 100D or 1000D problems?
-
In Eq. (4), POP only adjusts the step size along the negative gradient direction. How does it escape local minima as shown in Figure 1? If the gradient becomes zero, changing the step size will not help. Is POP able to escape local minima simply because it takes larger steps before getting stuck?