- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes AdaMult, an optimization algorithm for constrained learning whose main feature is that it normalizes the constraint violation in the dual update step in order to become more robust to scaling issues.
Critical strengths
The paper aims to reduce the algorithm's sensitivity to the choice hyper-parameter, and therefore the amount of hyper tuning that is typically required to achieved good performance. That is a meaningful goal and achieving it would be relevant and valuable to the community.
Critical weaknesses
The paper's main weakness/obstacle is its (lack of) clarity. I have a very hard time following the narrative and methodology, and the paper is full of phrases / sentences that feel over-hyped or over-confident, such as:
- "ADAMULT’S ingredients all earn their place"
- "ADAMULT does for the multipliers what Adam did for the model: it works out of the box"
Several claims are made regarding the lack of tuning needed for the proposed method, which are only demonstrated on a small set of tasks. Given the breadth of the machine learning and optimization literature, six tasks is not enough to claim that no tuning is necessary; the paper only demonstrates that a particular choice of hyperparameters performed well on the particular set of tasks reported in the paper.
I would highly recommend that the authors
- revise the tone of the paper to only make claims that are substantiated by evidence and leave reasonable doubt elsewhere
- demonstrate the behavior and performance of the proposed algorithm on a few small nonlinear optimization instances. This would help the reader under how the algorithm performs in practice
Questions for authors
- The "related work" section is one massive blob of text which makes it hard to read and understand
- The first half of section 3 only mentions multipliers associated to inequality constraints
- The algorithm tracks the log-magnitude of the multipliers, which may cause numerical issues if the log-state becomes too large. The algorithm appears to include several clipping steps to prevent such growth, which may however prevent convergence to the true optimum.
Response to Reviewer fnex
We thank the reviewer for their feedback and for recognizing the value of reducing hyperparameter tuning in constrained learning.
W1. "A very hard time following the narrative and methodology". The paper is built around one line of argument, in four steps. It states two requirements for a dual optimizer to serve as a default (Section 2), presents a method built to meet them (Section 3), proves one proposition for each requirement (Propositions 2 and 4), and tests its two empirical claims in one table each (Tables 1 and 2). The review points to two short passages whose presentation it questions, the related-work paragraph and the first half of Section 3, which we address next.
Q1. The related work is "one massive blob of text". We will split it into three paragraphs (the origins of AdaMult's three ingredients, parameter-free optimizers, and the methods closest to AdaMult).
Q2. "The first half of section 3 only mentions multipliers associated to inequality constraints". We will say at the start of Section 3 that equality multipliers follow the same update, with sinh in place of the exponential (currently stated later, at line 185).
W1, W3. "Only make claims that are substantiated by evidence". We share this standard, and the paper already states its results with their scope where it reports them, for example in the abstract (lines 19-21), the introduction (lines 93-94), Section 4.1 (lines 450 and 463) and the conclusion (line 478). In six places, a summary of these results leaves that scope implicit, and we will make it explicit in each, in most cases by adding the qualifier "on the six tasks tested". Two are the sentences the reviewer quotes, and the other four state that AdaMult makes tuning unnecessary (lines 13-15, 23-24 and 52, and the title of Section 4.1). We therefore respectfully disagree that the paper is "full of" such sentences.
W2. "Six tasks is not enough to claim that no tuning is necessary". We agree that no finite set of tasks can show this, and the paper claims it only for the tasks it tests. In particular, it acknowledges that "no default p is guaranteed to suffice in general" (line 364).
What the paper does show has two parts. Proposition 4 proves, for every problem and under mild conditions, that rescaling the constraints leaves AdaMult's primal iterates unchanged, which removes one reason to retune. The rest is, in the paper's words, "an empirical question" (line 369), and the six tasks are a small but demanding test of it. They range from one constraint to 50,000, cover inequalities and equalities, and include supervised learning, a 7B language model and reinforcement learning. One default is feasible and competitive on all six, where no baseline held to one configuration is (Table 1).
The newly added experiments on 808 synthetic problems (next answer) also show that, indeed, many problems are not solved by AdaMult at its default. However, they support our broader claim that AdaMult can transfer better than the baselines.
W4. "Demonstrate the behavior and performance of the proposed algorithm on a few small nonlinear optimization instances". We ran AdaMult and the baselines on 808 synthetic problems, which are quadratic and exponential programs with 10 or 100 variables and inequality or equality constraints. AdaMult's default, which was selected on this family of problems, solves 45% of them, against 32% for the best baseline at its best single configuration, thus supporting our thesis that one default can transfer across problems. The full table is in our response to all reviewers, and we will add the experiments to the paper.
Q3. The clipping steps "may however prevent convergence to the true optimum". A bound that is active at the solution would indeed prevent convergence to it. We will remove the upper bound: it was only aimed at guarding against overflow, and no AdaMult run on our tasks ever came near it. However, we will keep the lower bound: it holds the normalized multiplier of an inactive constraint at e−10, which is practically zero. Under this bound, Proposition 2 still proves that a point near the true solution is locally stable, and that this point converges to the true solution as zmin→−∞. In practice, no bound will then prevent the method from converging to essentially the true solution.
We hope this addresses the reviewer's concerns, and we are happy to clarify further.