- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies how to spend a limited optimization budget before deciding whether to return an answer or abstain. There is some sort of branching procedure which searches for a solution satisfying a 'branch/event identity'. There doesn't seem to be an actual description of what the branching procedure is, or what is actually being branched on, which makes it very hard to understand the rest of the paper. Unfortunately the manuscript does not explain its central objects or experimental procedure clearly enough to understand the contribution without reconstructing it from the code, which makes this review very hard to complete.
Critical strengths
Code for experiments with reproduction instructions are provided in the supplementary material. It seems to match the parts of the text I could understand.
Critical weaknesses
There is a significant amount of terminology which is undefined and uncited in the main text. Section 3 starts with "at transition s, a solver produces a dandidate x_0 with branch identity b_0 and event identity e_0". Unfortunately at this stage in the paper I have no idea what any of these things are. The paragraph, and the next paragraph then go on to write about the decisions the 'selector' makes, which is another completely undefined term. This paper seems to be lacking a very basic section which simply describes the system we are studying.
To make matters worse the introduction does not contain a single citation, and the related work section is tiny. As such, it is not possible to work out the meaning of most of the definitions in the paper from the text. Even if it is the case that I am struggling to understand because I have not worked in this specific part of optimization research before, I should be able to at least understand the problem and find a few resources to learn the essentials of the theory from the first sections of the paper.
Because of the issues I have stated above it was very difficult to read the final sections of the paper.
Questions for authors
I have no questions about the theory because my issues with the paper are all related to presentation and description.