- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper attempts to study how to better perform optimisation of some categorical black-box function, given a limited number of function calls. This is done so by assuming the true function to be some quadratic factor plus a neural branch, which can be learned with quadratic inference and residual correction procedure. Then, the surrogate can be searched for the best candidate to try next.
Critical strengths
-
The paper seems to tackle an interesting class of optimisation problem (although from the writing it is unclear which), and there seems to be a large amount of experiments ran for the proposed method.
-
I appreciate some attempts for the theoretical analysis of the algorithm, although the writing of that makes the implication of the results somewhat unclear.
Critical weaknesses
-
I think the paper in general could be structured and/or written better.
-
The first section of the paper makes it unclear as to what problem the work is solving is and what the proposed method is. I also think that later on, the paper also do not provide adequate explanation as to why their proposed method is different from the others, and why the authors expect the proposed methods to be an improvement over existing techniques. The class of problem being studied in the paper only appears in Sec 3, when it should probably be stated earlier for clarity.
-
In the described methods (Sec 3 and 4), given the problem, it is unclear why the structure for the decomposition of g(x) is chosen as such . It is then unclear as to why the proposed new structure/method makes sense and why there would expect to be improvement over existing works.
-
I also have issues with how the results are presented. I note these points separately in 2.
While I do note that I am not an expert in quantum-based optimisation, I do still believe the paper is not written clearly enough that the information can easily be followed or inferred, at least by a researcher who do not directly work within the area (I would still expect the contributions and the proposed methods/results to be stated more clearly). I would recommend a rewriting of the paper to make the ideas and claims clearer, which I do not think can be done over the rebuttals period.
-
-
I also have some doubts regarding the results.
-
In Table 1, it would seem that Trinity-D (the proposed method) seems to not be the best performing among the benchmarks, it is unclear as to why the negative results are significant and is noteworthy regardless. If they are able to outperform some of the methods that the author cares about, it needs to be explained better why the other algorithms' performance can be disregarded.
-
Additionally, Table 1 lacks error bars. While I believe (if I understand correctly) these do exist in the appendix, the statisitical significance of the reported results are probably important enough that they should be placed in the main text ahead of some of the other content anyway.
-
It is also unclear as to why these benchmarks are chosen. There are several other black-box optimisation methods (e.g., genetic algorithms, Bayesian optimisation, etc.) that the authors could also discuss. It is unclear why their proposed method provide an advantage over these techniques, and could potentially be resolved by additional results or more discussion for the nature of the tested problem.
-
Questions for authors
-
Can the authors reiterate more clearly as to what the proposed methods of the paper is (beyond just combining existing techniques together), and why these design choices are significant? This could likely be explained through more detailed ablation of the introduced components (which may already be done in the appendix somewhere or possibly in Table 2 but is unclear as to which result would suggest these), or a better justification of the chosen design choices over existing methods.
-
Given the presented results that do not look favourable to TRINITY-D, what are the takeaways of the results that is worth it for the presented paper?