- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper under review presents a benchmark for inverse design tasks and elaborate on the methodology of evaluation. The inverse design algorithms are invertible neural networks that have recently been published. The evaluation protocol unifies performance assessment for data driven inverse design over six data sets retrieved from recent publications. Variations in the amount of data and generated designs are presented. The multi-criteria evaluation contains inference speed along with the precision of predivtions.
Critical strengths
The inverse design problem is well picked and of broad interest. Working towards a unified evaluation protocol for such tasks is interesting.
Critical weaknesses
My main objection against the paper is its very limited contribution and zero novelty. Essentially, the paper only collects data sets and models from the literature and runs some evaluation. There are no deep thoughts how to expand the evaluation protocol beyond this. There are no real mistakes of flaws in this paper and the intention is ok, but just running some evaluation protocol with a few models and a few data set is very clearly miles away from what should be expected from an ICLR publication.
Also, one of the two metrics employed - the inference speed - is largely irrelevant as all these models are so small that one can use all of them in interactive mode. There is therefore not much value in this metric.
Looking into the contribution sections as written by the author, only the first point is a very limited real contribution. Already the second bullet is not of interest and the argument that the iMF is very recent and therefore has not been tested before can't really count. The third novelty is plotting a Pareto plot with - as I discussed above - with one largely irrelevant metric.
To be honest, I don't understand why this paper has been submitted here. This leaves me with high perplexity.
Questions for authors
I see no real way how my opinion on this paper can be changed, therefore I don't have any question.
Dear Reviewers,
We want to thank you for the time and care you put into reviewing our work and for giving us your feedback. We have decided to withdraw this submission and focus on strengthening our paper. Your comments and suggestions will help guide our revisions in order to prepare a future submission.
We appreciate your thoughtful engagement with our work.