- Rating
- 3
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper presents RECOB that allows benchmarking of chemical optimization through constructing datasets from >10 publications, where the data is of chemical experiment origin. The contribution includes the collated benchmarking datasets for their use in this specific paper. Evaluation protocols to compare the performance of different optimisation setups and their computational cost on these datasets.
Critical strengths
Through collating the variety of datasets, the paper's findings are more reliable, as often the best chemical optimisation approach can vary significantly by the problem of application. The protocols are comprehensive and the consideration of computational overhead is helpful.
Critical weaknesses
It is not clear to me that the assembled dataset is, or is to be made, available, which would reduce the value to the community if so. I feel the paper could be improved with data visualisation to assist in conveying key points to readers beyond the tables in the paper (and some graphs in the appendix).
Questions for authors
None
Response to Reviewer 3
We thank the reviewer for the positive assessment and these helpful suggestions.
(1) Dataset availability.
The assembled dataset and the packaged RECOB benchmark are already included in the Supplementary Materials submitted with the paper. We will also make the complete dataset and benchmark package publicly available on GitHub upon acceptance, so that other researchers can directly access and use RECOB in their own work.
(2) Data visualization.
We agree that additional visualizations would help readers better understand the benchmark and its key findings. Due to the page limit, the current manuscript relies more heavily on tables, with some additional visualizations provided in the Supplementary Materials. We will improve the visual presentation in the camera-ready version by adding more intuitive summary figures to illustrate the characteristics of the benchmark tasks and the comparative performance of different optimization algorithms.