- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The authors in this paper present a method which evaluate the inversed retrieval performances of perturbation prediction data.
Critical strengths
The experiment is easy to follow, and the figure is clear.
Critical weaknesses
The authors in this paper present a method which evaluate the inversed retrieval performances of perturbation prediction data, while the idea is interesting, I’m doubting its practical usage and the method seems very naïve.
- The authors should clarify the main contributions and application scenarios of this design. In most cases, the inverse prediction is not very helpful given the expectation of perturbation modelling. Moreover, the causal relationship is also not clear. If there is an unknown perturbation source, we know nothing about the real perturbation effect.
- The evaluated data are quite limited. The authors should consider large-scale datasets such as Tahoe and Xatlas.
- The name of the dataset is also incorrect. GEARS is a method, and it utilizes perturbation datasets from public resources.
- I also cannot find updated information on selected methods; e.g., the authors should also consider methods such as STATE for generating matching data.
- It will be better if we have more insights which can help us design better perturbation prediction or retrieval models.
Questions for authors
Please see my questions above, I am doubting the design of this question.