- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose using analogy to propose research ideas, based on the assumption that analogical transfer will identify more feasible hypotheses. The authors define the problem as receiving the top cited papers for a target paper, then building a relational graph from the references and uses analogical induction to predict edges (relationships) for the hypothesis. The authors validate their work on IdeaBench using two models against multiple baselines, and include an ablation experiment of removing analogical steps.
Critical strengths
The experiments include ablations on the main method and multiple baselines
Critical weaknesses
The results seem significant given the size of the corpus, but the improvements are measured using only one judge model in a single trial, so it is hard to tell how much specific model selection influences the validity of these results. The method of graph extraction and prediction through analogy need to be explored further. It is unclear how this approach compares to other edge prediction methods, or how sensitive it is to the specific domain. The method does not limit itself to IdeaBench since it can always select top five citations from any other paper, so a breakdown by research field could be informative.
Questions for authors
What other test data could you include in addition to IdeaBench? How did you evaluate the relationship extraction method, how does this perform across research fields?