- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper achieves local structure retrieval via the proposed 2-stage training and a coarse-to-fine retriever. The method is evaluated under both 0-shot and 1-shot.
Critical strengths
- Combination of this 2-stage training with the 0-shot and 1-shot settings makes sense.
- The improvements brought by the 2 stages are reasonably demonstrated.
Critical weaknesses
- i'm concerned about the coarse in retriever. How sensitive is the final performance to this candidate-pool size? since once missed at this stage, it can never get back. It would be good to report results with different candidate-pool size,for example, top-20/50/100 after coarse retrieval.
- Though the proposed method achieves the best results, the final performance is still far from perfect. Could the authors provide a systematic failure analysis to clarify where it's from?
- I think the presentation of the method could be clearer. The relationship between pretraining, one-shot fine-tuning, 0-shot/1-shot evaluation does take some effort to understand. It may be clearer to directly show them in fig. 2.
Questions for authors
- There are global citation formatting issues, ie, missing parentheses or spacing.
- How well does the method generalize to local structures that are unseen during pretraining, rather than downstream?