- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the lexical renaming problem. The authors employ the Vernier to analyze the problem. The multi-model evaluations demonstrate performance gains in open-source models like Qwen and Llama. The authors provide a useful analysis for understanding the lexical problem.
Critical strengths
- The experiments are comprehensive. The authors test multiple open-source models across different model families.
Critical weaknesses
-
The paper's presentation can be improved. For example, since the Vernier is an important technique in this study, the authors can draw a better figure to illustrate this method. The current version of Figure 1 is not very appealing, with elements too densely packed and appearing as if generated directly by an AI model. Additionally, Figure 1 could be moved earlier to page 2.
-
The contribution may need to be specified. It seems that the authors try to emphasize two aspects of the contribution: understanding and performance improvement. For example, the first two paragraphs of the introduction suggest that the authors try to understand the effects of weight updates on lexical failure. The last two paragraphs, however, show that the authors try to solve the lexical failure problem through a method. Such a writing style appears somewhat fragmented, leaving the readers confused about which contribution the author actually intends to emphasize.
-
The results are not well presented. For example, the text in Figure 2 is too small to be viewed. It is hard for readers to directly perceive the superiority of the method.
Questions for authors
Was the Vernier method originally proposed by the authors? If so, why are there some citations in lines 044 to 046 when introducing this method? If not, the authors should clearly state this.