- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates multilingual LLM tokenizer costs using cross-lingual token-count ratios into writing system and components of the tokenizers, across 15 languages and seven tokenizers. Authors analyze the tokenization disparities and simulates fixed-size vocab reallocations.
Critical strengths
- The problem is important as the cross-lingual tokenization disparities have implications for multilingual LLM costs and context window utilization and the experiments cover the seven tokenizers and 15 languages covering five script families.
Critical weaknesses
-
This paper doesn’t establish a sufficiently novel contributions beyond applying established tokenization statistics to a selected collection of languages. Prior studies have already examined multilingual tokenization disparities, compression, and vocab allocation, and adaptation. The incremental contribution appears limited in the context of literature, weakening the chance for ICLR.
-
The paper interprets relative character counts as a writing system component and relative chars per token as a tokenizer component. Char counts in translated sentences may also reflect lexical, morphological, and grammatical differences and chars per token is influenced by the information density and linguistic properties of the underlying text, not directly by tokenizer design. These raise concerns about the validity and interpretability of the proposed attribution and affects the claims of this paper. I would encourage authors to defend these cases if possible or revise the paper to make it better for future revisions.
Questions for authors
See "Critical Weaknesses"