acceptodds
Withdrawn from ICLR 2027

Evaluating Text Segmentation for Representation Quality in a Low-Resource Language

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text segmentation can directly affect the quality of downstream language representations, particularly in low-resource, non-delimited languages such as Myanmar, where syllable and word boundaries do not necessarily align. We evaluate five segmentation approaches—grammar-informed rules, dictionary lattices, statistical CRFs, and two neural language models—on 1,100 verified sentences across Government, News, and Novel domains. Beyond intrinsic segmentation performance, we assess downstream effects using three multilingual subword tokenizers and three embedding models. Grammar-informed segmentation remains stable across domains (Boundary F1: 0.771–0.815), while neural language models show greater degradation on literary text, with Boundary F1 decreasing by up to 0.168. Boundary accuracy is strongly correlated with semantic embedding quality (Pearson , ), while poor segmentation increases subword sequence length by up to 20%. These results show that segmentation quality is an important factor in downstream representation quality and that explicit linguistic knowledge remains valuable for robust low-resource language processing.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text Segmentation, Representation Quality, Low-Resource Language, Benchmark, Multilingual Embeddings

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_yCoKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper compares five Myanmar word segmentation systems on 1,100 annotated sentences across Government, News, and Novel domains. It evaluates intrinsic segmentation accuracy, subword expansion under three tokenizers, and similarity between embeddings of predicted and gold segmentations under three sentence encoders. The grammar-informed mmdt-tokenizer achieves the highest Word F1 on News and Novels, while Gemini leads on Government text. Gemma exhibits a substantial Boundary F1 decline on Novels. The paper reports a strong correlation between segmentation accuracy and embedding similarity and argues for the value of linguistic knowledge under domain variation.

Critical strengths
  • A practically relevant problem. The paper studies segmentation in an underserved language where word boundaries are difficult to identify.
  • A useful comparison across domains. Evaluating five systems on the same sentences provides a consistent view of their behavior across formal, journalistic, and literary text.
  • Complementary diagnostics. Word F1, Boundary F1, token counts, and multiple tokenizers and encoders help expose differences in segmentation granularity and model sensitivity.
Critical weaknesses
  1. The incremental contribution is not sufficiently established. The relationship between segmentation and downstream performance has been studied for many years in Chinese NLP, including task-oriented segmentation for machine translation (Chang et al., 2008). Tokenizer effects on multilingual representations and downstream performance are also well established (Rust et al., 2021), as the paper acknowledges. The potential novelty therefore lies in the Myanmar-specific comparison and cross-domain findings. The paper needs to explain more clearly what these experiments establish beyond the known importance of tokenization and how the findings inform practical segmentation choices.
  2. The representation metric does not independently establish semantic quality. Sections 3.4 and 4.3.2 measure cosine similarity between embeddings of predicted and gold segmentations of the same sentence. This measures proximity to a reference preprocessing convention, but does not demonstrate better semantic discrimination or downstream performance. For mE5, predicted-segmentation similarities occupy a narrow range of 0.988–0.998, while raw text already reaches 0.990–0.994 (Table 4). The reported correlation therefore does not establish a practically meaningful semantic improvement, and the statement that it confirms segmentation “directly improves” semantic fidelity is too strong.
  3. The robustness conclusion is broader than the evidence supports. The cross-domain Boundary F1 range is 0.044 for mmdt-tokenizer and 0.049 for Gemini. Their News-to-Novel Word F1 decreases are also relatively close: 0.110 and 0.130. The large 0.168 Boundary F1 decline applies specifically to Gemma. These results support observations about the evaluated implementations more strongly than a general rules-versus-neural conclusion. Confidence intervals and source-level analyses are needed to assess the reliability of the differences.
  4. The reliability and compatibility of the reference segmentation need clarification. The supplied main paper does not sufficiently describe independent annotation agreement, adjudication, or the conventions governing compounds and grammatical particles. These details matter because the systems produce markedly different segmentation granularities, and the qualitative examples include disagreements about precisely these boundaries. The paper states that annotation guidelines are provided in supplementary materials; those resources may resolve part of this concern.
Questions for authors
  1. Which Myanmar-specific or cross-domain findings constitute the main new empirical contribution beyond prior work on tokenization? What practical preprocessing decisions do they support?
  2. How are predicted, gold, and raw texts serialized and normalized before encoding? Are character preservation, unknown-token rates, and truncation checked? What independent evidence supports interpreting cosine-to-gold similarity as semantic quality?
  3. What observations and sample size are used for the reported correlations? The coefficients can approximately be reproduced from the 15 system–domain averages in Tables 3 and 4. Can the authors clarify the dependence assumptions and provide sentence-level analyses and confidence intervals for the main comparisons?
  4. Can the authors provide the annotation guidelines, annotator qualifications, independent agreement and adjudication results, corpus sampling procedure, and confirmation that prompt demonstrations are separate from evaluation sentences?
Reviewer_vrNmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper evaluates different text segmentation methods for Myanmar language, a low-resource language written in a continuous script. Several approaches, including rule-based and dictionary-based methods, a character-level CRF model, and proprietary and open-weight language models, are evaluated in terms of segmentation accuracy and the robustness of embedding representations.

Critical strengths
  • The paper is well written and easy to follow. The only minor presentational issue is that some content is unnecessarily repeated (e.g. the description of segmentation methods and the contributions / research questions).

  • The work focuses on a truly low-research language with interesting linguistic characteristics.

Critical weaknesses
  • The paper does not seem particularly well suited to the ICLR audience in terms of its topic or novelty. It would be more suited to a workshop focusing on low-resource languages.

  • The semantic similarity experiments in Section 4.3 should be reconsidered or better motivated. Since the language models are trained to process unsegmented text, it is unclear how they behave when given segmented text as input. Based on the results, mE5 appears to be quite robust to this variation, whereas LaBSE and MiniLM produce substantially different representations in the raw and gold settings (as indicated by low cosine similarity). It is unclear which representations would perform better on downstream tasks, but I suspect that the raw (unsegmented) representations may be more effective since that matches how the model is originally trained.

Questions for authors

None

Reviewer_DFFwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper compares different word segmentation approaches for Myanmar, including grammar-based, dictionary-based, statistical and LLM-based methods. The authors evaluate them across three domains and also investigate how segmentation affects subword tokenization and sentence embeddings.

I have to admit that I'm surprised by this paper. First, the related work section is extremely limited despite the considerable amount of research on segmentation and morphological analysis in morphologically rich languages, including work on joint segmentation and syntactic analysis (just look at the work of Reut Tsarfaty, for example), or more recent work on tokenization and morphology by Yuval Pinter and others. Of course, I'm not saying these languages all present the same segmentation problems. Thai and Khmer are particularly relevant because word boundaries are not explicitly marked by whitespace as well, while Japanese shares this property but also raises interesting questions about morphology and syntactic structure, less visible in the two others. Korean and Hebrew, on the other hand, have a long tradition of work on morphological segmentation and disambiguation, often jointly with other linguistic tasks. These are obviously different linguistic settings, but many of the underlying questions are closely related: what counts as a word, how linguistic knowledge can help resolve segmentation ambiguities, and how segmentation decisions affect downstream processing. I'm thus really surprised that the authors barely engage with any of this literature, especially when one of their main conclusions is about the importance of explicit linguistic knowledge. More importantly, without these connections, I have a hard time understanding what we actually learn from this paper that we didn't already know.

I'm not really convinced by the experimental setup either, especially with the LLMs. The authors give Gemini and Gemma five linguistic rules and four annotated examples and ask them to segment the sentences. But why? Why four examples? Why these rules? Why not give the models, say, 100 raw/gold segmentation pairs and see whether they can infer the annotation conventions through in-context learning? I'm not claiming this would necessarily work better, but at least comparing rules-only, examples-only and different numbers of examples would tell us something about what the models are actually doing. As it stands, I'm not sure what conclusions we can draw from this particular prompting configuration.

I also had some trouble agreeing with the cross-domain robustness conclusions because the authors emphasized the stability of the grammar-based approach compared to neural models while Gemini exhibited almost the same variation in Boundary F1 across domains (0.049 versus 0.044). The substantial degradation is mostly observed with Gemma. So why generalize this observation to neural approaches?

Finally, and this is probably my biggest concern with the experiments, the authors claim to investigate downstream representation quality on downstream tasks, but they don't actually evaluate any downstream task. They compute cosine similarities between embeddings obtained from predicted and gold segmentations. This tells us how much the representations change, but not whether one is semantically better than the other. Actually, raw unsegmented text already achieves cosine similarities above 0.99 under multilingual-E5. I'm again having trouble seeing what the strong correlation they report really tells us.

So, while the comparison of different Myanmar segmentation systems could be useful, I have a hard time seeing the contribution beyond that. The connection to previous research is largely missing, the LLM experiments don't really isolate what they claim to investigate, and the conclusions about representation quality is largely questionable.

Critical strengths

1° covers a language that has interesting properties but is extremely understudied

2° paper is well written

Critical weaknesses

1° Almost no engagement with previous works

2° the LLM experimental protocol is limited : how can we be sure the models didn't already learn some of the language properties at training time?

3° there's no real dowstream tasks experiments and the results showed are not convincing. Depending on the metrics, we can interpret them totally differently.

Questions for authors

1° why didn't you did some real in-context learning example with a growing sample of raw and gold segmented text so we could see what the model was really able to learn by itself.

2° It seems really unlikely that myanmar segmentation could be done with a couple of rules and exception while I'm certain than the grammar-based segmenter contains much more knowledge, what was really the goal of this LLM prompting experiment then ?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.