- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents VanuatuASR, a multilingual CTC speech recognition system for four Vanuatu languages, and investigates SSL adaptation using untranscribed audio from three additional languages. The system combines continued wav2vec 2.0-style SSL adaptation of OmniASR, language-balanced joint CTC fine-tuning, and a pooled character n-gram language model used in CTC prefix-beam decoding. This paper studies joint versus language-specific fine-tuning, pooled LM decoding under reduced labeled-data regimes, and the effect of adding untranscribed speech from related languages.
Critical strengths
- This paper focuses on an underserved empirical setting for studying multilingual ASR. The corpus covers four recognition languages with only 0.82 to 11.77 training hours each, plus 33.27 hours of untranscribed recordings in three additional languages.
Critical weaknesses
-
The methodological novelty is limited at ICLR, and the broader empirical contribution is not sufficiently established. The framework combines established components: wav2vec 2.0 continued pretraining, language-balanced multilingual CTC fine-tuning, and character n-gram LM decoding. This paper does not clearly establish what new methodological or empirical insight this combination provides beyond applying standard adaptation and decoding techniques to a new language set.
-
The main performance comparison does not isolate the proposed framework’s contribution. In Table 2, VanuatuASR without LM decoding is weaker than XLS-R (37.37% versus 36.05% macro WER), and its advantage appears only after adding the LM. However, the paper does not report XLS-R with the same LM and comparable decoding settings. Consequently, the claimed 10.89% relative improvement does not establish an advantage over an equivalently configured baseline. An OmniASR + LM baseline without SSL adaptation is also needed to determine whether the additional SSL stage benefits the final system.
-
The benefit of cross-language LM pooling is not demonstrated. This paper motivates a shared LM by arguing that individual languages lack sufficient text to train reliable language-specific LMs. However, it only compares the shared LM against decoding without an LM. Without language-specific LM baselines, the results establish the usefulness of external language modeling, but do not show whether pooling text across languages provides additional benefit. This leaves a central motivation for the shared LM unverified.
-
The recommendation about data collection exceeds the experimental evidence. Section 6.4 concludes that transcribing existing recordings is more beneficial than collecting additional untranscribed speech. The supporting experiment only expands SSL adaptation from 26.08 to 59.35 hours by adding three other languages under one model configuration. This result cannot distinguish insufficient data scale, language or domain mismatch, and training choices from a genuine limitation of unlabeled data. No direct comparison of annotation and collection strategies supports the broader recommendation.
Questions for authors
The following suggestions are intended to help strengthen the paper’s soundness:
-
How does VanuatuASR compare with XLS-R + LM and OmniASR + LM without SSL adaptation, using the same LM and comparable decoding settings?
-
Does the shared multilingual LM outperform language-specific LMs, particularly for the languages with the least training data?
-
The AI use statement is rather brief and could be more informative.