acceptodds
Under review as a conference paper at ICLR 2027

VanuatuASR: Open Multilingual Speech Recognition for Low-Resource Vanuatu Languages

Abstract

Multilingual automatic speech recognition (ASR) remains inaccessible to many languages that lack transcribed speech, digital text, and representation in large-scale pretraining corpora. We present VanuatuASR, to our knowledge the first unified multilingual ASR system for Vanuatu, a highly linguistically diverse region where speech and text resources are severely limited. Building on speech curated across seven Vanuatu languages totaling 65.92 hours, we investigate three practical questions: whether joint multilingual fine-tuning can match language-specific recognizers; whether a shared language model trained on pooled transcripts improves decoding when per-language text is too scarce to support individual models; and whether adding untranscribed speech from additional languages benefits self-supervised representation learning. Our system combines SSL on pooled unlabeled speech, language-balanced joint CTC fine-tuning on four transcribed languages, and shared character -gram LM decoding without language identifiers. Our results show that joint training matches four language-specific models with a single recognizer. The shared LM is the dominant source of improvement, reducing macro WER from 37.37% to 32.12% and remaining effective with as little as 25% of transcribed training data. Expanding the SSL pool with untranscribed speech from three additional languages, however, yields no measurable gain. VanuatuASR outperforms other state-of-the-art baselines under a common evaluation protocol, reducing macro WER by 10.89% relative to the strongest baseline. These results indicate that in severely low-resource settings, transcribing existing recordings and pooling scarce textual resources across languages delivers more value than collecting additional unlabeled speech.

Primary area:
datasets and benchmarks
Keywords:
Multilingual Automatic Speech Recognition, Low-Resource Languages

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PJN3OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents VanuatuASR, a multilingual CTC speech recognition system for four Vanuatu languages, and investigates SSL adaptation using untranscribed audio from three additional languages. The system combines continued wav2vec 2.0-style SSL adaptation of OmniASR, language-balanced joint CTC fine-tuning, and a pooled character n-gram language model used in CTC prefix-beam decoding. This paper studies joint versus language-specific fine-tuning, pooled LM decoding under reduced labeled-data regimes, and the effect of adding untranscribed speech from related languages.

Critical strengths
  • This paper focuses on an underserved empirical setting for studying multilingual ASR. The corpus covers four recognition languages with only 0.82 to 11.77 training hours each, plus 33.27 hours of untranscribed recordings in three additional languages.
Critical weaknesses
  • The methodological novelty is limited at ICLR, and the broader empirical contribution is not sufficiently established. The framework combines established components: wav2vec 2.0 continued pretraining, language-balanced multilingual CTC fine-tuning, and character n-gram LM decoding. This paper does not clearly establish what new methodological or empirical insight this combination provides beyond applying standard adaptation and decoding techniques to a new language set.

  • The main performance comparison does not isolate the proposed framework’s contribution. In Table 2, VanuatuASR without LM decoding is weaker than XLS-R (37.37% versus 36.05% macro WER), and its advantage appears only after adding the LM. However, the paper does not report XLS-R with the same LM and comparable decoding settings. Consequently, the claimed 10.89% relative improvement does not establish an advantage over an equivalently configured baseline. An OmniASR + LM baseline without SSL adaptation is also needed to determine whether the additional SSL stage benefits the final system.

  • The benefit of cross-language LM pooling is not demonstrated. This paper motivates a shared LM by arguing that individual languages lack sufficient text to train reliable language-specific LMs. However, it only compares the shared LM against decoding without an LM. Without language-specific LM baselines, the results establish the usefulness of external language modeling, but do not show whether pooling text across languages provides additional benefit. This leaves a central motivation for the shared LM unverified.

  • The recommendation about data collection exceeds the experimental evidence. Section 6.4 concludes that transcribing existing recordings is more beneficial than collecting additional untranscribed speech. The supporting experiment only expands SSL adaptation from 26.08 to 59.35 hours by adding three other languages under one model configuration. This result cannot distinguish insufficient data scale, language or domain mismatch, and training choices from a genuine limitation of unlabeled data. No direct comparison of annotation and collection strategies supports the broader recommendation.

Questions for authors

The following suggestions are intended to help strengthen the paper’s soundness:

  • How does VanuatuASR compare with XLS-R + LM and OmniASR + LM without SSL adaptation, using the same LM and comparable decoding settings?

  • Does the shared multilingual LM outperform language-specific LMs, particularly for the languages with the least training data?

  • The AI use statement is rather brief and could be more informative.

Reviewer_83HvOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

Authors prepare a multilingual ASR model for Vanuatu languages using a mixture of low-resource methods (Finetuning, SSL, LLM calibration). Further, authors perform studies of the effects of scaling these approaches in the low-resource setting to assist with further attempts for ASR adaptation in low-resource scenarios.

Critical strengths

Low-resource language support remains an open topic on discussion of ASR. Focus of the paper on languages outside typical domain of the task is a much needed expansion in the field. Taken in combination with development of a novel corpus - assuming public access - means the given work provides a healthy contribution to the field of speech representation. Further, authors' work to explore the viability of low-resource modeling approaches is greatly appreciated as developers for low-resource languages frequently struggle with how to optimize compute budget for training. In total, given thrust of work is in a sorely neglected area and vital for development of subfield of speech AI.

Critical weaknesses

While the focus of the work is healthy for the field, the findings of the paper are not particularly significant given the context of the conference. The approaches used for low-resource speech recognition (LLM, fine-tuning, SSL) are de facto approaches for speech modeling when data availability is scarce (indeed the original wav2vec 2.0 paper achieved significance results with a fraction of the authors' data). This is particularly summed up in their claim that: " For communities deciding how to allocate limited annotation resources, this offers a concrete recommendation: transcribing existing recordings yields greater benefit than collecting additional untranscribed speech." This is a well known recommendation in the field of speech-recognition and there are very few cases where it would not serve as a null argument.

As such, the methodology used in the paper is not particularly novel, mainly their approach/domain is. Were this a setting particularly concerned with speech/low-resource languages, this would not be an issue. But for the context of ICLR - exploring innovations in representations and their applications - the lack of methodological novelty makes this a poor fit for the conference. It is suggested the authors submit this valuable work to a more fitting venue. In doing so, their work would definitely benefit from greater focus on the data acquisition pipeline, as it addresses the looming question of speech data acquisition in tandem.

Questions for authors

What processes were used for the data harvesting pipeline?

Why focus on wav2vec 2.0 given the advent of HuBERT, w2v-BERT, wavLM, pseudolableing, and so on?

Why limit SSL training with Vanuatu languages? Theoretically the shared topological feature set would benefit from inclusion of other pacific languages.

Why CTC architecture? Were there not attempts with transformer and RNN-T decoder systems? The latter would benefit from a supposed 'intrinsic' LLM and allow further use of text data to be applied.

Reviewer_Af2bOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents dataset + multilingual models for ASR of various Vanuatu languages. Various pretrained models are adapted to those languages. The best performing model is derived from a CTC based omnilingual ASR model where the encoder is adapted via SSL on untranscribed speech and then further adapted via CTC objective on the transcriptions. Various ablations are performed: joint finetuning vs separate finetuning, effect of char LM with reduced supervision, scaling SSL adaptation to include additional Vanuatu languages without transcription. The dataset and models are planned to be released.

Critical strengths
  • the curated dataset seems valuable to the community, especially researchers interested in low resource language research
  • the experimentation, including ablation, is well designed
Critical weaknesses
  • there are gaps in the experimentation: none of the proposed models are leveraging LLM as a backbone (e.g. omnilingual llm-asr) and the character n-gram language model is only utilizing the target Vanuatu transcripts and not leveraging any other multilingual text.
  • there is an issue with the paper being presented as a model paper. If that is the case, then there is limited novelty since all the methods (SSL, CTC, multilingual finetuning vs monolingual finetuning) are well known. An alternative is to present the paper as a resource + benchmark for Vanuatu paper. In that case, a more thorough comparison of existing ASR models could be warranted.
Questions for authors

286: "For the transcribed speech, we use 80% for pretraining": do you mean training instead of pretraining? What does pretraining refer to?

Some experiments are meant to test zero-shot performance of ASR system. However, omnilingual ASR actually supports Bislama. Could you explain further this point?

348: "VanuatuASR achieves the best overall performance." Did you check that the baselines also use LM rescoring? (otherwise, it would not be an entirely fair comparison)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.