- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the mechanics of self-training with pseudo-labels for domain adaptation of low-resource ASR, specifically targeting the read-to-spontaneous speech shift in Hindi. It conducts an empirical study to determine how pseudo-label quality and quantity govern downstream WERs. It shows that once retained volume is controlled, downstream WER is perfectly rank-ordered by the error rate of the retained pseudo-labels, and the relationship is strongly concave. It also shows that, through a pre-registered prediction and a second iteration round, the structural ordering transfers across corpora but the linear slope does not, and that confidence filtering and iterative self-training act as substitutes rather than complements.
Critical strengths
- The empirical study to determine how pseudo-label quality and quantity govern downstream WERs is mathematically sound.
- The analysis of iteration saturation is insightful.
- Good experimental results. It demonstrates that a student trained on 10h labeled + 6.5h/30h pseudo-labeled data consistently outperforms its teacher trained on 128.5h of labeled data.
Critical weaknesses
- The scope is limited. As mentioned in the Limitations section, this study is conducted on Hindi only, adapting from a single read corpus using a single model size. While the two target pools provide some variation, it remains unclear whether the strong concavity and the specific threshold optimum generalize to other languages, different sizes of models, or different domain/acoustic conditions of the data.
- The exploration of filtering alternatives is limited. The paper mentioned soft-weighting but it only adopted hard thresholding. The comparison with hard thresholding against soft weighting or curriculum learning in the proposed framework was not conducted.
Questions for authors
- It is concluded that filtering and iteration are substitutes and advise against a second round. Since finding the optimal threshold requires evaluating multiple thresholds, how does the total compute cost of the threshold sweep compare to simply running two iterations of unfiltered self-training?