PronCoach: Perceive, Reason, and Correct Pronunciation
Abstract
Pronunciation correction aims to improve speech clarity through realistic pronunciation recognition and targeted pronunciation guidance. Reliable pronunciation coaching faces three challenges: (1) languages represent pronunciation differently; (2) fine-grained pronunciation-correction data and phone-level annotations are scarce, while ambiguity in phone perception leads to unstable expert annotations; and (3) feedback must account for weak forms, tone changes, accent variation, and other context-dependent phenomena. We present PronCoach, a pronunciation coaching framework for Mandarin and English audio without reference transcripts. We decompose coaching into a structured chain-of-thought process covering intended-content recovery, realized-pronunciation inference, contextual comparison, per-phone correction, intervention decisions, and feedback generation. Based on this decomposition, we fine-tune language-specific phonetic-perception models on expert annotations to infer probabilistic candidate realizations from acoustic and canonical-pronunciation evidence. An audio-language model (ALM) then uses the recording, intended content, and candidate evidence to generate aligned diagnoses and coaching feedback. These annotations supervise both complete coaching responses and intermediate tasks. To address the lack of a dedicated pronunciation-correction benchmark, we construct PronCoach-Bench with two complementary tracks. The Diagnostic Track evaluates correction localization, heard-realization agreement, and false corrections, jointly testing phonetic perception and correction decisions; the Clean-Speech Track tests correction restraint by measuring whether models avoid unnecessary feedback. The main PronCoach model achieves a heard-based overall F1 of 0.7528, compared with 0.4139 for the strongest evaluated public ALM baseline. Supplementary English evaluations yield the lowest phoneme error rate among the compared systems on two of three corpora. Furthermore, PronCoach demonstrates emergent cross-lingual correction capabilities, enabling it to perform pronunciation rectification across Mandarin and English representations. PronCoach-Bench will be publicly released soon.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.