- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper argues that the clean-speech codec WER understates the loss on degraded speech. Across ten operating points from six codec families on LibriSpeech test-clean with WHAM! noise, babble, and reverberation, it reports 4.5–22× amplification at 5 dB, fits a power law , attributes 6–96% of the loss to the codeword lookup via a quantizer-bypassed path, and proposes a pre-lookup latent-regression adapter that halves Mimi’s 5 dB WER while preserving vocabulary. The empirical coverage is broad, but the headline claims are not supported at the level asserted.
Critical strengths
The idea of quantitatively linking clean-speech and degraded-speech codec behavior is somewhat novel. It motivates deployment-aware evaluation and rate provisioning, which clean-only benchmarks cannot address.
Critical weaknesses
- The quantitative results of both the 'clean-to-deployment law' and the
quantizer-bypassed path attributesseem to vary largely across different codec families, code rates, SNR levels, and ASR models. There is a lack of a systematic schema for drawing practical guidance from these findings. - The adapter is indeed a post-hoc noise remover, which not only overfits to the training noise, but also removes background sound events that contain environmental information. I don't think this kind of setting still establishes a 'transparent' codec.
- Only WER is used as the evaluation tool, without alignment to human perception.
Questions for authors
- What is the precise definition of so-called "transparency"?