- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents an analytical framework, allocation profiles, for speech tokenisers. It characterizes how speech tokenisers partition acoustic space across phonetic categories, instead of probing what linguistic information tokens retain. Evaluating four widely used codecs and one custom-trained ablation model on TIMIT, it is found that adding RVQ depth universally causes levelling but largely preserves the initial ranking. It concludes that the phonetic organization is primarily settled at the first quantization level and by its architectural geometry, rather than by subsequent cascade depth.
Critical strengths
- The analysis is novel. It makes use of codeword-based allocation profiles instead of conventional index-based probing.
- The refinement decomposition is also well designed, by separating cascade refinement into amplitude and ranking.
- Mimi is used smartly with parallel acoustic and semantic streams for within-model comparison.
Critical weaknesses
- The major concern is that the analysis relies exclusively on TIMIT dataset, which a speech corpus of clean and reading style American English. Whether the findings in the paper are still valid for more diversified speech (different accents/languages, noisy conditions, conversational speech) is unknown.
- It is mentioned in the Conclusion that “whether it propagates that far, to phone recognition, minimal-pair discrimination or spoken language modelling, is the natural next question, not shown here”. However, without linking the allocation profiles to the performance of downstream tasks such as speech recognition, the contribution is limited.
Questions for authors
The toy RVQ codec is trained with a simple architecture and a small speech dataset. Whether this toy model may exhibit certain behaviors due to under-training and simplified capacity?
Thank you for your review and for your positive remarks.
We will submit a corrected version accordingly.