- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how neural audio codecs affect pitch structure and its measurement. Controlled synthetic stimuli and per-partial spectral analysis show that, in the tested EnCodec settings, the fundamental and lower partials are largely preserved while some upper partials shift toward the harmonic ladder of the nearest 12-TET fundamental. An estimator combining these partials can consequently report an intermediate pitch displacement. The authors investigate the source through decoder-only corpus interventions that keep test tokens fixed, examine consequences for tuning analysis of real recordings, and propose restricting pitch estimation to partials below the measured band edge.
Critical strengths
1. The paper provides useful mechanistic insight by locating changes in individual partials. Figure 1 and Table 1 distinguish an essentially preserved fundamental, two spectral lines at an edge partial, and displaced upper partials. This explains why an aggregate pitch estimate can differ from the displacement of any individual partial. The contribution therefore goes beyond reporting overall pitch error and identifies a concrete measurement issue when analyzing audio processed by a neural codec.
2. The decoder corpus intervention is convincing. The encoder and quantizer are frozen, so each fixed test input produces identical tokens across decoder arms; only the decoder’s fine-tuning corpus changes. Shifting the corpus by 33 cents moves the preferred grid by approximately 32.7 ± 2.8 cents, while the fundamental remains essentially unchanged and grid-preserving resampling controls do not produce a corresponding shift (Section 3.3). The reported spread is across seeds. This intervention, together with directionally consistent results in other decoders, supports an effect of the corpus on upper-partial placement rather than merely showing that fine-tuning reduces reconstruction error.
Critical weaknesses
1. The magnitude of the impact in commonly used pitch-analysis pipelines is not yet sufficiently quantified. The paper includes real-music experiments, but its main error magnitudes and mitigation results come from the authors’ full-band harmonic estimator. With pYIN, the proportion of Carnatic recordings whose preferred fit switches to 12-TET falls from 11% to 3%. For librosa, the main result is a count of recordings moving toward 12-TET, without comparable displacement magnitudes or a distortion-matched control. The practical impact therefore depends strongly on the analysis tool. More specific evidence is needed to identify which common workflows incur consequential errors and how large those errors are (Section 3.4; Appendix F; Table A31).
2. Mitigation is primarily validated for measurement accuracy, while improvements to reconstructed-audio fidelity remain limited. Fitting only partials below the band edge reduces pitch-estimation error but does not restore displaced upper partials, and direct validation is primarily limited to the authors’ estimator. Decoder fine-tuning provides a possible route to improvement, but has cross-corpus tradeoffs: fine-tuning on Carnatic music reduces scale-degree error on that corpus while increasing it on makam recordings. For applications requiring faithful preservation of input tuning, the paper should more clearly distinguish measures that improve analysis reliability from those that bring reconstructed partials closer to the input, including their conditions and costs (Sections 3.3 and 3.4; Appendices E and F).
Questions for authors
- How large is the impact in commonly used pitch analysis, and which settings require intervention? Using the existing real recordings and the already tested librosa pipeline, please report changes in estimated tuning in cents with uncertainty, and include a traditional-codec control with distortion comparable to EnCodec. Together with the existing pYIN results, please clarify which common workflows are more or less affected and when the changes are sufficient to alter practical analysis conclusions.
- What effective measures are currently available when faithful preservation of input tuning is the goal? Please distinguish reducing pitch-estimation error from bringing the decoded upper partials themselves closer to the input. For the former, could band-limiting or an equivalent procedure be directly evaluated in one of the already tested off-the-shelf tools, with its requirements for edge estimation and usable low-frequency information specified? For the latter, please use the existing bitrate and fine-tuning experiments to explain which settings reduce upper-partial displacement, the associated reconstruction-quality or computational costs, and whether the improvement transfers to other musical corpora. This would clarify how far the existing remedies go.