- Rating
- 3(+1)
- Confidence
- 1(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how the upsampling method affects the parameter reallocation benefit between convolutional rank and channel width in extremely compressed neural vocoders: under ZI, using DSC to save parameters for channel widening instead degrades quality, whereas switching to SR clearly improves the benefit. The authors define and compare rank-reduction cost, widening gain, and net reallocation benefit via A/B/C matched configurations, supported by rank sweeps, ZI→SR interpolation, architecture transfer, and multi-budget experiments, finding the interaction strongest at about 72K parameters and weaker at larger budgets. Based on this, they propose NanoVoco (SR, wide DSC refiner, DynamicTanh, 72.3K parameters), which achieves UTMOS 3.71 on LJSpeech, comparable to the evaluated Parallel WaveGAN and higher than FreGrad, and is deployed on STM32H747XI for real-time streaming synthesis using only on-chip memory. Overall, the experiments are solid and the engineering is complete, but evidence for perceptual quality and generality is insufficient, and the methodological novelty is weaker than the analysis work.
Critical strengths
-
The experimental isolation is solid and is the strongest part of the paper: the A/B/C controlled experiments separate the rank reduction loss, the widening gain, and the net benefit, and the rank sweep shows that the benefit is all negative under ZI and all positive under SR, making the core observation highly credible.
-
MCU deployment is a strong engineering contribution: the complete TTS pipeline actually runs on a Cortex-M7 (STM32H747XI, 480 MHz) using only on-chip Flash and SRAM; RTF 0.9876, no I2S underflow across 640 requests, and zero SDRAM usage, clearly adding to the paper's practical value.
Critical weaknesses
-
The perceptual evaluation is not sufficient to support the core quality claims. The biggest weakness is the lack of human listening tests: the claim that a 72.3K-parameter model is comparable in quality to much larger vocoders is based mainly on UTMOS, evaluated on only 128 utterances from LJSpeech. UTMOS can serve as a screening metric, but it is not sufficient in a vocoder paper that interprets relatively small score differences as architectural improvements. Moreover, several core comparisons involve external checkpoints with different training distributions and frontends, so I suggest adding a small-scale MOS/MUSHRA test comparing at least HiFi-GAN V2, Parallel WaveGAN, and one lightweight baseline.
-
The novelty and generality of the design principle are limited. Combining SR upsampling with separable convolution already has a precedent in nVOC-22, so the main novelty here lies in explaining and controlled-characterizing its interaction with rank-width reallocation, rather than in the component combination itself. This characterization is useful but remains largely empirical: it does not sufficiently explain why SR so markedly changes the return from widening a rank-constrained generator in the ~72K regime; the strongest effect is highly budget dependent, and most sweep and transfer experiments use only a single training seed. I therefore currently regard it as a careful characterization of an architectural phenomenon rather than a broadly established principle for efficient vocoding.
Questions for authors
-
Human perceptual evaluation: Could the authors provide human MOS/CMOS/MUSHRA comparing NanoVoco with the strongest relevant baselines? In particular, I would like to confirm whether the roughly 3.71 vs. 3.70 UTMOS difference with Parallel WaveGAN, and the improvements over compressed alternatives, are reflected in human judgments. This would materially strengthen my confidence in the main quality claim.
-
Generalization beyond LJSpeech: Does the upsampling–reallocation interaction persist on multi-speaker or otherwise substantially different datasets? The transfer experiments vary only architecture and adversarial supervision, not the speech distribution; one additional dataset would also help determine whether the effect is specific to LJSpeech.
-
Relation to nVOC-22: Could the authors more clearly distinguish this paper's contribution from nVOC-22? As I understand it, nVOC-22 already combines repetition upsampling with separable convolution, whereas this paper extends separable processing more broadly and focuses on the rank-width interaction. Please clarify which architectural elements are new and which part of the contribution should primarily be regarded as analysis rather than architecture.
4 comments
Question 1: "Could the authors provide human MOS/CMOS/MUSHRA comparing NanoVoco with the strongest relevant baselines?"
See our response to Critical Weaknesses 1.
Question 2: "Does the upsampling–reallocation interaction persist on multi-speaker or otherwise substantially different datasets? ..."
We trained six configurations on the multi-speaker VCTK dataset, using 100 speakers for training and nine held-out speakers for evaluation, following a setup similar to that of HiFi-GAN (see Fig. 6(a) in the revised paper).
| Model | UTMOS (mean ± std) |
|---|---|
| ZI-StdC | 2.1261 ± 0.5081 |
| ZI-DSC-fixwidth | 1.3803 ± 0.1805 |
| ZI-DSC | 2.6836 ± 0.3882 |
| SR-StdC | 2.2377 ± 0.4396 |
| SR-DSC-fixwidth | 1.5658 ± 0.3278 |
| SR-DSC | 2.8799 ± 0.3885 |
(Under ZI)
the rank-reduction cost is ZI-StdC - ZI-DSC-fixwidth = 2.1261 - 1.3803 = 0.7458
the widening gain is ZI-DSC - ZI-DSC-fixwidth = 2.6836 - 1.3803 = 1.3033.
(Under SR)
the rank-reduction cost is SR-StdC - SR-DSC-fixwidth = 2.2377 - 1.5658 = 0.6719 (< 0.7458)
the widening gain is SR-DSC - SR-DSC-fixwidth = 2.8799 - 1.5658 = 1.3141 (> 1.3033).
These results show that the upsampling–reallocation interaction persists on the multi-speaker VCTK dataset, with SR exhibiting a lower Rank-1 Cost and a slightly higher Widening Gain. However, the magnitude of the interaction is relatively small, particularly for Widening Gain. This suggests that the benefit of the interaction may also depend on dataset characteristics, as well as on the choice of upsampler.
Question 3: "Could the authors more clearly distinguish this paper's contribution from nVOC-22? Please clarify which architectural elements are new and which part of the contribution should primarily be regarded as analysis rather than architecture."
See our response to Critical Weaknesses 2.1.
Raising Score and Noting Assessment Uncertainty
Thank you to the authors for revising the paper and providing a response.
I will raise my score. My main basis is Fig. 2 and Fig. 3 in the revised manuscript: these two figures clearly address the main concerns I raised earlier, and the authors' additional clarifications are convincing to me. I therefore believe the revision represents a substantive improvement over the initial submission, and raising my score is appropriate.
At the same time, I would like to candidly note the uncertainty in my assessment, for the authors and the AC's reference:
-
Domain familiarity: I am not especially familiar with the specific subfield of this paper. My review therefore focuses mainly on the dimensions I can reliably assess, such as writing clarity, completeness of experimental reporting, consistency between figures and claims, and reproducibility information; my confidence is more limited on judgments such as domain novelty and the marginal contribution relative to the most related work.
-
The field is relatively old: This direction has a long research history, and judging whether the incremental contribution of this paper relative to existing work is sufficiently significant requires a deep grasp of the field's lineage. I am not fully confident on this point, so my score should be read as a judgment based on the dimensions noted above, rather than a comprehensive verdict on the paper's contribution to the field.
In light of the above, I suggest the AC appropriately weigh the opinions of reviewers more familiar with this subfield when calibrating novelty and contribution judgments; my raised score mainly reflects that the specific concerns addressed by Fig. 2/Fig. 3 have been resolved. If the authors or the AC consider it helpful, I can further specify exactly which points changed my view.
Thank you to the authors for their careful revision, and to the AC for organizing the review.
Dear Mce4,
Thank you so much for your acknowledgment of our revised manuscript!
We are very glad that our response helped clarify our work.
We sincerely appreciate your time and valuable insights, which have helped us improve our paper.
Best Regards, Authors
Thank you so much, Mce4!
We sincerely appreciate your recognition of our experimental isolation and MCU deployment.
We fully acknowledge the weaknesses and questions you raised and have revised our manuscript to address them based on your valuable feedback.
We agree that human listening tests would provide valuable perceptual evidence, and we acknowledge their absence as a limitation of this work. We are currently conducting a small-scale MOS listening test comparing HiFi-GAN V2, Parallel WaveGAN, and a lightweight baseline (V2-iSTFTNet). However, the process has taken longer than anticipated due to ethical considerations. We will report the results as soon as they become available.
In the meantime, we have expanded the objective evaluation to include M-STFT, MCD, Periodicity, ViSQOL, and PESQ, along with comparisons against lightweight iSTFT-based vocoders (see Table 1 in the revised paper). Although these objective metrics cannot replace human judgments, they provide complementary perspectives on synthesis quality and enable a more comprehensive evaluation.
nVOC-22 finds that SR-DSC is more effective than ZI-StdC. However, it does not experimentally investigate what interaction exists between the upsampler and StdC or DSC, why it occurs, when it is beneficial, or how we can use it. This work sets out to answer these questions.
(Analytically)
(Architecturally)
In the revised paper, we propose a supported explanation for why SR has lower rank-reduction cost and higher width-expansion gain (upsampling-reallocation interaction)
From a signal-processing perspective, sample repetition can be modeled as zero insertion followed by a fixed zero-order hold filter, which is essentially a low-pass filter. To find whether the interaction is related to the upsampler's built-in filtering, we design ZI-to-SR experiments.
The results show that StdC performance remains relatively stable as the upsampler progressively introduces fixed filtering, suggesting that this filtering provides little additional quality benefit to StdC. By contrast, both DSC configurations improve substantially, particularly the widened DSC.
These observations suggest that the importance of StdC's full filtering flexibility may diminish when the upsampler provides built-in filtering. Meanwhile, reallocating parameters to channel-width expansion appears to be more beneficial than retaining the full filtering flexibility under this setting.
Together, these observations are consistent with our proposed explanation for the upsampling–reallocation interaction (revised paper, Figs. 4 and 5).