- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposed ReVoc, a high-fidelity and efficient audio generation system via bridges and GANs. The authors evaluated the performance on text to speech synthesis and shows improvement on PESQ and other scores.
Critical strengths
-
The presentation of the paper is clear.
-
The metrics from the tables indciate the effectiveness of the method.
Critical weaknesses
-
My major concern of this work is that the innovation is limited. Authors proposed bridge and GAN, but they are some marginal improvement of existing methods.
-
Bridge may not be a key point that lead to a big significance to solve the audio generation problem.
-
The proposed methodolgy part is a bit weak. Bridge methods have been research in several speech generation, and enhancement works. In addition, for a text to speech problem, I am unsure whether bridge model will outperform diffusion/flow matching-based models.
-
4.1 proposed a mel vocoding and restoration model. This is a standard configuration of many speech audio generation works.
-
The work is about text-to-speech. Therefore, the title should be revised from audio generation to text-to-speech. Otherwise, authors should do more experiment on general audio and music generation as well.
Questions for authors
- See critical weaknesses.
3 comments
Response Regarding Methodological Contributions (Part 2)
We thank the reviewer for the valuable feedback. We would like to further clarify the key methodological contributions of ReVoC.
Building on prior research on bridges and adversarial learning, ReVoC investigates how to achieve high-quality waveform reconstruction with an efficient spectral backbone and a shared model across inference budgets. Its contributions concern the design of the spectral predictor, joint multi-NFE adversarial fine-tuning, and a restoration formulation applicable to both mel vocoding and codec restoration. We describe the evidence for these contributions below.
1. Spectral predictor design and reconstruction before adversarial fine-tuning
ReVoC uses a restoration bridge initialized from an acoustic estimate derived from the input. This formulation allows the model to refine the available acoustic structure through successive updates. We implement the predictor using a Vocos-style backbone, with a patch encoder that explicitly models local time–frequency structure before temporal processing.
The patch encoder makes a measurable contribution under the same restoration training setup. Our direct-flattening reference, labeled Vocos(Flow), uses the same temporal projection and backbone. After 100k Stage 1 updates, the two configurations obtain the following results using four-step EMA inference on the same 100 validation utterances.
| Stage 1 predictor | PESQ ↑ | M-STFT ↓ | UTMOS ↑ |
|---|---|---|---|
| Vocos(Flow) | 3.9119 | 0.9141 | 3.4076 |
| ReVoC-Small with patch encoder | 4.2394 | 0.8182 | 3.5560 |
The patch encoder improves PESQ by 0.3275, reduces M-STFT by 0.0959, and increases UTMOS by 0.1484. These improvements occur before GAN fine-tuning, providing direct evidence for the contribution of the spectral predictor design within the restoration framework.
2. Joint adversarial fine-tuning across inference budgets
A further methodological distinction is that ReVoC jointly fine-tunes one model for NFE = 1, 2, and 4. ReVoC makes these inference budgets available through the same checkpoint after adversarial fine-tuning.
Table 4 evaluates ReVoC-Base before and after joint fine-tuning on the same LibriTTS subset, with Stage 2 initialized from the Stage 1 checkpoint shown below.
| NFE | PESQ before → after ↑ | UTMOS before → after ↑ |
|---|---|---|
| 1 | 4.115 → 4.357 | 3.624 → 3.718 |
| 2 | 4.375 → 4.409 | 3.753 → 3.799 |
| 4 | 4.403 → 4.433 | 3.783 → 3.832 |
Joint fine-tuning improves both metrics at every evaluated NFE, with the largest gains at NFE = 1. The resulting checkpoint supports a selectable quality–computation trade-off without retraining or switching model weights. This is the practical purpose of our multi-NFE training strategy.
(continued below)
Response Regarding Methodological Contributions (Part 2) - Continued
3. Quality and efficiency relative to diffusion and flow vocoders
To address the question about diffusion and flow matching, we extract the relevant comparisons from Tables 1 and 11. These include the diffusion vocoder FreGrad and the flow-based vocoders PeriodWave and RFWave.
| Dataset | Model | NFE | PESQ ↑ | M-STFT ↓ | V/UV F1 ↑ | UTMOS ↑ | ViSQOL ↑ |
|---|---|---|---|---|---|---|---|
| LibriTTS | RFWave | 10 | 4.249 | 0.961 | 0.967 | 3.383 | 4.776 |
| ReVoC-Small | 2 | 4.301 | 0.808 | 0.975 | 3.646 | 4.921 | |
| ReVoC-Small | 4 | 4.342 | 0.794 | 0.977 | 3.720 | 4.929 | |
| LJSpeech | FreGrad | 50 | 3.775 | 1.344 | 0.943 | 3.934 | 4.574 |
| PeriodWave | 16 | 4.235 | 1.098 | 0.967 | 4.374 | 4.722 | |
| ReVoC-Base | 1 | 4.290 | 0.897 | 0.978 | 4.197 | 4.949 | |
| ReVoC-Base | 4 | 4.367 | 0.876 | 0.981 | 4.367 | 4.955 |
On LibriTTS, ReVoC-Small at NFE = 2 exceeds RFWave at NFE = 10 on all five metrics shown. On LJSpeech, four-step ReVoC-Base achieves higher PESQ, V/UV F1, and ViSQOL and lower M-STFT than the 16-step PeriodWave results, with a slightly lower UTMOS of 4.367 versus 4.374.
Computational efficiency is central to the design. We use a Vocos-style spectral backbone to investigate how much reconstruction quality can be obtained with inexpensive network evaluations. The following results are extracted from Table 9.
Measurements use FP32, batch size 16, and nominal one-second inputs on a single RTX 4090 GPU and eight pinned threads of an Intel Xeon Gold 6346 CPU. GMACs count neural operations per item over the complete rollout. CPU and GPU ×RT denote throughput relative to real time.
| Model | NFE | Params (M) | GMACs | CPU ×RT ↑ | GPU ×RT ↑ |
|---|---|---|---|---|---|
| BigVGAN-base | 1 | 14.02 | 32.95 | 1.3 | 101.4 |
| BigVGAN | 1 | 112.42 | 89.98 | 0.9 | 72.6 |
| Vocos | 1 | 13.53 | 1.27 | 233.5 | 8036.8 |
| RFWave | 10 | 18.14 | 25.27 | 1.3 | 94.3 |
| ReVoC-Small | 1 | 19.31 | 1.78 | 83.3 | 4776.2 |
| 2 | 19.31 | 3.55 | 57.3 | 2884.8 | |
| 4 | 19.31 | 7.10 | 35.8 | 1600.0 | |
| ReVoC-Base | 1 | 52.95 | 4.88 | 52.6 | 2563.9 |
| 2 | 52.95 | 9.75 | 31.5 | 1411.6 | |
| 4 | 52.95 | 19.50 | 17.7 | 738.3 |
ReVoC-Small and RFWave have similar parameter counts, at 19.31M and 18.14M, respectively. At NFE = 2, ReVoC-Small achieves the quality improvements shown above using 3.55 GMACs, approximately 14% of RFWave’s 25.27 GMACs. Its measured GPU throughput is approximately 30.6 times higher under this benchmark configuration.
Relative to Vocos, ReVoC introduces additional computation for iterative restoration. This computation provides measurable reconstruction gains and allows users to select the inference budget using a single checkpoint. The Small-model results demonstrate the value of the approach at a parameter scale close to existing efficient vocoders.
4. A shared restoration formulation for mel vocoding and codec restoration
Mel vocoding and codec restoration are established tasks, and Section 4.1 uses them to evaluate the restoration framework under two different types of acoustic degradation. Mel vocoding starts from an estimate derived from the mel spectrogram, while codec restoration starts from the native decoder output. Both are represented as complex STFT states and processed through the same restoration formulation.
This shared representation also supports joint training across tasks. In the supplementary experiment described in our response to Reviewer Wzfz, we continued Stage 1 training with a 1:1 mixture of the two tasks, followed by joint multi-task, multi-NFE GAN fine-tuning. The resulting unified ReVoC-Small checkpoint retained mel-vocoding performance close to the single-task model.
This experiment demonstrates that the formulation can support a single model across both tasks, while the main multi-NFE results establish that one checkpoint can retain and improve quality across inference budgets. We will make these methodological distinctions and their supporting ablations more explicit in the revision.
Response Regarding the Scope of ReVoC and the Paper Title (Part 1)
We thank the reviewer for raising the question of scope and suggesting evaluation beyond speech. ReVoC operates at the waveform level and addresses two tasks. In mel vocoding, it synthesizes a waveform from a mel spectrogram. In codec restoration, it refines the waveform produced by a native codec decoder such as EnCodec or Mimi. In mel-based TTS, the upstream text-conditioned model predicts mel spectrograms, and ReVoC serves as the vocoder. In codec-based TTS, the upstream model predicts codec tokens, and ReVoC can refine the waveform after codec decoding. The term audio generation in the title refers to conditional waveform generation and restoration across these two tasks.
1.Zero-shot on general audio mel vocoding.
As detailed in our response to Reviewer Wzfz, we evaluated the speech-trained ReVoC-Base checkpoint in the official MUSDB18 test split. The evaluation covers mixtures and four stems. The table reports the average scores across the five categories, with PESQ-WB included as a supplementary metric.
2.ReVoC as the waveform backend in TTS.
In our response to Reviewer BAqX, we evaluated ReVoC and Vocos on the same mel spectrograms predicted by F5-TTS Base and E2-TTS Base for 1,127 LibriSpeech-PC test-clean utterances. For each system, the acoustic model and its predicted mel spectrograms were held fixed. The table shows the results for Vocos → ReVoC-Base (NFE = 4).
The TTS results show how ReVoC performs as a waveform backend with predicted acoustic inputs. The MUSDB18 results extend the evidence to zero-shot music waveform reconstruction at 24 kHz.