acceptodds
Under review as a conference paper at ICLR 2027

Atom-TTS: A lightweight on-device TTS system adopting cascaded prediction and frame-driven vocoder

Abstract

On-device speech synthesis needs to balance speech quality under a low computing budget. However, existing methods struggle to model high-quality intermediate audio representations with a small number of parameters. Moreover, high-quality vocoders incur substantial inference overhead on edge devices with constrained computing resources. To address these issues, we propose Atom-TTS. Starting from acoustic prediction and waveform reconstruction, it reduces inference costs while maintaining synthesis quality. First, we propose the Cascaded Parameter Acoustic Predictor (CPAP), which adopts a TCN encoder for cascaded encoding and establishes explicit conditional relationships among parameters to efficiently predict intermediate representations. It compresses trainable parameters to 161k without degrading prediction quality. Next, we propose the Frame-Driven WORLD vocoder (FD-WORLD). It constructs frame-level spectra using spectral envelope priors, harmonic excitation and noise excitation, and generates waveforms via short-window inverse Fourier transform and overlap-add, boosting the audio synthesis speed of the vocoder by about 5 times. In experiments, Atom-TTS achieves an RTF of 0.0065 on a single-core CPU, corresponding to 155 times faster than real time. In a subjective listening test with 15 listeners on a shared corpus, it reaches a MOS of 3.24, outperforming a same-corpus baseline with 3.2 times more parameters by 1.32 MOS and another with 9.7 times more parameters by 0.99 MOS. The backend replacement is not distinguishable under natural listening. 80.5% of trials are judged as having no audible difference. Atom-TTS thus achieves a balance among quality, model size and efficiency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
text-to-speech, WORLD vocoder, on-device model, lightweight system

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LaiuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Atom-TTS, a lightweight TTS system consisting of a Cascaded Parameter Acoustic Predictor (CPAP) and a Frame-Driven WORLD vocoder (FD-WORLD). CPAP predicts WORLD acoustic parameters through a cascaded TCN architecture, while FD-WORLD replaces pulse-driven synthesis with frame-level spectral reconstruction. The experimental results show improvements in model size and inference efficiency while maintaining reasonable synthesis quality.

Critical strengths
  1. The paper focuses on a practical problem of lightweight TTS deployment, achieving a small model size and low inference latency.
  2. The paper provides useful ablation experiments on acoustic prediction and vocoder efficiency, with relatively detailed implementation and evaluation protocols.
Critical weaknesses
  1. The contributions seem more engineering-oriented. The acoustic parameter interface in Section 4.1 follows the existing WORLD representation. The frame-driven synthesis in Section 4.3 is also similar to Nercessian (2023) [A1], which already uses fixed-hop STFT/iSTFT synthesis with harmonic and noise excitation. While FD-WORLD clearly improves efficiency over the original WORLD, the paper does not sufficiently clarify what is new compared with existing frame-based WORLD synthesizers.
  2. The contribution of explicit conditional dependencies is not fully verified. Section 4.2 conditions MCEP and aperiodicity prediction on the predicted fundamental frequency and voicing. However, the cascade-versus-parallel comparison in Table 3 does not isolate the effect of this conditioning. The improvement may also come from the three-stage architecture itself.
  3. Insufficient support for some synthesis-quality claims. The subjective MOS results in Table 1 (Section 5.2) rank Atom-TTS above both sanoTTS variants, whereas UTMOS, SCOREQ, and DNS-OVRL in Table A6 consistently show the opposite ranking. The discussion in Appendix 11.3.2 does not fully explain this discrepancy. In addition, Section 5.4 concludes that FD-WORLD and WORLD are perceptually equivalent, although the switching task still detects audible differences.
  4. Formatting issues. The manuscript does not appear to fully follow the official ICLR 2027 formatting guidelines, particularly in the abstract and overall layout. The authors should check the formatting against the official ICLR template.

[A1] S. Nercessian, “Differentiable WORLD Synthesizer-Based Neural Vocoder With Application To End-To-End Audio Style Transfer,” in Proc. AES Convention 154, May 2023, Paper 10661.

Questions for authors
  1. How does FD-WORLD differ from the frame-based WORLD synthesizer in Nercessian (2023) [A1]? What is the main contribution beyond using fixed-frame synthesis?
  2. Could the authors remove the predicted F0 and voicing conditioning from the Spectral Encoder while keeping the three-stage CPAP architecture unchanged? This would help isolate the effect of the explicit conditional dependencies claimed in Section 4.2.
  3. Could the authors explain the opposite rankings between MOS in Table 1 and the automatic metrics in Table A6? Does the switching-detection result in Section 5.4 suggest that the perceptual equivalence claim should be qualified?
  4. Section 4.3 reports a frame duration of about 21 ms and a frame rate of 200 frames/s. However, the stated 512-sample Hann window at 16 kHz corresponds to 32 ms, and the 120-sample frame shift corresponds to approximately 133 frames/s. Please clarify these settings.

[A1] S. Nercessian, “Differentiable WORLD Synthesizer-Based Neural Vocoder With Application To End-To-End Audio Style Transfer,” in Proc. AES Convention 154, May 2023, Paper 10661.

4 comments

Thank you for your valuable suggestions. We will now respond to each of them:

  1. Thank you for raising the question regarding how our FD-WORLD compares to other frame-level methods.[A1] is indeed an excellent piece of work. During our initial research, we drew inspiration from the differentiable approach proposed in [A1]. Specifically, [A1] contributes by implementing differentiable synthesis, utilizing a multi-sinusoidal method to construct harmonic sources. In contrast, FD-WORLD does not aim for differentiability; instead, it employs integer-phase impulses to provide a harmonic structure, thereby reducing the computational cost of excitation generation during CPU-based inference. We reproduced the method based on the descriptions provided in the original paper and compared its inference efficiency with that of the FD-WORLD scheme. The experimental setup was as follows: the tests were conducted on a single-threaded CPU (i7-12700) with an output sample rate of 24 kHz. We used 50 test samples, with 3 warm-up runs per sample, followed by 10 measurement runs. The output was fed directly into a back-end vocoder, bypassing any front-end preprocessing stages; thus, the results do not represent an end-to-end Real-Time Factor (RTF). All experiments were implemented using Python. The results are as follows(DiffWORLD is [A1]): |Backends|RTF|Average time per sentence| |---|---|---| |WORLD(pyworld)|0.01468|63.84ms| |FD-WORLD(pytorch)|0.00226|9.85ms| |DiffWORLD(pytorch) 1024FFT/256hop|0.01661|72.22ms| |DiffWORLD(pytorch) 512FFT/120hop|0.01579|68.65ms|

It can be observed that the inference efficiency of [A1] is not superior to that of WORLD, and is, in fact, significantly lower than that of FD-WORLD. It should be noted that, although our implementation follows the methods and configurations described in the original paper, we cannot guarantee that this is the optimal implementation, as the authors have not made the source code publicly available. In terms of the computational complexity of excitation generation, let LL be the total number of samples in the output audio and KK be the number of harmonics. The synthesis complexity of FD-WORLD is O(L)O(L), whereas the synthesis complexity of [A1] is O(LK)O(LK). This is because [A1] requires calculating and superimposing KK harmonics at every sample point, while FD-WORLD only needs to accumulate the phase and generate an impulse at each point—a process that reduces both the actual number of operations and the amount of intermediate data.

  1. Thank you for your valuable suggestions. Following your feedback, we have reorganized our experiments. While maintaining the three-stage architecture of the CPAP model, we have set the F0 and V/UV inputs to the final layer to zero and removed the conditional dependence of MCEP and AP on F0. We are currently conducting ablation training with this modified setup, ensuring that the total number of trainable parameters remains unchanged. The experiments are underway, and we will upload the results within this week.

  2. UTMOS, SCOREQ, and DNSMOS are all learned quality prediction metrics rather than direct measurements of human perception. They rely on relationships between human ratings derived from specific datasets and evaluation protocols, rather than universal perceptual principles applicable to all listeners, all synthesis systems, and all real-world conditions. We believe this stems from a lack of generalization in the automated models employed. You may evaluate this yourself via our demo page—accessible by opening the audio_demo/index.html file (included in our supplementary materials) directly in your web browser. Furthermore, to demonstrate the authenticity of our MOS data, we have re-compiled the statistics and conducted additional analyses to substantiate the subjective quality advantages of our approach. |Comparision|Mean-diff. range| Atom wins|Ties|Atom loses| |---|---|---|---|---| |Atom - sanoTTS standard|+0.20~+1.53|199(66.3)|75(25.0%)|26(8.7%)| |Atom - sanoTTS tiny |+0.60~+1.73|235(78.3)|54(18.0%)|11(3.7%)|

Regarding the claim of complete equivalence, we incorporated three options into our ABX test: X=A, X=B, and "indistinguishable." Out of a total of 832 trials, participants responded "indistinguishable" in 670 instances, accounting for 80.5% of the total. In the remaining 162 trials—where participants chose either X=A or X=B—they provided the correct answer 90 times, representing a success rate of 55.6%. As this accuracy rate is not significantly higher than what would be expected by chance, we conclude that FD-WORLD and WORLD are perceptually equivalent for all practical purposes. Naturally, these data do not definitively prove that FD-WORLD and WORLD are perfectly equivalent in terms of perception; consequently, we have refined our phrasing—rather than asserting complete equivalence, we now state that there are no significant perceptual differences between FD-WORLD and WORLD.

  1. Thank you for pointing this out. Upon verification, we found that the 16 kHz label in Figure 1 was incorrect. The actual sampling rate for synthesis in Atom-TTS is 24 kHz; a 512-sample window corresponds to 21.33 ms, and a 120-sample frame shift corresponds to 5 ms (or 200 frames per second). We have corrected this in the figure. Additionally, for the automated evaluation metrics, all audio samples were resampled to 16 kHz to meet the input requirements of the evaluation models. This step was part of the pre-processing phase for evaluation, and the relevant code can be found in the supplementary materials.

  2. We have uploaded the actual Atom-TTS inference deployment code to the supplementary materials for your review.

AuthorsOpenReview ↗

Ablation studies on F0 and V/UV

Regarding Question 2, we have conducted the requested ablation studies by removing F0 and the V/UV (voiced/unvoiced) flag. Additionally, we performed two further experiments—one retaining F0 while masking the V/UV flag, and another retaining the V/UV flag while masking F0—and compared these against a baseline model (d1) with identical parameter counts and configurations. The results of these experiments are provided in Appendix D. Our analysis indicates that removing either F0 or the V/UV flag does not have a particularly significant impact on MCD (Mel-Cepstral Distortion) predictions, though minor differences in detail can be observed. However, there is a noticeable effect on AP (Aperiodicity). Removing F0, the V/UV flag, or both, leads to a reduction in AP estimation accuracy.

Thank you for the response and for running the additional experiments. The comparison with Nercessian (2023) helps explain where the speedup comes from. I would still like the paper to be clearer about what FD-WORLD adds to earlier frame-based WORLD methods. The new F0 and voicing ablation also answers my question. The clearest benefit is in aperiodicity estimation, with little change in MCD.

Overall, the response addresses some of my initial concerns. But my remaining concerns about perceptual quality are similar to those raised by Reviewer WxTQ. The additional MOS analysis strengthens the listening-test results, but the disagreement with the automatic metrics remains unexplained. Listeners often reported no audible difference in the ABX test. The differences detected in the switching task leave me unconvinced that the two vocoders are perceptually equivalent. I therefore maintain my original rating of 2 (weak reject).

Reviewer_WxTQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Atom-TTS, a lightweight speech synthesis system combining a Cascaded Parameter Acoustic Predictor (CPAP) with a deterministic Frame-Driven WORLD vocoder (FD-WORLD). The model predicts acoustic parameters through three sequential stages, while FD-WORLD replaces pitch-pulse-driven synthesis with frame-level reconstruction. With approximately 161K trainable parameters, the system reports a single-core CPU RTF of 0.00645 and 7.0 MB net memory overhead. Experiments include human listening tests, comparisons with lightweight TTS systems, architectural ablations, and backend evaluation. The results demonstrate promising efficiency, although the evidence supporting the reported synthesis quality requires further clarification.

Critical strengths
  1. Atom-TTS achieves an RTF of 0.00645 with approximately 161K parameters and 7.0 MB net memory overhead. The authors use a unified single-threaded CPU measurement protocol and provide implementation consistency checks between training and deployment. These results support a meaningful reduction in computational cost for the evaluated setting.

  2. The parameter-matched ablation in Table 3 shows that cascaded prediction reduces full-band MCD from 6.169 to 5.863 dB compared with parallel prediction. The backend comparison also uses identical acoustic parameters for WORLD and FD-WORLD, isolating the effect of waveform reconstruction. These experiments provide useful evidence for the design choices, although their perceptual benefits are not fully established.

Critical weaknesses
  1. Human MOS favors Atom-TTS (3.24) over the same-corpus sanoTTS baselines (1.92 and 2.25), but Table A6 consistently reverses this ranking on UTMOS (2.880 versus 3.000 and 3.477), SCOREQ (2.710 versus 3.306 and 3.774), and DNS-OVRL (3.343 versus 3.819 and 3.839). Although these metrics do not directly establish perceived naturalness, the disagreement requires stronger investigation. Appendix C.2 relies on five informally judged perturbations and a non-significant correlation, which does not adequately resolve the discrepancy. Furthermore, the sanoTTS baselines were retrained or distilled by the authors, making their training quality and validation important to the comparison. The inconsistent corpus descriptions add uncertainty about experimental provenance.

  2. The authors conclude that FD-WORLD is perceptually equivalent to WORLD, but 80.5% no-difference judgments and a non-significant forced-choice result do not establish equivalence. Switch detection remains 12.7 percentage points above control, WER increases from 0.0135 to 0.0173, and Appendix Table A7 reports substantial utterance-level variation in masking. These findings support perceptual similarity in many cases, but not the stronger equivalence claim. The backend timing values also differ between Section 5.4, Table 4, and Appendix A.6, while the deployment evaluation uses a desktop Intel i7-12700 rather than embedded hardware. This limits confidence in the broader on-device conclusions.

Questions for authors
  1. How do the authors explain the opposite rankings between human MOS and UTMOS, SCOREQ, and DNS-OVRL, and what additional evidence supports the subjective quality advantage?

  2. How were the retrained sanoTTS baselines validated, and which corpus was actually used given the alan-medium and en/male tony descriptions?

  3. What statistical evidence establishes perceptual equivalence between WORLD and FD-WORLD despite the detectable switching differences, WER increase, and utterance-level masking variation?

  4. Which experimental conditions explain the different backend timing values in Section 5.4, Table 4, and Appendix A.6, and how representative are the desktop CPU results of embedded deployment?

1 comment
AuthorsOpenReview ↗

Experimental results on the RK3588

We conducted tests on the RK3588 embedded platform with the following experimental setup:

Operating System: Ubuntu 22.04.5, Linux 6.1.84, aarch64, glibc 2.35

CPU: Single Cortex-A76 core

Threads: Single-threaded; no GPU/NPU usage

Atom-TTS runtime: Official ARM64 CPU ONNX Runtime 1.23.2

sanoTTS runtime: Official native scalar C implementation (https://github.com/Ampixa/sanoTTS)

The test dataset consists of 50 English sentences, evaluated over 5 rounds. For each sentence in each round, 30 measurements are taken, and the median value is selected as the result for that sentence in that round. Preliminary runs—such as warm-up iterations—are excluded from the official timing measurements. The overall Real-Time Factor (RTF) is calculated using a weighted approach based on the 250 collected data points: RTF = sum(total_ms) / sum(audio_ms). For the evaluation of sanoTTS, we utilized the official "heart-nano" model—the smallest model provided by the developers—along with their official implementation. We opted not to use our own trained or converted models, thereby ensuring that the results remain unaffected by potential errors introduced during our own processing steps. The final results are as follows:

MethodParametersSampling rateRTF
Atom-TTS(FD-WORLD)161k24kHz0.014377
Atom-TTS(WORLD)161k24kHz0.046240
SanoTTS heart-nano294k24kHz0.026259
Reviewer_VvfYOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Atom-TTS, a single-speaker TTS pipeline for CPU-only deployment that uses WORLD-style parameters as its intermediate representation. The front end, CPAP (161K parameters), It predicts the parameters in three stages (phoneme encoder → prosody encoder emitting F0 and V/UV → spectral encoder conditioned on the predicted F0 and V/UV), with scheduled sampling between ground-truth and predicted prosody. The back end, FD-WORLD, replaces WORLD's pitch-synchronous synthesis with a fixed-rate (5 ms hop, 512-point) frame-wise source–filter synthesis.

Critical strengths
  • very small CPU budgets offer a clear design goal
Critical weaknesses
  • WRONG template for ICLR. title font and abstract margin are incorrect.
  • Very difficult to read. Ideas are not presented clearly.
  • No demo page we can listen to at all. So we dont know how such low-parameter count tts performs.
  • The quality-superiority claim rests on a small volunteer MOS that every automatic metric contradicts.
  • No discussion on what machines are used to train, and for how many steps / how long
Questions for authors

see above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.