- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces Mugi, a unified framework for text-to-song generation, cover song generation, and timbrecontrollable music generation. Its main contributions are a joint autoregressive–diffusion training strategy, which allows acoustic flow-matching losses to update the autoregressive model, and a cover-oriented tokenizer that compresses multiple musical structure features into discrete melody tokens. This work evaluates Mugi using automatic metrics and human judgments, comparing it with open-source and commercial systems. The results suggest competitive performance across the three tasks, with improvements in some cover-fidelity metrics, while also revealing trade-off between structural fidelity, transcription accuracy, and overall generation quality.
Critical strengths
- This paper proposes a coherent multi-task formulation. Mugi addresses text-to-song, cover generation, and timbre control within a shared framework, provides a useful setting for studying how shared representations can support different music-generation objectives.
- This paper proposes a technically motivated joint-training strategy. The paper allows acoustic flow-matching losses to update the autoregressive model, and the ablation study provides preliminary evidence that continuous hidden-state conditioning and joint training can improve generation metrics. The coveroriented representation also explicitly incorporates multiple aspects of musical structure rather than relying solely on vocal pitch.
- The experiments in this paper incorporate human judgement to make the results more realistic.
- This paper presents an innovative and cutting-edge research.
Critical weaknesses
- The evaluation relies on automatic metrics, including SongBench, audio aesthetics, text consistency, phoneme error rates, and melody and pitch accuracy. However, these metrics may not fully capture overall song quality, such as structural coherence, arrangement quality, and vocal expressiveness. High metric scores do not necessarily translate into better human perception, and further analysis is needed to establish their alignment with subjective judgments.
- Using chord roots helps preserve the original song’s harmonic progression but may limit the flexibility of reharmonization. Strictly preserving the original harmony can constrain arrangement diversity, while the effectiveness of the current representation under substantial changes to lyrics, melody, or harmony remains to be validated. The current evaluation also does not sufficiently demonstrate the model’s controllability across diverse and complex editing scenarios.
Questions for authors
Each task is evaluated on 60 test examples. Could the authors provide analyses of sensitivity to test-set selection and statistical uncertainty, to establish whether the reported performance differences and model rankings are robust, particularly when the differences in aggregate scores are small?
Response to Reviewer jZJ2
We thank the reviewer for the encouraging assessment of our work.
1. Automatic metrics and human perception (W1)
We agree that automatic metrics cannot fully capture aspects such as structural coherence, arrangement quality, and vocal expressiveness. This is why our evaluation includes human judgments: the main text-to-song comparison is based on human Mean Opinion Score (Table 1), and we have additionally conducted human listening studies for cover generation and timbre control (please see the response to Reviewer fshb for the full results). Encouragingly, the human results are consistent with the automatic metrics: the systems that lead on the automatic melody metrics also receive the highest melody faithfulness ratings from human listeners. We will report these human results and discuss the scope of each automatic metric more explicitly in the revision.
2. Chord roots and reharmonization flexibility (W2)
We appreciate this point, and we discuss it in Section 7 of the paper. Conditioning on chord roots is a deliberate design choice for the cover task as we define it, which targets faithful preservation of the reference song. As noted in Section 7, this limits the freedom to reharmonize, and whether a cover should retain the original harmony is an open question: faithful reproduction and creative reinterpretation call for different levels of harmonic control. Exploring more flexible harmonic conditioning is a promising direction for future work, and we will explicitly state this trade-off when introducing the task.
3. Test-set size and statistical uncertainty (Q1)
We agree that statistical reliability should be established given the 60-example test sets. We ran an item-level paired bootstrap (B = 10,000 resamples of the 60 items; percentile 95% CI) over the per-example scores of the text-to-song comparison. All eight systems are scored on the identical 60 items (30 English + 30 Chinese) with no missing values, so every comparison is paired rather than between-sample. We report the aggregate SongBench Mean, the main metric we reported in the text-to-song generation task.
Table A. SongBench Mean over the 60 test items, with percentile-bootstrap 95% CI (B = 10,000).
We observed:
First, Mugi's interval is the narrowest of the eight systems (±0.043), because its per-item scores are less dispersed than those of the baselines; the aggregate is therefore not resting on a few unusually good items.
Second, Mugi ranks 3rd of 8 overall, behind the two commercial systems (Mureka-V9, Suno-v6); those gaps (−0.407 and −0.207) are several times the CI width and are stable under resampling. Among the five open-source systems, Mugi is ranked first in 99.3% of bootstrap resamples, and it is first among open-source systems in the English half (6.586) and the Chinese half (6.482) separately, so the ranking is not an artifact of one language.