acceptodds
Under review as a conference paper at ICLR 2027

Mugi: Multi-task Song Generation with Joint Autoregressive-Diffusion Training and Cover-Oriented Tokenization

Abstract

Song generation has advanced rapidly in recent years, but most existing studies still focus on text-conditioned generation, with limited support for tasks such as cover song generation. Meanwhile, recent systems increasingly adopt a two-stage paradigm in which an autoregressive (AR) language model predicts high-level representations and a diffusion model renders acoustic details. However, these two modules are typically trained separately and connected only through discrete tokens. In this paper, we present Mugi, a unified song generation model that supports text-to-song, cover song, and timbre-controllable song generation within a single framework. To support high-fidelity cover song generation, we introduce a Cover-Tokenizer that represents melodic information and linguistic content as two parallel token streams, enabling the generated song to faithfully preserve melodic, harmonic, and rhythmic characteristics while maintaining the phoneme-level temporal alignment of the lyrics. Furthermore, Mugi jointly trains the AR and diffusion stages by directly conditioning the diffusion model on the continuous hidden states of the AR model, optimizing the two stages together in a single end-to-end training process using both the AR language-modeling loss and the flow-matching objective. Extensive subjective and objective evaluations demonstrate strong performance across multiple song generation tasks, with objective results competitive with commercial systems.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Song Generation, Music Generation, Cover Song Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_jZJ2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Mugi, a unified framework for text-to-song generation, cover song generation, and timbrecontrollable music generation. Its main contributions are a joint autoregressive–diffusion training strategy, which allows acoustic flow-matching losses to update the autoregressive model, and a cover-oriented tokenizer that compresses multiple musical structure features into discrete melody tokens. This work evaluates Mugi using automatic metrics and human judgments, comparing it with open-source and commercial systems. The results suggest competitive performance across the three tasks, with improvements in some cover-fidelity metrics, while also revealing trade-off between structural fidelity, transcription accuracy, and overall generation quality.

Critical strengths
  1. This paper proposes a coherent multi-task formulation. Mugi addresses text-to-song, cover generation, and timbre control within a shared framework, provides a useful setting for studying how shared representations can support different music-generation objectives.
  2. This paper proposes a technically motivated joint-training strategy. The paper allows acoustic flow-matching losses to update the autoregressive model, and the ablation study provides preliminary evidence that continuous hidden-state conditioning and joint training can improve generation metrics. The coveroriented representation also explicitly incorporates multiple aspects of musical structure rather than relying solely on vocal pitch.
  3. The experiments in this paper incorporate human judgement to make the results more realistic.
  4. This paper presents an innovative and cutting-edge research.
Critical weaknesses
  1. The evaluation relies on automatic metrics, including SongBench, audio aesthetics, text consistency, phoneme error rates, and melody and pitch accuracy. However, these metrics may not fully capture overall song quality, such as structural coherence, arrangement quality, and vocal expressiveness. High metric scores do not necessarily translate into better human perception, and further analysis is needed to establish their alignment with subjective judgments.
  2. Using chord roots helps preserve the original song’s harmonic progression but may limit the flexibility of reharmonization. Strictly preserving the original harmony can constrain arrangement diversity, while the effectiveness of the current representation under substantial changes to lyrics, melody, or harmony remains to be validated. The current evaluation also does not sufficiently demonstrate the model’s controllability across diverse and complex editing scenarios.
Questions for authors

Each task is evaluated on 60 test examples. Could the authors provide analyses of sensitivity to test-set selection and statistical uncertainty, to establish whether the reported performance differences and model rankings are robust, particularly when the differences in aggregate scores are small?

1 comment
AuthorsOpenReview ↗

Response to Reviewer jZJ2

We thank the reviewer for the encouraging assessment of our work.

1. Automatic metrics and human perception (W1)

We agree that automatic metrics cannot fully capture aspects such as structural coherence, arrangement quality, and vocal expressiveness. This is why our evaluation includes human judgments: the main text-to-song comparison is based on human Mean Opinion Score (Table 1), and we have additionally conducted human listening studies for cover generation and timbre control (please see the response to Reviewer fshb for the full results). Encouragingly, the human results are consistent with the automatic metrics: the systems that lead on the automatic melody metrics also receive the highest melody faithfulness ratings from human listeners. We will report these human results and discuss the scope of each automatic metric more explicitly in the revision.

2. Chord roots and reharmonization flexibility (W2)

We appreciate this point, and we discuss it in Section 7 of the paper. Conditioning on chord roots is a deliberate design choice for the cover task as we define it, which targets faithful preservation of the reference song. As noted in Section 7, this limits the freedom to reharmonize, and whether a cover should retain the original harmony is an open question: faithful reproduction and creative reinterpretation call for different levels of harmonic control. Exploring more flexible harmonic conditioning is a promising direction for future work, and we will explicitly state this trade-off when introducing the task.

3. Test-set size and statistical uncertainty (Q1)

We agree that statistical reliability should be established given the 60-example test sets. We ran an item-level paired bootstrap (B = 10,000 resamples of the 60 items; percentile 95% CI) over the per-example scores of the text-to-song comparison. All eight systems are scored on the identical 60 items (30 English + 30 Chinese) with no missing values, so every comparison is paired rather than between-sample. We report the aggregate SongBench Mean, the main metric we reported in the text-to-song generation task.

Table A. SongBench Mean over the 60 test items, with percentile-bootstrap 95% CI (B = 10,000).

SystemMean95% CIhalf-widthEnglish half (30)Chinese half (30)
Suno-v66.741[6.667, 6.815]±0.0746.6686.815
Suno-v6-wild6.446[6.362, 6.528]±0.0836.4556.436
Mureka-V96.941[6.881, 7.001]±0.0606.9126.971
ACE-Step-1.56.087[6.002, 6.169]±0.0835.9286.247
HeartMuLa6.278[6.211, 6.343]±0.0666.2606.295
LeVo26.301[6.192, 6.398]±0.1036.4206.182
MiniMax-Music36.411[6.324, 6.496]±0.0866.3976.424
Mugi (Ours)6.534[6.491, 6.577]±0.0436.5866.482

We observed:

First, Mugi's interval is the narrowest of the eight systems (±0.043), because its per-item scores are less dispersed than those of the baselines; the aggregate is therefore not resting on a few unusually good items.

Second, Mugi ranks 3rd of 8 overall, behind the two commercial systems (Mureka-V9, Suno-v6); those gaps (−0.407 and −0.207) are several times the CI width and are stable under resampling. Among the five open-source systems, Mugi is ranked first in 99.3% of bootstrap resamples, and it is first among open-source systems in the English half (6.586) and the Chinese half (6.482) separately, so the ranking is not an artifact of one language.

Reviewer_zCfVOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Mugi combines a 3B autoregressive language model with a 0.6B chunk-wise diffusion model for text-to-song generation, cover generation, and voice-timbre control. Continuous language-model hidden states condition diffusion, and the acoustic loss updates both components. Cover-Tokenizer supplies separate musical and phonetic conditioning streams. Comparisons include released models and commercial services. Internal ablations test continuous conditioning, gradient flow across the connection, and the contribution of individual cover-conditioning features.

Critical strengths
  • Table 5 separates continuous conditioning from joint optimization under a shared 15,000-step training setup. Continuous states improve SongBench and MuQ similarity relative to discrete tokens. Enabling gradient flow then reduces PER from 0.110 to 0.075 compared with detached continuous states. This comparison supports the connection design beyond the final system's benchmark ranking.
  • Cover-pitch results show a measurable gain over the evaluated baselines. In Table 3, Mugi obtains MIDI MAE of 0.62 and F0 RPA of 0.724, compared with 1.03 and 0.585 for MiniMax-Music-Cover. Table 6 tests musical features and content tokens from a shared initialization. Pitch and lyric-content gains accompany a small decrease in SongBench.
  • Section 2.1 distinguishes Mugi's continuous connection from SongBloom's interleaved design, and the appendices document tokenizer construction, task sampling, and evaluation. Listening results also retain the gap to commercial services. Table 1 reports a mean of 3.50 for Mugi, compared with 3.55 for MiniMax-Music3 and at least 3.78 for the other listed commercial systems.
Critical weaknesses
  • Cover-preservation claims extend beyond the properties measured in Appendix A.7. Vocal pitch is evaluated after dynamic time warping, which can accommodate timing differences, and PER measures transcription accuracy rather than phoneme timing. Neither establishes preservation of accompaniment harmony or rhythm. Appendix A.4 measures feature reconstruction by Cover-Tokenizer, not preservation of those features in generated covers. I would restrict the current claim to demonstrated pitch and lyric-content results, with targeted harmony, beat, and phoneme-alignment measurements reserved for a revision that retains the broader claim.
  • Section 5.1 says all lyrics come from an API, yet Table 3 does not identify whether covers retain source lyrics and use both conditioning streams as described in Section 3.3. Tables 1 and 2 also group MiniMax-Music3 with open-source systems, despite Appendix A.3 describing API access to a closed-source model. Specify the cover lyrics, conditioning mode, and PER reference, and reconcile the MiniMax-Music3 grouping with the documented API version.
Questions for authors

Do Table 3 target lyrics match the SSLD-200 source songs, and are both Cover-Tokenizer streams supplied? How does API-generated lyric content apply to this evaluation and its Identity row?

1 comment
AuthorsOpenReview ↗

Response to Reviewer zCfV

We thank the reviewer for the careful reading and the precise questions on our evaluation protocol.

1. Scope of the cover-preservation claims

Thank you for pointing out the gap between the claims and the measured properties. We will restrict the claim wording to match the measurements, and we also add direct preservation measurements on generated covers, which we report below.

We introduce four new metrics computed between the generated cover and the reference song. chordABS and chordDTW measure frame-level chord-root accuracy at absolute times and after DTW alignment, respectively. dbtABS measures downbeat accuracy at absolute times with a 70 ms tolerance. Since Mugi generates covers autoregressively, a generated song may be uniformly shifted in time relative to the reference without any musical misalignment; dbtOFF therefore reports downbeat accuracy under the best global temporal offset, which we believe is the fairer measure for autoregressive cover generation.

VariantchordABSchordDTWdbtABSdbtOFF
F00.2300.5840.0650.214
F0+MIDI0.2260.5800.0720.419
F0+MIDI+Chord+Downbeat0.4510.8350.0870.525
F0+MIDI+Chord+Downbeat + Content0.4810.8340.1980.605

Adding chord roots raises chordDTW from 0.58 to 0.84, and adding downbeats raises dbtOFF from 0.42 to 0.53, with a further gain from content tokens. The preservation gains appear exactly when the corresponding conditioning feature is introduced, which provides generation-side evidence that the Cover-Tokenizer features cause the preservation of harmony and rhythm.

Furthermore, we organized a human listening study on cover generation, in which raters specifically evaluated melody faithfulness and lyric-position alignment against the reference song. Lyric-position alignment in particular cannot be captured by PER, which measures transcription accuracy rather than timing, so we assessed it directly with human raters.

ModelMelody faithfulnessLyric-position alignment
MiniMax-Music-Cover (commercial)4.103.95
Mugi4.174.11
SongEcho3.803.80
ACE-Step-1.5-xl-sft2.543.05

Mugi achieves the highest score on both dimensions, surpassing the commercial MiniMax-Music-Cover. Together with the chord and downbeat measurements reported above, these results show that the melodic, harmonic, rhythmic, and lyric-timing properties of the reference are preserved in the generated covers, and that this preservation is perceptually meaningful to human listeners.

We will add these metrics to Table 6 and update the claims in Section 3.3 and Appendix A.7 accordingly.

2. Lyrics source, conditioning mode, and the MiniMax-Music3 grouping

We apologize for the unclear description, and we will revise Sections 5.1 and A.3 to make the protocol explicit.

Lyrics and conditioning for covers. All cover experiments use the original lyrics of the SSLD-200 source songs, and both Cover-Tokenizer streams (melody tokens and content tokens) are supplied to the model. The MiniMax Lyrics Generation API is used only to generate new lyrics for the text-to-song and timbre-control test sets, it does not affect the cover evaluation. The PER reference for the cover task is the original source lyrics. We will state all of this in the revision.

MiniMax-Music3. Thanks for the careful reading, the description in our appendix was indeed inconsistent. The accurate status is as follows: MiniMax-Music3 is an open-source model, but the open-source release supports text-to-song generation only, while its cover capability is available exclusively through the API service. Accordingly, our text-to-song comparisons (Tables 1 and 2) use the open-source weights MiniMax-Music3, and the cover comparison (Table 3) uses the API service, as no open-source cover functionality is available. In the revision, we will introduce a separate "Partially open-source models" group in the appendix A.3 for systems of this kind, and annotate the corresponding tables to indicate which form of each system is used in each comparison.

Reviewer_fshbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Mugi is a unified full-song model for text-to-song, cover generation, and timbre control. A 3B Llama-3.2 AR LLM predicts 25 Hz Mix Music Codec tokens; a 0.6B Chunk-DiT renders VAE latents. Chunk-DiT is conditioned on continuous LLM hidden states (not discrete tokens), so L = L_CE + λ L_FM trains both stages jointly (§3.2, Eqs. 1–4, Fig. 1).

For covers, a Cover-Tokenizer emits two 25 Hz streams (§3.3, Fig. 2): a VQ melody stream trained on vocal F0, vocal MIDI, chord roots, and downbeats (Eq. 5), plus a CosyVoice S3 content stream for phonetic timing (OpenVoice timbre shift to cut leakage). Timbre control uses CAM++ embeddings with a centrality-based selection rule (§3.4).

Three 60-example EN/ZH test sets are used (§5.1). On text-to-song, Mugi reports the best open-source SongBench mean (6.53) and competitive AudioBox/MuQ vs commercial systems (Table 2); human MOS trails Suno/Mureka and is slightly below MiniMax-Music3 at fewer parameters (Table 1). On covers, it leads MIDI MAE / F0 RPA/RCA among compared systems (Table 3). Tables 5–6 support continuous conditioning + joint training and the multi-feature Cover-Tokenizer.

Critical strengths

1. Cover-Tokenizer. Prior cover systems often rely on vocal F0/MIDI alone (§2.2). Mugi's parallel melody stream (F0 + MIDI + chord roots + downbeats) plus CosyVoice content tokens (§3.3, Fig. 2) target accompaniment harmonic/rhythmic identity and lyric–melody timing. Table 6 shows monotonic melody-metric gains as features are added; Table 3 reports best MIDI MAE / F0 RPA/RCA among compared systems, including MiniMax-Music-Cover. This is the clearest advance for the cover setting.

2. Continuous conditioning / joint training are isolated. Table 5 compares discrete-token conditioning (separate training), continuous latents with stopped gradients, and joint continuous training. Continuous conditioning improves SongBench/MuQ; joint gradients further improve SongBench and cut PER (0.110 → 0.075). With competitive multi-task results at 3.6B generative parameters (Tables 1–4), the architecture is more than a packaging of known modules.

Critical weaknesses

1. Incomplete evaluation of the paper's distinctive claims. Cover and timbre are central, yet human evaluation covers only text-to-song (§6.1.1, Table 1; §A.8). Cover quality rests largely on automatic SongBench/AudioBox/MuQ plus pitch metrics (Table 3); style transfer and perceived cover quality under different target captions are not human-validated. Test sets are small (60 each). SongBloom, the closest joint AR–diffusion song system (§2.1), is not compared. It is hard to tell whether multi-task / joint-training gains are decisive or partly confounded by private data scale (5M songs; §4.2) and metric choice.

2. Joint AR–diffusion novelty is narrower than claimed, and reproducibility is provisional. Continuous AR states into a diffusion renderer with L_CE+λ L_FM is a sensible design, but SongBloom already explores joint AR+diffusion song generation, and cited image/speech work has similar ideas. The differentiation (parallel hidden-state conditioning, no discrete tokens to DiT) is real but incremental. Training data are proprietary; code/checkpoints are promised only after review. For a systems paper whose strongest numbers rely on a private corpus, that weakens the case until release or stronger public baselines.

Questions for authors
  1. SongBloom. §2.1 calls SongBloom the closest joint AR–diffusion song model. Can you provide a matched lyrics/captions comparison for text-to-song, or explain why not? A smaller matched subset would still help separate novelty from systems engineering.

  2. Cover / timbre listening tests. Will you add MOS for cover and timbre (musicality, melody faithfulness, style adherence, speaker similarity)? Automatic F0/MIDI metrics can reward pitch cloning that listeners still find unnatural.

  3. Ablation setup. Table 6 starts from a checkpoint without cover-task pretraining and trains 5k steps. Do the same feature rankings hold from the multi-task pretrained model, or under a budget closer to final SFT?

  4. Inference path. At inference, does Chunk-DiT condition on AR hidden states from sampled tokens (teacher-forcing gap)? How large is the train/test mismatch, and did you try scheduled sampling or self-conditioned hidden states?

  5. Commercial claim. Table 2: Mugi beats Suno-v6-wild on SongBench mean but trails Suno-v6 and Mureka-V9; Table 1 human MOS still favors commercial systems. Would you narrow the abstract's "competitive with commercial systems" wording to specific systems/metrics, or add human results that support the broader claim?

2 comments
AuthorsOpenReview ↗

Response to Reviewer fshb (Part I)

We sincerely thank you for the thoughtful comments and helpful suggestions. These questions helped us clarify several important aspects of the paper, and we respond to each point below.

1. Novelty of the joint AR-diffusion design, and reproducibility (W2)

We would like to clarify the scope of our contribution. We do not claim to be the first to connect an AR model and a diffusion renderer. Our contribution is a specific connection scheme with two properties that, to our knowledge, have not been demonstrated together in full-song generation:

  1. The diffusion renderer is conditioned purely on continuous, frame-aligned hidden states of the AR model, without any discrete intermediate token stream. In SongBloom, the diffusion stage receives discrete sketch tokens and one hidden vector per 0.64s patch; in Mugi, the Chunk-DiT consumes the hidden state at every AR step (25 Hz), so no information is forced through a discrete bottleneck between the two stages.
  2. Both stages are optimized jointly from the beginning of pretraining over all tasks, which Table 5 shows is responsible for an improvement compared with the detached variant.

We agree the framing in Section 2.1 could suggest a broader claim, and we will rephrase it in the revision along the lines above.

On reproducibility: we will release the model checkpoints, code, and the Cover-Tokenizer.

2. Comparison with SongBloom (Q1)

We added SongBloom to the text-to-song comparison, re-running it on the identical 60 test items and scoring it with the same pipeline and reference lyrics as every other system, so the comparison is fully paired. One setup difference is worth noting: SongBloom specifies style with a 10-second audio prompt rather than a text caption, so for each item we retrieved the MusicCaps segment (out of 326) whose MuQ-MuLan similarity to that item's caption was highest, and used it as the prompt. Mugi, by contrast, is driven directly by the text caption.

Table. Mugi vs. SongBloom on the lyric-to-song generation.

SystemSongBench MeanMel.Arr.Mus.Voc.Ins.Mix.Str.AudioBox CECUPCPQMuQ Sim.ASR PER ↓
Mugi (Ours)6.5346.5526.8935.7526.6206.8906.6176.4167.7257.9946.8048.4090.3910.064
SongBloom5.0245.1165.2284.0875.4255.5254.9674.8177.0777.5965.4787.9360.2640.135

Mugi outperforms SongBloom on every metric, with a clear margin on the SongBench mean (6.534 vs. 5.024) and on PER (0.064 vs. 0.135). We will add this comparison to Table 2.

3. Human evaluation for cover and timbre control (W1, Q2)

We thank the reviewer for this suggestion. We have conducted a human listening study on a 10-example test set for both tasks.

Cover generation. MiniMax-Music-Cover is a commercial system; all other compared systems are open-source. Bold indicates the best result among all systems; italic indicates the best among open-source systems.

ModelMusicalityCaption adherenceMelody faithfulnessLyric alignmentMean
MiniMax-Music-Cover (commercial)3.763.694.103.953.88
Mugi3.583.464.174.113.83
SongEcho2.512.883.803.803.25
ACE-Step-1.5-xl-sft2.172.652.543.052.60

Among open-source systems, Mugi achieves the best result on every metric, and it also achieves the highest melody faithfulness and lyric-position alignment overall, surpassing the commercial MiniMax-Music-Cover. Human raters confirm that Mugi's melody preservation is perceptually faithful. MiniMax-Music-Cover scores higher on musicality and caption adherence, which is consistent with the trade-off between reference adherence and musical aesthetics reported in Table 6.

Timbre control.

ModelMusicalityCaption adherenceTimbre similarityMean
Mugi (Ours)3.613.483.453.51
ACE-Step-1.5-xl-sft3.903.832.793.51
LeVo23.753.432.503.23

Mugi achieves the highest timbre similarity, the defining metric of the task, and ties ACE-Step-1.5-xl-sft on the mean score. We observe an interesting interaction in the human ratings: models with low timbre similarity tend to default to a small set of generally pleasant-sounding voices, which raters reward with higher musicality scores. Following the voice prompt faithfully, by contrast, means rendering whatever timbre the prompt contains, pleasant or not, so higher timbre fidelity can trade off against perceived musicality.

AuthorsOpenReview ↗

Response to Reviewer fshb (Part II)

4. Ablation initialization (Q3)

This is a fair question, and it is one we thought about when designing the study. To recap our setup: in Table 6, every ablation variant starts from a checkpoint pretrained without the cover task, and is then fine-tuned for cover generation under identical settings. This starting point is a deliberate choice rather than only a budget consideration.

We did consider running the ablation from the same multi-task pretrained checkpoint, fine-tuning each Cover-Tokenizer configuration from it (F0, F0+MIDI, F0+MIDI+chord+downbeat, and the melody+content variant). However, this introduces an unfairness, because the multi-task pretraining already uses one specific Cover-Tokenizer configuration. For example, if pretraining uses our final melody+content configuration, then the other four variants fine-tuned from that checkpoint are at a disadvantage, since the model has already been trained with the melody+content configuration for many more steps during pretraining. On the other hand, a fully matched alternative would be to pretrain each variant separately with its own cover configuration, but that costs five full pretraining runs, it is beyond what we can afford.

Therefore, we run the ablation in an SFT setting, starting from a checkpoint pretrained without the cover task and then fine-tuning with a number of steps similar to SFT stage. This is the only setup that is both fair and affordable: it requires only an SFT-level budget, no configuration is favored, and every feature's contribution is learned under the same conditions.

5. Inference path and the teacher-forcing gap (Q4)

Yes, at inference the Chunk-DiT conditions on hidden states computed from the AR model's own sampled tokens, so a teacher-forcing gap exists in principle. In practice, however, we do not observe its typical symptom: there is no progressive quality degradation over long generations. The audio samples on our demo page, which include full-length songs of up to 5 minutes, remain stable in both vocal quality and musical structure from beginning to end.

We attribute this stability to the supervision on discrete tokens, which anchors the AR model's generation. Since the hidden states are computed before these tokens and are thus bound to the token trajectory, they inherit this stability: as long as the sampled tokens remain on track, the conditioning signal seen by the Chunk-DiT stays close to what it was trained on.

Regarding the two remedies mentioned: for self-conditioning the AR model on its own hidden states, recent systems such as VibeVoice and VoxCPM2 show that related designs can be made to work in speech, and adapting them to song generation is a direction worth exploring. Scheduled sampling is also an interesting option that we leave for future work.

6. Commercial claim (Q5)

We agree the current wording is broader than the evidence supports. We will revise the abstract to state that Mugi is competitive with specific commercial systems, namely Suno-v6-wild on the SongBench metrics.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses

Dear Area Chair,

We thank the three reviewers for their constructive reviews. Here we summarize the main concerns and how we addressed them.

New results during the discussion period

  • SongBloom comparison (fshb Q1): We re-ran SongBloom on the identical 60 test items with the same scoring pipeline. Mugi outperforms it on every metric.
  • Human study on cover generation (fshb W1/Q2, jZJ2 W1): Mugi is the best open-source system on all metrics, and surpasses the commercial MiniMax-Music-Cover on melody faithfulness (4.17 vs. 4.10) and lyric-position alignment (4.11 vs. 3.95).
  • Human study on timbre control (fshb W1/Q2): Mugi achieves the highest timbre similarity (3.45) and ties the best baseline on the mean.
  • Direct harmony/rhythm preservation metrics (zCfV W1): four new metrics on generated covers (chordABS/chordDTW, dbtABS/dbtOFF). Preservation gains appear exactly when the corresponding Cover-Tokenizer feature is added (chordDTW 0.58 → 0.84 with chord roots; dbtOFF 0.42 → 0.53 with downbeats).
  • Statistical robustness analysis (jZJ2 Q1): paired bootstrap (B = 10,000) with 95% CIs. Mugi has the narrowest CI (±0.043) and ranks first among open-source systems in 99.3% of resamples, in both languages separately.

Clarifications

  • Novelty scope (fshb W2): our claim focus on the continuous connection scheme in AR+diffusion combination. Continuous 25 Hz hidden-state conditioning with no discrete bottleneck, trained jointly from pretraining.
  • Evaluation protocol (zCfV W2): cover experiments use the original SSLD-200 lyrics with both conditioning streams; the lyrics API is used only for the text-to-song and timbre test sets. MiniMax-Music3 is open-source for text-to-song but API-only for cover.
  • Ablation setup (fshb Q3): we explained why starting from a checkpoint without cover pretraining is the setup that is both fair (no configuration favored) and affordable.
  • Teacher-forcing gap (fshb Q4): although the teacher-forcing gap exists theoretically, in practice there is no degradation observed in long generations.

Planned updates

  1. Add all new results above to Tables 2, 3, and 6.
  2. Narrow the abstract's commercial comparison to specific systems/metrics, and restrict cover-preservation claims to the measured properties.
  3. Release model, code, and the Cover-Tokenizer.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.