- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents SketchSong, a hierarchical song generation framework built on the two-stage autoregressive paradigm of LeVo. Along the temporal dimension, the first-stage model predicts a low-rate sequence of discrete sketch tokens (MuQ-MuLan RVQ token) before generating mixed audio tokens, providing an explicit song-level plan. Along the track dimension, the second stage extends LeVo's vocal/accompaniment formulation to four explicit tracks (vocals, bass, drums, other instruments). Experiments on a 1M-song corpus show consistent improvements over a retrained LeVo baseline on objective metrics and listening tests, and ablations indicate the two components contribute complementarily.
Critical strengths
-
The two designs are well motivated, and the evidence matches the claims unusually well. Predicting a compact MuQ-MuLan-based sketch before the audio tokens is a sensible way to give the model a song-level plan, and the ablations line up with the motivation: sketch planning mainly helps structure, musicality and memorability, while four-track modeling mainly helps production quality and instrumental richness.
-
The sketch replacement experiment (Table 4) is probably the most convincing part of the paper. Fixing the prompt and swapping in a ballad or rock sketch shifts the output style consistently across 20 diverse prompts, which shows the sketch is a real control interface rather than an incidental gain.
Besides, the evaluation practice is honest and the paper is easy to read. The authors retrain LeVo on their own data and setup for a fair comparison while also reporting the released checkpoint, which is more rigorous than most song generation submissions I have seen. They are also candid about what the system does not do (no post-training, shared accompaniment codec, naive remixing), which makes the positive results more trustworthy.
Critical weaknesses
-
The framework is largely an incremental combination on top of LeVo. Using MuLan-type embeddings as a low-rate planning sequence ahead of audio token generation was already established by MusiCoT (CLAP as musical CoT). The paper cites it but does not articulate what is methodologically different beyond swapping the feature extractor.
-
The multi-track contribution is undermined by its decoding path. No dedicated codecs are trained for bass, drums, or other instruments. All three streams are decoded by LeVo's accompaniment codec, which was trained on mixed accompaniment and is therefore used out of distribution on isolated stems, and then remixed with a fixed ratio.
Also, there is no comparison with commercial systems (e.g., Suno v6) or stronger recent baselines (e.g., LeVo 2). Additionally, the reported PER metric is really poor.
Questions for authors
-
What exactly distinguishes the sketch planning here from MusiCoT's musical CoT, in terms of representation, granularity, and training schedule? Please state this explicitly in the paper.
-
Could you add comparisons with stronger baselines, e.g., LeVo 2, and commercial systems such as Suno-v6 if feasible? In addition, please consider including newer evaluation models such as SongBench in the objective metrics.
-
Have you trained or fine-tuned dedicated codecs for the bass, drums, and other tracks, even at a small scale? Relatedly, is there any evidence that reusing LeVo's two-stem codecs for four-track decoding is adequate, for example codec reconstruction metrics?
Other minor issues:
- The paper states it uses MuQ-MuLan but cites Huang et al. (2022), the original MuLan. These are different models (MuQ-MuLan, DOI: 10.1109/TASLPRO.2025.3602320). Which one was actually used, both for sketch extraction and for the MuQ-T/MuQ-A metrics? Please clarify and fix the citation.
- Why is PER so much higher than the released LeVo checkpoint for both the baseline and SketchSong? A brief diagnosis would be appreciated.
I find the framework clean and the sketch controllability analysis genuinely interesting, but the novelty over MusiCoT and the comparison set are not sufficient as presented. If these points are addressed convincingly in the rebuttal, I am willing to raise my score.