- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose in this paper a music source separation method that operates in the latent space and is capable of separating 12 active stems.
Critical strengths
The paper has several strengths:
- extending music source separation beyond the classic 4-stems task, tackling up to 12 stems mixtures;
- An extended and very solid perceptual evaluation ;
- A well designed demo page with impressive separation results
- the contribution of a re-annotated version of MoiseDB dataset to tackle 12 stems mixtures source separation
Critical weaknesses
The paper also has several weaknesses:
-
The paper is not always clear on the exact evaluation protocol. Are all test data different (unseen) from the data used for training ? Are the test data similar (same musical genre / type of mixing ..etc ) than the training data ? More problematic, a large part of the paper uses a held-out internal set for which very little information is given. Also, since it is an internal held-out set, the results are not reproducible despite the potential release of the code and weights.
-
The paper is not very clear on what are the key contributions. The authors mention that the latent-based source separation is one of them while many existing methods are also doing the source separation in the latent space. It seems that the main contribution is to extend source separation beyond 4-stems (and up to 12-stems mixtures) though this is not well dicussed in light of the existing literature.
-
Bibliography/ references: linked to the point above, the authors have only used a limited set of references for positioning their work. Though, they are a number of previous work that have tackled different settings than the classic vocal-drums-bass-others task. For instance (but many other exist in both supervised and unsupervised music source separation), there is work on Drum separation (for instance with 9 stems separation in [1]), in choir singing separation (4 homogeneous stems, see for example [2,3]) or in orchestral music (with many stems, see for example 4).
Other more minor points:
- It is not clear in the data used what is the amount of bleeding and how often it occurs (from most examples in the demo page it seems that the bleeding is absent).
- There are no indications of complexity (how heavy is the training of the model in terms of computing resources) ?
- It is not clear what is exactly the mix input creation (section 3. "Batch sampling during training").... but this goes again to the comment that it is not clear what is exactly the data used.
- Objective metrics: there are quite some explanations concerning residual damp or silence prediction without using specific metrics to evaluate this aspect. The metrics proposed in [5] for Predicted Energy at Silence (PES) or Energy at Predicted Silence (EPS) could probably be used to evaluate this aspect.
- the section on "the targets are spectrally entangled" is not very informative and the outcomes seem rather straightforward. If space is needed in a potential paper update, this section could be placed in appendix.
[1] B. Torres & al. "The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesis," in IEEE Transactions on Audio, Speech and Language Processing, vol. 34, pp. 84-95, 2026, [2] D. Petermann & al. “Deep learning based source separation applied to choir ensembles,” in Proc. Int. Soc. Music Inf. Retrieval Conf., 2020, pp. 733–739 [3] K. Schulze-Forster & al. "Unsupervised Music Source Separation Using Differentiable Parametric Source Models," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 1276-1289, 2023 [4] J. Garcia-Martinez et al., "The Spheres Dataset: Multitrack Orchestral Recordings for Music Source Separation and Information Retrieval," in IEEE Transactions on Audio, Speech and Language Processing, vol. 34, pp. 2622-2634, 2026 [5] K. Schulze-Forster & al. "Phoneme Level Lyrics Alignment and Text-Informed Singing Voice Separation," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 2382-2395, 2021
Questions for authors
- Can you clarify the exact evaluation protocol : Are all test data different (unseen) from the data used for training ? Are the test data similar (same musical genre / type of mixing ..etc ) than the training data ? will the data be distributed ? how does the method perform on out-of-domain data ?
- what are the key contributions of the paper in light of an extended bibliography ?
- what is the complexity of the method (training) ?
3 comments
I would like to thank the reviewers for their answers. Most of my comments are satisfactorily addressed and taken this new insights, I will slightly raise my score.
We thank reviewer 3Wk5 for their engagement and for raising their score in recognition of our work.
We are pleased our rebuttal addressed most of their comments, and would be happy to address remaining concerns they may still have.
Comment: 3Wk5
We thank reviewer
3Wk5for their detailed review. We group our responses by topic.Evaluation protocol and data (W1, Q1)
The data sources are licensed multi-track songs obtained from external vendors (disjoint from MoisesDB). Our splits contain 13,741 / 783 / 765 songs (train / validation / evaluation), corresponding to 649.8h / 37.3h / 36.4h with ≥ 2 stems overlap; the sets are distinct by artist (App. K, Tab. 7).
To evaluate out-of-domain performance, we use MoisesDB, with all baselines retrained on our corpus: it includes unseen artists, potentially unseen instruments, different production setups and a different taxonomy, and was never used for design, tuning or model selection (Sec. 5.1). There, a song-clustered paired bootstrap places DeFormer +0.58 dB above BS-RoFormer (p = 0.026; App. F.1), and the blind listening study, drawn from MoisesDB, rates it highest (Sec. 5.3). Our MoisesDB results can be reproduced with our released model checkpoint.
In addition, every Table 2 configuration evaluated on MoisesDB shows similar trends as on our internal set: Fusion (+0.29 dB) and De-Mix (+0.69 dB), the latter significant at the window, song and artist level (p ≤ 0.01; Tab. 9, App. N).
Key contributions (W2, Q2)
Our main contributions are:
Separating at the bottleneck is a design choice that lets cross-source joint attention be used effectively. We agree with the reviewer's point on the "latent-based" framing and rephrased it in the revision (Sec. 1, 4.1).
Related work (W3)
We thank the reviewer for the references, and agree that our related work could be expanded. Given the 9-page limit, we focused the related work on the approaches we found to be most relevant to our setting: general multi-instrument separation over diverse music. The suggested works address separation beyond four stems in specific, more homogeneous settings (drum kit components [1], choir voices [2, 3], orchestral instruments [4]); following the reviewer's suggestion, we added them to the related work (Sec. 2).
Minor points