acceptodds
Under review as a conference paper at ICLR 2027

DeFormer: Latent-based music source separation beyond four stems.

Abstract

In music source separation, we aim to decompose a stereo music recording into a set of individual audio tracks, each isolating a specific instrument or group of instruments. Currently, most systems separate a given mixture considering a four-stem convention: drums, bass, vocals and a residual other that absorbs every other instrument. In this work, we introduce DeFormer, a music source separation model capable of separating a given mixture to 12 stems, decomposing at the latent representation level. Expanding the task from 4 to 12 demands greater sensitivity to instrument-specific cues due to finer distinctions in the target distribution, e.g., sparsity, spectral (or timbral) entanglement, etc. DeFormer addresses these challenges by combining a novel latent-separation architecture with a training recipe adapted to sparse and skewed supervision. On our held-out set, DeFormer reaches 6.59 dB mean cSDR over 12 targets, 1.06 dB ahead of the closest of four baselines we retrain at the same target count. DeFormer also demonstrates a notable gap in subjective performance on both general audio quality and adjacent source bleeding, measured in a conducted human study involving domain-expert annotators with professional music production backgrounds. Code and model checkpoints will be released upon acceptance. Samples page: https://deformerpaper-dev.github.io/DeFormer

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
music source separation, improved spectral sequence modeling, custom attention

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_3Wk5OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose in this paper a music source separation method that operates in the latent space and is capable of separating 12 active stems.

Critical strengths

The paper has several strengths:

  • extending music source separation beyond the classic 4-stems task, tackling up to 12 stems mixtures;
  • An extended and very solid perceptual evaluation ;
  • A well designed demo page with impressive separation results
  • the contribution of a re-annotated version of MoiseDB dataset to tackle 12 stems mixtures source separation
Critical weaknesses

The paper also has several weaknesses:

  • The paper is not always clear on the exact evaluation protocol. Are all test data different (unseen) from the data used for training ? Are the test data similar (same musical genre / type of mixing ..etc ) than the training data ? More problematic, a large part of the paper uses a held-out internal set for which very little information is given. Also, since it is an internal held-out set, the results are not reproducible despite the potential release of the code and weights.

  • The paper is not very clear on what are the key contributions. The authors mention that the latent-based source separation is one of them while many existing methods are also doing the source separation in the latent space. It seems that the main contribution is to extend source separation beyond 4-stems (and up to 12-stems mixtures) though this is not well dicussed in light of the existing literature.

  • Bibliography/ references: linked to the point above, the authors have only used a limited set of references for positioning their work. Though, they are a number of previous work that have tackled different settings than the classic vocal-drums-bass-others task. For instance (but many other exist in both supervised and unsupervised music source separation), there is work on Drum separation (for instance with 9 stems separation in [1]), in choir singing separation (4 homogeneous stems, see for example [2,3]) or in orchestral music (with many stems, see for example 4).

Other more minor points:

  • It is not clear in the data used what is the amount of bleeding and how often it occurs (from most examples in the demo page it seems that the bleeding is absent).
  • There are no indications of complexity (how heavy is the training of the model in terms of computing resources) ?
  • It is not clear what is exactly the mix input creation (section 3. "Batch sampling during training").... but this goes again to the comment that it is not clear what is exactly the data used.
  • Objective metrics: there are quite some explanations concerning residual damp or silence prediction without using specific metrics to evaluate this aspect. The metrics proposed in [5] for Predicted Energy at Silence (PES) or Energy at Predicted Silence (EPS) could probably be used to evaluate this aspect.
  • the section on "the targets are spectrally entangled" is not very informative and the outcomes seem rather straightforward. If space is needed in a potential paper update, this section could be placed in appendix.

[1] B. Torres & al. "The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesis," in IEEE Transactions on Audio, Speech and Language Processing, vol. 34, pp. 84-95, 2026, [2] D. Petermann & al. “Deep learning based source separation applied to choir ensembles,” in Proc. Int. Soc. Music Inf. Retrieval Conf., 2020, pp. 733–739 [3] K. Schulze-Forster & al. "Unsupervised Music Source Separation Using Differentiable Parametric Source Models," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 1276-1289, 2023 [4] J. Garcia-Martinez et al., "The Spheres Dataset: Multitrack Orchestral Recordings for Music Source Separation and Information Retrieval," in IEEE Transactions on Audio, Speech and Language Processing, vol. 34, pp. 2622-2634, 2026 [5] K. Schulze-Forster & al. "Phoneme Level Lyrics Alignment and Text-Informed Singing Voice Separation," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 2382-2395, 2021

Questions for authors
  • Can you clarify the exact evaluation protocol : Are all test data different (unseen) from the data used for training ? Are the test data similar (same musical genre / type of mixing ..etc ) than the training data ? will the data be distributed ? how does the method perform on out-of-domain data ?
  • what are the key contributions of the paper in light of an extended bibliography ?
  • what is the complexity of the method (training) ?
3 comments
AuthorsOpenReview ↗

Comment: 3Wk5

We thank reviewer 3Wk5 for their detailed review. We group our responses by topic.

Evaluation protocol and data (W1, Q1)

The data sources are licensed multi-track songs obtained from external vendors (disjoint from MoisesDB). Our splits contain 13,741 / 783 / 765 songs (train / validation / evaluation), corresponding to 649.8h / 37.3h / 36.4h with ≥ 2 stems overlap; the sets are distinct by artist (App. K, Tab. 7).

To evaluate out-of-domain performance, we use MoisesDB, with all baselines retrained on our corpus: it includes unseen artists, potentially unseen instruments, different production setups and a different taxonomy, and was never used for design, tuning or model selection (Sec. 5.1). There, a song-clustered paired bootstrap places DeFormer +0.58 dB above BS-RoFormer (p = 0.026; App. F.1), and the blind listening study, drawn from MoisesDB, rates it highest (Sec. 5.3). Our MoisesDB results can be reproduced with our released model checkpoint.

In addition, every Table 2 configuration evaluated on MoisesDB shows similar trends as on our internal set: Fusion (+0.29 dB) and De-Mix (+0.69 dB), the latter significant at the window, song and artist level (p ≤ 0.01; Tab. 9, App. N).

Key contributions (W2, Q2)

Our main contributions are:

  • (i) an analysis of what makes separation beyond four stems hard, namely spectrally overlapping targets under sparse, skewed supervision (Sec. 3; App. C), together with a failure analysis showing that the weakest stems collapse to silence and that their content gets routed to timbrally similar stems (Sec. 5.2);
  • (ii) DeFormer, which extends cross-source interaction in the separation process: stem streams attend to one another at the bottleneck latent (De-Mix) and to the mixture through the skip connections (Fusion). DeFormer outperforms all retrained baselines in both objective evaluation and a subjective study with 80 music professionals;
  • (iii) a reproducible 12-stem evaluation on public data: we release our models together with a MoisesDB segment list, its mapping to our taxonomy and our scoring script, and document the taxonomy, which maps 98 instrument labels onto 12 stems with no residual "other" stem (App. A). The internal data itself cannot be released.

Separating at the bottleneck is a design choice that lets cross-source joint attention be used effectively. We agree with the reviewer's point on the "latent-based" framing and rephrased it in the revision (Sec. 1, 4.1).

Related work (W3)

We thank the reviewer for the references, and agree that our related work could be expanded. Given the 9-page limit, we focused the related work on the approaches we found to be most relevant to our setting: general multi-instrument separation over diverse music. The suggested works address separation beyond four stems in specific, more homogeneous settings (drum kit components [1], choir voices [2, 3], orchestral instruments [4]); following the reviewer's suggestion, we added them to the related work (Sec. 2).

Minor points

  • Bleeding in the data: stems are delivered as individual tracks. 5.9% of stems carry more than one of the 19 instrument families, and 1.06% more than one of our 12 stem types (App. H). These rates describe stem composition, not recording bleed; since annotators labeled three randomly drawn 5 s segments of each stem rather than the full track (App. B), bleed elsewhere in a track was not measured (App. K).
  • Complexity (Q3): DeFormer (156M) was trained for 150k updates at batch 256 on 4 s segments, about 5 × 10^21 FLOPs, or roughly 11k H100 GPU-hours. The fixed 12-output model produces all stems together, at 4.6× real time on one H100 (App. L).
  • Mixture creation: each example is built around a uniformly sampled anchor stem type, so rare stems appear as often as common ones. A 4 s window is drawn from the anchor's activated regions, and all stems of the track active in it are summed; this is now described in Sec. 3.
  • Silence and bleeding metrics: following [5], we computed PES on MoisesDB; EPS is undefined here, as no system emits exactly silent frames, so we add thresholded miss and false-activation rates (−60 dBFS). De-Mix reduces leakage into silent stems: relative to the Fusion-only model, PES drops by 5.3 dB and false activations from 22.6% to 14.5%, at an unchanged miss rate. Against BS-RoFormer, DeFormer misses active content no more often (7.3% vs. 7.8%) and leaves only a faint residual in silent frames, at about −68 vs. −74 dBFS (PES −21.2 vs. −27.6 dB); expert listeners nonetheless rate it better on bleeding (4.08 vs. 3.64; Sec. 5.3). These metrics are now in Sec. 5.2 and App. M (Tab. 8).
  • Spectral entanglement section: we respectfully prefer to keep it in the main text, as it motivates our design choices: the SD/SU blocks, attention along frequency, and cross-source attention with aligned relative positions (FIS-RoPE). We made this link explicit in Sec. 3.

I would like to thank the reviewers for their answers. Most of my comments are satisfactorily addressed and taken this new insights, I will slightly raise my score.

We thank reviewer 3Wk5 for their engagement and for raising their score in recognition of our work.

We are pleased our rebuttal addressed most of their comments, and would be happy to address remaining concerns they may still have.

Reviewer_bzCmOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed DeFormer, a system for music source separation. The contribution of this work is the data processing, and the model architecture. Demos are available.

Critical strengths
  • The proposed method achieved a good separation result compared to other systems.

  • There are many good engineering tricks in training objectives.

  • It is nice to have a 12-stem separation system.

Critical weaknesses
  • Section 3, the authors claim that they annotated 15k songs. Are all these songs clean stems, or are they a mixture of stems? There is a lack of description of where do the audio files come from.

  • For model architecture, it looks like that this is only a marginal improvement of the BSRoformer. What is the major difference between this work and BSRoformer?

  • It looks that the major improvement comes from the losses, fusion, and engineering tricks. From Table 3, the model does not improve over BSRoformer.

  • The title of DeFormer is a bit misleading, as DeFormer does not improve over BSRoformer. BSRoformer can also extend beyond four stems. All improvement in this work looks like from data, loss, and other engineering tricks.

  • Section 4.1 and 4.2 are weak and lack innovations.

  • The authors should claim more contributions on the data processing pipeline, as this is more important than DeFormer from the results.

Questions for authors

See critical weaknesses.

3 comments
AuthorsOpenReview ↗

Comment: bzCm

We thank reviewer bzCm for their thoughtful review. We group our responses by topic; each heading lists the weaknesses (W) it addresses.

Data (W1)

The data sources are licensed multi-track songs obtained from external vendors, delivered as individual stem tracks; training mixtures are formed by summing stems (Sec. 3, App. E). Most stems contain a single instrument family: 5.9% of stems carry more than one of the 19 families, and 1.06% more than one of our 12 stem types (App. H). Since annotators labeled three randomly drawn activated regions of each stem rather than the full track (App. B), some stems may contain additional parts that were not heard. "Clean" is also approximate: as in commercial productions, some stems carry heavy production effects (e.g., long delays on electric guitar). Our splits contain 13,741 / 783 / 765 songs (train / validation / evaluation), distinct by artist, with electronic and pop as the most common of 39 genres. These details are now in App. K (Tab. 7).

Innovation, Difference from BS-RoFormer and source of the improvement (W2, W3, W5)

DeFormer's backbone is SCNet, not BS-RoFormer (Sec. 4.2), and its improvement stems from our architectural components rather than from the data. Table 2 is a controlled ablation on the same data, adding one component at a time: scaling SCNet to 348M parameters adds 0.08 dB and our objective 0.13 dB, while with the objective held fixed, Fusion adds 0.34 dB and De-Mix a further 0.96 dB at less than half the parameters. The architectural components thus account for 1.30 dB of the 1.43 dB gain over the scaled SCNet.

To rule out the data as the source of the gain, we also trained DeFormer (K=4, its 12-stem configuration untuned) on MUSDB18-HQ alone, the training data behind the published SCNet L and BS-RoFormer results (Tab. 3, bottom; App. O). It reaches 10.06 dB, in the range of their reported 9.69 and 9.80 dB, and 10.98 dB on MoisesDB, on par with public checkpoints trained on more data.

Our main comparison is Table 1, where all baselines, including BS-RoFormer, are retrained for 12 stems on the same corpus and schedule. DeFormer (156M) outperforms BS-RoFormer (329M) by 1.06 dB on our test set and 0.58 dB on MoisesDB, and expert listeners rate it higher in both audio quality (4.10 vs. 3.57) and bleeding (4.08 vs. 3.64; Sec. 5.3).

These components follow from an analysis of the expanded task. Extending the taxonomy from 4 to 12 stems is not simply a matter of adding targets: the additional targets are highly entangled, with strong spectral and timbral overlap (Fig. 1; Fig. 9 in App. D). Within each stream, we build on prior work: SCNet's SD/SU blocks compress mid and high bands more strongly, on the premise that lower frequencies carry more information, and from BS-RoFormer we adopt only the idea of attending along frequency as well as time.

What these approaches leave out is explicit interaction between targets. In its multi-target form, as retrained here, BS-RoFormer adds one mask head per stem on top of a shared backbone; SCNet splits the channels of its final layer into targets. Targets thus relate only implicitly, and contested content is often claimed by several targets (bleeding) or by none (silence). Our components are designed to bridge this gap. In De-Mix, target streams attend to one another at the bottleneck, each with its own projections, while FIS-RoPE aligns their positions. In Fusion, each stream attends to the shared encoder skip with full-sequence attention, replacing SCNet's convolutional fusion.

(W5) We respectfully disagree that these sections lack innovation: Sec. 4.2 introduces De-Mix, FIS-RoPE and our Fusion module (Figs. 3–4; App. D), whose contribution is isolated above, with De-Mix's joint attention also halving the parameter count (Tab. 2).

The name and scope of DeFormer (W4)

The name reflects the method: DeFormer (i) De-mixes the given stems using our custom transformer components. We agree that BS-RoFormer, like all our baselines, can be extended beyond four stems; this is exactly what we do in our controlled experiments (Tab. 1). We plan to release DeFormer's code and 12-stem weights, providing the community with an openly available model evaluated beyond the four-stem setting, the focus of our title.

Contribution of the data pipeline (W6)

We thank the reviewer for this suggestion and agree that the data pipeline is a major contribution in its own right. That said, the data pipeline is not the core contribution of our work: as described above, we identify a gap in existing methods, where targets do not interact during separation, and specifically design our components to bridge it. Since all systems in Tables 1 and 2 share the same data and pipeline, the gains in our controlled experiments are attributable to these components. We will make the data pipeline's contribution more prominent in the final version.

Reviewer_bzCmOpenReview ↗

to authors

Thought from the model perspective the novelty is incremental, I think there are interesting findings in multiple aspects, from data, multi-instrument, and model. I am satisfied with the comments.

We thank reviewer bzCm for their engagement and for raising their score in recognition of our work.

We are pleased they found our rebuttal satisfactory.

Reviewer_rcp5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces DEFORMER, a music source separation model extending from 4 to 12 stems. The paper overcomes challenges of 4-stem separation models. Separation happens at the latent level in the De-Mix module. They annotated ~15k songs to expand the task from 4 to 12 stems. The model consists of the DEFORMER architecture, which uses SCNet's Sparse Downsampling (SD) and Sparse Upsampling (SU) blocks as the structural encoder-decoder skeleton. It inserts a dedicated De-Mix module at the bottleneck that projects into NstemsN_{\text{stems}} representations (zs1,…,zsnz_{s_1}, \dots, z_{s_n}) and applies joint dual-path (temporal + frequency) attention across the stems. It uses attentive Fusion Modules on the skip connections. Apart from the annotated dataset, DEFORMER was evaluated on MoisesDB and MUSDB18-HQ, and compared with prior 4-stem models.

Critical strengths

Separating 12 distinct stems end-to-end within a single model architecture is a substantial practical and empirical contribution.

Isolating the contribution of each component (Table 2) strengthens the understanding of the proposed method's impact.

Significant effort on creating a 15k-song dataset, expanding the task from 4 to 12 stems. It is a sufficient contribution.

Clear motivation and well-presented figures make the method easy to understand.

Critical weaknesses

Incremental novelty. The method is comprised of existing components from prior works in the domain.

Moderate generalization. On one of the two known benchmarks (MoisesDB), the method clearly outperforms related works. However, on MUSDB18-HQ, performance is within a narrow margin of related works and worse in some cases.

The dataset creation lacks sufficient detail regarding the origin of the audio sources, the collection methodology, and the total duration (in hours or track lengths) of the musical material.

Ιt is unclear whether the songs represent complete studio recordings or short excerpts, what genres and arrangements are covered, and how the ~15k annotated songs were split across training, validation, and testing.

The combination of modules from other works is convincing and logical. However, it lacks the theoretical grounding that is crucial for a top-tier conference.

Questions for authors

Can the authors provide a breakdown or error analysis explaining why the proposed method performs substantially better on MoisesDB than on MUSDB18-HQ?

Beyond empirical trial-and-error, what theoretical intuition or analytical grounding explains why these specific components interact synergetically rather than redundantly?

Which specific module or modification do you consider the core architectural contribution that distinguishes this approach from direct pipelines of prior work?

Will the 15k-song dataset and evaluation split be made publicly accessible to enable reproducible benchmarking?

3 comments
AuthorsOpenReview ↗

Comment: rcp5

We thank reviewer rcp5 for their time and thoughtful review. We group our responses by topic; each heading lists the weaknesses (W) and questions (Q) it addresses.

Dataset details (W3, W4)

The data sources are licensed songs obtained from external vendors. Production quality varies, from finished mixes to sketches and incomplete recordings. Our splits contain 13,741 / 783 / 765 songs (train / validation / evaluation), corresponding to 649.8h / 37.3h / 36.4h with ≥ 2 stems overlap; the sets are distinct by artist. Fig. 2 shows the activated hours per stem type, and Fig. 8 (App. C) the co-occurrence of each stem with all others. Summed over stems, this yields 2,764h, counting repetitions of stems separately. The most common of 39 genres are electronic (~17%), pop (~13.5%) and cinematic (~4.9%), with a long tail of other genres. The annotation methodology is specified in App. A and B; corpus details are now in App. K (Tab. 7).

Novelty and grounding (W1, W5, Q2, Q3)

Extending the taxonomy from 4 to 12 stems is not simply a matter of adding targets: the additional targets are highly entangled, with strong spectral and timbral overlap (Fig. 1; Fig. 9 in App. D). For processing within each stream, we build on prior work: SCNet's SD/SU blocks compress mid and high bands more strongly, on the premise that lower frequencies carry more information, and, following BS-RoFormer, our attention blocks attend along frequency as well as time.

What these approaches leave out is explicit interaction between targets. Common separators either separate a single latent only at the output (channel groups, e.g., HTDemucs and SCNet; per-target mask heads, e.g., multi-target BS-RoFormer), or decode each target independently (e.g., Banquet's per-query passes), so targets relate only implicitly and contested content is often claimed by several targets (bleeding) or by none (silence). Addressing this is our core contribution (Q3): in De-Mix, target streams attend to one another at the bottleneck, each with its own projections, while FIS-RoPE aligns their positions. Standard attention over concatenated streams could in principle learn this alignment; in practice, building it into the architecture simplifies the modeling task. In Fusion, each stream attends to the shared encoder skip with full-sequence attention, replacing SCNet's convolutional fusion.

Each component improves on top of the previous one (Tab. 2): our objective adds 0.13 dB, and with the objective held fixed, Fusion adds 0.34 dB and De-Mix a further 0.96 dB while halving the parameter count. For reference, the public 4-stem state of the art improved by about 1.5 dB from HTDemucs to SCNet (Tab. 3). Against the closest retrained baseline (BS-RoFormer), DeFormer improves by 1.06 dB on our test set and 0.58 dB on MoisesDB (Tab. 1), and expert listeners rate it higher in both audio quality (4.10 vs. 3.57) and bleeding (4.08 vs. 3.64; Sec. 5.3).

Generalization (W2, Q1)

To rule out the data as the source of our gains, we trained DeFormer (K=4, its 12-stem configuration untuned) on MUSDB18-HQ alone, the training data behind the published SCNet L and BS-RoFormer results (Tab. 3, bottom; App. O): it reaches 10.06 dB, in the range of their reported 9.69 and 9.80 dB, and transfers to MoisesDB (10.98 dB) on par with public checkpoints trained on more data. Trained for 12 stems on our corpus and folded into four, it also lies within 0.25–0.50 dB of these checkpoints on MUSDB18-HQ (Q1).

On MoisesDB, all baselines are retrained on our corpus and taxonomy; the set is out of distribution (unseen artists, potentially unseen instruments, different production setups and taxonomy) and was never used for design, tuning or model selection (Sec. 5.1). There, a song-clustered paired bootstrap places DeFormer +0.58 dB above BS-RoFormer (p = 0.026; App. F.1), and the blind listening study, drawn from MoisesDB, rates it highest. Every Table 2 configuration evaluated on MoisesDB shows that Fusion (+0.29 dB) and De-Mix (+0.69 dB) largely retain their gains, De-Mix being significant at the window, song and artist level (p ≤ 0.01; Tab. 9, App. N).

Taxonomy mismatch is a further factor. When curating the MoisesDB subset (Sec. 5.1, App. I), we verified that each stem type has at least 50 examples and that its labels align with our taxonomy (e.g., contrabass and tuba: bass in MoisesDB, strings and wind in ours). Out-of-distribution instruments nonetheless remain, most apparent in keys and sfx, the stems with the highest timbral variance.

Release (Q4)

We will not release the internal data. We will release our model checkpoints and code, along with the MoisesDB segment list, its mapping to our taxonomy and our scoring script (App. J), so that our MoisesDB results can be reproduced.

I still have concerns about the novelty. However, the extension from 4 to 12 stems and the improvements on some benchmarks are satisfying. In addition, the dataset with 15k songs is a substantial contribution. The reproducibility partially addresses my concerns about releasing the data. The comments are clear, and I am raising my rating to weak accept.

We thank reviewer rcp5 for their engagement and for raising their score in recognition of our work.

We would be happy to address further specific concerns they may still have.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Comment Nov 13

We thank all reviewers for their time and for their constructive, actionable feedback. We have uploaded a revised manuscript addressing the main concerns (changes highlighted in red\textcolor{red}{\text{changes highlighted in red}} in the PDF):

  • Corpus and evaluation protocol: corpus details, artist-disjoint splits, durations and genres (App. K, Tab. 7); mixture construction (Sec. 3); MoisesDB serves as an out-of-distribution test set, never used for design, tuning or model selection (Sec. 5.1).
  • Positioning: an explicit list of contributions (Sec. 1), related work extended to separation beyond four stems and to explicit source coupling (Sec. 2), and the link between our data analysis and design choices (Sec. 3).
  • New results: the ablation evaluated on MoisesDB, where Fusion and De-Mix retain their gains (Tab. 9, App. N); paired significance tests (App. F.1, N); silence and leakage metrics (Sec. 5.2, App. M); training and inference cost (App. L); and DeFormer trained on MUSDB18-HQ alone, which matches the published results of SCNet and BS-RoFormer on the same data (10.06 vs. 9.69 and 9.80 dB) and transfers to MoisesDB (Tab. 3, App. O).

Our main contributions are:

  • (i) an analysis of what makes separation beyond four stems hard, namely spectrally overlapping targets under sparse, skewed supervision, together with a failure analysis of the weakest stems;
  • (ii) DeFormer, which extends cross-source interaction in the separation process: stem streams attend to one another at the bottleneck (De-Mix) and to the mixture through the skip connections (Fusion), outperforming all retrained baselines in objective evaluation and in a blind study with 80 music professionals;
  • (iii) a reproducible 12-stem evaluation on public data: our models, a MoisesDB segment list, its mapping to our taxonomy and our scoring script.

DeFormer is the result of a substantial effort, from annotating 15k songs with 397 music professionals, through designing DeFormer and retraining all baselines for 12 stems (about 11k H100 GPU-hours for DeFormer alone), to a blind listening study with 80 domain experts. We expect its release to benefit the community well beyond source separation, for example as a source of pseudo-supervision for music information retrieval, transcription and generative modeling, or as a practical tool for music education. We believe that DeFormer merits recognition as such a contribution, and will use the discussion period to address any concerns the reviewers may still have.

AuthorsOpenReview ↗

General Comment Nov 16

We would like to thank all the reviewers for their involvement and recognition of our work, and are glad to see that we have a consensus toward acceptance.

We would be happy to use the remaining time of the discussion period to address any concerns still standing, and welcome further comments from the reviewers and the AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.