- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a novel loss that can supervise positive-weight combinations of two latent codes thoughout their mixing cone. It is shown that latent substraction can recover a source from a mixture when the codes of the interfering sources are available. It is also shown that the resulting latent structure also benefits several aspects of conditional rectified flow. It is a strong paper, well written despite a lenghty and far too detailed appendix.
Critical strengths
The paper has several strengths:
- the proposed approach is novel and a clear extension of previous work
- the method presentation is detailed, well substantiated and clear
- the experimental plan is solid and diverse, with appropriate and reimplemented baselines
Critical weaknesses
Despite its strengths, the paper suffers from some weaknesses:
- it includes a very lengthy appendix that contains many details, both methodological/theoretical and experimental (> 30 pages of appendix). Obviously, this appendix cannot be read in details during reviewing. Even if it may contain only related and interesting information, a more focused appendix on the major aspects would have been more appropriate.
- The experimental settings are not totally clear, and in particular considering the training dataset (it seems that there is only a vague statement in the appendix concerning the data used for training, without the possibility to easily reproduce the experiences). Are all results provided with a system trained on the exact same training set ?
- All results are only given using objective metrics which are not well linked to perceptual quality. There are no perceptual tests to evaluate the quality of separation and there is no demo sites where separation examples would be provided.
- The related work section is concise and clear, but it would be interesting to also discuss other types of work which exploit parallel encoding sections depending on the type of source, inducing also a kind of additive property for source separation (see for instance the work of Bie & al, "Learning Source Disentanglement in Neural Audio Codec," ICASSP 2025).
Questions for authors
- Can you provide more details on the exact training dataset for the different experiments ?
- Are specific experiments demonstrate the clear out-of-domain scenario (such as for instance trained on speech, tested on music/general audio )?
- Can you provide perceptual results or comprehensive demo page ?
2 comments
On Perceptual Evaluation (Weakness 3 & Question 3)
We fully agree that listening tests are essential for evaluating audio quality. To this end, we have developed an interactive companion demo page where reviewers and readers can perceptually evaluate our proposed model across diverse configurations and experimental tasks. The listening experience takes approximately 5 to 10 minutes and is accessible at this link.
On Modular/Disentangled Codecs with Parallel Sub-Codebooks (Weakness 4)
We thank the reviewer for pointing out this relevant line of work; contrasting it with our approach strengthens the positioning of our paper.
Specifically, Bie et al. (ICASSP 2025, SD-Codec) retain a unified encoder–decoder backbone (based on DAC) but partition the discrete bottleneck into parallel domain-specific RVQ sub-codebooks assigned to predetermined semantic categories (speech, music, and sound effects), while related modular codecs employ parallel encoder branches or class-specific decoders. While effective for domain-specific tasks with fixed source classes, this paradigm differs fundamentally from our approach and incurs notable trade-offs:
- Category and Stem Supervision: SD-Codec requires clean, isolated stems for each fixed semantic category during training to supervise branch specialization (), and cannot separate or compose arbitrary sources within the same category (e.g., two musical instruments or two speakers) or outside its predefined taxonomy.
- Discrete Unit-Weight Sums vs. Continuous Cone Homomorphism: Although SD-Codec trains its decoder on equal-weight sums of quantized domain embeddings () at fixed LUFS loudness targets, discrete RVQ representations do not support continuous scalar rescaling () or continuous positive-weight combinations across arbitrary mixing angles and gains . Indeed, for source separation, SD-Codec does not rely on direct waveform decoding of latent arithmetic, instead applying post-hoc STFT magnitude masking with the noisy mixture phase. By contrast, enforces continuous decoder homomorphism across the entire cone , directly enabling arbitrary crossfading laws (Sec. 4.3) and pure waveform-level latent subtraction without masks (Sec. 4.2).
- Bitrate Scaling and Generative Bottleneck Complexity: Partitioning the latent space into domain-specific RVQ streams multiplies the representation bitrate by (e.g., bitrate for three-source mixtures in SD-Codec) and fragments the bottleneck into coupled discrete hierarchies that are cumbersome for downstream continuous generative priors.
In contrast, our framework preserves a single, unified, source-agnostic continuous latent space of fixed dimensionality with no specialized sub-networks. Linearity is induced as an intrinsic geometric property of the decoder's inversion manifold via self-supervised random pairing without clean stems or category labels, natively supporting arbitrary mixing ratios, continuous crossfading, and straight-trajectory Rectified Flow generation (Sec. 4.4).
We have incorporated this discussion into Section 2 (Related Work) of the revised manuscript.
We thank the reviewer for their thorough analysis and valuable suggestions. We address each concern below.
On the Length and Structure of the Appendix (Weakness 1)
We share the reviewer's perspective: a 30+ page supplement, motivated by full transparency and reproducibility, can become cumbersome during review. To ensure the appendix is concise, focused, and immediately accessible, we will implement the following structural revisions:
This restructuring will reduce the page count while preserving full scientific rigor.
On the reproducibility of the experimental settings (w2, q1 and q2)
Yes, all models we trained (every autoencoder variant and their downstream flow models) were trained on the exact same training dataset, using identical preprocessing, architectures, training schedules, and random seeds. Furthermore, our headline separation and interpolation benchmarks were deliberately evaluated on datasets never used in training (MUSDB18-HQ and LJSpeech) to test zero-shot algebraic generalization. Below is a concise summary of the data split across training and evaluation:
univ,softnorm, Implicit, Trig Mixup).Reproducibility & Manuscript Revisions: