- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose in this paper a method for music morphing. The method generates instruction-conditioned morph paths between a source and a target. This is done by learning a per-song style token (that can shift global style without changing the song structure and content), the structural latent being model separately. At inference, the two songs (input and target) need to be tempo- and beat-aligned.
Critical strengths
The paper strengths are:
- the topic, music morphing, is interesting and still rather underexplored ;
- the proposed method is specific although it exploits known concepts both in audio and image processing
- the accompanying web page with a selection of audio examples that illustrate the behavior of the method
Critical weaknesses
The paper suffers from several weaknesses:
- The work is poorly positioned in the state of the art in audio/music processing in general, and music transformation and editing in specific. Without even considering the large share of work in text-to-music generation, or the seminal (and rather old) papers on audio morphing, the following work on the specific topic of timbre transfer are amongst the interesting work in this domain:
M. Alinoori & al. “Music-STAR: a Style Translation system for Audio-based Re-instrumentation”. In: Proc. ISMIR. 2022. O. Cifka & al. “Self-Supervised VQ-VAE for One-Shot Music Style Transfer”. In: Proc. ICASSP. 2021. L. Comanducci & al. “Timbre transfer using image-to-image denoising diffusion implicit models”. In: Proc. ISMIR. 2023. S. Li & al. “Music style transfer with time-varying inversion of diffusion models”. In: Proc. AAAI. 2024. S. Rouard & al. al. “Audio Conditioning for Music Generation via Discrete Bottleneck Features”. In: Proc. ISMIR. 2024 T. Baoueb & al. Wavetransfer: A flexible end-to-end multi-instrument timbre transfer with diffusion, Proc. of IEEE-MLSP, 2024.
- The experimental plan is questionnable. First, the baselines are all image morphing algorithms, and are not sufficient to demonstrate the merit of the proposed approach. Second, only objective metrics are used. They are no perceptual tests which could have better highlight the merits and limitations of the morphing method. The examples in the accompanying web page are illustrative but more information on how they were chosen would be needed (are they cherry picked exemples ? randomly drawn from a large set of text-prompt pairs ?)
- The paper is also difficult to reproduce. There are details on the architecture, algorithms and methods, but since the code, the model checkpoints, the training data are not distributed, reproducibility remain uncertain.
- The proposed method needs to tempo and beat-align the two audios (input and target), which potentially limits the space of morphing possibilities. Though, it is not totally clear what this is needed, especially if one of the audio is not rhythmic (e.g. wih no clear tempo).
minor point:
- referring to left / right for the "input/target" pair of signals may be confusing since in audio left/ right usually refers to the left and right channels of a stereo audio signal.
Questions for authors
- Can the experimental plan be strengthen ?
- Can the reproducibility be improved ?
3 comments
Final Reminder
Hello Reviewer 9zzt,
As the discussion period ends today, please let us know if the response adequately address your concerns. We would be happy to clarify anything.
Thanks,
Authors of MusicMorpher
Summary of how each concern is addressed
Thank you for your thoughtful and detailed review. You asked for perceptual evidence, audio baselines and clearer positioning, and we have added all three to the revision:
- Listening test: 75 listeners cast 1,744 votes and preferred MusicMorpher 69% of the time (77% vs. FlowMorph, 61% vs. FreeMorph; p < 0.01). See Sec. 5.3 and App. E.
- Audio baselines: MusicMorpher outperforms MorphFader and a naive crossfade on both MPS and FAD (Table 1, Tables 10–11). The crossfade scores poorly on MPS, which suggests the metric does not reward simple mixing.
- Positioning: Thank you for the references. Sec. 2.3 now discusses SoundMorpher, classic audio morphing, and all six style-transfer works you suggested. Those methods map one input to one target style, whereas we morph continuously between two full pieces.
- Representative examples: The project page now includes 10 random, unselected pairs and real recordings (#random, #real-data).
- Reproducibility and beat alignment: The paper gives the full algorithm and hyperparameters, and we will release MPS. Beat alignment is optional; removing it changes MAR only slightly (Table 6).
We would be grateful to hear if any concern remains. If not, we would sincerely appreciate your reconsidering the score.
Response to Reviewer 9zzt
W1 (positioning). Thank you for the pointers. We added SoundMorpher and MorphFader (Sec. 2.3), and we have added a paragraph on classic audio morphing (signal-model interpolation of spectral envelopes and partials) and the timbre/style-transfer works you list (Music-STAR, Cífka et al., Comanducci et al., Li et al., Rouard et al., Baoueb et al.) in Sec. 2.3. These methods map one input to a single target style or timbre, and classic morphing targets individual sounds rather than full polyphonic mixes. MusicMorpher instead produces a continuous path between two complete pieces, where every intermediate must itself be coherent music.
W2/Q1 (baselines, perceptual test, examples). Added an audio baseline (MorphFader, Table 1) and a naive crossfade (Tables 10–11), both outperformed by MusicMorpher, plus a listening study with a 69% preference for MusicMorpher. The original examples were cherry-picked and are now labelled as such (cherry-picked). We added 10 unselected random pairs (random), real recordings (real-data), and all methods side by side (compare).
W3/Q2 (reproducibility). The model, data, and the code tightly coupled to them are under license. We give the full algorithm, hyperparameters, and baseline details so the method can be reimplemented on open T2A models (e.g., Stable Audio Open), and we will release MPS.
W4 (beat alignment). Alignment is optional. It keeps time-indexed latents on the same metrical positions, much as a DJ matches tempos before a transition. Without it, MAR rises only slightly (App. Table 6: MusicFM 0.0664 → 0.0701, MuQ 0.0934 → 0.1038, CLAP 0.0968 → 0.0978). We focus on music, which is almost always rhythmic, where alignment helps. For non-rhythmic audio, alignment brings no benefit but causes no harm: the step can simply be skipped, and the style-token path does not depend on a beat grid.
Minor (left/right). Left and right are the ends of the morph axis, not stereo channels. We have clarify this at first use in the text at Page 5.