acceptodds
Under review as a conference paper at ICLR 2027

MusicMorpher: Music Morphing via Style Tokens

13 views

Abstract

Continuous-strength instruction editing lets users specify what to change and by how much. Such methods provide this control for images, but remain underdeveloped for music because they require source-target examples or morph paths for supervision, which are hard to obtain. For example, image-based Kontinuous Kontext derives such trajectories from FreeMorph outputs, but requires post-filtering since interpolating zero-noise latents is non-smooth. We find image-based methods transfer poorly to music: they can act akin to a volume slider and ignore musical traits like tempo and beat alignment. We therefore identify music morphing, rather than editor fine-tuning, as the upstream bottleneck. We introduce MusicMorpher, a music morphing method that generates instruction-conditioned morph paths between a source and target. We do so by learning a per-song style token, applied through modulation so that it shifts global style without touching temporal structure, and interpolating it together with each song's inverted structural latent, each in its own space. At inference, we tempo- and beat-align songs before style-token computation to prevent rhythmic discontinuities. We compare MusicMorpher against SDEdit, RF-Inversion, FreeMorph, and FlowMorph on morph quality and smoothness across varying rhythmic alignment, introducing trajectory-level measures absent from prior single-sample metrics. Results show our method produces higher-quality, smoother morphs, and, without supervised instruct-edit training, extends to continuous-strength semantic edits reaching quality-alignment trade-offs beyond baseline sweeps; our analyses explain prior methods' inconsistent results. Examples at https://musicmorpher.github.io/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Music Generation, Music Editing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9zztOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose in this paper a method for music morphing. The method generates instruction-conditioned morph paths between a source and a target. This is done by learning a per-song style token (that can shift global style without changing the song structure and content), the structural latent being model separately. At inference, the two songs (input and target) need to be tempo- and beat-aligned.

Critical strengths

The paper strengths are:

  • the topic, music morphing, is interesting and still rather underexplored ;
  • the proposed method is specific although it exploits known concepts both in audio and image processing
  • the accompanying web page with a selection of audio examples that illustrate the behavior of the method
Critical weaknesses

The paper suffers from several weaknesses:

  • The work is poorly positioned in the state of the art in audio/music processing in general, and music transformation and editing in specific. Without even considering the large share of work in text-to-music generation, or the seminal (and rather old) papers on audio morphing, the following work on the specific topic of timbre transfer are amongst the interesting work in this domain:

M. Alinoori & al. “Music-STAR: a Style Translation system for Audio-based Re-instrumentation”. In: Proc. ISMIR. 2022. O. Cifka & al. “Self-Supervised VQ-VAE for One-Shot Music Style Transfer”. In: Proc. ICASSP. 2021. L. Comanducci & al. “Timbre transfer using image-to-image denoising diffusion implicit models”. In: Proc. ISMIR. 2023. S. Li & al. “Music style transfer with time-varying inversion of diffusion models”. In: Proc. AAAI. 2024. S. Rouard & al. al. “Audio Conditioning for Music Generation via Discrete Bottleneck Features”. In: Proc. ISMIR. 2024 T. Baoueb & al. Wavetransfer: A flexible end-to-end multi-instrument timbre transfer with diffusion, Proc. of IEEE-MLSP, 2024.

  • The experimental plan is questionnable. First, the baselines are all image morphing algorithms, and are not sufficient to demonstrate the merit of the proposed approach. Second, only objective metrics are used. They are no perceptual tests which could have better highlight the merits and limitations of the morphing method. The examples in the accompanying web page are illustrative but more information on how they were chosen would be needed (are they cherry picked exemples ? randomly drawn from a large set of text-prompt pairs ?)
  • The paper is also difficult to reproduce. There are details on the architecture, algorithms and methods, but since the code, the model checkpoints, the training data are not distributed, reproducibility remain uncertain.
  • The proposed method needs to tempo and beat-align the two audios (input and target), which potentially limits the space of morphing possibilities. Though, it is not totally clear what this is needed, especially if one of the audio is not rhythmic (e.g. wih no clear tempo).

minor point:

  • referring to left / right for the "input/target" pair of signals may be confusing since in audio left/ right usually refers to the left and right channels of a stereo audio signal.
Questions for authors
  • Can the experimental plan be strengthen ?
  • Can the reproducibility be improved ?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 9zzt

W1 (positioning). Thank you for the pointers. We added SoundMorpher and MorphFader (Sec. 2.3), and we have added a paragraph on classic audio morphing (signal-model interpolation of spectral envelopes and partials) and the timbre/style-transfer works you list (Music-STAR, Cífka et al., Comanducci et al., Li et al., Rouard et al., Baoueb et al.) in Sec. 2.3. These methods map one input to a single target style or timbre, and classic morphing targets individual sounds rather than full polyphonic mixes. MusicMorpher instead produces a continuous path between two complete pieces, where every intermediate must itself be coherent music.

W2/Q1 (baselines, perceptual test, examples). Added an audio baseline (MorphFader, Table 1) and a naive crossfade (Tables 10–11), both outperformed by MusicMorpher, plus a listening study with a 69% preference for MusicMorpher. The original examples were cherry-picked and are now labelled as such (cherry-picked). We added 10 unselected random pairs (random), real recordings (real-data), and all methods side by side (compare).

W3/Q2 (reproducibility). The model, data, and the code tightly coupled to them are under license. We give the full algorithm, hyperparameters, and baseline details so the method can be reimplemented on open T2A models (e.g., Stable Audio Open), and we will release MPS.

W4 (beat alignment). Alignment is optional. It keeps time-indexed latents on the same metrical positions, much as a DJ matches tempos before a transition. Without it, MAR rises only slightly (App. Table 6: MusicFM 0.0664 → 0.0701, MuQ 0.0934 → 0.1038, CLAP 0.0968 → 0.0978). We focus on music, which is almost always rhythmic, where alignment helps. For non-rhythmic audio, alignment brings no benefit but causes no harm: the step can simply be skipped, and the style-token path does not depend on a beat grid.

Minor (left/right). Left and right are the ends of the morph axis, not stereo channels. We have clarify this at first use in the text at Page 5.

AuthorsOpenReview ↗

Final Reminder

Hello Reviewer 9zzt,

As the discussion period ends today, please let us know if the response adequately address your concerns. We would be happy to clarify anything.

Thanks,

Authors of MusicMorpher

AuthorsOpenReview ↗

Summary of how each concern is addressed

Thank you for your thoughtful and detailed review. You asked for perceptual evidence, audio baselines and clearer positioning, and we have added all three to the revision:

  • Listening test: 75 listeners cast 1,744 votes and preferred MusicMorpher 69% of the time (77% vs. FlowMorph, 61% vs. FreeMorph; p < 0.01). See Sec. 5.3 and App. E.
  • Audio baselines: MusicMorpher outperforms MorphFader and a naive crossfade on both MPS and FAD (Table 1, Tables 10–11). The crossfade scores poorly on MPS, which suggests the metric does not reward simple mixing.
  • Positioning: Thank you for the references. Sec. 2.3 now discusses SoundMorpher, classic audio morphing, and all six style-transfer works you suggested. Those methods map one input to one target style, whereas we morph continuously between two full pieces.
  • Representative examples: The project page now includes 10 random, unselected pairs and real recordings (#random, #real-data).
  • Reproducibility and beat alignment: The paper gives the full algorithm and hyperparameters, and we will release MPS. Beat alignment is optional; removing it changes MAR only slightly (Table 6).

We would be grateful to hear if any concern remains. If not, we would sincerely appreciate your reconsidering the score.

Reviewer_FDPXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2(−1)
Soundness
2
Read review
Paper summary

This paper proposes MusicMorpher, a framework for music morphing that focuses on controlling how strongly an instruction is applied, rather than simply determining what should change. Music morphing is particularly challenging because discontinuities in beat, tempo, and timbre can lead to audible structural breakdown. To address this, MusicMorpher independently interpolates style tokens and noisy latents obtained by inverting two endpoint songs, using the interpolated style token as conditioning during flow sampling. The authors also introduce Morph Path Smoothness, a new metric that evaluates how monotonically and linearly intermediate samples progress away from the source endpoint in an embedding space. Experimental results show that MusicMorpher outperforms adapted image-morphing baselines in audio quality, semantic alignment, and morph-path smoothness across all evaluated conditions.

Critical strengths
  1. The paper clearly defines the problem of music morphing and provides well-motivated arguments for addressing the limitations of existing approaches.

  2. The proposed method incorporates musical characteristics into its preprocessing and modeling design. The authors also introduce new metrics to evaluate morphing quality and smoothness, providing a more comprehensive evaluation of the generated morph trajectories.

Critical weaknesses
  1. While the monotonicity and smoothness of the morph trajectory are important, the fundamental goal of music morphing is to meaningfully interpolate the musical characteristics of two endpoints. However, the paper does not sufficiently evaluate how effectively style and temporal structure are interpolated individually. The results in Table 11 of the Appendix provide limited evidence in this regard. More detailed analyses of interpolation quality and the independent roles of style and structure would strengthen the paper. For example, does interpolating only the style token produce meaningful stylistic changes while preserving temporal structure? Conversely, does interpolating only the structural latent produce meaningful structural transitions while maintaining a consistent style?

  2. Although the paper introduces MPS to evaluate morph-path smoothness, its relationship with existing audio evaluation metrics, such as FAD, CLAP, LPAPS, Chroma similarity, and KAD, is not sufficiently explored. Analyzing how these metrics vary along the morph trajectory could provide additional insight into what MPS captures. More importantly, Section 5.1 motivates MPS by noting that existing smoothness metrics correlate poorly with human preferences. However, the paper does not clearly demonstrate whether MPS achieves a stronger correlation with human-perceived smoothness. A direct comparison with human judgments and existing first-order smoothness metrics would better justify the proposed metric.

Questions for authors
  1. What exactly does the term "real point" on a morph trajectory refer to in the Introduction?

  2. The Related Work section states that the method: "Our problem differs not in performance but in definition: rather than traversing a given axis, we construct the axis itself from an open-vocabulary instruction." Still, this capability is not clearly introduced or explained in the Introduction. Does it mean a text prompt? Also, the paper instead emphasizes generating "semantically and structurally coherent music morphs" by independently interpolating style tokens and structural latents. Could the authors clarify how open-vocabulary instructions are incorporated into the proposed framework and what makes the style interpolation specifically semantic?

  3. Would the method produce equivalent results if the two endpoints (A_L) and (A_R) were swapped and the interpolation direction reversed? Does the proposed method exhibit any directional asymmetry?

  4. Figure 1 is not sufficiently intuitive, particularly the noise interpolation component. A clearer visualization of how the inverted latents are interpolated and subsequently used for generation would improve the presentation.

  5. The Method section could better emphasize the motivation and rationale behind the key design choices before introducing their technical details. Currently, implementation details, such as slerp, are discussed extensively before the overall reasoning behind the framework becomes sufficiently clear. Presenting the main design principles first, followed by the technical details, would make the method easier to follow.

  6. Why should the distance from the source endpoint increase monotonically along a morph trajectory? Since the proposed method uses spherical interpolation, the resulting trajectory does not necessarily exhibit linear or monotonic progression in the evaluation distance space. Could the authors further justify this assumption, particularly given that a perceptually smooth morph may follow a nonlinear trajectory?

4 comments
AuthorsOpenReview ↗

Response to Reviewer FDPX

W1 (style vs. structure). Removing either component raises MAR/MSR (Sec. 5.4). Audio: style-only (style-only) changes semantics but loses shared temporal structure; noise-only (noise-only) keeps structure but barely changes semantics, sounding like a crossfade. Both are needed.

W2 (MPS vs. other metrics). MPS agrees with humans better than first- and second-order smoothness (App. F.2). FAD and KAD measure quality, CLAP measures text alignment, and Chroma and LPAPS measure pairwise similarity; none of them measures path shape. MPS complements them, so we report it with FADₛᵤₘ.

Q1 ("real point"). A point on the path that is perceptually plausible music. We replaced the phrase with "perceptually acceptable data point".

Q2 (instructions; "semantic"). A morph is always defined by two endpoints. For editing, an LLM rewrites the source caption using a free-form instruction, the target is generated and beat-aligned, and we morph from source to target. The phrase "open-vocabulary" is removed. The style token is a pooled global embedding that carries genre, instrumentation, and mood, while the noise latent carries timing; this distinction is now defined in Sec. 4.2.

Q3 (swap). Near-symmetric, since slerp(a, b; λ) = slerp(b, a; 1 − λ) and each endpoint is inverted independently. The only asymmetry is which song sets the tempo. See all 9 pairs swapped at swap.

Q4/Q5 (Fig. 1; Method). Fig. 1 is redrawn as Align, Encode, Interpolate, Denoise. Sec. 4 now opens with the intuition (structure lives in the inverted noise, semantics in the style token), and the slerp details are consolidated in Sec. 4.4.

Q6 (monotonic/linear). Agreed: the "straight line" sentence was an overstatement. We have revised this to: "on average over many pairs, such a path can be reparameterized to a straight line in normalized distance-to-left; we use linearity as a reference, not a requirement." Monotonicity and the endpoint bound apply to the average path over many pairs, not to individual paths, and are the minimal conditions for λ to act as a strength control.

AuthorsOpenReview ↗

Final Reminder

Hello Reviewer FDPX,

As the discussion period ends today, please let us know if the response adequately address your concerns. We would be happy to clarify anything.

Thanks,

Authors of MusicMorpher

AuthorsOpenReview ↗

Summary of changes addressing your review

Thank you for taking the time to revisit your review. We reiterate again your questions helped us strengthen the paper, and we have tried to address each one directly:

  • Does MPS reflect perception? (W2) MPS agrees with human listeners more often than the smoothness metrics on every backbone (62.6–65.2% vs. 48.0–58.6%; App. F.2, Fig. 9). In our listening test, MusicMorpher was also preferred 69% of the time (1,744 votes, p < 0.01).
  • Are both style and structure needed? (W1) The ablations in Sec. 5.4 show that removing either one worsens MAR and MSR. You can hear the difference on the project page (#style-only, #noise-only).
  • Clarity (Q1–Q6): We addressed all six questions. We redrew Fig. 1, reorganized Sec. 4 to begin with the intuition, defined the key terms, and softened the "straight line" claim.

We hope these changes address your concerns, and we would be grateful if you would consider updating your score.

Thank you for the authors' revisions. Several of my concerns are addressed. However, my main concern (w1) remains unsolved, as the ablation studies demonstrate improvements in overall smoothness but do not directly test how style and temporal structure are interpolated or whether they fulfill their intended role. Since this is closely related to the paper's main technical contribution, I will maintain my score.

Reviewer_ZoKJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MusicMorpher, a method for constructing a morph path between two audio endpoints, or between a source audio and a target generated from an edit instruction. The method first aligns the tempo and beats of the endpoints, extracts a per-song style token representing global characteristics, and inverts each endpoint into a noisy latent that anchors temporal structure. The style tokens and inverted latents are then interpolated separately using slerp. The authors also introduce Morph Path Smoothness (MPS), an embedding-based metric intended to quantify the linearity of a morph trajectory.

Critical strengths
  1. Separately interpolating the global style token and inverted latent provides an intuitive mechanism for modifying global characteristics while anchoring temporal structure. The component ablations show that removing either component increases MAR.

  2. The paper provides a clear geometric motivation for using slerp instead of linear interpolation in both the LayerNorm-normalized style-token space and the approximately Gaussian high-noise latent space.

  3. The quantitative evaluation is large-scale, covering 4,500 text-prompt pairs and three alignment conditions, resulting in 13,500 morph paths. Multiple audio embedding backbones and alignment settings are considered.

Critical weaknesses
  1. Related work and baseline coverage omit relevant audio-morphing methods such as SoundMorpher [1] and MorphFader [2]. Even if the settings are not directly comparable, these works should be discussed and the differences in novelty clarified.

  2. There is no formal subjective evaluation. Music-morphing quality, smoothness, and resemblance to both endpoints are difficult to establish using objective embedding metrics alone.

  3. MPS is not validated against human judgments. The paper mentions preliminary evidence that an existing metric correlates poorly with human preference, but provides neither details of this study nor evidence that MPS correlates better.

  4. Table 8 shows that FreeMorph + Inversion achieves lower MAR than MusicMorpher on all three embedding backbones, although with worse FAD. This quality-smoothness trade-off should be included in the main comparison. A naive crossfade baseline is also needed to determine whether MPS rewards simple mixing because of its embedding-space linearity.

  5. Reproducibility is limited. Licensing restrictions may justify withholding the data and checkpoints, but do not by themselves explain why the implementation code cannot be released.

  6. The evaluation uses only endpoints synthesized by the same base model used by MusicMorpher. Experiments on real-world or out-of-distribution music are needed to demonstrate generalization to arbitrary input audio.

References

[1] Niu, X., Zhang, J. and Martin, C.P., 2024. Soundmorpher: Perceptually-uniform sound morphing with diffusion model. arXiv preprint arXiv:2410.02144.

[2] Kamath, P., Gupta, C. and Nanayakkara, S., 2025, April. Morphfader: Enabling fine-grained controllable morphing with text-to-audio models. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1-5). IEEE.

Questions for authors
  1. How does MusicMorpher differ from prior audio-morphing methods such as SoundMorpher and MorphFader? Could the authors clarify the novelty relative to these works?

  2. Could the authors provide a formal listening test evaluating audio quality, morph smoothness, and resemblance to both endpoints?

  3. How do the authors reconcile the main smoothness claims with the lower MAR achieved by FreeMorph + Inversion in Table 8? Could they also report MPS and FAD for a naive crossfade baseline?

  4. Could the authors clarify the scope of the “training-free” claim, given that the style-token adapter is trained jointly with the base model? Can the implementation code be released independently of the training data and checkpoints?

  5. Does MusicMorpher generalize to real-world audio inputs rather than endpoints generated by the same base model?

2 comments
AuthorsOpenReview ↗

Response to Reviewer ZoKJ

Q1 (SoundMorpher, MorphFader). Both are now in Sec. 2.3, and MorphFader is a baseline (Table 1, implementation in App. B, morphfader). MorphFader linearly mixes attention Q/K/V, which, like FreeMorph, tends to superimpose the two songs; MusicMorpher instead slerps a normalized style token and the inverted noise paths, and beats MorphFader on MPS and and beats MorphFader on MPS and FADₛᵤₘ. SoundMorpher needs per-endpoint embedding optimization, a per-pair LoRA, and a binary search, whereas MusicMorpher needs none of these. In practice this cost is prohibitive: in our test, generating a single 10-sample morph path between one source–target pair took SoundMorpher about 2 hours, versus a few minutes for MusicMorpher. At that rate, our 13,500-path evaluation would take roughly 27,000 GPU-hours, and such per-pair cost also makes SoundMorpher impractical for interactive use or as a large-scale data pipeline, which is our target setting.

Q2 (listening test). Added in Sec. 5.3 and App. E: listeners preferred MusicMorpher in 69% of comparisons overall (77% vs. FlowMorph, 61% vs. FreeMorph). All three are significant under a two-sided binomial test against 50% (overall and vs. FlowMorph p<10−11p < 10^{-11}; vs. FreeMorph p<0.01p < 0.01).

W3 (MPS vs. humans). Added in App. F.2 and Fig. 9: MPS agrees with listeners more often than first- and second-order smoothness on all backbones (e.g., MusicFM 64.7 vs. 58.3/52.0; CLAP first-order is below chance at 48.0).

Q3 (FreeMorph+Inv. MAR; crossfade). FreeMorph+Inv. gets a low MAR because its path barely changes (a noisy volume fade, freemorph), and its FADₛᵤₘ is worse. This is why we always report MPS with FADₛᵤₘ. The naive crossfade (Tables 10–11, crossfade) has a decent FADₛᵤₘ but a poor MPS. MusicMorpher is the only method that is best on both.

W5/Q4 (code; "training-free"). The model, data, and the code tightly coupled to them are under license. We give the full algorithm, hyperparameters, and baseline details so the method can be reimplemented on open T2A models (e.g., Stable Audio Open), and we will release MPS. "Training-free" means no training at morph time (no DiT fine-tuning, LoRA, or optimization); the style encoder is part of standard pretraining. We will make this scope explicit in the intro.

Q5 (real audio). MusicMorpher only inverts and encodes the endpoints, so it applies to any audio. See morphs between real Song Describer recordings at real-data.

Thank you for the detailed response and revision. The added listening study, human validation of MPS, audio baselines, and expanded related work address my main concerns, and I have raised my score accordingly. However, I still find licensing a weak reason for not releasing the implementation code.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overview (All Reviewers and AC)

We thank the AC and reviewers ZoKJ, FDPX, and 9zzt for their thoughtful feedback and the chance to improve our manuscript. Changes are in blue in the PDF, and new audio is on the project page (https://musicmorpher.github.io), organised into labelled suites.

Main themes. The comments mostly ask for additional evidence and clearer presentation, not changes to the method: (1) a listening test and human validation of MPS; (2) audio-native baselines; (3) broader related work; (4) clarity of Fig. 1, the Method section, and some wording; (5) scope (real audio, endpoint symmetry, beat alignment, example selection). All are addressed.

Key results.

  • Listeners prefer MusicMorpher: 69% overall (77% vs. FlowMorph, 61% vs. FreeMorph; 75 listeners, 1,744 votes; binomial p < 10⁻¹¹).
  • MPS matches humans better than first- and second-order smoothness on all backbones (62.6–65.2% vs. 48.0–58.6% agreement).
  • Audio baselines: MusicMorpher beats MorphFader and a naive crossfade on both MPS and FADₛᵤₘ. The crossfade scores poorly on MPS, so MPS does not reward mixing.
  • Generality: works on real recordings and is near-symmetric under endpoint swap.
  • Transparency: original examples are labelled cherry-picked, and 10 randomly drawn pairs are added.

An invitation. Every concern now has new evidence behind it (human listeners, audio-native baselines, real recordings, and unselected examples), and we warmly invite the reviewers to listen at https://musicmorpher.github.io and, if it resolves your concerns, to reconsider your assessment.

AuthorsOpenReview ↗

General Reminder

Dear Reviewers,

We have provided responses to all reviewer comments and updated the manuscript PDF accordingly. As the discussion period approaches its conclusion, we would appreciate your feedback on whether our responses adequately address your respective concerns.

We would be happy to provide any additional clarification if needed. Thank you for your time, consideration, and continued engagement with our work.

Best regards,

The Authors of MusicMorpher

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.