acceptodds
Under review as a conference paper at ICLR 2027

Armonia: Edit-Compatible Diffusion Inversion for Text-Guided Music Editing

Abstract

Text-guided music editing aims to modify specified attributes of a recording while preserving musical content unrelated to the intended change, enabling flexible music creation. Inversion anchors this editing process to the source music by mapping it to a state that initializes generation under the target text. However, a state that enables faithful source reconstruction is not necessarily a suitable starting point for the requested edit. This distinction motivates us to move part of the editing control into inversion and prepare an initialization that is better suited to the requested edit. To this end, we propose *Armonia*, a training-free method that refines inversion states using output-level attention distillation derived from source- and target-conditioned model responses. To improve refinement efficiency and effectiveness while reducing the risk of off-manifold drift, Armonia uses temporal weighting to focus optimization on selected inversion steps and local projection to limit deviations from each DDIM inversion proposal. The resulting state is then used to initialize the original target-conditioned denoising process, leaving the pretrained diffusion model unchanged. Compared with SOTA, Armonia achieves an average relative improvement of 11.5% in CLAP across three datasets while retaining competitive structural preservation.

Primary area:
generative models
Keywords:
music editing, diffusion models, diffusion inversion, text-guided editing, training-free methods

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wVusOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(−1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Armonia, a training-free framework for text-guided music editing that addresses the reconstruction–editing objective gap. Unlike conventional inversion methods focused on source reconstruction, Armonia incorporates target guidance during DDIM inversion using output-level attention distillation, Temporal Weighting, and Local Projection to construct an edit-oriented initialization. Experimental results show strong target alignment (CLAP and REL), while source-preservation performance remains less competitive.

Critical strengths
  1. The paper presents an interesting perspective on music editing by arguing that inversion should produce an edit-oriented initialization rather than one optimized solely for source reconstruction. The authors also clearly formulate two key research questions that guide the method design. Incorporating target information during inversion is a well-motivated approach to addressing this problem.

  2. The proposed method combines attention distillation, Temporal Weighting, and Local Projection in a coherent framework. The overall design is interesting, and the method demonstrates comparable qualitative results.

Critical weaknesses
  1. The paper argues that source attention control produces low audible distortion but fails to fully achieve the requested timbre change. However, it does not clearly explain or provide sufficient evidence that this failure is caused by the inversion initialization rather than the mechanism itself. Further analysis would be needed to strengthen the motivation for the proposed approach.

  2. The fundamental goal of music editing is to modify the requested attributes while preserving unrelated musical content. However, the proposed method does not explicitly distinguish between editing-relevant and preservation-relevant information. In particular, the attention distillation objective assumes that differences between source- and target-conditioned attention outputs provide edit-specific guidance, but these differences may also reflect unrelated musical characteristics such as melody, rhythm, or phrasing. Temporal Weighting only controls when and how strongly refinement is applied, while Local Projection constrains latent displacement without explicitly preserving musical content. Therefore, it remains unclear whether the latent refinement effectively achieves both objectives. Further experimental analysis is needed to demonstrate that the intended attributes are modified while unrelated musical content is preserved.

+) Some figures could be improved in terms of clarity and presentation. In particular, several labels and descriptions in Figures 1 and 2 appear cluttered or are placed too closely together, making distinct components or concepts difficult to distinguish. A cleaner and more compact layout would improve readability.

Questions for authors
  1. The main text mentions that a link to edited music samples is available in the supplementary material, but I could not find the corresponding audio links in the provided Appendix. Could the authors clarify where these samples can be accessed?

  2. Why is latent refinement performed starting from z_1 rather than z_0? Is there a specific reason for not applying refinement directly to the initial source latent?

  3. What motivates the use of source-conditioned queries and target-conditioned keys and values in the attention distillation objective? Is this design intended to preserve source content while incorporating the desired target attributes? In particular, source queries may also contain timbre-related information, while target keys and values may encode melody or rhythm. How does this configuration distinguish between information that should be preserved and information that should be modified?

Reviewer_iAh2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the task of text-guided music editing (for example, modifying specified attributes of a piece of music) without additional training, using a pretrained text-conditioned music generation model. In particular, it tackles the challenges of editing methods that rely on diffusion model inversion. Naive inversion performs the transformation without using information from the target text condition. In contrast, the proposed method leverages this information by incorporating it into an output-level attention distillation loss function, thereby performing inversion in a way that makes editing easier. In evaluations across multiple datasets, the proposed method, Armonia, demonstrated an advantage in CLAP scores.

Critical strengths
  • One of the strengths of this work is that it validates the performance of the proposed method from multiple perspectives, using both objective and subjective evaluations rather than relying on a single dataset or metric.
Critical weaknesses
  • On the other hand, I judge that this paper does not meet the acceptance bar at its current stage of submission, for the following reasons:
    • The validation of the novelty and effectiveness of the proposed loss function is insufficient. I consider the main technical contribution of this paper to be the design of the loss function based on Output-Level Attention Distillation. However, in the comparison between SA map and SA output in Table 3, the improvement in CLAP score is marginal (from 0.352 to 0.356), and performance degrades on some metrics. Based on this result alone, it is difficult to conclude that the proposed loss function is fundamentally superior to existing methods based on attention maps. In addition, the paper does not provide sufficient theoretical or experimental analysis of why using the attention output rather than the attention map leads to improved editing performance. I believe it is necessary to explain more clearly what has been fundamentally improved relative to existing attention control methods.
    • The trade-off between editing performance and source preservation, and the fairness of the baseline comparison. Although Armonia achieves the best performance in CLAP score, it is sometimes inferior to existing methods such as Melodia in terms of source preservation performance. This leaves the question of whether the improvement of the proposed method is merely the result of increasing the editing strength. Furthermore, while the baselines use the recommended hyperparameters from their original papers, Armonia explores multiple settings, which raises concerns about the fairness of the comparison. I believe it is important to tune the hyperparameters of each method and compare the Pareto frontier between editing performance and source preservation. In particular, it is necessary to verify whether Armonia still shows an advantage under conditions that achieve comparable CLAP or LPAPS values. In particular, the advantage over Melodia cannot be sufficiently judged from Fig. 6.
Questions for authors
  • Would it be possible to provide a qualitative discussion of why SA output achieves better performance than SA map?
  • For comparisons with every method, please compare the Pareto frontier across multiple metrics, such as CLAP and source preservation metrics.
Reviewer_kngEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Armonia is a training-free method for text-guided music editing with AudioLDM 2. During DDIM inversion, selected latents are refined by a few Adam steps on an MSE between the source-branch self-attention output and a mixed output in which source queries attend to detached target-conditioned keys and values. A Gaussian timestep window and a per-step projection budget limit the refinement, and standard target-conditioned denoising follows. MelodiaEdit, MusicDelta and ZoME-Bench are used with eight zero-shot baselines, five objective metrics and a 67-participant listening test.

Critical strengths
  1. The central effect is shown in a clean matched comparison. Under identical settings (Table 2), refinement raises CLAP from 0.337 to 0.430 over plain DDIM inversion without degrading LPAPS, Chroma or CQT1-PCC, and listeners rank Armonia first in relevance on all three datasets.
  2. The ablations are extensive, and the higher runtime (about 2.2x Melodia) and weaker preservation are reported openly.
Critical weaknesses
  1. The two methodological contributions are close to existing diffusion techniques that the paper does not position against. The "reconstruction-editing gap" claimed as the first contribution is the premise of editability-oriented inversion such as SPDInv and ENM Inversion (both cited) and LORE (2025), which optimizes inverted noise with an attention-based loss toward the target. Eq. 8 has the form of the attention distillation loss of Zhou et al. (CVPR 2025), which optimizes latents to match a self-attention output whose queries read another source's keys and values, a construction shared by K/V-swapping methods (Cross-Image Attention, Visual Style Prompting). Neither Zhou et al. nor LORE is cited. Applying these ideas to music inversion is useful, but the paper should state what is specific to music. The mechanism argument (p.5) is untested, and the source condition appears to be an empty prompt, so the loss contrasts unconditional and target-conditioned responses.
  2. The evidence does not show a better trade-off than the strongest baseline. In Fig. 6, Melodia matches or exceeds Armonia at inversion ratios 0.6-0.8, and Armonia leads only at 0.9-1.0, contrary to the text's claim that full inversion is "best balanced". On MusicDelta, all preservation metrics are worse than Melodia's. Armonia's rho and inversion ratio were selected by CLAP on the evaluation sets (App. D), while baselines use defaults. At full inversion, the MusicDelta CLAP gain over Melodia falls from 13.1% to about 6% (Table 6). CLAP, LPAPS and KAD share one CLAP checkpoint, results are single-run, and the listening study gives no intervals, case-selection rule or loudness normalization.

Minor (not decisive): citations are all narrative-style without parentheses, references contain casing and venue errors (e.g., "Musiclm", "Ismir"; SDEdit cited as arXiv), and the TL;DR (10.8%) and abstract (11.5%) disagree.

Questions for authors
  1. Can you report CLAP-versus-preservation curves against Melodia and SteerMusic on all datasets, with operating points selected on a held-out split?
  2. Is c_src empty in the main experiments? If so, how does Armonia compare with inverting under the target prompt or with CFG during inversion?
  3. With learning rate 1e-3, one Adam step already gives a mean-squared displacement of about 1e-6 = rho. How often is the projection active, and how does K = 3 differ from one step?
  4. On the demo page, Melodia files are named "ours_melodia_(ours)" or "ours_ours", DDIM files "ddim_worse", and the page is titled as a timbre-transfer demo. How were columns produced and examples selected? Was loudness normalized in the listening study?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.