acceptodds
Withdrawn from ICLR 2027

MuseReason: From Musical Evidence to Counterfactual Editing via Bidirectional Affective Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Music emotion understanding and emotion conditioned music generation have made significant progress in recent years, but they are usually modelled as relatively independent tasks. Existing music emotion understanding methods mainly focus on emotion state prediction and lack an explicit characterisation of the musical evidence supporting emotional judgements. Emotion conditioned music generation methods typically take the target emotion directly as a condition, with less reasoning about which musical attributes should be adjusted based on the source musical state and which constraints should be maintained. This paper proposes MuseReason, a bidirectional emotion reasoning framework for music emotion understanding and counterfactual editing. MuseReason uses temporal music representations to extract verifiable musical evidence. It then combines the musical evidence with the target affect to predict structured music adjustments and preservation constraints for counterfactual editing. The experimental results show that MuseReason demonstrates capabilities in emotional understanding, structured output, and temporal emotional modelling. Structured music adjustment can support counterfactual editing by considering emotional changes while maintaining the source music. MuseReason extends music emotion modelling from a single emotional prediction to a unified framework that connects musical evidence, emotional reasoning, structured music adjustment, and generated result verification.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
music understanding, affective computing, audio language models, multimodal reasoning, counterfactual music editing, controllable music generation

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 2 reviews, above 0% of the others.
Reviewer_K8DVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

MuseReason feeds a frozen MERT-v1-330M encoder through a 32-query temporal projector into Qwen3-8B, which has three LoRA adapters.

  • Forward reasoning outputs musical "evidence" (tempo, mode, energy, rhythm, timbre, structure), a free-text appraisal, global valence/arousal and a trajectory summary.
  • Inverse reasoning takes the source state and a target affect and outputs a list of adjustments (attribute, operation, magnitude, region) plus a preservation set.
  • Outcome reasoning judges a generated edit. From the edited-audio tokens, measured outcomes and the target, it predicts affect-improved / preservation-satisfied / accepted / failure-mode / next-action.

A control compiler turns the plan into inputs for a frozen ACE-Step 1.5 editor. Supervision comes from DEAM, PMEmo2019, MERGE, MusicQA and MusicBench. The counterfactual-supervision construction comprises 3,412 MusicBench edit pairs, 3,246 DEAM target-affect plans and 12,984 ACE-Step candidates. Evaluation covers:

  • Affect estimation: V/A MAE, quadrant accuracy and trend F1 against audio-language models.
  • Plan matching: attribute, operation, magnitude and preservation scores for the inverse step.
  • Editing: "Joint success", meaning verifier-measured affect improvement with content ≥ 0.25 and chroma ≥ 0.85.
  • Outcome assessment: macro-F1 on the five judgment fields.

Headline numbers are V/A MAE 0.263/0.311, Joint success 68.77% (best baseline 39.41%), and outcome "Accepted" F1 64.58.

Critical strengths
  • Linking "why does this sound sad" to "what should change to make it calm, and what must stay" is a sensible goal. The explicit separation of predicted plan, compiled controls and post-edit measurements (Table 6, Fig. 6) makes the pipeline easy to inspect.

  • Partitions are made at the source level, PMEmo is split by artist, and verifier-based reranking is reported separately as selection-assisted (App. D.4).

Critical weaknesses
  1. The outcome-consistency module (contribution 3) is evaluated on labels that are deterministic functions of its own inputs, and it does not see the plan it is meant to check.

    • The labels are a rule on the inputs. Eq. 26 defines A = 1[Δaff > 0], P = content ≥ 0.25 ∧ melody ≥ 0.85 and Accepted = A ∧ P. Failure mode and next action are a lookup on (A, P) (Table 11). MEASURED_OUTCOME (App. A.1) contains target_distance_improvement (= Δaff), content_similarity and melody_similarity. A deterministic implementation using the same measurements and thresholds would reproduce all five reference labels exactly.
    • The model scores well below that. With full feedback it reaches 66.79 / 58.07 / 64.58 / 25.95 / 35.08 macro-F1 (Table 12). Table 14's "Accepted Rule" and "Mapping" scores measure internal consistency; they are not a rule baseline.
    • What the ablations show. "No feedback" removes the measurements that determine the labels, and "shuffled" breaks their correspondence with the sample. These ablations show sensitivity to the feedback, but not musical reasoning beyond the deterministic mapping.
    • The plan is not an input. The specified outcome interface (Eq. 17, Fig. 6, App. A.1, Algorithm 1) takes (Zcf,ocf,a⋆)(Z_{cf}, o_{cf}, a^\star) but not Δ^\hat\Delta or p^\hat p. As specified, it cannot compare "intended musical adjustments and their realised results", as contribution 3 and Sec. 5 claim.
  2. The editing comparison does not isolate the reasoning.

    • The conditioning text barely matters. Five conditions run through the same frozen ACE-Step editor: Reference Plan, Target Text, Target VA, Direct Music LLM and Qwen2.5-Omni. They give very similar results on DEAM (Joint 20.45–21.19%, content 0.2650–0.2665, melody 0.9320–0.9323; Tables 3–4). Even the condition called Reference Plan does no better than plain target text; its provenance and compilation need clarification.
    • A possible editor-control confound. MuseReason jumps to content 0.8811 and Joint 68.77%. App. A.4 lists "source preservation strength" among the controls the compiler can set, but the controls actually used for each condition are not reported. The large preservation gap could therefore come from a different editor setting rather than from better plans, and the paper gives no way to tell. A comparison at matched preservation strength is needed.
    • Sensitivity to the threshold. The content threshold (0.25) sits close to these conditions' mean content similarity (≈ 0.265–0.267). Without per-example distributions or a threshold-sensitivity analysis, it is unclear how much the threshold drives the Joint gap. MuseReason's Joint and Improve rates coincide on all three datasets, so for it the preservation criterion removes no affect-improving edits at the reported precision.
    • Unmatched durations. Instruct-MusicGen, MusicGen-Melody and M2UGen produce 5 s outputs that are not duration-matched (Table 10).
    • No human judgment. Affect change is measured only by an independently trained MuQ-based verifier, validated on held-out DEAM annotations. The same verifier is used for outcome supervision and for the reranking analysis. As Sec. 5 acknowledges, there is no listener study, so perceived affect change and source identity are unvalidated.

Further issues with contributions 1–2:

  • Affect estimation. MuseReason is trained with affect supervision from DEAM/PMEmo. Table 1 compares it with audio-language models without specifying their adaptation or prompting protocol. Even so, MuseReason trails on quadrant accuracy (46.40 vs 57.32 for Kimi-Audio, 52.23 for Qwen2.5-Omni), descriptor F1 (52.00 vs 54.33) and valence-trend F1 (29.92 vs 42.81). A supervised MER baseline on the same splits (e.g. MERT or MuQ plus a regressor) is missing. The verifier's trajectory CCC of 0.757 on DEAM suggests such a baseline could be strong, though it is not a matched comparison.
  • Audio dependence. Fig. 3 supports strong audio dependence for temporal trends (−69.3% / −79.9% relative under zero audio). For static predictions the dependence is much weaker: quadrant accuracy falls only 3.0% and descriptor F1 only 4.6% (relative), and V-MAE gets 16.2% worse. The grounding claim in Sec. 4.2 should distinguish the two.
  • Inverse plans. Preservation sets have very low agreement with the reference sets (F1 9.04 / 7.08 / 5.80, Table 9), and zero audio barely changes attribute F1 (50.12 → 46.97) or preservation F1 (9.04 → 8.43). Multiple valid plans may exist, but the paper does not show that the predicted preservation sets are appropriate alternatives.
  • Provenance of the reasoning targets. It is not stated how the appraisal texts and the DEAM target plans were authored (rules, an LLM, people), or whether anyone checked them.
  • Sample sizes and uncertainty. Editing evaluation counts and uncertainty are not reported. The rounded percentages are consistent with denominators of 269 (DEAM) and 142 (PMEmo/MERGE), which would be small. The reference audio set for MERT-based FAD is not specified; Table 5 reports DEAM FAD 455.03 for MuseReason against about 5,837–5,865 for the controlled baselines.
  • Missing related work: preservation-constrained music editing (MusicMagus, MEDIC, InstructME, AUDIT), emotion-to-attribute generation (EMOPIA, EmoGen, MusER), and explainable MER via mid-level features (Chowdhury et al., 2019).
  • Typos: "Control Complier" (Fig. 2), "procided" (Sec. 4.2).
Questions for authors
  1. (Could change my score.) Please report the ACE-Step mode and all compiled controls, including source-preservation strength, for each condition in Tables 3–4. How do the conditioning strategies compare at matched preservation strength?
  2. (Clarification.) How is the "structured diagnostic" Reference Plan constructed and compiled?
  3. (Could change my score.) How does a deterministic rule baseline do on outcome assessment? Does the implemented reasoner receive Δ^,p^\hat\Delta, \hat p, even though Eq. 17 omits them?
  4. (Clarification.) How were the appraisals and DEAM target plans generated, and were they checked by people?
  5. (Suggestion.) A blinded listening test of affect change and source identity, with rater and item counts and CIs.
  6. (Suggestion.) A supervised MER baseline on the same splits, and the full adaptation/prompting protocol for the Table 1 baselines.
Reviewer_YL5gOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper connects music affect understanding, structured adjustment, counterfactual editing, and outcome assessment. It employs mismatched and zero audio, feedback perturbations, and conditioning comparisons with the same frozen editor to examine the role of musical evidence and structured plans.

Critical strengths

The audio and feedback controls help check whether the model actually uses its inputs. This is more informative than comparing final scores alone. The main results are also clearly separated from verifier-based candidate reranking. (Sections 4.2–4.5; Appendix D.4)

Critical weaknesses

The editing controls are not described clearly enough for a fair comparison. The compiler can change source-preservation strength, but the actual values for each condition are not reported clearly. It is therefore difficult to separate the benefit of reasoning from the benefit of execution settings. (Appendix A.4) Also, Joint Success only requires movement towards the target affect and satisfaction of preservation thresholds. It does not mean that the target affect is reached. Without a listening study, the perceptual meaning of this success rate remains unclear. (Appendix D.4; Section 5)

Questions for authors

Are editing strength and preservation strength matched across conditions? Please report the actual compiled controls. Does the main advantage remain when requiring a minimum affect improvement or a small distance to the target affect?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.