- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MUUNE, which combines music-text data construction, music knowledge adaptation and unified representation supporting audio of up to 300 seconds learning with a multimodal large language model. In the evaluation, authors construct a scalable multi-view supervision pipeline and atomic counterfactual evaluations with public-data extensions.
Critical strengths
-
The paper proposes a well-designed data construction pipeline that generates Multi-View Retrieval Text from audio using both Audio Language Models and MIR tools, enabling the creation of diverse texts (views) as well as counterfactual evaluation data.
-
The proposed full-song encoding approach, which represents an entire song by encoding the song within one music-adapted language-model sequence, enabling retrieval based on full-song music understanding rather than short audio excerpts. The experiments further demonstrate the effectiveness of this design, showing substantial gains when extending the input length from 30 seconds to 300 seconds.
Critical weaknesses
-
It remains unclear whether the gains come from music-specific reasoning learned by the Music Reasoner or simply from adapting an MLLM into a retrieval model. For example, a comparison with a baseline trained as an MLLM retriever without the reasoning component would help more clearly isolate the contribution of the Music Reasoner.
-
The relationship between the newly constructed training/evaluation datasets and the proposed method is not entirely clear.
Questions for authors
If my understanding is correct, in Table 1, SDD orig. and JMC use the original text captions, whereas the other datasets are based on the proposed Multi-View text generation pipeline. Since the Multi-View texts are constructed using the same methodology as that used for creating the training data (e.g., CL-Subset and CL-Full), there may be some concern that the evaluation setting is more favorable to the proposed approach. To avoid potential misunderstanding, it would be helpful to explicitly indicate which evaluation datasets are derived from the same Multi-View construction pipeline, for example through a marker or note in the table.
Thank you for your positive assessment of our work and for suggesting clearer annotations of the evaluation-text sources. We clarify below the relationship between our data pipeline and evaluation, and the contribution of music knowledge adaptation.
1. Evaluation-text sources and the motivation for our benchmarks
Your understanding is broadly correct, but the evaluation sets have different text sources. SDD_orig uses the original human-written descriptions, and JMC-30s uses the official captions. JMC-fullsong retains the original captioning prompt but regenerates descriptions from longer audio. Music4All-US10K preserves the original tags and constructs additional text views from these tags and associated attributes. In-house, SDD-MV, and MTG-Test-MV use the MusicMeta multi-view pipeline. We will explicitly annotate these distinctions in Table 1 and the appendix.
Public resources for full-song music understanding and retrieval remain underdeveloped, particularly when queries must specify multiple fine-grained musical attributes. SDD and JMC provide valuable starting points, but additional evaluation resources are needed to assess richer song-level conditions. Developing a scalable data-construction pipeline and multi-view, multi-attribute retrieval benchmarks from public audio is therefore an important contribution of this paper, rather than simply a means of evaluating our model.
Our evaluation is also not restricted to texts generated by the MusicMeta pipeline. Alongside SDD_orig, the JMC settings, and the original-tag-based Music4All-US10K evaluation, Table 2 includes independent public benchmarks covering short-clip retrieval and music-theoretic conditions. These evaluations examine performance across different text sources, granularities, and protocols, rather than relying exclusively on the multi-view benchmarks.
2. The data pipeline provides useful supervision beyond MUUNE
First, its benefits transfer across architectures. Table 1 adapts CLAP and MuQ-MuLan using the same CL-Subset and CL-Full contrastive data. Both models achieve higher overall retrieval scores than their public-weight counterparts. This shows that the supervision is useful for conventional dual encoders as well as our MLLM-based framework, and is not specific to the MUUNE architecture.
Second, the benefits extend to independent public benchmarks. Table 2 evaluates data-adapted models on MusicCaps-test, FGMCaps, and MusicBench A/B. In particular, MuQ-MuLan improves on all four tasks after CL-Full adaptation. This provides evidence that the training data support transfer beyond evaluation texts constructed by the same pipeline.
Third, the annotations undergo human quality assessment. Tables 14 and 15 report audits of structured tags, music-theoretic attributes, and multi-view queries. The audits provide direct evidence that the pipeline can deliver rich attribute coverage with low query hallucination rates. Thus, its usefulness is supported by both downstream experiments and annotation-quality assessment, rather than assumed from the generation procedure alone.
Together, these results establish that our pipeline supplies quality-audited supervision that is useful across architectures and evaluation settings. Furthermore, CLAP and MuQ-MuLan remain below MUUNE in overall retrieval performance even after adaptation with the same CL-Full contrastive data. This supports the effectiveness of combining music knowledge adaptation with unified representation learning, beyond the contribution of the contrastive dataset alone.
3. Music knowledge adaptation is distinguished from direct MLLM retrieval
Regarding whether the gains come from the Music Reasoner or merely from converting an MLLM into a retriever, Table 3(a) already includes MLLM baselines trained directly for retrieval without our music-metadata SFT.
The Baseline provides the most direct controlled comparison: Baseline, SFT-Subset, and SFT-Full share the same backbone initialization, eight-layer EH, and CL-Subset contrastive data. Their mean retrieval scores increase from 33.63% to 37.92% and 42.42%, respectively. The improvement therefore demonstrates the contribution of music knowledge adaptation beyond direct retrieval training. Qwen3 AuT and MOSS-Music provide supplementary comparisons under different initialization conditions.
Importantly, music knowledge adaptation and inference-time MMOT generation are distinct. The former incorporates musical knowledge into model parameters; the latter expresses that knowledge as explicit analysis text. The SFT comparison uses direct encoding without MMOT, so its gains do not depend on generating an intermediate reasoning sequence at inference. The continued improvement from SFT-Subset to SFT-Full further demonstrates that scaling music-specific supervision strengthens retrieval representations. This is precisely why music knowledge adaptation is a central component of MUUNE.