- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents a new multi-resolution architecture for learning audio representations, based on JEPA, but with an encoder that is trained across “dynamically aggregated resolutions”. This architecture seems to improve the tradeoff between representing local and global features, and probes trained on this model seem to outperform most baselines in most tasks.
Critical strengths
The paper is technically very good, and very well presented. The model architecture and training procedure are clearly understandable, and the results obtained are objectively very good.
Moreover, the paper tackles an interesting problem, and addresses a real gap: in the audio domain, the representations are often biased towards encoding local vs. global details based on the scale of the input received by the encoder. The evaluation is not perfect (see weakness 2 below) but sufficient to demonstrate that the method works well.
Critical weaknesses
Weakness 1.
Not releasing training code, model weights, or specifics of the training data often leads to sterile publications that do not interact well with present and future literature, especially for a paper whose main contribution is presenting a new model, without any particularly novel components. If this paper is published as-is, future researchers are going to be expected to compare blindly with numbers written in this paper, which they will not be able to reproduce.
I recognize and appreciate the authors’ thoroughness in describing the architectural and training details. Unfortunately, this does not overcome the fact that the table presenting the main results is physically impossible to reproduce.
I strongly believe that reproducibility is a main pillar of science, and papers claiming state-of-the-art methods with numbers that cannot be reproduced end up being a net-negative contribution to the community. I understand the practical concerns that prevent authors from releasing model weights trained on private data. However, it is reasonable, in my opinion, to expect researchers who have access to 250k hours of private music data, to also be able to reproduce their experiments on open source datasets that are roughly 1% the size.
Moreover, in this period of time, there are major ethical concerns with training generative models with illegal, unlicensed data, especially in music. I believe that top-tier conferences like ICLR should set the bar for ethical and open research practice.
Therefore, I strongly believe that this paper should be rejected, unless the authors either (i) release the model weights, or (ii) reproduce their experiment on open-source, licensed data.
Weakness 2.
Layer selection is clearly a crucial part for the evaluation of models like this. Different tasks have been shown to be more easy to solve for layers at different depths. An additional evaluation in which layers are selected per-task, instead of per-model, would make the results much stronger and reveal a more complete picture.
Questions for authors
On evaluation: for baselines that have short contexts (like MERT, 5s), how do you extract features for longer samples (such as GTZAN, 30s and GiantSteps, 2m)?
The performance scaling between the three sizes (small, base, large) is very unexpected. It seems like some part of the training procedure was specifically tuned for the large model? For a paper that presents a model with no open source code, loss curve plots in the appendix would help understanding the weird scaling behavior of the architecture.
If I understand correctly, the four-query fusion version requires 4 forward passes, and 4x compute. Does the shared multi-resolution encoder produce features that fuse better than 4 independent models? I think a matched-compute comparison would be useful, where the four-query fusion is compared with an average of the four separate fixed-resolution models (3.1, 6.25, 12.5, 25hz).
Smaller comments:
On the topic of layer selection (see weakness 2), I also believe that it would be useful to:
- add (possibly to the appendix) the layers that were selected by your ID algorithm for each of the baselines, instead of only for your model;
- specify on what data was the ID computed for your model and the baselines;
- add your choice of ID algorithm (from the appendix I seem to understand it was TwoNN) to the main paper, in section 3.2.
Minor suggestions:
The context-detail tradeoff is a core point of the paper. When it is mentioned in line 152 (start of 2.2) it required for me to go back to the introduction and recall exactly how you define it. To avoid that, it would be nice to spend a few words re-explaining it shortly at the start of 2.2.
Unless I missed it, it looks like fig. 2 is never referenced. It would be nice to reference it (even multiple times if needed) close to the most complex parts of the method explanation, to give the reader an immediate visual cue on how to mentally picture what they are reading.
At the start of 3.1 (lines 227-228) the sentence “training samples a random 5-82s crop, …” is highly cryptic, due to the fact that “training” can be either a noun or an adjective and “samples” can be either a noun or a verb. At a first read, it looks like “training samples” is the subject and then the sentence would end without a predicate.
2 comments
I understand the reasons to keep data provenance confidential.
Unfortunately, this is not really a matter of "easier" or "harder" to reproduce. I don't think all authors should be expected to fully release data, weights, and code for all experiments.
The problem is that the main table claiming state of the art results is absolutely impossible to reproduce, not only hard or inconvenient. We have to remember that deep learning is an intrinsically data-centric science, and methods/models/algorithms are only meaningful when paired with the data they are meant to fit.
Other companies have solved the copyright/confidentiality problem in the past in at least two ways: (i) training and testing smaller versions of their models on open source data; (ii) releasing weights for the models trained on closed source data so that independent validation would be possible.
I understand that this can be a somewhat subjective matter, therefore I will raise my score from 1 to 2, to indicate being open to discussion about this with the AC and ultimately relying on their judgement.
Response to review
Thank you for the thorough review. We have updated the paper based on the below response to the comments and questions.
We thank the reviewer for the positive assessment of the technical quality and motivation of the paper. We also share their concerns about data in music AI and the importance of reproducibility. We understand that releasing code, weights, and data would make the work easier to reproduce.
However, there are multiple reasons for keeping data provenance and code confidential. This work was conducted within a large organization working with the music industry, and further details could compromise confidentiality and also the anonymity required by double-blind review. We also clarify that MultiJEPA is a representation-learning model, not a generative music model.
The ICLR 2027 author guidelines encourage code release and reproducibility, but do not require public release of model code or weights. Such releases are valuable, but not always possible in an industrial setting. We respectfully ask the reviewer to assess the paper on its scientific merits and the limitations we have disclosed, rather than treating non-release as grounds for rejection on its own. Previous music SSL work, including MERT, MuQ, PuPuJEPA, and MusicFM, has also reported training-data scales without full provenance disclosure. We believe our controlled experiments, detailed methodology, and analyses remain useful contributions despite the release constraints.
Repeating the full study on public data is not feasible during the discussion period. It would change the training-data regime, likely affecting absolute performance against external baselines. We'd also like to point out that an open large-scale music dataset with appropriate license conditions does not currently exist to the authors' knowledge. More importantly, our main conclusion comes from fixed- and multiresolution models trained on the same data, not from their comparison with external models. We have no reason to expect that changing datasets would remove the observed context–detail tradeoff, although we do not claim to have established data-independent results.
Questions and minor comments
Long clips: We process them in chunks and average representations, as in previous work.
Scaling: Larger music SSL models do not consistently outperform smaller ones, as also seen in MERT and PuPuJEPA. The cause remains unclear. JEPA loss curves are not necessarily informative about downstream quality because the EMA teacher changes during training.
Independent-model fusion: We agree that this is an important baseline. We have run it, and MultiJEPA fusion wins on 14/16 metrics, as detailed above.
Layer selection: We will report the selected layers, the data used for selection, and clarify that we use task-independent TwoNN selection, rather than choosing a different layer for each task.
Presentation: We will restate the context–detail tradeoff in Section 2.2, reference Figure 2 more clearly, and fix the ambiguous crop-sampling sentence.
We also respectfully question whether the inability to release data and weights should, on its own, warrant a clear rejection. The reviewer describes the paper as technically very good, well presented, and sufficiently evaluated to demonstrate that the method works. While we recognize the reproducibility limitations, ICLR does not require the public release of data or model weights. We therefore hope the reviewer will reconsider whether these limitations justify the rating, given their otherwise positive technical assessment.