acceptodds
Under review as a conference paper at ICLR 2027

MultiJEPA: Learning Music Representations Across Temporal Resolutions

Abstract

We present MultiJEPA, a multi-resolution joint-embedding predictive architecture for learning representations of signals whose semantics occur at multiple scales. Rather than committing to a single resolution, MultiJEPA trains one encoder across dynamically aggregated resolutions. In music, semantics span timescales from the millisecond-scale precision required for onset and beat localization to harmonic context and musical form unfolding over tens of seconds to minutes. Music is therefore an attractive domain for studying how temporal resolution relates to learned semantics. Specifically, a controlled five-resolution analysis reveals a strong context–detail tradeoff: coarse representations favor tonal and structural understanding, while fine representations favor beat tracking and transcription, with no single resolution dominating. MultiJEPA recovers this fixed-resolution frontier with a single encoder queried at four temporal rates. Simple four-query fusion exploits complementary information across resolutions, outperforming the compared audio-only SSL encoders on 15 of 16 probing metrics spanning nine downstream tasks. We further propose a short fine-to-coarse self-alignment phase that organizes these views into a nested representation and improves cross-resolution probe transfer. Together, these results establish resolution as an explicit design axis for music understanding models, with MultiJEPA as a concrete method to extend the pareto frontier of the context-detail tradeoff.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Self-Supervised Learning, Representation Learning, Joint-Embedding Predictive Architecture, Music Information Retrieval, Multi-Scale representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_k7RMOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a new multi-resolution architecture for learning audio representations, based on JEPA, but with an encoder that is trained across “dynamically aggregated resolutions”. This architecture seems to improve the tradeoff between representing local and global features, and probes trained on this model seem to outperform most baselines in most tasks.

Critical strengths

The paper is technically very good, and very well presented. The model architecture and training procedure are clearly understandable, and the results obtained are objectively very good.

Moreover, the paper tackles an interesting problem, and addresses a real gap: in the audio domain, the representations are often biased towards encoding local vs. global details based on the scale of the input received by the encoder. The evaluation is not perfect (see weakness 2 below) but sufficient to demonstrate that the method works well.

Critical weaknesses

Weakness 1.
Not releasing training code, model weights, or specifics of the training data often leads to sterile publications that do not interact well with present and future literature, especially for a paper whose main contribution is presenting a new model, without any particularly novel components. If this paper is published as-is, future researchers are going to be expected to compare blindly with numbers written in this paper, which they will not be able to reproduce.

I recognize and appreciate the authors’ thoroughness in describing the architectural and training details. Unfortunately, this does not overcome the fact that the table presenting the main results is physically impossible to reproduce.

I strongly believe that reproducibility is a main pillar of science, and papers claiming state-of-the-art methods with numbers that cannot be reproduced end up being a net-negative contribution to the community. I understand the practical concerns that prevent authors from releasing model weights trained on private data. However, it is reasonable, in my opinion, to expect researchers who have access to 250k hours of private music data, to also be able to reproduce their experiments on open source datasets that are roughly 1% the size.

Moreover, in this period of time, there are major ethical concerns with training generative models with illegal, unlicensed data, especially in music. I believe that top-tier conferences like ICLR should set the bar for ethical and open research practice.

Therefore, I strongly believe that this paper should be rejected, unless the authors either (i) release the model weights, or (ii) reproduce their experiment on open-source, licensed data.

Weakness 2.
Layer selection is clearly a crucial part for the evaluation of models like this. Different tasks have been shown to be more easy to solve for layers at different depths. An additional evaluation in which layers are selected per-task, instead of per-model, would make the results much stronger and reveal a more complete picture.

Questions for authors

On evaluation: for baselines that have short contexts (like MERT, 5s), how do you extract features for longer samples (such as GTZAN, 30s and GiantSteps, 2m)?

The performance scaling between the three sizes (small, base, large) is very unexpected. It seems like some part of the training procedure was specifically tuned for the large model? For a paper that presents a model with no open source code, loss curve plots in the appendix would help understanding the weird scaling behavior of the architecture.

If I understand correctly, the four-query fusion version requires 4 forward passes, and 4x compute. Does the shared multi-resolution encoder produce features that fuse better than 4 independent models? I think a matched-compute comparison would be useful, where the four-query fusion is compared with an average of the four separate fixed-resolution models (3.1, 6.25, 12.5, 25hz).

Smaller comments:

On the topic of layer selection (see weakness 2), I also believe that it would be useful to:

  • add (possibly to the appendix) the layers that were selected by your ID algorithm for each of the baselines, instead of only for your model;
  • specify on what data was the ID computed for your model and the baselines;
  • add your choice of ID algorithm (from the appendix I seem to understand it was TwoNN) to the main paper, in section 3.2.

Minor suggestions:

The context-detail tradeoff is a core point of the paper. When it is mentioned in line 152 (start of 2.2) it required for me to go back to the introduction and recall exactly how you define it. To avoid that, it would be nice to spend a few words re-explaining it shortly at the start of 2.2.

Unless I missed it, it looks like fig. 2 is never referenced. It would be nice to reference it (even multiple times if needed) close to the most complex parts of the method explanation, to give the reader an immediate visual cue on how to mentally picture what they are reading.

At the start of 3.1 (lines 227-228) the sentence “training samples a random 5-82s crop, …” is highly cryptic, due to the fact that “training” can be either a noun or an adjective and “samples” can be either a noun or a verb. At a first read, it looks like “training samples” is the subject and then the sentence would end without a predicate.

2 comments
AuthorsOpenReview ↗

Response to review

Thank you for the thorough review. We have updated the paper based on the below response to the comments and questions.

We thank the reviewer for the positive assessment of the technical quality and motivation of the paper. We also share their concerns about data in music AI and the importance of reproducibility. We understand that releasing code, weights, and data would make the work easier to reproduce.

However, there are multiple reasons for keeping data provenance and code confidential. This work was conducted within a large organization working with the music industry, and further details could compromise confidentiality and also the anonymity required by double-blind review. We also clarify that MultiJEPA is a representation-learning model, not a generative music model.

The ICLR 2027 author guidelines encourage code release and reproducibility, but do not require public release of model code or weights. Such releases are valuable, but not always possible in an industrial setting. We respectfully ask the reviewer to assess the paper on its scientific merits and the limitations we have disclosed, rather than treating non-release as grounds for rejection on its own. Previous music SSL work, including MERT, MuQ, PuPuJEPA, and MusicFM, has also reported training-data scales without full provenance disclosure. We believe our controlled experiments, detailed methodology, and analyses remain useful contributions despite the release constraints.

Repeating the full study on public data is not feasible during the discussion period. It would change the training-data regime, likely affecting absolute performance against external baselines. We'd also like to point out that an open large-scale music dataset with appropriate license conditions does not currently exist to the authors' knowledge. More importantly, our main conclusion comes from fixed- and multiresolution models trained on the same data, not from their comparison with external models. We have no reason to expect that changing datasets would remove the observed context–detail tradeoff, although we do not claim to have established data-independent results.

Questions and minor comments

  • Long clips: We process them in chunks and average representations, as in previous work.

  • Scaling: Larger music SSL models do not consistently outperform smaller ones, as also seen in MERT and PuPuJEPA. The cause remains unclear. JEPA loss curves are not necessarily informative about downstream quality because the EMA teacher changes during training.

  • Independent-model fusion: We agree that this is an important baseline. We have run it, and MultiJEPA fusion wins on 14/16 metrics, as detailed above.

  • Layer selection: We will report the selected layers, the data used for selection, and clarify that we use task-independent TwoNN selection, rather than choosing a different layer for each task.

  • Presentation: We will restate the context–detail tradeoff in Section 2.2, reference Figure 2 more clearly, and fix the ambiguous crop-sampling sentence.

We also respectfully question whether the inability to release data and weights should, on its own, warrant a clear rejection. The reviewer describes the paper as technically very good, well presented, and sufficiently evaluated to demonstrate that the method works. While we recognize the reproducibility limitations, ICLR does not require the public release of data or model weights. We therefore hope the reviewer will reconsider whether these limitations justify the rating, given their otherwise positive technical assessment.

I understand the reasons to keep data provenance confidential.

Unfortunately, this is not really a matter of "easier" or "harder" to reproduce. I don't think all authors should be expected to fully release data, weights, and code for all experiments.

The problem is that the main table claiming state of the art results is absolutely impossible to reproduce, not only hard or inconvenient. We have to remember that deep learning is an intrinsically data-centric science, and methods/models/algorithms are only meaningful when paired with the data they are meant to fit.

Other companies have solved the copyright/confidentiality problem in the past in at least two ways: (i) training and testing smaller versions of their models on open source data; (ii) releasing weights for the models trained on closed source data so that independent validation would be possible.

I understand that this can be a somewhat subjective matter, therefore I will raise my score from 1 to 2, to indicate being open to discussion about this with the AC and ultimately relying on their judgement.

Reviewer_HbUvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes MultiJEPA, a self-supervised learning framework that learns music representations at multiple temporal resolutions. The method extends JEPA with a Transformer Resampling Block to aggregate fine-grained spectrogram tokens into coarse tokens, followed by a self-alignment phase that matches pooled finegrained resolution features with native coarse resolution representations. Experiments across different downstream tasks show that it outperforms the compared audio-only self-supervised learning baselines on multiple evaluation metrics.

Critical strengths
  1. The research problem of the paper is relevant and clear. Experiments on different datasets and settings also demonstrate the motivation.
  2. The proposed multi-resolution architecture is straightforward and practical.
  3. Extensive experiments have been conducted across multiple datasets with different baselines. It also includes analyses of model scaling, cross-resolution alignment, probe transfer, and long-context generalization. These experiments give useful evidence about how the framework performs at different temporal resolutions.
Critical weaknesses
  1. The proposed cross-resolution self-alignment objective is not sufficiently justified. Equation 2 matches block averaged finegrained resolution features with coarse teacher features, but does not explicitly preserve finegrained information within each block. The paper states that alignment decreases fine-task performance raises further concerns about this. In addition, the paper does not examine the sensitivity to the alignment weight or compare the proposed asymmetric objective with alternative alignment strategies. More analysis is needed to establish the robustness of the proposed design.
  2. Ablation and sensitivity analysis experiments are not comprehensive enough. For instance, some key components such as TRB are not ablated, and key parameters such as sampling rates, masking ratios, token budget, and RoPE scaling etc. are never validated and justified. These missing experiments make it difficult to assess the robustness of the proposed method and determine whether simpler, more computationally efficient designs could achieve comparable performance.
  3. The computation and complexity analysis is not sufficiently evaluated in the paper. The authors should provide complexity comparisons such as FLOPs, training time, latency, memory usage among fixed-resolution models, four-query fusion models, and inference with fair and consistent parameters and experiment setups.
Questions for authors
  1. How sensitive is four-query fusion to the selected resolutions? Could the authors report results for different subsets of two or three resolutions and compare them with the full four-query fusion? This would help determine whether all four queries are necessary and how performance changes with inference cost.
  2. How is the stability of the proposed model compared to baselines? Can authors also provide the variance metrics for the experiments?
2 comments
AuthorsOpenReview ↗

Response to review

Thank you for the thorough review. We have updated the paper based on the below response to the comments and questions.

Weaknesses

  1. Fine detail and alignment: The JEPA objective continues during self-alignment, so fine-grained features are still trained. The alignment loss matches block averages; it does not directly constrain variation within blocks. We observe a slight drop in transcription, but not in beat tracking. We agree that the reason for this tradeoff is not fully established; an alignment-weight sweep would be useful but is too costly for the discussion period.

  2. Ablations and hyperparameters: Masking, token budget, and other settings were selected empirically and shared across the controlled models. The fixed 2,048-token budget keeps encoder cost comparable across resolutions. Extensive sweeps are expensive and intertwined with EMA dynamics. The TRB was chosen as a flexible learned pooling operator; simpler alternatives such as mean pooling are possible but lose information. We do not claim this frontend or these hyperparameters are universally optimal.

  3. Compute: We agree and will add training and inference FLOPs. Four-query fusion costs four forward passes, while the shared encoder avoids four separate pretraining runs.

Questions

  1. Fusion subsets: Two- or three-rate fusion is an interesting cost–performance tradeoff. We expect coarse and fine views to be particularly complementary, but have not measured the optimal subset. We focus here on the simple, parameter-free four-query fusion rather than claim results for untested combinations.

  2. Stability and variance: As in other EMA-based JEPAs, training stability depends on hyperparameters. Models with frozen targets may be less sensitive in this respect. We report five probe seeds where applicable, but not multiple costly pretraining seeds. We will make this limitation explicit.

We appreciate the reviewer's recognition of the relevance of the problem, the extensive evaluations, and the absence of substantive soundness concerns. We acknowledge that the suggested ablations would provide useful additional insights, but believe that the existing experiments already support the main conclusions of the paper. We hope these clarifications address the reviewer's concerns and provide grounds to reconsider the rating.

Thank you for the response but some aspects such as additional analysis and experiments for alignment weight study are still not addressed. I choose to keep the scores.

Reviewer_auS6OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces MultiJEPA, a multiresolution joint-embedding predictive architecture for music representation learning. A shared encoder, trained through masked latent prediction with deep supervision at four depths, operates at four temporal token rates (3.1–25 Hz). A Transformer Resampling Block pools a common fine-grained patch grid before encoding; with a fixed budget of 2048 tokens, coarser rates cover longer audio crops (approximately 10–82 s; Tab. 1). Experiments with five fixed-resolution JEPAs (3.1–50 Hz) reveal a context–detail tradeoff: coarse resolutions favor key estimation and structural segmentation, while fine resolutions favor beat tracking and transcription (Tab. 2, Fig. 4). MultiJEPA largely preserves these task-specific profiles, and averaging representations from its four resolution queries reportedly outperforms the compared audio-only SSL baselines on 15 of 16 linear-probe metrics (Tab. 2, Fig. 1). A brief self-alignment stage matches block-pooled fine-resolution features to native coarse-resolution features, improving cross-resolution agreement, probe transfer, and full-track structure scores for the fine-resolution query, with a small performance loss on fine-grained tasks (Figs. 5–7). All models are trained on approximately 250,000 hours of internal audio; neither the training data nor the model weights will be released.

Critical strengths
  1. A controlled, fixed-budget resolution sweep establishes temporal rate as an important design variable. All five models use the same data, training recipe, and token budget, making the contrasting trends easy to interpret: coarser rates favor key estimation and structural segmentation, finer rates favor beat and onset detection, and genre classification and tagging remain largely unchanged. This is the paper’s most convincing finding: it does not hinge on comparisons with external baselines and provides concrete evidence for treating temporal resolution—and the context length it affords—as an explicit design choice.

  2. Strong probing performance across coarse- and fine-grained tasks (Tab. 2). The proposed recipe substantially outperforms the compared audio-only SSL encoders on structural segmentation (approximately +5–7 relational mIoU), onset transcription (approximately +10 F1), and F0 estimation, while remaining competitive on genre classification and tagging. These results support JEPA-style encoders with deep supervision and long contexts as strong backbones for music representation learning. However, as discussed below, the gains reflect the full training recipe and data scale; they cannot be attributed specifically to multiresolution training.

Critical weaknesses

1. The novelty and empirical benefit of multiresolution training are not clearly established.

Most architectural components are adopted from prior work: masked latent prediction (I-JEPA/V-JEPA), multi-depth supervision (V-JEPA 2.1), the Transformer Resampling Block (SAME), and continuous 2D RoPE. The distinctive contributions therefore rest on multiresolution training and cross-resolution self-alignment, but neither is clearly positioned against prior work. For multiresolution training, the paper omits music-audio literature sharing the motivation that music spans multiple timescales and combining complementary representations across scales, including multiscale pooling for tagging (Hamel et al., ISMIR 2011), spectrogram-pyramid feature learning (Dieleman & Schrauwen, ISMIR 2013), and multi-level, multi-scale aggregation of CNNs trained on different input lengths (Lee & Nam, IEEE SPL 2017); these are directly relevant to the paper's fusion finding. It also omits once-for-all sequence compression (Chen et al., ICASSP 2023), which already trains a single speech SSL model to operate across a range of frame rates with marginal loss relative to fixed-rate variants. Self-alignment, presented as part of Contribution 1, is a form of cross-resolution feature distillation, closely related to FlexiViT's teacher distillation (Beyer et al., 2023) and DINO's local-to-global multi-crop matching (Caron et al., 2021), both cited by the authors, and in audio to ATST (Li et al., IEEE/ACM TASLP 2024), which distills a clip-level teacher into a frame-level student. Bringing these ideas to music would still be a valuable contribution if it yielded clear gains, but the evidence for such gains is limited.

The experiments also do not isolate the benefit of multiresolution training. The best fixed-resolution model for each metric outperforms the best multiresolution query or fusion on 13 of 16 metrics, with gaps of approximately 2–4 points on OpenMIC, both key datasets, and structure (Tab. 2). Fixed-resolution models already outperform the audio-only baselines on 15 of 16 metrics, suggesting that the headline performance largely reflects the shared recipe and data scale; deep supervision alone contributes several points (Tab. 7). Matched-rate comparisons are mixed, and Fig. 1 does not resolve the issue: its nondominated set includes the fixed 3.1-Hz model, while no fusion of fixed-resolution encoders is evaluated to distinguish multiresolution fusion from generic ensembling. The frontends also differ: fixed-resolution models vary the input patchifier size, whereas MultiJEPA pools a shared patch grid through the TRB. Without a single-rate TRB control, performance differences cannot be attributed specifically to multiresolution training. Finally, the practical advantage of using one encoder instead of several remains unquantified because training compute is not compared. Together, these gaps weaken Contributions 1–2 and the conclusion’s claim to “match or exceed specialized fixed-resolution models.” The related work, contribution list, abstract, and conclusion should distinguish the demonstrated strength of the overall recipe from the unproven benefit of multiresolution training itself.

2. Overstated performance claims

The headline claim of “15 of 16 metrics spanning nine downstream tasks” overstates the breadth of the results, and the comparisons are not conducted under matched conditions. The 16 metrics fall into correlated groups: Harmonix structure contributes four (two protocols × two metrics, accounting for 25% of the total), while MTAT (AUROC/mAP), key estimation (GiantSteps/FMAK), beat tracking (two tolerances applied to the same predictions), transcription (frame/onset), and F0 estimation each contribute two. Only GTZAN and OpenMIC contribute one each. The task count is also inconsistent: the nine column groups in Tab. 2 count key estimation twice, yielding eight distinct task types, whereas Tab. 6 lists ten dataset entries. At the task-group level, fusion wins in eight of nine groups, but some gains are marginal (MTAT: +0.1 AUROC / +0.3 mAP; GTZAN: +1.0), with no standard deviations reported and only a single pretraining seed. MultiJEPA is trained on approximately 250,000 hours of internal audio, potentially more than the baselines use, yet the baseline training-data sizes, parameter counts, and checkpoint details are not reported. Neither the training data nor the model weights will be released. Without results summarized by task group, variance estimates, and sufficient information to assess baseline comparability, the magnitude and reliability of the advantage over prior encoders remain unclear.

Questions for authors
  1. The token grid is two-dimensional (time × 8 frequency positions), and resampling is applied only along the time axis. Thus, “temporal resolution” refers to the temporal rate of the latent token grid rather than the sampling rate of the input audio. How are the eight frequency tokens at each time step handled by the downstream probes: are they averaged into a single 1024-dimensional vector or concatenated into an 8192-dimensional vector? The probing protocol does not specify this, whereas the intrinsic-geometry and PSD analyses average over frequency. If concatenated, the features provide higher-dimensional probe inputs than those of baselines with one-dimensional token sequences (MERT, MuQ, and MusicFM), potentially benefiting frame-level tasks such as transcription and F0 estimation (And How to handle PuPuJEPA?). Please report the probe input dimensionality for each model and, if applicable, results using frequency-averaged features.

  2. The 25-Hz multiresolution query outperforms the fixed 25-Hz model on structure, GiantSteps, and F0, whereas the 3.1-Hz query underperforms its fixed-resolution counterpart (Tab. 2). What explains this asymmetry? Could the fine-resolution gains stem from exposing the shared encoder to longer-context crops, and could the coarse-resolution losses reflect fewer training updates at each rate? Please also report the rate-sampling probabilities and clarify whether they were tuned.

  3. Given that the training data and model weights will not be released, do the authors plan to release the training and evaluation code, model configurations, and probing scripts? Please also report the training-data sources and sizes, parameter counts, and exact checkpoints used for the external baselines. These details would help readers reproduce the evaluation protocol and assess baseline comparability.

  4. The fine-grained evaluation covers beat tracking, transcription, and F0 estimation but omits chord recognition, a standard MIREX task requiring temporally localized harmonic information. Key estimation, by contrast, is evaluated at the song or clip level. Adding chord recognition would help assess whether fine-resolution queries retain harmonic information and whether fusion benefits tasks requiring both context and temporal detail.

  5. During self-alignment, the fine-resolution student processes crops of the same duration as those used by the teacher at the selected resolution. Fine-resolution queries therefore receive longer contexts and more tokens (up to 16,384) than during pretraining (Tab. 5). Does the continued-pretraining control use the same crop lengths and token counts for its fine-resolution queries? If not, how much of the 25-Hz query’s gain in full-track structure performance—from approximately 33 to 43 relational mIoU (Fig. 5)—is attributable to longer-context exposure rather than the alignment loss itself? A control using the same long crops and token counts without the alignment loss would help distinguish these effects.

2 comments
AuthorsOpenReview ↗

Response to review

Thank you for the thorough review. We have updated the paper based on the below response to the comments and questions.

1. Novelty and benefits of multiresolution training

Thank you for the additional references. We will expand the related work, including the connections to DINO and ATST. We agree that self-alignment is related to cross-view distillation. Our objective specifically matches pooled fine-resolution features to native coarse features at corresponding positions in the same audio. FlexiViT is also relevant, although it focuses on distillation across patch sizes rather than this fine-to-coarse temporal alignment.

We agree that the external baseline comparisons do not isolate the effect of multiresolution training. Our controlled comparison is instead between fixed-resolution and multiresolution JEPAs trained with the same data, backbone, objective, and budget. It shows that one shared encoder largely preserves the complementary strengths of specialized resolutions. While fixed-resolution models can be better on individual metrics, MultiJEPA lets us combine these views without training four separate encoders.

We have now completed the requested fusion-of-specialists comparison. We fused the four separately trained fixed-resolution models using the same procedure as MultiJEPA. MultiJEPA's four-query fusion wins on 14 of 16 metrics (see updated paper). The two exceptions are zero-shot structure metrics. This directly addresses whether the gains are due only to averaging complementary models. We will add the full results to the paper.

Four specialist models require four separate pretraining runs, whereas MultiJEPA trains one shared encoder. We recognize that four-query fusion still requires four inference passes. We will report training and inference FLOPs to make this distinction clear.

A single-rate TRB control could isolate the frontend's contribution, but the TRB was chosen for flexible aggregation, not as a claimed source of performance gains. We do not attribute the improvements specifically to the TRB, and do not believe that this additional control is needed to support the central findings.

2. Performance claims

We will clarify the distinction between 10 dataset entries, 8 task types, and 16 metrics. The 15/16 claim is explicitly about metrics against the compared audio-only SSL baselines, not 15 independent tasks. We also avoid counting correlated metrics individually in Fig. 1; the complete set is shown separately in the appendix. We will make the wording more precise.

We recognize that some gains on GTZAN and MTAT are small, as is common on mature benchmarks. Other improvements, particularly in structure, transcription, and F0, are substantial. External baselines provide a reference for representation quality; the matched comparison within our own models is the main evidence for multiresolution training.

The paper reports averages over five probe seeds where applicable, but only one pretraining seed. Large-scale SSL papers commonly report single pretraining runs because of the cost, although we agree that this limits conclusions about training variance. We will add the available baseline checkpoint, parameter-count, and training-data-scale details.

Questions

  • Frequency tokens: The eight frequency tokens are concatenated, as in MATPAC and PuPuJEPA (8,192 d for a 1,024-dimensional encoder). We will specify probe input dimensions. All of our fixed- and multiresolution models use the same feature construction, so this does not affect their controlled comparison.

  • Rate asymmetry: Rates are sampled uniformly, without tuning. Each rate receives roughly four times fewer direct updates than a standalone model, which may explain some coarse-rate losses; fine queries may benefit from longer-context learning in the shared encoder. These remain possible explanations, not causal claims.

  • Reproducibility: We will report baseline checkpoints and their published data scales. Internal confidentiality constraints prevent us from releasing the training data, provenance, weights, or codebase.

  • Chord recognition: We agree it could add useful evidence. We already test local harmonic/pitch information through transcription and multi-F0, and global harmonic information through key estimation. We therefore believe the present suite covers the main tradeoff studied here.

  • Self-alignment and longer context: The reviewer is right that longer-context exposure may contribute to the gains. The alignment phase uses 81.92-second views, which we will clarify. We do not claim that the loss alone explains the improvement; the phase is best understood as long-context, cross-resolution distillation, similar in spirit to a short alignment stage such as GRAM in DINOv3. It improves 25-Hz full-track relational mIoU from 33.5 to 43.1 (versus 35.2 with continued pretraining), alongside improved cross-resolution agreement. We will adjust the claims accordingly.

Thank you to the authors for the thorough response and for the additional fusion-of-specialists experiment. The direction this paper pursues, treating temporal resolution as an explicit design choice in music representation learning, is likely to be a valuable contribution to the Music and Audio community.

However, I still do not see a clear novelty, either methodologically or experimentally. Most of the architectural components come from prior work. The benefit of multiresolution training itself remains only partially isolated, because of the frontend difference between the fixed- and multi-resolution models and the unmatched comparison with external baselines. I am therefore maintaining my current score. I hope the promised clarifications, the full fusion results, the compute comparison, the probe-dimensionality details and the expanded related work will be reflected in the final version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.