acceptodds
Under review as a conference paper at ICLR 2027

Demand Geometry and Sparse Interfaces for Continuous Music Tokens

Abstract

Continuous music tokenizers support reconstruction and downstream modeling, but different tasks may benefit from different low-dimensional views of the same tokens. For frozen tokenizers, we study tradeoffs among active input width, shared capacity, and demand-specific utility. A minimax certificate quantifies unavoidable compromise and rules out exact common optimality for the tested reconstruction and prediction operators. Using target-canonical normalization, we define demand relations invariant to invertible linear input and target transformations under fixed whitening supports and ridge strengths. For each tokenizer, Atom Atlas assigns each known demand of shared orthonormal atoms. Within a fixed union subspace, chosen dense-view projectors admit exact realization by shared atom subsets if and only if they commute pairwise. Support-aware certificates bound operator utility loss from approximate realization for fitted operators. Demand relations recur across three tokenizers on Moises and two on Slakh with a different teacher. Across six primary MUSDB/Moises settings at , Atlas retains 84–97% of the dense-over-shared reduction in worst-demand operator regret. With matched nonlinear predictors on Moises, Atlas improves future-token prediction and prediction of frequency-band teacher embeddings over the shared view. On the SAME-L band task, native Atlas retains 85.3% of the improvement from the shared view to native dense, supporting prediction directly from stored atom coordinates.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Continuous music tokens, Audio representation learning, Demand geometry, Sparse interfaces, Shared dictionaries, Subspace compatibility

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_73iWOpenReview ↗
Rating
2
Confidence
2(+1)
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Atom Atlas. Each demand selects KK coordinates from a shared MM-atom orthonormal bank, and analyzes the tradeoff between shared capacity, demand-specific utility, and sparse coordinate access.

Critical strengths

The experimental section is extensive.

Critical weaknesses

My main concern is clarity and presentation, particularly in the introduction. It is a bit difficult for me to follow up why this problem, what others do, and what the authors propose consequently. If the author intends to trade a compressed introduction for later contents, I think it is compressed too much. For example, in 'Locodec (Luo et al., 2026) represents speech by low-rate spherical vectors with progressively decodable coordinates. Our stereo-music adaptation tests whether one order serves demands ranging from local fidelity to source, frequency, and long-range structure.', I don't see a clear motivation for this paper's design. Substantial paper logic flow modification may be needed.

Also, the authors should polish the figure presentation.

Questions for authors

Can the authors ripple out the introduction around the core issues, limitations of existing work, and the consequent motivations for proposed solutions? For example, why a shared K-dimensional view is insufficient and why K-of-M sparse access is the appropriate solution, before introducing the geometric machinery?

7 comments
AuthorsOpenReview ↗

Response to 73iW (1/1): Motivation and figure presentation

Response to 73iW (1/1): Motivation and figure presentation

Thank you for identifying where the motivation and the transition to our design needed a clearer explanation. We have added a concrete shared-view example before the tokenizer-specific background and polished Figures 1–3 and their captions to clarify the interface choice and the interpretation of the results. The revised Introduction presents the interface question before the geometric machinery.

Motivation and explanatory order

The opening asks how a frozen tokenizer should expose latent directions to different downstream demands at a fixed consumer input width KK. A common prefix or shared rank-KK view supplies the same directions to every demand. When demands prefer different directions, this can create a utility compromise. The short introductory example makes this issue concrete before the Locodec background.

KK-of-MM access separates the two budgets involved in that compromise. One shared bank defines MM atom coordinates, while each demand reads a subset of KK. Different demands can select different subsets while each consumer still receives KK coordinates. This motivates representing demand-specific views by addresses into one reusable bank. Dense task-specific views in the same shared span provide the reference for quantifying the utility retained under this coordinate-subset constraint.

For illustration, consider two demands favoring orthogonal axes, with one active coordinate per consumer. A single shared rank-one view cannot attain both demand-specific optima. With normalized axis utilities, its best worst-demand regret is 1/21/2. A bank containing both axes lets each demand read its own coordinate, with active width one and shared capacity two. This explains why allowing different compact views can help while each consumer's input width remains fixed.

The revised Introduction places the Locodec context after the short example, so the tokenizer-specific background follows the interface motivation. The operator analysis then formalizes shared-view compromise and sparse realization, while the consumer experiments assess predictive utility for the stated tasks and baselines.

Figure presentation and result navigation

We refined the presentation within the original three-figure structure, retaining every plotted value:

  • Figure 1 labels the common directions supplied by a shared view and the KK-of-MM selection used by Atlas. It distinguishes active width KK from shared capacity MM, and identifies the matched demands whose relations are compared across tokenizers without axis alignment.
  • Figure 2 separates certified operator bounds, with midpoint markers, from cross-tokenizer relation correlations. The plot now identifies Pearson/Spearman correlations and the tokenizer abbreviations; the caption retains the fitted-operator scope and pairing-control pointer.
  • Figure 3 adds arrows for native Single-to-Dense headroom and the improvement retained by Atlas, making the 85.3% retention explicit. The arrows represent differences of mean errors. Native/oriented means and the paired-song bootstrap interval remain consistent with Appendix Table 21.

The added relation controls and width/MIDI comparisons appear in Figures 5 and 8, with direct pointers from Sections 5.2 and 5.4. Main Table 3 collects six predictive contrasts by task, budget, baseline, metric and interval; Appendix Table 25 provides the expanded summary. We also checked appendix figure placement and table typography for readability.

Reviewer_73iWOpenReview ↗

Response for Authors

Thank authors for the response. I acknowledge that the additional example and revised figures improve the presentation to some extent. However, my main concern regarding the clarity and accessibility of the paper, especially in introduction, remains insufficiently addressed.

The introduction is still too compressed and difficult to follow for readers unfamiliar with the topic. In particular, the illustrative example explains why different tasks may require different representations, but does not sufficiently motivate the specific need for the proposed sparse interface over conventional task-specific projections. I believe the paper requires a substantial restructuring of its introductory logic, rather than incremental wording changes, to clearly establish the problem, motivation, and significance of the proposed approach. Due to the substantial revision needed for illustrative clarity, I would lower my score to 1. Thank you.

AuthorsOpenReview ↗

Response to 73iW: From task-specific views to shared coordinate access

Thank you for re-reading our revised paper and clarifying the remaining concern. We agree that showing why demands favor different views does not by itself explain the additional motivation for coordinate-subset access, since conventional task-specific dense projections already provide different views. In response, we have reorganized the Introduction so that the problem, the shared-coordinate goal, and the role of the analysis form a continuous argument. The motivation now develops before the Locodec background and the contribution list. The specific changes are as follows:

  1. Problem and existing alternative. We retain the opening explanation that a common prefix gives every demand the same compact view, although source, frequency and temporal requirements may favor different directions. The second paragraph now explicitly acknowledges task-specific dense projections as a flexible way to provide different KK-dimensional views.

  2. Shared coordinates and the research question. The second paragraph distinguishes the shared capacity MM from the KK coordinates read by each consumer. It explains how selected index sets identify each demand's view and how their overlap records direction reuse. The question is then how much of the dense-view utility this coordinate-subset organization retains.

  3. Connection to Atlas and the analysis. In the third paragraph, Atlas's shared orthonormal dictionary follows from this goal. Dense views in the same MM-dimensional span permit task-specific linear combinations, making the coordinate restriction and the role of the realization analysis explicit. The analysis characterizes when the chosen dense-view subspaces admit exact realization and bounds the operator utility lost through approximation. Demand relations connect this account to comparing the structure of view preferences across tokenizers without aligning axes. We retain the original contribution sequence, with matched nonlinear consumers providing the separate evaluation of predictive utility.

Dense and sparse interfaces can both reuse computed coordinates. The comparison includes cached and precomposed dense implementations, as well as learned projections and full-width inputs (Section 5.4 and Appendices R–T). Precomposed Dense and Atlas have similar observed deployment costs in the measured workloads. We retain that empirical scope while making the central motivation clearer: organizing demand-specific utility through a shared coordinate interface and quantifying its trade-off relative to flexible dense views. Thank you for prompting us to explain this connection more carefully.

Reviewer_NPSKOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

For a frozen continuous music tokenizer, giving every downstream demand the same low-dimensional view forces a utility compromise. The authors bound this compromise via a minimax certificate, define a target-canonical normalization to compare demand relations across Locodec, SAME-L, and Music2Latent, and propose Atom Atlas. Atom Atlas rotates a shared union subspace via approximate joint diagonalization into shared orthonormal atoms so each demand can read a smaller subset of atoms.

Critical strengths
  1. Separating active consumer input width from shared subspace capacity is a clean way to study frozen tokenizers. It avoids assuming that one coordinate order works best for reconstruction, source separation, and prediction at the same time.

  2. The cross-tokenizer result in Section 4 is a strong finding. After target-canonical normalization removes linear coordinate differences, Locodec, SAME-L, and Music2Latent show very similar demand-relation matrices across music tasks.

  3. The support-aware bound in Proposition 3 is tight. On the in-domain prediction tasks in Section 5.2, Atom Atlas keeps most of the improvement of dense task-specific views over a single shared view while predicting directly from stored atom coordinates.

Critical weaknesses
  1. The practical need and end-to-end gain of the restricted interface are not yet clear. Bitrate, token frame rate, and channel width have different costs, and the paper does not tie its chosen input widths to a concrete music deployment budget. Existing models like Music2Latent and MERT simply expose full-width features. In Table 10, Atom Atlas saves a few hundred kilobytes of interface storage across 25 demands relative to factorized dense projections, which leaves out the tokenizer and consumer models. Also, as Appendix F.3 notes, precomposed dense projections can match Atlas's single-demand compute, and coordinate caching changes the multi-demand tradeoff. These interface savings are valid, but their impact on end-to-end memory and latency is not measured.

  2. Evidence for adapting task-specific atom subsets to new downstream tasks is mixed. The results on construction-related tasks are positive, but the excluded-task studies in Section 5.4 and Appendices O and P do not show a clear edge over an equally optimized shared subset. On Jamendo, optimized shared subsets get higher mean AUROC, and the comparisons reuse development folds. On GTZAN and NSynth, the assignment differences are inconclusive, and on VocalSet the adapted subsets do not beat construction-only subsets.

  3. Section 5 is hard to follow because it mixes different metrics, interface budgets, and evaluation setups in dense prose. Even with Table 1, readers have to flip to the appendices constantly to check the exact scope of each number. Also, on page 10, the paper states that the 1,080.6-hour private corpus was used under internal company data governance, without clarifying the underlying rights or licensing basis, or how non-commercial terms on public resources (such as MoisesDB, Music2Latent, and MERT) were handled.

Questions for authors
  1. What is a realistic music workload where reading a small subset of shared atoms is needed? On such a workload, how does Atlas compare to full-width inputs and precomposed dense projections in accuracy, end-to-end memory, and latency when shared-coordinate caching is considered?

  2. Given the mixed results on excluded tasks in Section 5.4 and Appendices O and P, what do the current experiments actually show about the benefit of task-specific atom subsets over an equally optimized shared subset? Also, how does the accuracy-cost tradeoff compare to a simple learned dense projection?

  3. Can you reorganize Sections 5.2 to 5.4 around a compact summary table that lists the task, interface budget, baseline, metric, and confidence interval? Also, please clarify the rights and licensing basis behind the private corpus and the compliance with non-commercial terms for the public datasets and checkpoints used.

4 comments
AuthorsOpenReview ↗

Response to NPSK (1/3): Workload, width and measured costs

Response to NPSK (1/3): Workload, width and measured costs

Thank you for your detailed assessment of the practical workload, the shared-interface baselines, and the presentation of the evidence. Your distinction between interface storage and complete inference costs helped us specify a more informative set of comparisons. Below we explain the resulting changes to the deployment study, task-adaptation evidence, presentation, and data-use statements.

Deployment workload and efficiency

We agree that coordinate counts alone cannot describe a deployment tradeoff. We have therefore added full-width and learned-dense accuracy controls and measured inference from audio through the loaded codec and consumers.

Our measured example is offline piano/guitar/bass activity analysis from one frozen encoder, with one or three prediction heads active and all three resident. In the Atlas path, each head receives K=16K=16 selected coordinates from a shared M=104M=104 atom bank. We compare this path with Full256, precomposed Dense and cached alternatives. Active consumer width is the controlled interface budget; tokenizer clock and precision stay fixed.

The Moises band-conditioned TCN complements this multi-head example by studying one/four active bands across several widths. These widths define controlled accuracy/cost comparisons; application requirements would determine the operating point in a deployment.

In the same environment, we refit Single/Dense/Atlas and add Full256 and native learned shared/specific projections, retaining all K=8/16/26 settings and three seeds at 4,000 updates. Dense/Atlas share the frozen M=104M=104 union; learned maps use native 256-D coordinates. The compact views match active width; union capacity, projection parameter counts and total compute differ. Mean cosine errors are:

MethodK=8K=8K=16K=16K=26K=26
Single0.3306330.3227290.314865
Within-union Dense0.3096850.3018620.301269
Atlas0.3149770.3060410.303278
Learned shared dense0.3123500.3039150.299822
Learned specific dense0.3084440.3011140.296931

The same Full256 reference, 0.296060, is reused across widths. Atlas has lower mean cosine error than Single at every width. Learned shared projections have lower point means, with the K=8K=8 interval spanning both directions and the K=16/26 intervals favoring the learned projection. Learned specific projections provide the lower-error reference at all three widths. Appendix R gives the complete pointwise intervals. Comparisons among compact views match active width and the final consumer update budget, with projection capacity specified separately.

Appendix S summarizes all tested modes, including precomposed Dense and shared-coordinate caching; its accompanying evidence retains all 50 cached and 150 audio rows. For cached-window inference at K=26K=26, precomposed Dense takes 0.8670/0.8634 ms for one/four active bands versus cached Atlas at 0.8846/0.8806 ms. Audio-call medians are 304.537/304.307 ms versus 304.297/304.594 ms, with about 3,595 MiB peak PyTorch allocated memory including the full loaded codec. Calls encode 70 seconds but predict only one 96-token window; models and files are warm.

The native learned shared/specific controls are measured in this same audio path. At K=26K=26, their one/four-band medians are 304.185/304.328 ms and 304.521/305.120 ms, respectively. These measurements place the learned-dense accuracy results alongside audio-to-window latency and peak PyTorch allocated memory, including the resident codec and consumers. Atlas and precomposed Dense have similar observed costs under this workload.

Interface-state storage is reported separately from peak allocated memory. At K=8K=8, four registered views use 32,768 Dense versus 106,752 cached-Atlas bytes. The 25-demand factorized-storage ledger describes a different demand inventory and storage configuration. The revised comparisons therefore identify both the interface state and the measured resident-model memory.

For the MIDI workload, audio calls read and encode a 120-second crop and score every frame after burn-in, providing the full-crop counterpart to the band-window measurement.

AuthorsOpenReview ↗

Response to NPSK (2/3): Dedicated timing and frozen-bank adaptation

Response to NPSK (2/3): Dedicated timing and frozen-bank adaptation

A dedicated-device timing repeat covers all 13 implementations in five randomized blocks (520 cells; 14,950 timings), reusing frozen seed-1234 models and three training songs. Direct Dense/direct Atlas/cached Atlas 120-second audio medians are 0.595609/0.597613/0.596669 seconds for one instrument and 0.703957/0.703293/0.703212 for three, aggregated over block–song medians. Full256 medians are 0.596725/0.701751 seconds for one/three instruments. Peak allocation remains about 3,635 MiB; Dense/Atlas interface state is 49,152/106,880 bytes. Independent process-log checks identify only the benchmark process in all 520 sampled case intervals. Atlas, precomposed Dense and Full256 fall in a similar observed deployment-cost range under these conditions. Appendix T.3 reports every implementation, paired block–song ratios and the warm, resident-model measurement scope. These comparisons characterize the measured accuracy and cost of the K=16K=16 sparse interface alongside compact dense and full-width alternatives.

Scope of task adaptation

The evidence distinguishes shared-address reselection from the additional value of task matching. On Jamendo, the clearest supported change is reselection of an address shared by the two tasks. The Slakh MIDI follow-up evaluates task-matched addresses against a common atom address and shared native-space controls, so each conclusion is attached to its specific comparator.

Slakh MIDI evaluates supervised adaptation of construction-excluded target definitions in the frozen Moises bank. Training labels select addresses; source mean, whitening, union and dictionary remain fixed. We retain the 320/80/80 split and 120-second crops for 128-pitch piano/guitar/bass activity. Construction exclusion refers to the MIDI target definitions: Slakh audio had prior uses, and bass also appears in the source-construction interventions.

Ten interfaces include common/swapped/random atoms, Single variants, union Dense, native learned maps and Full256. Independent width-192 TCNs receive 4,000 updates/head and three original paired seeds. All 84 fits/90 heads are sealed before test preparation; Full passes the validation-skill gate. A reference based only on training-label priors yields test AP 0.124657; comparisons between interfaces are evaluated separately.

Original three-seed comparisonMean AP difference [95% interval]
Atlas minus common atoms+0.003152 [0.001651, 0.004639]
Atlas minus swapped supports+0.003599 [0.001691, 0.005544]
Atlas minus adapted native Single+0.001139 [−0.002783, 0.004955]
Learned specific minus learned shared dense+0.022714 [0.018454, 0.027086]

Within the frozen bank, each matched address selects the 16 highest diagonal utilities for its target, while the common address minimizes the three-target worst-demand operator regret by MILP. Both use the same bank and active width and feed independently trained task heads. Neural AP is then evaluated separately from the operator criterion. The common/swapped comparisons support an AP benefit from matching addresses in this primary fixed-bank study.

Adapted native Single assesses another shared-interface family, with its interval spanning both directions. Original three-seed mean AP is 0.487884 for Atlas, 0.492828 for within-union Dense and 0.511135 for Full256, placing the within-bank result alongside broader accuracy references. Under this fixed-update MIDI protocol, Atlas also has higher mean AP than both native learned-dense baselines (Appendix Table 38). The band task favors the learned-specific projection, so the projection-family comparison is reported separately for each task.

The native specific and shared dense baselines use three and one maps, respectively, so this contrast also changes projection capacity. The clearest address gains are in bass; piano/guitar and Atlas-minus-common F1 intervals permit effects in either direction.

AuthorsOpenReview ↗

Response to NPSK (3/3): Robustness, presentation and resource terms

Response to NPSK (3/3): Robustness, presentation and resource terms

To make the aggregation and uncertainty easier to inspect, we now state the metric rules explicitly. AP is computed over all scored frame–pitch cells within each song, then averaged equally over songs, individual seed scores and instruments; predictions are not ensembled. F1 uses the fixed zero threshold on class-weight-adjusted logits, without fitting calibration parameters. The 10,000 paired-song intervals condition on fitted models and omit training/search uncertainty. A retrospective 36-comparison analysis with approximate simultaneous AP intervals also retains the highlighted overall effects. Appendix T.1 compares the overall simultaneous AP intervals from the three- and five-seed analyses side by side; the full instrument families and seed/omission checks accompany the sources.

We also checked sensitivity to the observed training seeds. Two seeds added after original test inspection produce 56 new fits/60 heads, for 140 systems/150 heads in total. This extends training-seed coverage on the same inspected split. Five-seed Atlas-minus-common/swapped AP is +0.003404 [0.001969, 0.004793]/+0.004035 [0.002219, 0.005860]; specific-minus-shared is +0.023143 [0.018716, 0.027659]. Their overall intervals remain positive in the separate simultaneous family, and AP point differences retain their direction in every observed seed and each omission. Atlas-minus-adapted-Single is +0.001357 [−0.002369, 0.004947], with an interval spanning both directions. All 12,000 scalar rows accompany the sources; local raw-logit audits verify these outcomes.

We froze 140 systems before parsing extra-cohort MIDI: 13 official-test songs (primary) and 80 official-validation songs (secondary), excluding indexed project selections. Models and addresses remain fixed. Atlas-minus-common AP is +0.002692 [−0.001403, 0.007304] / +0.003572 [0.002059, 0.005173]; Atlas-minus-swapped is +0.006792 [0.002580, 0.011165] / +0.007392 [0.005041, 0.009761], respectively. The primary common-atom interval and both adapted-Single intervals span both directions; Full256 provides the higher-AP reference in both cohorts. Appendix T.2 reports the separate cohorts and six overall AP contrasts; complete scalar outcomes accompany the sources. These extend the same MIDI target family to additional songs identified within the bounded project-history audit.

Appendix M retains the earlier excluded-task comparisons, with Jamendo selection in Appendix N. The earlier Jamendo task-matching, GTZAN and NSynth comparisons have intervals allowing effects in either direction; VocalSet's selected address improves over the swapped control, while construction-only supports provide the higher technique-accuracy reference. Address-adaptation outcomes thus depend on the task and comparator.

Presentation and licensing

We appreciate your request for a compact summary that readers can follow without repeatedly reconstructing budgets and metrics from the appendices. We have added main Table 3 within the original results flow, together with direct pointers to the expanded appendix summary and follow-up protocols.

Table 2 retains its original task/interface order and includes artist-bootstrap uncertainty. New main Table 3 identifies task, budget, baseline, metric and interval for six predictive contrasts, retaining the learned-specific baseline, both Slakh/MERT intervals and the adapted-Single control. Appendix O (Table 25) gives the expanded summary, while Appendices R and T provide the detailed width and MIDI comparisons. Sections 5.2–5.4 retain their original argument, add the functional and denominator results, and point to seed/cohort sensitivity and measured workloads in the corresponding appendices. These additions make the evidence visible while preserving the original method and main-figure structure.

We have supplemented the Ethics Statement with the requested provenance and use information. Our organization purchased the private corpus from a third-party data vendor, with permission for research model training. This study is solely research, without use in products or commercial services, and private audio is not redistributed. We identify MoisesDB v0.1's CC BY-NC-SA 4.0 terms, Music2Latent and MERT-v1-95M's CC BY-NC 4.0 terms, and SAME-L's Stability AI Community License. Appendix U records the resource-specific sources. The Reproducibility Statement retains the original conditional release commitment under upstream licenses and rights review, with an added description of the supplementary evidence.

Thank you for the substantial additional experiments and clarifications. These revisions strengthen my confidence in the paper’s contribution and clarify its scope, but they do not fundamentally alter the practical performance-to-cost trade-off. I maintain my weak-accept recommendation.

Reviewer_TA2oOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how a frozen continuous music tokenizer should be exposed to several downstream demands at a fixed consumer input width. Demands are encoded as PSD utility operators on whitened token coordinates, so the loss of a shared rank-K view is a normalized regret. A minimax bracket quantifies the compromise that no shared view can avoid, and the paper then asks whether a larger shared union can give each demand its own K-of-M coordinates. Exact realization of the demand-specific dense views by subsets of one orthonormal bank holds if and only if the corresponding projectors commute, and support-aware bounds cover the fitted case. The empirical work spans three tokenizers, MUSDB, Moises and Slakh: regret brackets, cross-tokenizer relation correlations, retention of the dense-over-single gain, and matched nonlinear consumers.

Critical strengths

1. The framing works. Asking what a fixed-width interface must lose, rather than which tokenizer is better, turns a vague design question into a bracketed quantity, and the bracket for Locodec is tight enough to be a certificate rather than an illustration.

2. Proposition 1 does what it claims. Removing the dependence on axis alignment is exactly what blocks most cross-tokenizer comparison, and it is a cleaner instrument than the usual CCA or CKA route.

3. The robustness work is real: eighteen construction refits, three independently trained Locodec models, AJD against direct refinement, and consumer intervals that include zero on Slakh are all reported rather than dropped.

Critical weaknesses

1. The headline numbers are ratios of small differences. Retaining 84 to 97% of the dense-over-single reduction divides by a quantity that is itself small and that the paper conditions on being positive, so closure is least stable precisely where the two views are closest. The absolute figures sit in appendix tables; in the main text the practical gain reads as a cosine error of .31486 falling to .30327. Only worst-demand regret is reported, so I cannot see whether Atlas helps some demands and hurts others.

2. Table 2 carries no uncertainty, and its own pattern argues against the intended reading. The other task's subset scores best on Mood/theme, above both the task-specific and the optimized shared address, which is hard to square with addresses being matched to tasks. With two folds and two seeds, an increment of about .015 AUROC is not separable from noise as presented.

3. The cross-tokenizer agreement may be close to a tautology. Proposition 1 makes the relations invariant to invertible linear changes of input and target, so two tokenizers related by an invertible linear map have identical relation matrices by construction, and three continuous autoencoders trained to reconstruct the same audio are plausibly in that neighbourhood. Nothing in the design is a null that should fail, so correlations near .98 cannot separate "demand geometry is a property of the signal" from "these tokenizers are linear views of the same signal".

Questions for authors

1. Can you add bootstrap intervals to Table 2, and explain why the other task's subset beats the task-specific one on Mood/theme?

2. What does the relation correlation look like for a tokenizer that should not agree, for example speech Locodec, a discrete codec, or a deliberately nonlinearly distorted token space? That is the control that would tell me how much of the .98 is structural.

3. On Slakh/MERT the Atlas–Single prediction intervals include zero. Does the retention claim survive there, or should it be read as holding only on Moises?

3 comments
AuthorsOpenReview ↗

Response to TA2o (1/3): Absolute gains and Table 2 uncertainty

Response to TA2o (1/3): Absolute gains and Table 2 uncertainty

Thank you for the careful review and for identifying where absolute improvements, uncertainty, and the interpretation of relation agreement needed further clarification. Your comments led us to add quantitative controls and to distinguish more clearly between fitted-operator guarantees and consumer prediction results. We address these points individually below.

Absolute gains and variation across demands

We agree that retention is easier to assess when the absolute effect and denominator variability are visible. The headline 84–97% concerns closure of worst-demand normalized operator regret; nonlinear prediction retention is a separate readout. Across all six primary operator cells, Single-to-Dense headroom ranges from 0.0797 to 0.2442 and the absolute Single-minus-Atlas reduction from 0.0666 to 0.2352 (Appendix Table 26). The percentage is therefore interpreted alongside the scale of the available improvement.

For nonlinear prediction, the main discussion now states absolute errors, Dense headroom, retention uncertainty and denominator stability alongside the original result. Appendix O gives the complete joint gain, headroom and retention intervals. For the originally submitted native SAME-L band experiment, Single/Dense/Atlas cosine errors are 0.314862/0.301271/0.303275. Atlas reduces error by 0.011587 [0.009784, 0.013371], against Dense headroom of 0.013590; retention is 85.3% [81.0%, 89.6%]. The oriented condition gives an absolute improvement of 0.010518 [0.008765, 0.012267] and retention of 84.5% [78.7%, 90.2%]. The joint 10,000-draw song bootstrap retains every denominator; all are positive. These paired-song bootstrap intervals quantify uncertainty across songs, conditional on fitted models. The Moises rows of main Table 3 and the expanded appendix summary (Table 25) use the separately identified same-environment refit, explaining its different interval despite nearly identical rounded means.

To address your question about demands beyond the worst case, Appendix O now shows all 150 demand-level changes across the six primary cells. Every demand has lower Atlas regret than Single. These are descriptive fitted-operator comparisons: readouts reuse training operators; other families use evaluation operators. We do not treat these 150 comparisons as independent samples or equate operator improvements with nonlinear prediction gains. Construction-data and tokenizer-seed sensitivity are examined separately in Appendix Tables 17–18.

Table 2 uncertainty and task specificity

Following your request for uncertainty in Table 2, we added 10,000 paired artist-bootstrap intervals for every mean, retaining the original table number and task/interface order. Resampling preserves the two outer folds and averages four individual fold–seed AUROCs, without prediction ensembling. Key differences are:

ContrastGenre AUROC difference [95% interval]Mood/theme AUROC difference [95% interval]
Shared minus construction+0.01519 [0.00327, 0.02526]+0.01155 [0.00119, 0.02720]
Own minus swapped+0.00578 [−0.00450, 0.01156]−0.00440 [−0.01373, 0.00478]
Own minus shared−0.00200 [−0.00621, 0.00231]−0.00155 [−0.00926, 0.00630]

Here, “own” identifies the address selected by that task's mean inner-fold AUROC; “swapped” uses the address selected for the other task. These labels identify the inner-fold selection objective. The reported scores come from fresh consumers evaluated on outer-fold artists, where the selected address can have a different ranking.

On the reused Jamendo development folds, shared reselection improves over construction supports. Own-versus-shared and own-versus-swapped intervals allow effects in either direction for both tasks, placing the Mood point ordering within the reported uncertainty. The pointwise artist-bootstrap intervals condition on the fitted consumers and selected addresses; training and adaptive-search uncertainty remain outside their scope. We have revised the discussion to distinguish shared reselection from task-specific address assignment.

AuthorsOpenReview ↗

Response to TA2o (2/3): Relation controls and predictive relevance

Response to TA2o (2/3): Relation controls and predictive relevance

Controls for relation agreement

Your question distinguishes coordinate invariance from the empirical meaning of cross-tokenizer agreement. We therefore assessed relation correlations together with predictive performance under feature distortions and disrupted pairing.

We added feature controls on the same 188 training/25 validation songs and 32 reserved demands. Prediction experiments are offline: causal readouts restrict token indices; encoder context can include future audio. Among the controls suggested in your question, we tested deliberately nonlinear token distortions. Affine transformations preserve relations as predicted by Proposition 1; invertible cubic transformations retain weighted r=0.9801–0.9950 and projector r=0.9356–0.9698; this distortion largely preserves the observed agreement. Independent Gaussian features and whole-song mismatch give the following comparison:

FeaturesMean validation canonical NMSE rangeCross-tokenizer weighted-correlation range
Correct pairing0.9898–0.99090.9805–0.9953
Song-mismatched real tokens1.0025–1.00680.9795–0.9967
Independent Gaussian1.0002–1.00150.9938–0.9985

These are observed ranges across tokenizer/seed conditions, not confidence intervals. NMSE one is the training-target-mean reference. Correct pairing has lower mean validation NMSE than both controls, while relation agreement remains high across all three conditions. All nine Gaussian weighted comparisons yield one-sided structured-permutation p=1/1,152; this tests demand-label correspondence, not music specificity. Appendix P summarizes every control condition and its unnormalized amplitudes; the accompanying numeric evidence retains all seeds and per-demand errors. A fixed-alignment cross-tokenizer linear mapping has validation R2R^2 of 0.029–0.292; this characterizes the tested alignment and linear map family.

Figures 6–7 in Appendix Q show that nominal 25% real-feature energy already gives relation correlations of 0.9900–0.9992 with full-real fits; Gaussian relations also reproduce across disjoint training halves. Figure 5c shows how a target/sampling-only Gaussian approximation predicts the null relations with r=0.9948–0.9997. It normalizes expected Gram entries under a scalar covariance approximation, rather than exactly integrating finite-sample whitening/ridge or producing a denoised PSD operator. Together, these controls place the observed recurrence in the context of the shared target/sampling background. We therefore assess the real-input relation scores through the view-ranking diagnostic below. The invariance and sparse-realization results retain their fitted-operator scope. Section 5.2 now interprets the original correlations alongside this control and utility evidence.

We additionally test whether a relation score helps anticipate which borrowed interface gives lower validation error. Within each tokenizer, we fit every target readout through each of 32 original rank-4 views, using training moments only. Within each target, relations rank the other 31 views against negative validation NMSE. The equal-tokenizer mean weighted-minus-target-only Spearman difference is +0.088092 [0.014452, 0.115394], from 10,000 paired draws of the 25 validation songs. All tokenizer point differences are positive; SAME-L's individual interval spans both directions. The main relation discussion now includes the mean Spearman associations and their difference. Appendix Q reports the primary and secondary contrasts; complete score and bootstrap records accompany the sources. This supplies a functional complement to the recurrence statistic, with its scope set by the retrospective linear readouts, previously used validation songs, and historical ridge selection.

AuthorsOpenReview ↗

Response to TA2o (3/3): Slakh/MERT scope and MIDI activity evidence

Response to TA2o (3/3): Slakh/MERT scope and MIDI activity evidence

Slakh/MERT scope

The prediction-retention ratio is used for comparisons with positive Single-to-Dense headroom. For the original Slakh/MERT task, mean Dense headroom is nonpositive for both tokenizers, so main Table 3 and Appendix Table 25 report absolute prediction-error differences and intervals. Atlas-minus-Single normalized-MSE differences are +0.000069 [−0.000119, 0.000298] for Locodec and −0.000110 [−0.000233, 0.000018] for SAME-L. Both intervals remain compatible with effects in either direction for this MERT task. The prediction-retention results are reported for the Moises tasks and the separately evaluated FMA transfer (Appendix J), alongside absolute effects for Slakh/MERT.

New MIDI activity evidence: absolute and instrument-level effects

The new Slakh MIDI activity study evaluates a separate target family alongside the original MERT regression. Appendix Table 25 reports Atlas-minus-common/swapped mean AP of +0.003152 [0.001651, 0.004639] and +0.003599 [0.001691, 0.005544], about 0.315/0.360 percentage points. The primary mean covers all three prespecified instruments; its descriptive decomposition shows gains concentrated in bass, with piano/guitar intervals allowing effects in either direction. The overall Atlas-minus-common F1 interval likewise spans both directions. Against adapted native Single, Atlas AP is +0.001139 [−0.002783, 0.004955]; benefits against common atoms therefore depend on the shared-interface comparator. Appendix T gives the protocol, all ten interfaces and the overall/instrument contrasts; complete instrument, seed and AP/F1 records accompany the sources. MIDI and MERT remain separate target-family comparisons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.