acceptodds
Under review as a conference paper at ICLR 2027

DISENTANGLING AUTHORED SPACE IN 7.1.4 MUSIC: A LARGE-SCALE CORPUS AND A NEURAL SPATIAL FILTERING CODEC

Abstract

Spatial intelligence has so far dealt with space that has a physical referent; the space people hear in music is authored, since every channel assignment in a 7.1.4 master is a mixing decision. Learning it has lacked a corpus, a representation that holds the space apart from the content, and a ground truth to check one against; to our knowledge, the largest channel-based 7.1.4 music dataset available for research has ten tracks. We produce 125,557 tracks and 8,392 hours of 7.1.4 music from Creative Commons–licensed stereo with an upmixing model trained on human masters, which is closer to them than a commercial upmixer on all five objective metrics and is rated by listeners between the two. Since a mix shapes its content through the relations between channels, the space can be carried as the filtering that turns a mono reference into twelve channels: our neural spatial filtering codec (NSFC) leaves the reference to a pretrained mono codec and transmits the space at a nominal 6 kbps (13.75 kbps in total), optionally with a flow-matching postfilter. Without ground truth, we test the separation by its behaviour, in downstream tasks built from the corpus that fix the reference and change only the spatial codes, so that they apply to any representation learned on it that carries the space apart from the content. Within the domain of the corpus, the stream carries spatial structure beyond static per-channel gains, transfers across musical content and, when substituted, moves the mix in the intended direction. Authored space thus becomes something that can be learned, separated from content and tested.

Primary area:
datasets and benchmarks
Keywords:
spatial music, multichannel audio coding

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ro4yOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Audio mixes in 7.1.4 format are shaped by how professionals decide to distribute sound around and above the listeners. Therefore they often reflect creative mixing decisions rather than necessarily reproducing a physical acoustic scene. This paper investigates whether this spatial distribution of the format can be represented separately from its musical content.

The authors introduce a corpus of 125,557 tracks in 7.1.4 format, generated from Creative Commons stereo recordings using an upmixing model trained on a private collection of human-produced immersive mixes. Then they propose a Neural Spatial Filtering Codec that represents each twelve-channel mix through a mono reference and a compact spatial code description with codes that can be swapped to steer the mix.

Critical strengths
  • The separation between mono reference and spatial side information (in the evaluation protocol, Section 5) provides a concrete mechanism to test whether a codec has learned a spatially meaningful representation and its therefore a useful experimental design that enables controlled tests on the replacement of spatial codes.
  • Broad evaluation on the proposed architectures: waveform and spectral metrics, spatial metrics, embedding-based distributional metrics, and listening tests on the calibrated 7.1.4 loudspeaker system. The subjective evaluation is particularly relevant because objective metrics often don't correlate with human hearing.
  • The large generated 7.1.4 corpus could enable research at a much larger scale.
Critical weaknesses

If I understand correctly, the paper claims that the space can be separated from the content, in a 7.1.4 mix, by representing the space as a set of filters applied to a mono signal. The 7.1.4 music that is needed to test this claim is generated with an upmixer, which does that by applying filters to a mono signal, so the training and evaluation data have approximately the same structure that the codec assumes, by construction. The results show that the codec can invert the transformation done by the upmixer, but not that it would generalize to human-made 7.1.4 mixes. I seem to understand that the authors have access to 200 test tracks mixed in 7.1.4 by humans, but never run the codec on them.

If this is the case, this looks like a significant evaluation that the authors would need to run to test whether the claim actually generalizes to real music.

In fact, the codec seems to not generalize strongly on the only human mixes that the codec is tested on, AlbumDB. The authors provide an explanation that is absolutely plausible, but not directly verifiable without a comparison with other human mixes.

Questions for authors

The weakness described above is my only real concern with the soundness of an otherwise well constructed paper. I don’t think that it is an absolutely necessary condition for acceptance that the codec generalizes to all human samples, but if it doesn’t, it should be disclosed and discussed more openly. Could the authors evaluate the codec and the Section 5 tests on the 200 internal human test masters?

Reviewer_4vbLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that the spatial layout of a 7.1.4 music mix is "authored" rather than physical, and tries to make it learnable. It has three parts. (1) A corpus: an upmixing model, trained on 2,000 internal hand-mixed 7.1.4 masters, converts Creative Commons stereo from FMA, MTG-Jamendo, ccMixter and Dogmazic into 125,557 tracks (~8,400 h) of 7.1.4 audio, with licence screening, deduplication, quality filtering and a fixed split. On internal test tracks the upmixer is closer to the human masters than Halo Upmix on five objective metrics, and in a listening test with 24 listeners it is rated between Halo Upmix and the human masters. (2) NSFC, a codec that sends the content as a mono reference through a pretrained DAC, and the spatial information as a separate 6 kbps RVQ stream. The decoder predicts complex ratio filters plus a bounded mask that spread the reference over the twelve channels. An optional flow-matching postfilter (NSFC-Flow) trades waveform fidelity for perceptual quality. Results are reported against the original mix on 363 test tracks, with a mask ablation and per-channel Opus as a high-rate reference. (3) Since there is no ground truth for authored space, disentanglement is tested by behaviour on validation tracks: whether the spatial codes encode controlled channel-emphasis conditions, transfer across tracks, and steer the output when swapped (e.g., mirrored mixes). The authors report positive results within the corpus domain, and limited control over the height channels and on the ten AlbumDB human masters.

Critical strengths
  1. A large, carefully documented multichannel music resource. Public 7.1.4 music data is essentially limited to AlbumDB (10 tracks). The corpus provides 125k tracks with per-track licence and attribution tracking, fingerprint-based deduplication, documented exclusion rules, and a leakage-aware split. Even though the audio is generated, a resource built this carefully could be useful for upmixing and multichannel coding research, and the curation pipeline itself is reusable. This is the main reason I see value in the submission.

  2. Honest reporting of limitations. The paper reports where things do not work: the height channels never reach the target emphasis, steering fails on the AlbumDB masters, and the codes still carry content information. It also gives pre-specified test families, cluster bootstrap intervals, and per-channel breakdowns. This makes the claims easier to verify, although some of these results also weaken the main claim (see weaknesses).

Critical weaknesses
  1. The corpus and the disentanglement claim are circular, and do not hold on real human mixes. Every channel in the corpus is produced by one upmixer as a time–frequency mask applied to the stereo mid signal (Eq. 1). NSFC then models each channel as a filter applied to a mono reference. So the "space" the codec learns to separate is, by construction, the filtering that generated the data, and the behavioural tests mostly confirm the generator's structure rather than anything about authored spatial organisation. The only real human 7.1.4 data in the paper, AlbumDB, points the same way. There, mirroring does not steer the output (cosine 0.11 vs. 0.86 in-domain), and oracle per-channel gains reconstruct better than NSFC (0.364 vs. 0.570). The corpus statistics are also far from typical 7.1.4 practice (0.26% centre energy, 1.9% height energy). Meanwhile, the evidence that the upmixer captures human mixing practice is weak: against Halo Upmix, the listening-test differences are +0.05 (p = 0.42) for quality and +0.13 (p = 0.15) for spatial, neither significant, and the analysis was defined after the test. Since the title and abstract are about learning authored space, this is the main reason for my recommendation.

  2. Both the codec and the disentanglement evaluation lack meaningful comparisons. NSFC is compared only with its own ablation and per-channel Opus at 14× the bitrate. There is no comparison with parametric multichannel coding (MPEG Surround / MPEG-H style downmix + spatial parameters, which the paper itself cites as the closest idea) or with learned spatial codecs such as SpatialCodec, so it is unclear what the learned stream adds. The disentanglement tests use 32 validation tracks, which were also used for checkpoint selection, a classifier fitted on 16 training tracks, and controlled conditions that are simple static channel gains. They include no baseline representation either. The stream also retrieves the same track in 54.7% of cases with the spatial condition held fixed, which shows it carries content as well. With these gaps, the codec contribution reads as an incremental variant of existing spatial codecs, and the disentanglement claim is not established.

Questions for authors
  1. Since the upmixer builds every channel as a mask on the mid signal, and NSFC models channels as filters on a mono reference, how do you rule out that the tests just recover the upmixer's own structure? Have you run the same tests on 7.1.4 data that was not generated this way?

  2. In the listening test the upmixer is not significantly better than Halo Upmix (p = 0.42 and 0.15). Would you soften "rated between the two" in the abstract accordingly?

  3. The corpus has 0.26% centre energy and 1.9% height energy, which is quite different from typical 7.1.4 music. How does the internal training set compare on these statistics?

  4. Why are there no comparisons with a parametric multichannel coder or a learned spatial codec (e.g. SpatialCodec) at a similar rate?

  5. Will the upmixing model be released? Without it or the internal masters, the corpus can't be reproduced or extended.

  6. Running the disentanglement tests with a simple baseline (e.g. hand-crafted inter-channel level/phase cues) would show what the learned stream adds.

Reviewer_2ojsOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper points out that "authored" 7.1.4 music has remained outside of spatial audio research and representation learning. To address this, the paper proposes 1. stereo-to-7.1.4 upmixer to solve data scalability problem and constructs paired corpus of 125k tracks with trained upmixer, 2. neural spatial filtering codec (NSFC) that transmits a mono-downmix reference through a pretrained DAC and the spatial structure as another RVQ stream.

Critical strengths
  • The paper is well-motivated. Framing authored space which has no physical reference as an object of representation learning is novel and important. With only 10 tracks of 7.1.4 music publicly available, a large corpus with licence verification, deduplication and a frozen split enables further future research.

  • Given that authored space has no ground truth to compair against, the paper's attempt to evaluate the representation behaviourally is reasonable and welcome effort. The subjective evaluation is also carefully conducted, and shows proposed system works reasonably well.

Critical weaknesses

The critical problem of the paper is that the Upmixer that produced the paired corpus and the NSFC codec which is trained with that corpus shares same inductive bias, so with results only from the codec can not be the evidance that it also works for authored space of real human mixes. The upmixer is defined so that every 12 channel is a time-frequency filtering of a single mid signal (from Eq. (1)), and also for NSFC where the filters are applied to Mono downmix mm stream. So the spatial structure of the corpus (for all training / valid / test set) is of a form thet codec is built to express. I think that's one reason why NSFC reconstructs better than oracle panning within the corpus (according to L416) but no longer works for AlbumDB (L1331-1332), which is the true human masters.

It seems like all NSFC codec metrics in Table 2 are computed on the 63 corpus test tracks generated by the Upmixer. There exists 200 internal human test masters (which is used for training/testing the Upmixer) but were not used for codec evaluation. The claim in the title and abstract that "authored space" to be learned is at present supported only for space "authored by the model".

And also I think the Upmixer evaluation of Table 1 has similar issue. The objective metrics in Table 1 are computed only on the 200 internal tracks, and since the proposed model was trained on test split of the human-mixed set from the same team, the model could be biased to well estimate the test set. The objective metrics should also have been calculated with out-of-domain AlbumDB set which has human masters.

The codec evaluation has no baseline (Table 2). One possible simple baseline of can be: send the stereo downmix of 7.1.4 through DAC (or another stereo codec) and run the Upmixer on the receiving side. This would reconstruct corpus tracks up to stereo coding error.

Also, the "own codes" column of Table 15 shows that reconstruction itself of Ltf and Rtf fails. This also makes the results of Table 2 unreliable, because the metrics are aggregated for all channels.
The results could be dominated by the high-energy channels and so cannot expose this failure, and per-channel table that exists for the Upmixter (Table 5) is absent for the codec. Per channel results of codec should be reported for both produced corpus and internal 200 test set (and AlbumDB)

Questions for authors

Questions

  1. Can you report NSFC's objective metrics on the 200 internal human test masters (and its comparison with oracle panning), and AlbumDB? Can you also report channel-wise metrics (in format of Table 5)? I am particularly interested in Ltf/Rtf/Ltr/Rtr.

  2. Can you tell whether the failur of the Ltf/Rtf in Table 15 comes from the encoder/codebook side or from the decoder prior?

  3. Can you provide a comparison against a "stereo + DAC -> upmixer" baseline?

  4. What are the upmixer's objective metrics on the AlbumDB tracks?

  5. When codes (i.e., content, spatial) of real mixes are swapped and applied to each other, how well does the downmix of the output preserve the reference? Say, we have 7.1.4 mix y1 and y2 and its Mono downmix x1 and x2. From NSFC, we can get content of y1 and spatial info of y2, and get final decoded mix of y'1. If we downmix this to Mono x'1, is this become similar to x1?

  6. The agreement between the 12->2 downmix of the Upmix output and the input source stereo was used only to filter out the botom 0.1%. Can you report its distribution over the whole corpus and internal 200 test set? Since the carrier is mid only and antiphase content cannot be placed, I'd like to know how large the mismatch in the side component is.

Clarification

  1. L311: Are the diagonal informations "inter-channel"? Should it be something like "intra-channel level"?

  2. Do you plan to release the upmixer weights?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.