- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Audio mixes in 7.1.4 format are shaped by how professionals decide to distribute sound around and above the listeners. Therefore they often reflect creative mixing decisions rather than necessarily reproducing a physical acoustic scene. This paper investigates whether this spatial distribution of the format can be represented separately from its musical content.
The authors introduce a corpus of 125,557 tracks in 7.1.4 format, generated from Creative Commons stereo recordings using an upmixing model trained on a private collection of human-produced immersive mixes. Then they propose a Neural Spatial Filtering Codec that represents each twelve-channel mix through a mono reference and a compact spatial code description with codes that can be swapped to steer the mix.
Critical strengths
- The separation between mono reference and spatial side information (in the evaluation protocol, Section 5) provides a concrete mechanism to test whether a codec has learned a spatially meaningful representation and its therefore a useful experimental design that enables controlled tests on the replacement of spatial codes.
- Broad evaluation on the proposed architectures: waveform and spectral metrics, spatial metrics, embedding-based distributional metrics, and listening tests on the calibrated 7.1.4 loudspeaker system. The subjective evaluation is particularly relevant because objective metrics often don't correlate with human hearing.
- The large generated 7.1.4 corpus could enable research at a much larger scale.
Critical weaknesses
If I understand correctly, the paper claims that the space can be separated from the content, in a 7.1.4 mix, by representing the space as a set of filters applied to a mono signal. The 7.1.4 music that is needed to test this claim is generated with an upmixer, which does that by applying filters to a mono signal, so the training and evaluation data have approximately the same structure that the codec assumes, by construction. The results show that the codec can invert the transformation done by the upmixer, but not that it would generalize to human-made 7.1.4 mixes. I seem to understand that the authors have access to 200 test tracks mixed in 7.1.4 by humans, but never run the codec on them.
If this is the case, this looks like a significant evaluation that the authors would need to run to test whether the claim actually generalizes to real music.
In fact, the codec seems to not generalize strongly on the only human mixes that the codec is tested on, AlbumDB. The authors provide an explanation that is absolutely plausible, but not directly verifiable without a comparison with other human mixes.
Questions for authors
The weakness described above is my only real concern with the soundness of an otherwise well constructed paper. I don’t think that it is an absolutely necessary condition for acceptance that the codec generalizes to all human samples, but if it doesn’t, it should be disclosed and discussed more openly. Could the authors evaluate the codec and the Section 5 tests on the 200 internal human test masters?