- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper redefines monaural-to-stereo upmixing as language-driven stereo imaging creation rather than a sole reconstruction problem. The authors propose "Width as a Command": language directors translate user requests into element-level width and translation scripts, monaural source separation routes scripts to time-frequency regions, classification-based parametric stereo skeleton predicts IID/IC, conditional diffusion models generate locally centered IID motions, and a fixed renderer outputs stereo while preserving monaural channels. The experiments and ablation studies verify that local IID motions outperform side-channel energy as width descriptors.
Critical strengths
- The formulation of the task essence demonstrates profound insight, since it explicitly identifies the one-to-many uncertainty caused by downmixing and the mean regression issue of point-wise regression.
- The localized centering IID oscillation decouples "width" and "global translation" with clear acoustic semantics and facilitates editing, while the mid-channel algebraic preservation and post-translation design provide controllability and content fidelity guarantees, preventing generative models from arbitrarily altering mono-channel content.
- The experiments effectively demonstrate the role of diffusion and width representations.
Critical weaknesses
- Human experiments are missing. ABX, MOS, or professional mixing evaluations are absent.
- Training and evaluation data are limitted, where the universality of the model remains unknown, especially for long audio.
- Metric interpretations show potential conflicts, for example, Schroeder's Side-FAD is minimal while Fréchet-PS remains high, maybe indicating the proposed evaluation system has not yet demonstrated consistency with perceived quality.
Questions for authors
- Could you provide more evaluation results in subjective/expert experiment or universality experiment (e.g., long audio)?
- Could you provide more detailed explanation of the conflicts between the results of different evaluation metrics?
Thank you for identifying the task formulation and the centered-swing design as the core of the work; that is exactly the framing the revision now foregrounds (Section 1). Section, table, and appendix numbers below refer to the revised PDF, changed text is set in blue.
1. Human evaluation and audio length. We have added a blinded listening test of the system's instruction following (Appendix D). It is a two-alternative forced choice in the style of the Artificial Analysis Music Arena: each question shows the written instruction and two renders of the same mono excerpt from the same frozen model, differing only in the width map, and the listener picks the one that follows the instruction. Twelve listeners answered ten questions each (120 judgments, 100 songs, loudness-matched, order randomized). The instructed mix was chosen on 97 of 120 questions (81%, Wilson 95% CI 73-87%, p < 1e-11), and every listener scored at least 6 of 10. To address length, the stimuli were rendered at three durations, 5 s, 15 s, and 30 s; accuracy was 78%, 83%, and 81% respectively (chi-square p = .83), so longer excerpts neither helped nor hurt, and the 30 s renders already exercise the long-form inference path of Section 5.4 (windowed generation with overlap merging, Appendix A.5). Per-element accuracy ranged from 70% (hi-hat/cymbals) to 94% (lead vocal). We are currently recruiting additional listeners with five or more years of music experience and will extend the table as results arrive.
2. Conflicts between metrics. The two distributional metrics measure different necessary conditions, and we have added Appendix C.1 to state this with pseudocode. Both are Frechet distances between Gaussians fitted to pooled per-frame vectors over all 300 excerpts; they differ only in the vector considered. Side-FAD embeds the side channel alone with VGGish, so it tests whether the side channel sounds like the side channel of real music, and it never sees the left and right channels together. Frechet-PS uses the six-band IID and IC of each frame, so it tests whether the stereo image has the statistics of real stereo. Schroeder is the case that pulls them apart: its side channel is the mid passed through all-pass filters, so it contains the right instruments with the right spectral envelope and scores the best Side-FAD (0.56), while its IID is near zero and its IC follows a fixed comb that never varies with the music, so its Frechet-PS is high (31.13). Combing both metrics we read Schroeder as 'fake' stereo. So the metrics are not inconsistent with each other; a good upmix has to satisfy both.