- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes AutoSynth, an audio-conditioned autoregressive model that jointly models MIDI performance events, fixed synthesizer parameters, and variable-length modulation routes as a unified sequence. Training proceeds in two stages: supervised fine-tuning (SFT) on large-scale audio–program pairs, followed by GRPO post-training. The authors also show that by cascading an off-the-shelf text-to-audio model (Stable Audio 3), the system can generate synthesizer programs whose sounds match a text description. They evaluate on a cross-generator Gemini–Woosh set and on FSD50K against Synth Permutations, DDSynth-RL, and CTAG, and run a 10-person expert listening study. They report better semantic alignment and content quality than the synthesizer-program baselines.
Critical strengths
- The proposed method is well motivated, and jointly modeling MIDI, synthesizer parameters, and modulation routes with an autoregressive Transformer is novel.
- The code is released and the appendices are detailed, which supports reproducibility.
- The authors construct a cross-generator Gemini–Woosh test set to avoid leakage from the generators used to build the training data.
Critical weaknesses
- Although the method is novel, the evaluation is not fully convincing. The baselines use different synthesizers (Surge XT, Dexed, and SynthAX), so performance differences may reflect the synthesizer rather than the method. In addition, most of the evaluation metrics (CLAP, wMFCC, SOT, MSS, RMS, and AudioBox CE and CU) are also used as rewards during GRPO. This makes the comparison unfair, since these metrics are susceptible to reward hacking and may yield higher scores without real improvement.
- There is no ablation study on the joint modeling, and the model’s editability is not evaluated. For example, MIDI accuracy (e.g., note-level F1) should be reported, as well as whether editing the MIDI preserves the intended timbre.
- Since the subjective evaluation is important, its rating criteria should be described in the main text rather than only in the appendix. Figures 3 and 4 report “Alignment” and “Content quality” scores without explaining the rating scale or criteria.
Questions for authors
- Can you provide a baseline (e.g., Synth Permutations or DDSynth-RL) retrained on Helm’s fixed parameters, so that all methods share the same synthesizer?
- How was Synth Permutations retrained (training data and number of steps)? Does it reproduce the in-domain results reported in the original paper?
- Can you report MIDI accuracy (e.g., note-level F1) and program statistics, such as the number of active routes and the fraction of parameters set to extreme values, before and after GRPO?