- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper tackles whole-song rhythm quantization (mapping expressive musical performance timing into score note values and metric beats) in an extreme data-limited regime (the standard ASAP benchmark contains only 822 training performances from 176 works). The authors propose AMT-Q, which adapts a pretrained 360M symbolic-music foundation model (the Anticipatory Music Transformer) to rhythm quantization without adding any task-specific parameters. By feeding performance events as the anticipated control stream and interleaving them with score events, the model acts as a fixed-lookahead transducer.
To combat severe exposure bias during whole-song autoregressive rollouts, the authors develop a three-stage curriculum: (1) teacher forcing, (2) score-token dropout selected via whole-song validation rollouts, and (3) on-policy DAgger retraining on model-generated contexts anchored to reference weights.
Recognizing that standard whole-song strict note F1 conflates global scale differences, cumulative drift, and local rhythmic quantization, the authors introduce a suite of disentangled metrics: scaled note F1, inter-onset-interval (IOI) F1, scaled IOI F1, and Theil-Sen local time-scale estimation.
Validation on the 59-performance ASAP test set demonstrates that AMT-Q achieves 9.33% strict note F1 and 77.93% IOI F1, matching or outperforming task-specific state-of-the-art baselines. Crucially, controlled comparisons confirm that musical pretraining more than doubles strict F1 over random initialization (8.7% vs. 3.2%), and AMT-Q outputs align with the reference score's time scale substantially more frequently (43/59 vs. 23/59).
Critical strengths
- Parameter-Free Foundation Model Adaptation with Rigorous Pretraining Controls: Adapting the Anticipatory Music Transformer into a streaming rhythm quantizer without any new parameters is conceptually elegant. More importantly, the authors conduct an exemplary controlled study: holding the architecture, training updates, data augmentations, and checkpoint selection rules identical, musical pretraining achieves a statistically significant 2.7x increase in strict F1 over random initialization across works, while text pretraining (SmolLM2) fails to bridge the gap. This offers conclusive evidence on the transferability of generative symbolic priors to low-resource transcription.
- Methodological Rectification of Rhythm Quantization Evaluation: The paper makes a consequential contribution to the evaluation methodology of symbolic music. By dissecting whole-song strict F1 into global scale mismatch, cumulative temporal drift, and local rhythmic correctness via IOI F1 and Theil-Sen slope estimation, the paper resolves long-standing ambiguities in why past systems failed and provides an actionable blueprint for future benchmarking.
Critical weaknesses
- Low Absolute Strict Accuracy and Persistent Whole-Song Drift: Although AMT-Q surpasses prior systems on IOI F1 and time-scale adherence, the absolute strict note F1 remains below 10% (9.33% with DAgger, median 4.75%). This confirms that while the model successfully captures local rhythmic ratios (IOI F1 ~78%), whole-song autoregressive rollouts on complex classical pieces still frequently suffer from irreversible global drift, limiting its current readiness for commercial automated notation software without manual beat anchors.
- Evaluation Restricted to a Single Small Benchmark: The experimental validation is conducted exclusively on the ASAP dataset (14 test works, 59 performances). While ASAP is the standard benchmark in recent literature, demonstrating whether AMT-Q generalizes across other genres (e.g., jazz with complex syncopation, pop, or non-piano MIDI datasets) would provide broader validation for the proposed framework.
Questions for authors
- Section 5.3 notes that the Spearman rank correlation between whole-song validation strict F1 and test strict F1 across checkpoints is only 0.02. What accounts for this near-zero correlation? Is it driven by the small number of works in the validation (16) and test (14) sets, or by high sensitivity to drift in specific pieces? How can practitioners perform reliable model selection in this regime?
- Table 1 shows that while DAgger increases mean strict F1 over works from 9.6% to 14.7%, the median strict F1 slightly drops from 5.14% to 4.75%, and IOI F1 slightly decreases from 78.81% to 77.93%. Does DAgger introduce higher variance across different musical compositions?
- Since the primary bottleneck for strict F1 is long-term drift rather than local interval quantization, have the authors considered incorporating periodic global re-synchronization anchors (e.g., bar-line or downbeat estimators) during sliding-window inference?
2 comments
Reply to Nsh9 (2/2)
Checkpoint Selection
"Section 5.3 notes that the Spearman rank correlation between whole-song validation strict F1 and test strict F1 across checkpoints is only 0.02."
"What accounts for this near-zero correlation?"
"Is it driven by the small number of works in the validation (16) and test (14) sets, or by high sensitivity to drift in specific pieces?"
"How can practitioners perform reliable model selection in this regime?"
Strict F1 on 14 to 16 works is dominated by a few pieces that either lock onto the beat or drift early in the piece, so a checkpoint's rank on validation is only weakly correlated to its rank on test. With AMT-Q 3B we trained four runs with different seeds and evaluated all 20 checkpoints of each on test:
| Run | Step selected on validation | Strict F1, mean of its 20 checkpoints | Validation-to-test rank correlation |
|---|---|---|---|
| Seed 42 (AMT-Q 3B) | 18000 | 19.86 | −0.03 |
| Seed 1 | 38000 | 18.96 | +0.13 |
| Seed 2 | 20000 | 19.03 | +0.57 |
| Seed 3 | 18000 | 19.65 | +0.26 |
| All four runs | 19.38 (all 80 checkpoints) |
Validation ranks checkpoints poorly, but the choice matters little: the checkpoints selected on validation average 19.89 strict F1 on test, against 19.38 over all 80 checkpoints. For practitioners we recommend selecting on validation but comparing systems by the mean over several checkpoints or seeds, and reporting scaled and IOI F1 beside strict F1.
DAgger and Variance Across Pieces
"while DAgger increases mean strict F1 over works from 9.6% to 14.7%, the median strict F1 slightly drops from 5.14% to 4.75%, and IOI F1 slightly decreases from 78.81% to 77.93%."
"Does DAgger introduce higher variance across different musical compositions?"
With the 3B model we present DAgger as an optional extension rather than part of the recommended system: it raises IOI F1 but lowers strict and scaled F1. The pattern the review noticed, a higher mean with a lower median, comes from strict F1 being close to all-or-nothing per piece. With AMT-Q 3B, four of the 14 ASAP test works dominate the mean, and 7 of the 59 recordings are above 50%. On ATEPP the mean lead over Beyer & Dai is +12.24 points per work and the median lead +3.00. This is why we report medians and IOI-based measures beside the mean.
Re-Synchronization Anchors
"have the authors considered incorporating periodic global re-synchronization anchors (e.g., bar-line or downbeat estimators) during sliding-window inference?"
Unfortunately, it appears that existing beat-tracking systems aren't robust enough on classical music for this strategy to be effective. We tested re-anchoring to a beat tracker during decoding.
Re-anchoring to a beat tracker: every 69 notes, the output is shifted by whole beats so its beat count matches the tracker (PM2S on ATEPP, librosa on Gangqinpu). Strict F1 (%), mean over works (ATEPP) or songs (Gangqinpu), without and with re-anchoring; Holm p of the change.
| Test set, system | Original | Re-anchored | Holm p |
|---|---|---|---|
| ATEPP (232 works), AMT-Q 3B | 29.35 | 18.21 | < 0.0001 |
| ATEPP, Beyer & Dai | 17.11 | 13.62 | < 0.0001 |
| ATEPP, Zeng et al. | 12.57 | 10.92 | < 0.0001 |
| Gangqinpu test split, AMT-Q 3B | 27.44 | 30.30 | 0.0081 |
It helps only where the beat tracker is reliable: on the metronomic Gangqinpu songs it raises AMT-Q 3B's strict F1, while on classical ATEPP, where the tracker errs, it lowers strict F1 for every system, significantly after Holm correction. Better beat or downbeat estimators would make this route more useful.
Reply to Nsh9 (1/2)
We thank the reviewer for the positive assessment and the careful questions, and for the strengths it identifies:
Since the submission we applied the same method to a 3.39B-parameter music foundation model, AMT-Q 3B, and evaluated it on two larger out-of-distribution test sets never used for fine-tuning (ATEPP, Gangqinpu; see our general response). Below we address each concern brought up by the reviewer.
Low Absolute Accuracy and Whole-Song Drift
We agree that absolute strict F1 is still low, and we state this as a limitation. With AMT-Q 3B it rises to 19.68 on ASAP, 25.05 on ATEPP and 26.15 on Gangqinpu, but whole-song drift remains the main failure: strict F1 is close to all-or-nothing per piece. On ATEPP, 497 of the 2,489 recordings are above 50% strict F1, while the median Gangqinpu song scores 8.47. We make no claim of readiness for notation software without beat anchors.
Limited Test Set
We now evaluate on two larger out-of-distribution test sets never used for fine-tuning: ATEPP (2,489 classical recordings, with noisier performances obtained from automated transcription) and Gangqinpu (11,425 songs, mostly pop piano arrangements, the genre the review mentions).
Strict note F1 (%) of AMT-Q 3B and the two released models, all fine-tuned on ASAP only; ATEPP and Gangqinpu never used for fine-tuning.
Results on other metrics can be found in parts 1 and 2 of the general reply.