- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
With in-sequence (anticipatory) conditioning, conditioning events are placed earlier in the token sequence than their musical time, so sequence order and musical time disagree. The paper formalises this as delta conditioning with laminar groups. It then proposes a two-level, H-Net-style hierarchy: tokens are compressed into note groups and then into onset-synchronous time groups. The main Transformer uses the original musical time as rotary (RoTE) coordinates, while the input order, causal mask, and next-token objective stay unchanged. Experiments on deduplicated Lakh MIDI cover melody-conditioned accompaniment, random conditioning, and audio-conditioned drum generation. Evaluation uses per-token-type perplexities, the PHALAR coherence score, and a 12-participant listening test with participant-level permutation tests.
Critical strengths
S1. The same-architecture ablation shows large, consistent effects. Across three parameter allocations, realignment reduces time-token perplexity PPLt from 2.57 to 1.80, 2.82 to 1.88, and 2.58 to 1.85 (−28% to −33%), and improves PHALAR by +2.3 to +2.9 (Table 1). In the random-conditioning listening test against the model without realignment, the result is 67/19/14 (win/tie/loss). A participant-level exact permutation test gives a Holm-corrected p = 0.0117 (Table 8; I recomputed the four Holm values and they match). This directly supports the abstract's claim of improvement over "an otherwise identical model without realignment".
S2. The mechanism is simple and causally correct. Only the attention coordinates change. The mask, slots, and KV-cache order are untouched (App. F, Eq. 32), so the method can be reused in other interleaved conditional generators. It extends naturally to mixed audio–MIDI sequences. With audio conditioning, PHALAR rises from 5.40 to 6.35 against a human reference of 6.51, computed on the 2,454 windows that are valid for both models (H.3). The appendix is also careful about data deduplication (Choi et al.'s list) and data accounting, and about consent and blinding in the listening study.
Critical weaknesses
W1. The attribution to the order/time conflict is not isolated. Table 1's flat baseline ("AMT"; from Appendix G.2/G.3 it appears to be the Vanilla Transformer trained on the same representation with the same 10 s lookahead) uses no realignment, yet:
- its PPLt is 1.83, essentially equal to the best realigned model (Balanced, 1.80);
- its total PPL of 1.71 is better than every realigned hierarchical model (1.78–1.84), and its instrument/pitch/duration perplexities are also the best;
- the hierarchical variants without realignment are much worse than this flat model (PPLt 2.57–2.82, PHALAR 2.63–3.25).
The large ablation gap is therefore more consistent with realignment repairing timing information lost by hierarchical compression than with the order/time conflict itself harming modelling, as Fig. 1 and Prop. 1 suggest. The p.5 argument that realignment cannot be done at the token level seems to follow from the representation choice of placing TIME at the end of a group, not from a general limitation. The missing control is a flat Vanilla Transformer with token-level musical-time RoPE, where each token causally inherits the most recent known time, or alternatively AMT's original time-first triplets.
W2. The evidence for an advantage over the flat baseline is thin.
- The only advantage is PHALAR 5.57 vs. 4.27 in the symbolic row of Table 1. The text does not state the task, the number of songs, or per-system completion rates.
- Failed or unfinished generations, invalid MIDI, and silent windows are excluded from PHALAR (G.4, p.23) without reporting per-system retention. If retention differs, the means mix generation quality with how often a system produces something scorable. A common-set analysis is needed. The audio experiment already does this.
- The comparison has no confidence intervals and uses a single seed. The flat baseline is absent from the listening test, and the accompaniment-task listening result is not significant (41/25/34, Holm p = 0.273).
- p.7 says realignment "retains the efficiency of grouped modeling", but no runtime or FLOPs are reported, and Appendix F concedes that sequence-length savings do not determine end-to-end runtime.
Secondary concerns (less decisive than W1–W2):
W3. Some claims are broader than the evidence. The abstract says that "on both symbolic and audio-conditioned generation tasks, temporal realignment substantially improves timing prediction". In the audio-conditioned setting, PPLt moves only from 1.91 to 1.87 (−2%). The audio "w/o realignment" model gives audio latents no absolute time embedding at all (H.2). That comparison is therefore "shared timestamps vs. none" rather than RoTE vs. alternative ways of injecting time. Moreover, the audio experiments cover only synthesised accompaniment-to-drums on 100 test songs.
Other statements also go beyond the evidence:
- p.2 states that temporal realignment "consistently outperforms non-realigned baselines for both settings". The only non-realigned comparisons are the same-architecture ablations, and the accompaniment listening comparison is not significant.
- p.7 introduces the baseline as "the state-of-the-art MIDI generation model" and states that the best realigned model "outperforms the AMT baseline in generation quality". The baseline is retrained on this paper's representation (see Q1), and the advantage holds only on PHALAR, while the baseline's total PPL is better (1.71 vs. 1.78). "Outperforms a flat baseline on PHALAR" would be accurate.
- The introduction claims "broader applicability" and the conclusion calls the method "a general framework", but the evaluation covers only Lakh MIDI and FluidSynth-rendered accompaniment.
W4. No listening examples. The submission provides no demo page, audio examples, or supplementary material. For a generation paper whose subjective evidence is a 12-listener test, readers cannot judge the claimed timing improvements or the failure modes by ear. Anonymised paired examples (with/without realignment and the flat baseline, under both conditioning tasks) would help.
Questions for authors
- Baseline identity. Is "AMT" in Table 1 the Vanilla model of Appendix G.2 (same representation, 10 s lookahead) rather than the original AMT (arrival-time triplets, δ = 5 s)? If so, please rename it and state exactly which AMT mechanisms are kept.
- Symbolic PHALAR row. Which task and how many songs does it cover? Please report per-system completion and exclusion counts, and results on the common set of windows valid for all systems.
- Flat + musical-time control. Please add a flat Vanilla Transformer with causal token-level musical-time RoPE. If the hierarchy plus realignment remains clearly better, or offers a measured efficiency advantage, I would raise my score.
- Lookahead sweep. Without retraining, can you report PPLt at inference lookaheads of 5/10/20 s for all models? A gain that grows with the order/time displacement would directly support the conflict hypothesis.
- Audio baseline and PHALAR calibration. Please add an audio "w/o" baseline with absolute time embeddings, PHALAR scores for mismatched or time-shifted pairs as calibration anchors, and song-level bootstrap CIs.
I would raise my score to 3 if Q3/Q4 show benefits beyond the flat model that grow with displacement, or if a listening test against the flat baseline favours the proposed model.
Minor issues
- Eq. 6 requires strictly monotone coordinates, but under anchored conditioning several events share the same t̃ (Eq. 24). The tie-breaking rule is stated differently on p.3 and p.14.
- OMCAT's RoTE is applied to feature embeddings, whereas here it is applied to the main network's Q/K. Please state the difference.
- Per-model parameter counts are not given, and KV heads differ (4 vs. 6).
- There is a duplicated sentence in H.1, "conditioning,the" on p.7, and "transcribed" is used loosely in the Fig. 4 discussion.
- Related work: Moonbeam (Guo & Dixon, 2025, arXiv) also uses onset time as a RoPE coordinate for symbolic music and is worth discussing. It is arXiv-only, so I do not count its absence against the paper.
Overall. Realignment is clearly effective within the proposed hierarchical model, and the mechanism is clean. However, the general claim about the order/time conflict, and the need for the hierarchy, are not established against a flat model that already matches the timing prediction without realignment. The advantage over that baseline rests on a single under-specified metric.
6 comments
Response to Reviewer rW8T (2/3)
Evidence for an advantage over the flat baseline.
Symbolic PHALAR is computed on accompaniment, 200 test windows of 30 s, as now stated in Appendix G.4. Every model completed all 200 items of both tasks (1,598 of the 1,600 generations on the first attempt, the rest on the second), and all models are scored on the same 955 five-second windows (of 1,000), so retention does not differ across systems. Song-level paired bootstrap intervals for PHALAR (10,000 resamples) and a paired permutation test for FMD show a significant difference from H-Net only for H-Net w/o realignment; the flat models do not differ significantly from it:
| Difference from H-Net | PHALAR, 95% bootstrap CI | FMD, permutation p |
|---|---|---|
| H-Net w/o realignment | −1.63 [−1.88, −1.38] | +6.4 (p ≤ 0.001) |
| Vanilla | +0.12 [−0.03, 0.27] | −1.4 (p = 0.16) |
| Vanilla w/o realignment | −0.04 [−0.22, 0.15] | −1.4 (p = 0.17) |
The gap between our model and the flat baseline observed in our original submission (5.57 vs. 4.27) therefore does not hold in the retrained comparison. Two things changed since our original submission: training now includes span conditioning with equal task weights, and inference presents every window as a training window, including the music that follows it. The flat baseline improved the most, while our model changed little, and we cannot attribute the change to either factor alone. We removed the claim that our model outperforms the baseline in generation quality.
The new listening test (Table 3) includes both flat models. In span infilling, removing realignment from H-Net lowers both criteria significantly, H-Net ties Vanilla, and it is rated significantly smoother than Vanilla w/o realignment. In accompaniment, Vanilla receives the highest ratings, but no difference from H-Net is significant after correction.
Mean ratings with 95% confidence intervals; in parentheses, the paired difference from H-Net; † Holm-adjusted p < 0.05 (Table 3):
| Task | Model | Smoothness | Soundness |
|---|---|---|---|
| Accompaniment | H-Net | 3.90 ± 0.26 | 3.35 ± 0.37 |
| H-Net w/o realignment | 3.83 ± 0.30 (−0.07) | 3.39 ± 0.36 (+0.04) | |
| Vanilla | 4.08 ± 0.21 (+0.19) | 3.75 ± 0.33 (+0.40) | |
| Vanilla w/o realignment | 3.85 ± 0.31 (−0.04) | 3.65 ± 0.37 (+0.29) | |
| Span | H-Net | 3.34 ± 0.27 | 3.17 ± 0.31 |
| H-Net w/o realignment | 1.98 ± 0.22 (−1.36†) | 2.38 ± 0.38 (−0.80†) | |
| Vanilla | 3.38 ± 0.28 (+0.04) | 3.38 ± 0.30 (+0.21) | |
| Vanilla w/o realignment | 2.87 ± 0.33 (−0.47†) | 3.21 ± 0.31 (+0.04) |
Claims broader than the evidence; audio comparison. We revised the abstract (the listening claim is now restricted to span infilling, where it is significant), the introduction ("consistently outperforms" and "broader applicability" removed), Section 4.1 (no "state-of-the-art" baseline or "outperforms AMT"), and the conclusion (no "general framework"). For audio conditioning, we agree that time-token perplexity changes little and that the model without realignment receives no timing information for the audio latents, so the comparison contrasts shared timestamps with none rather than alternative ways of injecting time. We now state this limitation in Section 4.2, treat the audio results as supporting evidence, and focus the analysis of the paper on symbolic conditioning, where all of the controls above apply. We therefore did not add the absolute-time audio baseline or calibration anchors for the audio experiment. For symbolic accompaniment, PHALAR now comes with the song-level intervals above, and our scorer also reports, on the same 955 windows, a mismatched-pair floor (each generation re-paired with another song's conditioning) of 0.85 and the human ceiling of 5.98.
Response to Reviewer rW8T (3/3)
Listening examples. The demo page at https://listening-examples-iclr-2027.vercel.app now contains, for every clip of both tasks, the outputs of all four symbolic models and the original.
Significance in light of the new results.
We want to state plainly what the retrained comparison changes: our model no longer outperforms the flat baseline in generation quality, and we removed that claim. We believe the significance of the paper rests on three contributions that the new results support. First, the formal framework: delta conditioning and the distinction between ordering time and musical time describe a family of in-sequence conditioning schemes, including prefix, block-wise, and anticipation conditioning (Appendix A), and show that moving conditioning ahead of its musical time makes sequence order and musical time disagree; the span analysis (Table 2) shows the effect where the framework locates it, at the start of each masked span, where models without realignment degrade at the longest lookahead. Second, realignment is effective at different levels and in different architectures: applied to tokens in a flat Transformer and to time groups in the hierarchical model, it improves timing prediction in both, and in the hierarchical model it is also crucial for generation quality (PHALAR, FMD, and the listening test on span infilling). Third, the hierarchy improves efficiency at little cost in quality: with realignment, H-Net stays close to Vanilla, with a slightly higher perplexity and no significant difference in PHALAR, FMD, or the listening ratings, while it needs only about a quarter of the forward FLOPs, trains in 12.3 instead of 21.4 GPU-hours, and generates about 1.5× faster (159 vs. 109 tokens/s when filling a 4,096-token window on one H100 at batch size 1). The revised introduction and conclusion state the contributions in these terms.
Reply to questions
Baseline identity.
Yes, it was the Vanilla model trained on our representation with the 10-s lookahead, not the original AMT (arrival-time triplets, δ = 5 s). We renamed it Vanilla w/o realignment and describe it as a flat Transformer with anticipation-style (delta) conditioning on our representation; it keeps AMT's interleaving of controls ahead of their sounding time, but not its tokenization.
The other questions, on the symbolic PHALAR row, the flat control with musical-time RoPE, the lookahead sweep, and the audio baseline and PHALAR calibration, are answered in the responses above.
Minor fixes
Per-model parameter counts are now given in Appendix G.2 (132.9M for H-Net and 134.0M for Vanilla), together with their key/value heads (4 and 6). The tie-breaking rule is now stated once in Section 2.2: groups that share an ordering time are ordered by a rule specified by the conditioning scheme, which must place a conditioning group before every target group that depends on it; the mono-constraint allows conditioning groups to share an ordering time and keeps them in musical-time order, which covers anchored and block-wise conditioning, and Appendix A states the anticipation case accordingly. Appendix F now states the difference from OMCAT, which applies RoTE to feature embeddings, whereas we rotate only the queries and keys of the main Transformer. Moonbeam is discussed in Section 5, the paragraph with "conditioning,the" was rewritten, and "transcribed" in the attention-map discussion (now Appendix H) was replaced. We could not locate the duplicated sentence in Appendix H.1 in our source and would appreciate a pointer if it persists in the revision.
Response to Reviewer rW8T (1/3)
We thank the reviewer for an exceptionally careful review; the requested controls changed the paper. We retrained all symbolic models under identical settings and added the flat Transformer with token-level musical-time RoPE that the reviewer asked for, the lookahead sweep (as a new analysis of span infilling, Table 2), FMD and statistical tests, a new listening test that includes both flat models, compute measurements, and an updated demo page. We also revised the claims that went beyond the evidence. The revised PDF has been uploaded; a version with all changes marked (additions in blue, deletions in red) is provided as supplementary material. Point-by-point answers follow.
Naming in the revision:
H-Netis our hierarchical model andVanillaa flat decoder-only Transformer, each trained with and without temporal realignment;Vanilla w/o realignmentis the baseline previously labeled "AMT". ForVanilla, realignment places every token at the musical time in force at that token.Flat Transformer with token-level musical-time RoPE. We added this control (
Vanilla): each token's rotary position is the musical time in force at that token, i.e., the time of the most recentTIMEtoken read up to and including it, so tokens of a time group share a position and their order is left to the causal mask. All four symbolic models (H-NetandVanilla, each with and without realignment) were retrained with identical data and settings, and the baseline previously labeled "AMT" is nowVanilla w/o realignment. The reviewer is right that realignment does not require the hierarchy: it also improves the flat model's timing prediction, and we removed the claim that realignment cannot be applied at the token level. We now present realignment as architecture-agnostic and the hierarchy as an efficiency choice. On accompaniment,H-Netmatches both flat models in PHALAR and FMD, with no significant difference, at only about a quarter of the forward FLOPs, while the flat models keep a lower overall PPL, which we state. The efficiency advantage is also measured in wall-clock time (Appendix G.2): training took 12.3 instead of 21.4 GPU-hours, and generation runs about 1.5× faster (159 vs. 109 tokens/s when filling a 4,096-token window on one H100 at batch size 1), although both savings are smaller than the FLOPs saving.Lookahead sweep. We report it as a new analysis of span infilling, where the model fills masked spans between given segments, split by position within each masked span, with lookaheads of 5, 10, and 20 s at inference and no retraining (Table 2). At the start of a masked span, the model has just read the given music and the conditioning placed ahead of it, and must decide when the first notes enter. There, the time-token perplexity of models without realignment rises at the longest lookahead, whereas with realignment it stays nearly constant; for the flat model, the gain from realignment at the start of a masked span grows with the displacement. In the middle and at the end of a masked span, where the preceding target notes are available, the differences are small, and averaged over all tokens the effect is diluted (Table 8 in the appendix), which is why the overall PPL hides it. We believe this is the direct evidence for the order/time conflict that the reviewer asked for, and it holds beyond the hierarchical model.
Time-token perplexity by position within the masked span and by lookahead (Table 2):