Plan, Generate, Replan: Hierarchical Symbolic Music Generation with Boundary-Aware History Correction
Abstract
Music creation is an iterative process: composers repeatedly revise what has been written and decide how the music should develop next. Supporting this process requires modeling how music evolves under user instructions. We introduce the Music World Model (MWM), a hierarchical latent world model for controllable long-form symbolic music generation. MWM represents musical content as latent states and requested changes as explicit actions. A phrase-level planner predicts structural context, while a segment-level executor recursively generates musical states under this context and the requested action. Users can update actions at phrase boundaries, and the planner refreshes its predictions from generated history to account for how the music has evolved. However, mismatches between new plans and preceding context can enter this history and influence subsequent plans, causing structural drift. We therefore introduce Boundary-Aware History Correction (BHC), an inference-time mechanism that combines context anchoring with persistent correction memory to reconcile planned slow trajectories with preceding context. BHC carries correction information across phrase boundaries while keeping learned weights and external actions fixed, without future reference music. To support learning these action-conditioned dynamics, we construct Slakh-Action from Slakh2100 Redux, aligning multi-track symbolic music with segment-level states, phrase-level summaries, and structural action annotations. An anonymous interactive demo is available at https://demo-czv.pages.dev/
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.