Position: Pretrained LLMs Should Move from Linear to Graph-Structured Execution

Large language models (LLMs) typically execute their layers in a fixed linear sequence. Post-training methods modify this structure through skipping, looping, reordering, or parallel execution, offering different capability and resource benefits without training a new LLM from scratch. However, these operations are often studied separately. This position paper argues that research on post-training LLM restructuring should systematically explore how multiple structural operations can be combined to pursue their different benefits within the same LLM. We introduce model execution graphs to realize this position, jointly specifying which pretrained layers execute, how often, and how they are connected. We also develop an evaluation framework that assesses multiple capabilities and resource costs and examines whether composition provides benefits beyond applying the operations separately. Experiments across pretrained LLMs show that combined operations can achieve simultaneous resource savings, but these savings can accompany task-performance losses and do not necessarily confer an advantage over simpler alternatives. The same graph can also affect task performance differently across LLMs. Finally, we outline future research directions for designing and evaluating model execution graphs.