Evolving Anatomy Graphs Steer Longitudinal Image Forecasting Beyond Pixels

Longitudinal image forecasting aims to predict how a patient’s anatomy will change, yet image similarity can reward a forecast that changes nothing. On our brain MRI benchmark, copying the baseline outperforms every tested fore- caster on PSNR and SSIM despite predicting no progression. The forecasters also miss the change itself: on three core brain structures, none reaches the change correlation of a regression on elapsed time (0.324). We introduce GraphMorph, which makes regional geometry an explicit forecast. Each anatomical region is encoded as a spatial node in an anatomy graph, retaining shape and location in- formation while allowing its evolution to depend on other regions. Conditioned on the requested interval, clinical covariates and available anatomical history, the model evolves these nodes under supervision from measured follow-up anatomy and decodes them into spatial maps of the predicted future anatomy, the anatom- ical request. A separately trained renderer deforms the patient’s baseline scan using the request as both spatial control and its primary supervision target. The anatomical forecast therefore determines not only what the renderer receives, but what it learns to reproduce, allowing forecast accuracy and rendering fidelity to be assessed separately. On brain MRI and knee radiographs, GraphMorph achieves regional-change correlation 0.314 over 32 brain regions, 0.422 on the core structures and 0.230 for knee joint space, versus 0.230, 0.268 and 0.119 for the strongest of ten brain and six knee image forecasters. Substituting measured follow-up anatomy into the same frozen renderer raises core-brain change corre- lation from 0.422 to 0.704, showing that the generated change responds to the supplied anatomy