acceptodds
Under review as a conference paper at ICLR 2027

Evolving Anatomy Graphs Steer Longitudinal Image Forecasting Beyond Pixels

12 views

Abstract

Longitudinal image forecasting aims to predict how a patient’s anatomy will change, yet image similarity can reward a forecast that changes nothing. On our brain MRI benchmark, copying the baseline outperforms every tested fore- caster on PSNR and SSIM despite predicting no progression. The forecasters also miss the change itself: on three core brain structures, none reaches the change correlation of a regression on elapsed time (0.324). We introduce GraphMorph, which makes regional geometry an explicit forecast. Each anatomical region is encoded as a spatial node in an anatomy graph, retaining shape and location in- formation while allowing its evolution to depend on other regions. Conditioned on the requested interval, clinical covariates and available anatomical history, the model evolves these nodes under supervision from measured follow-up anatomy and decodes them into spatial maps of the predicted future anatomy, the anatom- ical request. A separately trained renderer deforms the patient’s baseline scan using the request as both spatial control and its primary supervision target. The anatomical forecast therefore determines not only what the renderer receives, but what it learns to reproduce, allowing forecast accuracy and rendering fidelity to be assessed separately. On brain MRI and knee radiographs, GraphMorph achieves regional-change correlation 0.314 over 32 brain regions, 0.422 on the core structures and 0.230 for knee joint space, versus 0.230, 0.268 and 0.119 for the strongest of ten brain and six knee image forecasters. Substituting measured follow-up anatomy into the same frozen renderer raises core-brain change corre- lation from 0.422 to 0.704, showing that the generated change responds to the supplied anatomy

Primary area:
applications to neuroscience & cognitive science
Keywords:
longitudinal image generation, controllable generation, disease progression forecasting, anatomical graphs, medical imaging

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2DrTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses longitudinal medical image forecasting by predicting a patient's future anatomical scan from baseline imaging, clinical covariates, and available prior-visit history. It highlights a critical evaluation pathology ("identity collapse"), where simply copying the baseline scan outperforms ten existing generative forecasters on standard pixel-similarity metrics (PSNR/SSIM) while failing to capture subtle regional progression. To overcome this, the authors introduce GraphMorph, a sequential three-stage framework that decouples predicting future regional geometry from rendering the future scan: (1) a spatial anatomical autoencoder encodes named segmentation masks into 3D/2D spatial tensor nodes within an anatomy graph, (2) a graph-conditioned temporal evolver advances these nodes over the requested interval under direct segmentation supervision to decode future occupancy maps ("the anatomical request"), and (3) a frozen, registration-pretrained deformation network (DiffuseMorph) steered by a ControlNet-style branch warps the baseline scan, using the predicted request as both spatial conditioning and the primary anatomical alignment target. Across multi-cohort 3D brain MRI (ADNI, OASIS-3, AIBL), 2D knee radiographs (OAI), and an extension to 2D Potts grain growth, the authors pair extensive regional-change and adherence evaluations with controlled anatomical substitutions to separate forecast accuracy from rendering fidelity.

Critical strengths
  1. The paper shows that copy-forward beats every image forecaster on PSNR/SSIM and that no pixel-space forecaster matches an elapsed-time regression on core brain structures. The change-sensitive protocol (copy-forward, interval regression, tabular references, oracle substitutions) is a useful contribution independent of the method.

  2. Separating the anatomical request from the frozen renderer allows forecast accuracy and rendering adherence to be measured separately. The appendices audit cross-segmenter reliability, input controls, SyN baselines, scale sweeps, and preprocessing artifacts. The paper discloses most of its own weaknesses.

Critical weaknesses
  1. Δ_Region relies on a ~2.3× multiplier fitted on test predictions; without it (s = 1), rankings reverse in favor of Δ-LFM on 32 brain regions and Diffusion/Flow Matching on knee. Under an independent segmenter (FastSurfer), the raw MAE advantage over copy-forward vanishes or reverses and the correlation margin over Δ-LFM shrinks by more than half. On the knee, joint-space MAE is worse than copy-forward (9.53 vs. 8.91 pp) and tabular masks beat the native spatial request by +0.063.

  2. Five brain baselines receive neither elapsed interval nor covariates; only GraphMorph and the tabular reference receive prior-visit history on the knee. The authors' own control shows that removing history drops R_chg^(32) from 0.313 to 0.224, statistically indistinguishable from Δ-LFM, while the cross-region attention mechanism, the paper's named contribution, adds only +0.019. Cited 3D longitudinal baselines (TADM-3D, AG-LDM, TimeFlow) are omitted and MambaControl is evaluated without its core module.

Questions for authors
  1. Would the authors report Δ_Region using the validation-fitted scale as the primary metric, with s = 1 rankings in the main tables?
  2. Can the no-history GraphMorph control be included in Tables 1 and 3, and all knee methods trained on the same registration-gated 25,691 pairs?
  3. Can the three confounded Stage 2 ablation arms (core-6, original 32-region, full-32 decode) be retrained under the deployed schedule and batch size?
  4. What mechanism makes zero-padding to (192, 224, 192) vs. (128, 160, 128) change interior predictions, and why was the larger canvas the validation default?
  5. What is the source of the 8 mm header-origin shift affecting 24.2% of test pairs but 0% of training, and can the corrected table be reported?

Suggestions for future work: Cross-region spatial message passing rather than pooled FiLM, and non-deformational progression (new lesions, intensity drift).

5 comments

We thank the reviewer for the close reading, and for recognising that the change-sensitive evaluation protocol is a contribution independent of the method and that separating the anatomical request from the renderer lets forecast accuracy and rendering adherence be measured separately. We answer the concerns below.

Q1. ∆Region relies on a ~2.3× scale fitted on test predictions, and without it the rankings reverse. Would you report the validation-fitted scale as primary, with the unscaled rankings in the main tables?

A: We thank the reviewer for this question. The revised Tables 1 and 3 report ΔRegion\Delta_{\mathrm{Region}} with the validation-fitted scale as the primary metric, together with the s = 1 values in the column ΔRaw\Delta_{\mathrm{Raw}} (Table R1). With the validation-fitted scale, GraphMorph has the lowest ΔRegion\Delta_{\mathrm{Region}} in all three settings (1.030 over 32 brain regions, 0.825 on the brain core and 1.194 on the knee).

Why we scale? A forecast can be wrong in its overall amount of change, or in which regions and patients change most. One scale fitted on validation removes the first error, so ΔRegion\Delta_{\mathrm{Region}} measures only the second.

Without scaling (s = 1), ΔRegion\Delta_{\mathrm{Region}} also counts the first error, the error in the overall amount of change, and there we do worse. Our forecasts predict about 0.43 of the measured change, where 1 would be exact. ∆-LFM predicts 1.34 of it on the brain and Diffusion 0.74 on the knee, which are both closer to 1 (Table R2). This is why ∆-LFM has the lowest ΔRegion\Delta_{\mathrm{Region}} over 32 brain regions at s = 1 and Diffusion on the knee, while we still lead on the brain core (Table R1). The revised limitations state this under-prediction. It does not affect regional change correlation (RchgR_{\mathrm{chg}}), our headline metric, because a correlation does not change when every forecast is multiplied by the same scale factor.

Table R1. ΔRegion\Delta_{\mathrm{Region}} at s = 1 and at the validation-fitted scale (lower is better).

MethodBrain 32, s = 1Brain 32, validation scaleBrain core, s = 1Brain core, validation scaleKnee, s = 1Knee, validation scale
GraphMorph1.1831.0300.9060.8251.3111.194
∆-LFM1.1531.1280.9760.953——
CardiacAging1.5151.3081.5331.176——
Diffusion1.1801.1610.9930.9461.2381.226
Flow Matching1.1621.1400.9490.9131.2391.220
Noise-Free Flow1.2721.1511.0140.9111.4781.273
Pix2Pix1.4191.4181.6711.6251.2531.243
DiffuseMorph1.2391.2211.1441.0881.4871.338
BrLP1.2531.1941.0641.044——
MambaControl backbone1.3081.2841.2891.336——
ImageFlowNet (ODE on brain, GraphODE on knee)1.4031.3311.5791.4691.2701.261
Copy-forward2.0002.0002.0002.0001.9981.998

Table R2. Amplitude bias, the forecast change size divided by the measured change size.

MethodBrainKnee joint space
GraphMorph0.430.43
∆-LFM1.34—
CardiacAging0.34—
Diffusion0.840.74
Flow Matching0.760.68
Noise-Free Flow0.490.32
Pix2Pix0.650.85
DiffuseMorph0.760.38
BrLP1.87—
MambaControl backbone1.55—
ImageFlowNet0.540.77
Oracle control (F)1.05—
AuthorsOpenReview ↗

Response Part II

Q2. Under an independent segmenter (FastSurfer), the MAE advantage vanishes. On the knee, MAE is worse than copy-forward (9.53 vs 8.91 pp) and a tabular forecaster beats GraphMorph by 0.063.

A: We thank the reviewer for these careful observations.

MAE is the distance between the forecast change and the measured change. It therefore counts every error together, the wrong amount of change, the wrong patients and regions, and the noise in the measurement. The measurement alone is noisy. On the brain, the two segmenters agree only at r = 0.554 on the measured change (Sec. 5.1). On the knee, consecutive changes of the same joint are anticorrelated at −0.43 (Appendix E.5).

1. MAE under FastSurfer and on the knee. In both cases the measurement noise dominates the MAE and biases it towards a forecast of no change. Under FastSurfer, our MAE differs from copy-forward by −0.039 pp [−0.152, +0.075] on the pre-specified sample (Table 2). On the knee it is 9.53 against 8.91 pp (Table 3). The MAE advantage over copy-forward therefore holds only on the brain under SynthSeg (2.61 against 3.49 pp, Table 1), and the revised limitations state this.

In contrast, correlation asks only whether the patients and regions with more measured change receive more forecast change, so a forecast of no change cannot score on it. Under FastSurfer the margin over ∆-LFM is 0.077 smaller, and it remains positive on the pre-specified sample (+0.037 [+0.002, +0.078], Table 2). GraphMorph still reads 0.328 against 0.291 for ∆-LFM when a different segmenter measures the results (Appendix L.5).

2. Tabular masks. On the knee, a tabular regressor forecasts the anatomy better than our graph (+0.063 [+0.041, +0.084] in RchgR_{\mathrm{chg}} on validation, Appendix E.3). On the brain, with 32 regions, our forecast delivered as a spatial request renders higher than the tabular forecast (0.319 against 0.286 on validation, Table 5a). Both knee images come from the same GraphMorph renderer, which accepts any anatomical forecast. The revised limitations state this with the measured difference (Table S6).

AuthorsOpenReview ↗

Response Part III

Q3. The baselines receive fewer inputs than GraphMorph. Without history GraphMorph falls to 0.224, indistinguishable from ∆-LFM, and the named contribution, cross-region attention, adds only +0.019. TADM-3D, AG-LDM and TimeFlow are omitted, and MambaControl lacks its core module.

A: We thank the reviewer for this analysis. We go through the points in the order raised.

1. Inputs of the baselines. We ran each baseline from its original code where available, and in their original form some take no elapsed interval or covariates. To check whether this explains the gap, we gave the elapsed interval and covariates to Flow Matching and Noise-Free Flow (Table R3). The added inputs raise both on the core, Flow Matching from 0.217 to 0.316 and Noise-Free Flow from 0.184 to 0.314, and GraphMorph stays ahead with history inputs (0.447) and without them (0.380). Over 32 regions they read 0.165 and 0.153, against 0.322 and 0.226. On the knee, adding the same inputs to Flow Matching does not close the gap either (0.129 against our 0.230).

2. History. ∆-LFM itself uses up to two prior visits (Table S36), so that comparison sets our model without history against a baseline with it. With history on both sides the comparison is 0.322 against 0.228 (Table S23), and on the 185 test pairs that have no prior visit we lead ∆-LFM by 0.238 to 0.179 (Table S7). On the three core brain structures, GraphMorph without history still reads 0.380, against 0.263 for the best image forecaster in Table 1 and 0.316 for Flow Matching with the added inputs, and on the knee it reads 0.154 against 0.119 for Flow Matching, so on the core and on the knee the lead does not depend on history. The revised Tables 1 and 3 include this model (Table R4).

3. Cross-region attention. Its gain is small but consistent, +0.019 [+0.010, +0.028] over three seeds (Table 5). Attention is one component of the forecaster. The contribution of the paper is the decomposition, forecasting the anatomy first and then rendering it. When the forecast is replaced by the identity, the core RchgR_{\mathrm{chg}} falls from 0.447 to −0.005 (Table 4). Delivered as a spatial request, our forecast is ahead of a tabular forecaster by +0.033 [+0.019, +0.050] over 32 brain regions. This gain comes from the spatial form, because the same graph forecast reduced to masks loses 0.039 [0.026, 0.053] (Table 5). The revised discussion states the size of the attention gain.

4. TADM-3D, AG-LDM and TimeFlow. Our comparison covers ten image forecasters from the main families, namely deformation, diffusion, flow matching, neural ODE and direct prediction. TADM-3D and AG-LDM are closest to two of them, Noise-Free Flow, which also predicts change from the baseline, and BrLP, whose framework and preprocessing AG-LDM extends. We retrained AG-LDM on our split with its released code, and it reads 0.105 over 32 regions and 0.028 on the core (Table R3). TADM-3D, retrained in the same way from its released code, reads 0.031 and −0.040. We do not run TimeFlow, because it extrapolates a registration between two earlier scans, which 185 of our 669 test pairs do not have, and its registration family is represented by DiffuseMorph (Table 1).

5. MambaControl. It is run as its released backbone, because the Fourier graph module is not in the released code (Appendix I.6), and Table 1 labels it MambaControl backbone.

Table R3. RchgR_{\mathrm{chg}} of the comparators with the added inputs and of the added baselines.

Comparator changeBrain, 32 regionsBrain, coreKnee joint space
Flow Matching with elapsed interval and covariates0.1650.3160.129
Flow Matching, original†0.1210.2170.119
Noise-Free Flow with elapsed interval and covariates0.1530.314—
Noise-Free Flow, original†0.0880.1840.087
TADM-3D, retrained on our split0.031−0.040—
AG-LDM, retrained on our split0.1050.028—
GraphMorph0.3220.4470.230

† As reported in Tables 1, 3 and S23.

Table R4. RchgR_{\mathrm{chg}} of GraphMorph with and without history inputs.

MethodBrain, 32 regionsBrain, coreKnee joint space
GraphMorph0.3220.4470.230
GraphMorph, no history inputs0.2260.3800.154
∆-LFM (uses history)0.2280.219—
Best other image forecaster0.228 (∆-LFM)0.263 (CardiacAging)0.119 (Flow Matching)
Reviewer_dVCfOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces GraphMorph, a novel framework for longitudinal image forecasting that leverages evolving anatomy graphs to predict regional and spatial changes beyond pixel-level differences. Extensive evaluations on brain MRI and knee radiograph datasets demonstrate that GraphMorph outperforms existing image-forecasting and direct-prediction baselines by effectively modeling structural anatomy dynamics.

Critical strengths

Formulates longitudinal forecasting around structured anatomical graphs rather than relying solely on noisy pixel-space or voxel-level differences, yielding significantly higher biological and structural interpretability.

Demonstrates robust cross-domain generalizability by successfully transferring from brain MRI data to knee radiograph joint-space progression tasks with superior performance metrics.

Incorporates rigorous anatomical request supervision and rigorous ablation studies, confirming the distinct contribution of cross-region attention and graph-based evolution over standard diffusion or direct-prediction baselines.

Critical weaknesses

The model relies heavily on pre-computed segmentation masks and robust initial registration pipelines, making its performance vulnerable to upstream segmentation errors or outliers in clinical deployment.

While regional volume correlation and structural alignment improve, standard image quality metrics like PSNR and SSIM fail to distinctly capture these progression improvements, suggesting a disconnect between perceptual fidelity and clinical anatomical accuracy.

The computational overhead and complexity introduced by the multi-stage pipeline, combining anatomical autoencoders, graph evolvers, and specialized renderers. This limit its straightforward adoption and scalability in real-time clinical workflows.

Questions for authors

See weakness

1 comment

We thank the reviewer for recognising the formulation of forecasting around anatomy graphs and the interpretability it gives. We answer the three concerns below.

Q1. Pre-computed masks and registration make performance vulnerable to upstream errors and outliers in clinical deployment

A: We measured how much upstream errors cost (Table R1). Our brain masks come from SynthSeg, a widely used off-the-shelf segmenter whose hippocampal Dice is 0.79 against 0.906 for a recent task-specific model (Sghirripa et al., 2025; Gong et al., 2026), so every reported result already includes realistic segmentation error.

Table R1 compares GraphMorph's result without and with each upstream error, three that occurred in our data and three that we introduced on purpose. For registration, an 8 mm placement offset lowers the core regional change correlation (RchgR_{\mathrm{chg}}) from 0.483 to 0.430. For outliers, keeping a subject flagged by our segmentation checks changes the result by 0.002. For segmentation error, keeping the knee visits where two segmenters disagree costs 0.007, a one-voxel shift of every region boundary lowers the core RchgR_{\mathrm{chg}} from 0.447 to 0.370, and a two-voxel shift lowers it to 0.200. GraphMorph stays ahead of every compared method with the three errors that occurred in our data and with the one-voxel shift (0.316 at most), and the two-voxel shift marks the limit of the error it tolerates.

Table R1. GraphMorph's regional change correlation without and with an upstream error. † Each region is eroded or dilated, with the direction drawn randomly. A one-voxel shift changes hippocampal volume by 43% (Dice 0.71 against the unshifted mask).

Data and regionsUpstream errorWithout the errorWith the errorChange
Brain, core8 mm placement offset (162 pairs)0.4830.430−0.053
Brain, 32 regionsFlagged outlier (one subject)0.3240.322−0.002
Knee, joint spaceSegmenters disagree (4.7% visits)0.2300.223−0.007
Brain, core1-voxel shift†0.4470.370−0.077
Brain, core2-voxel shift†0.4470.200−0.248
Brain, 29 regionsFastSurfer masks as input0.3550.285−0.070

Q2. PSNR and SSIM do not capture the progression gains, suggesting a disconnect between perceptual fidelity and anatomical accuracy

A:
Yes, and this disconnect is the problem the paper addresses. When only a small part of the image changes, image similarity rewards a forecast that stays close to the input, so copy-forward, which predicts no change, has the highest PSNR (29.89, Table 1). The revised Figure 5 shows this across all methods. PSNR and SSIM therefore cannot show progression gains, which is why we evaluate the change directly with regional metrics. GraphMorph also has the highest PSNR and SSIM among the forecasters on the brain (29.18 and 96.80), so its anatomical accuracy does not come at the cost of image fidelity.

Q3. The autoencoder, graph evolver and renderer pipeline limits adoption and scaling in real-time clinical workflows

A: The pipeline adds little cost at inference (Table R2). One forecast takes 0.406 s and 2.44 GB of memory on one GPU, 19× faster than ∆-LFM and 536× faster than Diffusion. GraphMorph runs a deterministic forward pass through a 2.05M-parameter model with no sampling, whereas most image comparators run 16 to 1,000 sampling steps. Only the two single-pass image networks are faster, CardiacAging (0.093 s) and Pix2Pix (0.127 s). At this speed, one GPU forecasts all 669 brain test pairs in under five minutes. A forecast is needed once per visit, for a follow-up that is months or years ahead, so sub-second inference is well within what a clinical workflow requires.

Table R2. Inference cost per brain forecast.

MethodSampling stepsTime per forecast (s)Peak GPU memory (GB)Parameters
GraphMorph1–12 evolver steps0.4062.442.05M
DiffuseMorph10.4255.451.08M
Pix2Pix10.1270.0816.65M
CardiacAging10.0931.0815.48M
ImageFlowNetODE solver4.460.3230.67M
MambaControl backbone160.7954.70260.40M
BrLP503.315.25803.83M
∆-LFM50 × 27.786.14574.01M
Noise-Free Flow30065.17.142.33M
Flow Matching30065.27.152.33M
Diffusion1,0002187.172.33M
Reviewer_VhN9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This manuscript propose decoupled approach for generating evolution of medical images ( brain MRI knee radiographs here ). The decoupling focus on learning the evolution of the anatomy underlying the image as graph latents separate from the rendering process. PSNR and regional correlation evaluation shows some evidence of improving over learning coupled progression directly.

Critical strengths
  • Decoupling the anatomy change underlying the image , and rendering process of the anatomy makes sense.

  • comparisons to recent and broad range of methods is conducted.

Critical weaknesses
  • The problem scope is small and conditions are demanding ( requiring accurate segmentation of the anatomy and strong encoders and decoders ). This makes ICLR not the proper venue for this manuscript, but more of MICCAI , or other medical imaging journals.

  • The generalization of the followed approach of learning a temporal graph model is questionable. Given that the training happen in three stages, any broken link results in catastrophic cascade failure of the final results.

  • The evaluation of using regional correlation is not standard in generative and reconstruction papers, even for temporal reconstruction. It seems tailored to the proposed method. What does 0.230 and 0.119 correlation means to me or to the community ? will the doctors decide to use the proposed method because its correlation is 0.311 instead of 0.272 with regional anatomy? Table 1 results are not infromative.

  • The presentation can be improved. The writing of the paper is LLM-looking, most sentences are very long with colon (:) , seemi colon (;) and em dashes (---) in the middle making it hard to understand. Most paragraphs are referencing the appendix, making the main paper not informative as is. The figures and plots in the paper are not informative either (se Figure 3 and Figure 4). Furthermore, the extensive AI use is clear for the entire paper, making reading hard. This is not a professional way of writing a manuscript.

Questions for authors

How does this generalizes to other modalities or when the anatomy segmentation is not available?

6 comments

We thank the reviewer for recognising that separating the change in anatomy from the rendering of the image makes sense, and that the comparison covers a recent and broad range of methods. We then answer each point below.

Q1. The scope is small and needs accurate segmentation and strong encoders and decoders. This makes ICLR not the proper venue for this manuscript, but more of MICCAI?

A: We thank the reviewer for this comment. We consider ICLR the right venue, for two reasons.

Longitudinal image forecasting is an established ML task with recent work at ML venues. ∆-LFM and CRONOS forecast follow-up 3D scans with flow matching (Chen et al., ICLR 2026; Disch et al., ICLR 2026), deep causal models generate counterfactual brain MRI under interventions on age and brain volume (De Sousa Ribeiro et al., ICML 2023; Melistas et al., NeurIPS 2024), and DDL-CXR generates a patient's follow-up chest X-ray from the previous one (Yao et al., NeurIPS 2024).

This paper also targets a failure mode that is not specific to medical images. When image changes affect only a small fraction of the image, image similarity rewards a forecast that predicts almost no change (see the copy-forward rows of Tables 1 and 3). We test it on Alzheimer's disease and osteoarthritis, but the same problem arises in any forecasting task where a small, structured change sits inside a mostly unchanged signal. Examples are land-cover change in satellite imagery and microstructure evolution in materials, which our grain-growth experiment tests (Appendix F).

GraphMorph does not need a state-of-the-art segmenter, a strong encoder or a strong decoder.

We deliberately use widely adopted, off-the-shelf tools rather than the most accurate task-specific ones. For brain MRI we use SynthSeg (Billot et al., 2023), released in 2021. On manually labelled hippocampi it reaches a Dice of 0.79 (Sghirripa et al., 2025), whereas a recent model trained for this task reaches 0.906 (Gong et al., 2026). For the knee, the masks were produced with UniverSeg (Butoi et al., ICCV 2023) without task-specific training. We also test how the quality of the input segmentation affects the result (Table R1). With every region boundary shifted by one voxel, GraphMorph still reads 0.370, above every compared method (0.316 at most).

Table R1. GraphMorph's regional change correlation without and with an upstream error. † Each region is eroded or dilated, with the direction drawn randomly. A one-voxel shift changes hippocampal volume by 43% (Dice 0.71 against the unshifted mask).

Data and regionsUpstream errorWithout the errorWith the errorChange
Brain, core8 mm placement offset (162 pairs)0.4830.430−0.053
Brain, 32 regionsFlagged segmentation outlier (one subject)0.3240.322−0.002
Knee, joint spaceTwo segmenters disagree (4.7% of visits)0.2300.223−0.007
Brain, core1-voxel boundary shift†0.4470.370−0.077
Brain, core2-voxel boundary shift†0.4470.200−0.248
Brain, 29 regionsFastSurfer masks as input (292 pairs)0.3550.285−0.070

We take the encoder and decoder to be the anatomy autoencoder (Sec. 3.1) and the renderer (Sec. 3.2). Neither is a demanding requirement, because neither is large or heavily trained. The whole deployed model has 2.05M parameters. The autoencoder trains on segmentation masks only, and the renderer is a pretrained DiffuseMorph registration network that stays frozen, with only a 131^3 convolution and a control branch trained to steer it.

These light components are sufficient, because they add little error (Table R2). Given the measured follow-up anatomy, the renderer reaches 0.703 on the brain, close to the ceiling of 0.770. With our forecast in its place the correlation is 0.447, so the remaining error comes from the forecast, not from the encoder or the renderer. The knee shows the same pattern (0.755 with the measured anatomy against 0.230 with the forecast).

Table R2. Regional change correlation at each stage of the pipeline.

SettingRegional change correlation, brain coreRegional change correlation, knee joint space
Measurement ceiling†0.7700.650
Autoencoder round trip0.9840.97
Oracle control (F)0.7030.755
GraphMorph0.4470.230

If the reviewer was referring to a different encoder or decoder, we would be glad to clarify that component as well.

AuthorsOpenReview ↗

Response Part II

Q2. Generalization is questionable. With three training stages, any broken link causes catastrophic cascade failure.

A: The approach generalizes across tasks, source cohorts, segmenters and input conditions. Across tasks, we apply our method to Alzheimer's disease on brain MRI, osteoarthritis on knee radiographs and simulated grain growth in materials, and it leads every image forecaster on all three (Tables 1 and 3, Appendix F). Across source cohorts, the brain lead holds within each of ADNI, AIBL and OASIS-3 (Table S25). Across segmenters, the lead over ∆-LFM holds when FastSurfer, which took no part in training, measures the result (0.328 against 0.291, Appendix L.5). Across input conditions, the lead over ∆-LFM holds for patients without a prior visit (0.238 against 0.179 over 32 regions, Table S7).

Multi-stage training is a mature and widely used design in image generation, for example latent diffusion (Rombach et al., CVPR 2022), ControlNet (Zhang et al., ICCV 2023) and BrLP (Puglisi et al., 2025). In our method each stage can be checked on its own, because each stage outputs an anatomy that we compare directly with the measured anatomy. A broken link can therefore be detected and corrected at its own stage, before the next stage is trained on it. We report this check for every stage (Table R2), and it locates most of the error in the forecast, not in the links between stages.

Q3. The evaluation of using regional correlation is not standard. Why regional correlation metrics? It seems tailored to the proposed method. Would doctors choose a method for 0.311 over 0.272 correlation?

A: Correlation between predicted and measured change is a common way to evaluate longitudinal forecasts (Bass et al., NeurIPS 2020; Lachinov et al., IEEE TMI 2024; Farki et al., ISBI 2026). We compute it for each anatomical region to tell how well a forecast separates patients who will change more from those who will change less, because PSNR and SSIM cannot tell whether a forecast captures the change.

A doctor would not choose a method on correlation alone, so the paper also reports what a clinician would check next. On the three core brain structures, GraphMorph's forecast volume change is off by 2.61 percentage points, against 3.24 for the second best method and 3.49 for predicting no change, so it reduces the no-change error by 25% against 7% (Table 1). It also has the lowest error relative to how much each patient actually changes (ΔRMAE\Delta_{\mathrm{RMAE}} 0.664 and ΔRegion\Delta_{\mathrm{Region}} 0.825, against 0.752 and 0.911 for the second best method). Its forecasts are also the closest to the real follow-up scan among the forecasters (PSNR 29.18, SSIM 96.80).

We have revised Table 1 so that its main finding is visible at a glance. The columns are now grouped into change metrics and image metrics, and the caption states the main finding.

Q4. LLM-looking writing, long sentences with colons, semicolons and em dashes, paragraphs that rely on the appendix, and uninformative Figures 3 and 4

A: We agree that the presentation needs to be clearer, and we revise it as follows. We have rewritten the main text and the appendix in short sentences, without semicolon and colon chains or em dashes. Each main-text paragraph now states its result with the number, and the appendix reference only points to the full table. The new analyses are collected in one place (Appendix L). The main text now also states the error decomposition by stage (Sec. 5). The captions of the main tables and of Figures 3 to 5 now state their finding, the metrics and reporting conventions are defined once (Sec. 4).

Q5. How does it generalize to other modalities, or when anatomy segmentation is unavailable?

A: GraphMorph applies to any modality in which the regions to forecast can be segmented, and it is retrained for each one. Retrained on 2D knee radiographs and on simulated grain growth, it leads every image forecaster, as it does on 3D brain MRI (Tables 1 and 3, Appendix F).

The method needs region masks, but not a task-specific segmenter. Our knee bone masks (Appendix J.2) were produced with UniverSeg (Butoi et al., ICCV 2023), which segments from a few labelled examples without task-specific training, and the knee lead holds when poorly segmented visits are kept (0.223 against 0.113, Appendix E.1). With no anatomical regions at all, a variant that divides the brain into a regular grid reads 0.079 against 0.322 (Appendix A.4), so the regions are what the method relies on.

thanks for the reply, by the points I raised above still apply.

Comments on the paper

1 comment
Area_Chair_YmY3OpenReview ↗

Please Participate in the Discussion Phase

Dear Reviewers,

The authors have posted their rebuttal and responses addressing your review comments and questions.

Please carefully read their responses (as well as the updated manuscript, if applicable) and actively engage in the reviewer-author discussion phase. If you have any remaining concerns or follow-up questions, please post them so the authors can address them before the discussion period closes.

Your timely participation and updated assessments are essential to helping us reach a fair and well-informed recommendation for this submission.

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.