acceptodds
Under review as a conference paper at ICLR 2027

Longitudinal Medical World Models for Patient-Specific Disease Dynamics

Abstract

Patients with brain tumours are followed with serial MRI at irregular intervals, and each scan informs whether treatment continues or changes. Anticipating how the tumour will evolve before the next scan could support these decisions. This paper shows that such forecasts are best built from disease dynamics shared across patients, adapted to each patient’s treatment and history. We present Longitudi- nal Medical World Models (LMWM). LMWM learns one latent transition from many patients and adapts it with two additive signals: a treatment response field that encodes where and when each recorded course acts, and a history map of the deviations that recur in the patient’s own past forecasts. Without these signals, LMWM is exactly the shared model, so the effect of treatment and history can be measured directly. Evaluating such forecasts also needs care: image similar- ity favours unchanged predictions, and copying the current scan already attains a follow-up lesion Dice of 72.9%. We therefore score the predicted change with a signed change F1 (cF1). On patient timelines built from twelve public brain tu- mour collections, which align serial MRI with treatment records, LMWM raises cF1 from 31.6% for the shared model to 45.1%, above twelve image-generation baselines, and is the only method whose lesion overlap exceeds that of copying the input. These results point to a simple view of patient-specific forecasting: learn disease dynamics once across patients, adapt them to each individual, and judge the forecast by the change it predicts.

Primary area:
datasets and benchmarks
Keywords:
Longitudinal Modelling, World Model, Disease Dynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8paNOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

The paper proposes latent flow matching controlled by treatment adaptation and patient history, as well as time conditioning to produce the next state. Additionally, they collected multiple public datasets and are expected to share a homogenous test space for world models in GBM and Metastasis.

Critical strengths

Their dataset is huge and the comparison methods comprehensive. They actually show an improvement over trivial baselines like just keeping the last image and blurring the lesion, which is not the case for most baselines. To the best of my knowledge it is a novel conditioning approach well suited to the problem.

Critical weaknesses
  • It is unclear why the authors combined metastasis and GBM datasets and the value of the dataset:

Both exhibit vastly different growth trajectories and cannot be distinguished from the MRI[1]. In some cases pathology reports are necessary to distinguish them. Given that the growth is not predictable from the images, it at least explains why the performance of all methods is really bad and often worse than just using the last baseline image. Even growth dice of their method is 0.18. One thing to note is that the registration between the cases does not look pixel accurate in Fig 6. Hence pixel wise metrics might be misleading and at least perceptual ones like LPIPS are preferable, where their method performs worse than baseline. In the figure there is SSIM and DICE and in the Table is PSNR, LPIPS and Change Agreement. The dice is reported in the appendix. Why not report the SSIM as well? In this regard, in Figure 6, is it actually predicting any growth? To me all the models are either copying the lesion or just fading it away and not accurately predicting the necrosis. Given these qualitative results the clinical impact of this work is questionable. In particular their result in FIg S2 row 1, where the lesion obviously was resected and the prediction that the lesion just stopped taking contrast agent makes no sense. At least the other methods just keep the lesion, which is a more likely outcome than what the methods of the authors provide.

  • Flow matching is not well motivated:

From the text it looks like this is just an iterative refinement as there is no noise component in the flow matching. This makes the entire system deterministic. However, given that the results are not accurate, does it make sense to just provide a single future and not at least a likelihood of the next state. Also there is no motivation given, no ablation to other methods, even though this is their main workhorse to transition from one state to the next. The citations in this section are also generic flow matching papers and do not tell me if this is a good method for state transitions.

Minor comments:

  • The related work sections could be significantly improved by highlighting the difference to existing work. It is unclear how this work is different from preceding works and why those baseline methods were chosen. The related work section just lists some works, but does not define which gap the authors are filling with their work.
  • The AI Use statement does not actually follow the guidelines

[1] Ocaña-Tienda, Beatriz, and Victor M. Perez-Garcia. "Mathematical modeling of brain metastases growth and response to therapies: A review." Mathematical Biosciences 373 (2024): 109207.

Questions for authors

Clarifications:

  • Can you elaborate on the dataset, that you propose? Are the additional sequences relevant? Did a clinician look over any of the artifacts that you produced? Why does it make sense to combine metastasis and GBM? Does your system perform better or worse after including these extra datasets?

  • Do you use noisy inputs in the flow matching or do you just deterministically refine the prediction and why does it make sense to do it like this?

Future work:

  • Can you elaborate, why the flow matching is superior to other methods, like RNNs or single step prediction models like Transformers or even just a CNN?
  • Can you elaborate on the eligibility of the baselines. They all use the same conditioning, however it is unclear if this conditioning is causing issues. it would be prudent if they used the same conditioning as you do to isolate the contribution of each component or at least use their conditioning on your method.
12 comments
AuthorsOpenReview ↗

Response I

We thank the reviewer for recognising the scale of the dataset, the breadth of the comparison and the novelty of the conditioning, and that LMWM improves over copying and blurring the input where most baselines do not. We answer the questions below.

Q1 Metastasis and GBM grow differently and can look alike on MRI. Why combine them, and what is the dataset's value?

Reply: We appreciate the clinical viewpoint and thank you for the reference. Our goal is a standard benchmark for tumour progression, and each disease needs a large cohort to support analysis of its progression. Glioma and brain metastases are the two brain tumours that are close in setting, serial MRI under treatment, and large enough for this analysis, so we harmonize them in one resource that places scans, lesion masks and treatment records on each patient's timeline.

We agree that the two diseases grow differently, so the benchmark keeps them as separate cohorts and never mixes them in a forecasting task. In clinical use the disease type is known from the first diagnosis, and our model forecasts progression rather than distinguishing tumour types.

The two cohorts do show the difference as Ocaña-Tienda and Perez-Garcia (2024) describe, with a median changed region of 2.3% of the brain in glioma against 0.5% in metastases (Table S6). In this paper, forecasting is trained and evaluated on glioma only (Section 2.1). Metastasis scans enter only the autoencoder training, because their treatment records are scattered, and all methods that use an autoencoder share the same one trained on the same data.

Q2 Growth Dice is 0.18 and methods are often worse than the last scan; Figure 6 registration is not pixel accurate, so should LPIPS, where LMWM loses to the baseline, be preferred?

Reply: Thank you for the insight. Predicting tumour growth from images is hard but feasible, as shown by deep learning on pre-treatment MRI that predicts where glioblastoma will later progress (Tran et al., npj Digital Medicine 2025), and by patient-specific growth models calibrated on MRI that predict tumour infiltration (Lipková et al., IEEE TMI 2019) and post-radiotherapy tumour size (Rockne et al., Physics in Medicine and Biology 2010). In our test, LMWM ranks first among all methods at telling which tumours will grow, with growth AUC 76.2% against 50% for chance (Table S1).

Core growth Dice is low because it scores only new tumour more than two voxels outside the baseline tumour (Table S3), a small region. Copying the last scan scores 0%, and LMWM's 18.3% is the highest of all methods (Table S1).

The registration results were checked and verified by clinicians, and cases with clear registration errors were removed. The registration cannot be perfect and comes with many problems, such as acquisition differences, the tumour condition, thick slices, truncated coverage, resampling distortion and inter-visit offsets, as stated in Appendix D.7 and Figure S4. These problems can cause pixel differences, which also make the task more difficult.

Image similarity is misleading for this task, as the paper states (Abstract), so we rank methods by change metrics rather than by pixel similarity. LPIPS measures perceptual appearance rather than structure (Zhang et al., CVPR 2018). Copying the input gets the best LPIPS while predicting no change (Table S1), which shows that LPIPS is not sensitive to tumour progression. The change metrics proposed in the paper are therefore preferred for measuring tumour change.

The low scores of current methods come from both the difficulty of the task and the limited research on it, and this is why we propose a benchmark to support further research on tumour progression forecasting.

AuthorsOpenReview ↗

Response II

Q3 Figures show SSIM and Dice, while the table shows PSNR, LPIPS and change agreement. Why not report SSIM too?

Reply: Thank you for the suggestion. We left the 3D tumour SSIM out of Table 1 because it saturates at 0.985–0.990 for every method and at 0.987–0.995 across horizons, since the zeroed background enters the average (Appendix B.6). Following your suggestion, Table 1 now reports it for every method, also listed in Table R1.

Table R1. 3D tumour SSIM of every method.

MethodSSIM ↑ (masked volume)SSIM ↑ (averaged over TT)
Copy input0.9887 ± 0.01100.5167 ± 0.2762
Δ-LFM0.9890 ± 0.00980.4796 ± 0.2150
MambaControl0.9876 ± 0.01190.4183 ± 0.1746
ImageFlowNet0.9895 ± 0.01010.5273 ± 0.2501
GlioPREDICT0.9867 ± 0.01230.4495 ± 0.2093
CardiacAging0.9886 ± 0.01070.5191 ± 0.2581
CounterSynth0.9881 ± 0.01130.5068 ± 0.2689
DiffuseMorph0.9886 ± 0.01080.5127 ± 0.2650
MeWM0.9875 ± 0.01190.4817 ± 0.2362
BrLP0.9866 ± 0.01330.4587 ± 0.2252
TADM0.9876 ± 0.01130.4688 ± 0.2460
SADM0.9846 ± 0.01150.3138 ± 0.1605
TaDiff0.9862 ± 0.01060.3793 ± 0.2041
LMWM°0.9895 ± 0.00980.4854 ± 0.2476
LMWM†0.9873 ± 0.01170.5332 ± 0.2588
LMWM0.9901 ± 0.01030.5967 ± 0.2195

On the masked volume, every method falls between 0.9846 and 0.9901, while cF1 spans 6.9% to 45.1% (Table 1), so this SSIM separates the methods little. LMWM has the highest value, 0.9901. Averaged over the tumour region TT only, SSIM separates the methods more (0.3138 to 0.5967), and LMWM is again the highest.

Q4 Does Figure 6 predict any growth? In Figure S2 row 1 the lesion was resected, yet LMWM shows it losing contrast, while other methods keep the lesion.

Reply:
Figure 6 (top) and Figure S2 (bottom) show tumours that shrink. LMWM is the only method whose forecast shrinks the tumour, as the observed change (the residual ground truth) shows, although by less than observed, while the other methods either blur the tumour or change its intensity.

Figure 6 (bottom) and Figure S2 (top) show resection cases. The other methods keep the lesion unchanged or over-blur it, whereas LMWM tries to remove only the tumour region, although the predicted cavity is still imprecise. The extent of the resection is known only from the follow-up scan, so no method receives it as an input.

Across the test set, LMWM also forecasts growth best, with growth AUC 76.2% (Table S1), and it has the highest cF1 inside necrosis (52.7%) and inside the resection cavity (46.3%) (Table R2). We will add more cases to supplement Figures 6 and S2.

On clinical impact, the paper shows that current methods mostly copy or blur the lesion, as you observe, and a blurred copy beats ten of the twelve baselines on cF1 (Table 1). We read this as the reason to develop the task with a benchmark that measures change. LMWM moves beyond copying, with growth AUC 76.2% against 50.0% for the copy (Table S1).

Table R2. cF1 (%) inside each lesion class.

MethodWhole tumourNecrosisEnhancingOedemaCavity
Copy input0.00.00.00.00.0
Δ-LFM40.646.548.139.840.3
MambaControl41.446.747.241.042.0
ImageFlowNet28.129.130.127.827.1
GlioPREDICT27.031.132.526.927.9
CardiacAging21.322.924.720.922.8
CounterSynth19.021.522.019.020.0
DiffuseMorph6.96.17.86.58.5
MeWM16.218.520.216.118.6
BrLP15.818.521.514.119.5
TADM13.415.116.812.115.1
SADM17.518.323.715.318.9
TaDiff19.823.824.219.718.6
LMWM45.152.753.644.946.3

LMWM has the highest cF1 in every lesion class, including necrosis (52.7% against 46.7% for MambaControl) and the resection cavity (46.3% against 42.0%). So its change forecasts hold in each tissue class, including the resected tissue of Figure S2 row 1.

AuthorsOpenReview ↗

Response III

each tissue class, including the resected tissue of Figure S2 row 1.

Q5 Flow matching has no noise and is deterministic refinement. Why a single future with no likelihood, and why no motivation or ablation against other transition models?

Reply: Thank you for the question. The transition is deterministic by design. Flow matching does not require noise as its input. It learns a mapping between two distributions, as rectified flow transports between two arbitrary distributions (Liu et al., ICLR 2023), and data-to-data bridges such as I²SB (Liu et al., ICML 2023) learn a flow directly between two data states.

We start the flow from the current state because most of the brain does not change between visits (median changed region 2.3%, Table S6), so the flow only has to learn the change. Table R3 tests this choice and the transition itself. It adds noise to the input ziz_i, and it replaces the flow with the same transformer or a 3D CNN that regresses the next latent directly, under our conditioning and under the shared conditioning of the baselines.

LMWM gives a single forecast for a given treatment course, and a different course gives a different forecast, because the treatment field is an input. We agree that this is a limitation, and our next work will extend the transition to output several possible futures (e.g., stable disease or progression) for the same patient and treatment course, each with its likelihood.

On the citations, the paper already cites and discusses flow-based state transitions for longitudinal imaging, Δ-LFM (Chen et al., 2026) and ImageFlowNet (Liu et al., 2025a), in Appendix G. The revision adds Temporal Flow Matching (Disch et al., 2025), which also models only the change between scans, to Section 2.3. We also move the related work from the appendix into the main text, so that these differences are easier to find.

Table R3. Transition ablation. Components not listed follow LMWM.

ModelConditioningTransitionPSNR ↑LPIPS ↓SSIM ↑cF1 ↑cPrec ↑cRec ↑ρ ↑
LMWMoursflow w/o noise19.383.660.990145.163.839.349.7
LMWMoursflow w/ noise19.044.170.990535.667.929.540.1
transformeroursdirect regression18.963.920.989742.869.037.243.2
transformershareddirect regression17.924.810.989028.173.320.635.8
3D CNNoursdirect regression18.774.520.988937.464.533.937.1
3D CNNshareddirect regression17.584.860.987827.972.422.734.6

The flow from ziz_i without noise gives the best change agreement. Adding noise lowers cF1 from 45.1% to 35.6% and ρ from 49.7% to 40.1%, which fits our motivation, since the flow then has to rebuild the unchanged brain as well as the change. Direct regression also falls below the flow, with cF1 42.8% for the same transformer and 37.4% for a 3D CNN. Our conditioning improves every transition, raising cF1 from 28.1% to 42.8% for the transformer and from 27.9% to 37.4% for the 3D CNN, so the treatment field and history add to any transition.

Q6 Related work lists papers without stating the differences, the gap, or why these baselines were chosen

Reply: Thank you, we agree. We added a new paragraph to the related work section, shown below.

Existing methods feed treatment or patient history to one network as extra inputs. LMWM instead learns one shared transition that predicts disease change from the current latent, and designs the treatment and history corrections from what each input carries. Surgery, radiotherapy and chemotherapy act at different places and on different time scales, so treatment enters as a response field with a spatial code and a learned time kernel for each class (Section 2.4). A patient's history is informative where the forecast errors of completed intervals recur in the same direction, so history enters as a map that keeps these consistent errors and cancels conflicting ones (Section 2.5). This adapts the forecast to each patient's care and course without per-patient fitting, and shows what each correction contributes (cF1 from 31.6% to 37.2% and 45.1%, Table 1).

The gap we fill is that existing generators do not show how treatment and patient history change shared disease dynamics, or what each contributes to the forecast. We chose the baselines to cover the three lines of work this gap touches. Δ-LFM, ImageFlowNet, TADM, SADM, BrLP and MambaControl forecast a follow-up scan from the current scan and elapsed time, so they test the shared transition. TaDiff, GlioPREDICT and MeWM condition on treatment, so they test the treatment field. CardiacAging, CounterSynth and DiffuseMorph test whether generic generation or registration forecasts change.

Reviewer_6ZPLOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Longitudinal Medical World Models for forecasting brain-tumor MRI evolution from current imaging, treatment information, and patient history. Its main contribution is a structured adaptation of frozen shared dynamics: a spatial and temporal treatment response field modifies the transition, while a gated history branch incorporates recurring past forecast errors and clinical records without per-patient parameter fitting. The paper also includes twelve public imaging collections and introduces change-focused evaluation. On held-out glioma patients, the full model achieves the highest reported change F1 among twelve baselines, although the results establish observational forecasting improvements rather than causal treatment prediction or clinical readiness.

Critical strengths
  1. The model offers a clear way to personalize a shared disease forecast. Figure 3 separates treatment information from recurring errors in the patient’s earlier forecasts, while preserving the shared model when these inputs are absent. Table 1 shows that both additions help: cF1 increases from 31.6% to 37.2% with treatment and to 45.1% with history. This provides useful evidence that individual treatment and disease history contain information beyond the current scan. Besides, all figures are very beautifully made! These are some of the best figures I ever see in this line of work.
  2. The evaluation exposes a misleading shortcut in longitudinal forecasting. Table S1 shows that simply copying the current scan achieves 72.9% follow-up Dice, exceeding every baseline. Table 1 also shows that blurring the input outperforms ten baselines on cF1. Including these simple references alongside growth, regression, and volume-change measures makes the comparison more informative than image similarity alone.
Critical weaknesses
  1. The experiments do not clearly show whether the proposed history mechanism is better than simpler ways of using the same information. In Table 1, the full model adds the previous latent change, the consistent residual map, and clinical text together. The improvement therefore does not establish how much the residual map contributes, or whether directly providing earlier scans and records would work similarly. Comparisons using the same history inputs with simpler conditioning, together with separate removal of each history component, would help establish the value of the proposed design.
  2. The clinical meaning of the reported improvement remains uncertain. Although Table S1 shows better overall scores, the full model still has a 67% false-growth rate and only 18.3% core growth Dice. These measurements also depend on an automated segmenter whose agreement with stored test annotations is approximately 68.1% Dice (Appendix B.3), so some apparent forecasting errors may come from the evaluator.
Questions for authors
  1. Could you address weakness 1?
  2. Could you address weakness 2?
6 comments
AuthorsOpenReview ↗

Response Part I

We thank the reviewer for recognising that LMWM personalizes a shared forecast while keeping the shared model intact, and that the evaluation exposes the copy shortcut, and for the kind words on the figures. We answer the questions below.

Q1 Table 1 adds previous latent change, residual map and clinical text together. How much does the map contribute, and would directly providing earlier scans and records work as well?

Reply: Thank you for the suggestion. We build the map from forecast errors that recur across a patient's completed intervals, because they mark what the shared model misses for this patient (Section 3.5). Table R1 tests this design in two ways, starting every row from the same treated model (LMWM†). First, it removes each history input in turn. Second, it replaces the map with simpler uses of the same history, an unweighted mean of the errors, or the earlier scans and record text given directly without the map.

Table R1. Removing each history input, and replacing the map with simpler uses of the same history.

ModelHistory inputsPSNR ↑LPIPS ↓SSIM ↑cF1 ↑cPrec ↑cRec ↑ρ ↑
LMWMchange + map + text19.383.660.990145.163.839.349.7
LMWMmap + text18.724.110.989643.364.038.046.5
LMWMchange + text19.074.080.989342.159.136.445.8
LMWMchange + map19.313.750.989944.863.437.348.4
LMWMchange + mean error + text19.014.030.988242.660.236.245.0
LMWMchange + earlier scans + text19.873.390.991030.776.120.846.2
LMWM†none18.294.940.987337.263.335.443.7

Change is the latent change between the two preceding scans. Map is the consistent map of Eq. 5. Text is the treatment record text. Mean error is the plain average of the same forecast errors, without duration weighting or the coherence gate. Earlier scans are up to two earlier latents, given directly to the history branch.

Each history input helps, and the map helps most. Removing the map lowers cF1 from 45.1% to 42.1%, more than removing the latent change (43.3%) or the text (44.8%). The map also beats simpler uses of the same history, the plain mean of the errors (42.6%) and the earlier scans given directly (30.7%).

AuthorsOpenReview ↗

Response Part II

Q2 The full model has 67% false growth and 18.3% core growth Dice, read by a segmenter at 68.1% Dice. What is the clinical meaning, and how much error comes from the evaluator?

Reply: Thank you for the question. Clinically, the forecast tells which patients' tumours will grow, and LMWM does this best among all methods (growth AUC 76.2% against 71.5% for Δ-LFM, Table S1). The 67% false growth depends on how readily each method predicts growth, from 38% for LMWM° to 97% for MambaControl, so Table R2 sets every method to the same 50% false growth and compares how much true growth each one finds. The 18.3% core growth Dice is low because core growth covers only new tumour more than two voxels outside the baseline tumour (Table S3). This region is small, so every method scores low, and LMWM's 18.3% is the highest.

The evaluator scores every method in the same way, so the ranking of methods holds. Table R3 measures its share of the error. It re-scores the forecasts against expert masks on MU, RHUH, UCSD and SAILOR (Table S5), and reports the evaluator's own limit, the score it reaches when it segments the acquired follow-up.

Table R2. Growth AUC, growth recall at 50% false growth, and core-growth AUC (%).

MethodGrowth AUC ↑Recall ↑ at 50% false growthCore-growth AUC ↑
Δ-LFM71.576.2 ± 4.966.2
MambaControl64.767.0 ± 4.959.7
ImageFlowNet47.565.6 ± 6.558.9
GlioPREDICT59.659.0 ± 5.853.5
CardiacAging43.662.3 ± 8.354.6
CounterSynth46.167.2 ± 4.963.3
DiffuseMorph45.754.7 ± 6.753.5
MeWM50.659.4 ± 8.255.3
BrLP59.456.6 ± 7.956.2
TADM53.159.4 ± 6.057.0
SADM53.773.6 ± 8.566.2
TaDiff56.973.6 ± 14.362.9
LMWM°64.971.7 ± 4.762.9
LMWM†68.877.4 ± 5.166.7
LMWM76.282.1 ± 4.670.2

At the same 50% false growth, LMWM finds 82.1% of true growth, against 76.2% for Δ-LFM, the strongest baseline. It also has the highest core-growth AUC, 70.2%. Treatment and history each add recall, from 71.7% for LMWM° to 77.4% for LMWM† and 82.1% for LMWM. So at a matched false-growth rate, LMWM still detects the most growth.

Table R3. False growth and core growth Dice (%), scored against evaluator masks and against expert masks. nn: cases in each column.

MethodFalse growth ↓ (evaluator)False growth ↓ (expert)Growth Dice ↑ (evaluator)Growth Dice ↑ (expert)
n=66n=6n=212n=66
Δ-LFM7610014.514.1
MambaControl9710010.710.9
LMWM676718.318.7
Evaluator limit–50–31.6

Against expert masks, core growth Dice stays close to the evaluator score for every method (LMWM 18.7% against 18.3%), and the ranking is unchanged. The evaluator limits what any forecast can reach, because even the acquired follow-up, segmented by the evaluator, scores only 31.6% against the expert masks. LMWM reaches 18.7%, which is 59% of this limit. The expert false-growth column covers 6 cases, and on these the evaluator alone already gives 50% false growth.

Reviewer_6ZPLOpenReview ↗

Response to rebuttal

I appreciate the authors' efforts and the detailed rebuttal. My concerns are sufficiently addressed. I have increased my rating to 4.

Reviewer_JHjjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a generative model that synthesizes future MRI scans from a patient's current scan and the disease and treatment history of that patient. The authors argue that existing methods do not utilize the longitudinal data available in medical imaging datasets, despite their importance for predicting disease progression. To address this limitation, they propose a model that incorporates the historical data about disease and treatment into the generative process itself. Their results demonstrate that leveraging this additional context improves the accuracy of future MRI prediction.

Critical strengths
  • The main contribution of the paper is to show that, by incorporating a patient's historical data, beyond just the previous scan, predictions of future states can be significantly improved. This historical data combines both the treatment history and previous imaging modalities into a unified conditioning for a generative model that predicts future imaging states. Previous methods seem to only rely on per-patient attributes as conditioning, which does not provide enough information to make useful future predictions.

  • To train and evaluate the model, the authors put together an extensive dataset of longitudinal brain tumor data. This is the main building block of the proposed method, which relies on the existence of such per-patient historical data to train the generative model. The authors plan to release the dataset (through code that reproduces the data from the sources), which could be helpful for future works in the field.

Critical weaknesses
  • Some design choices are not well established, making the overall model design vague:

    • The authors propose training the generative model that predicts future states by using a flow-matching objective that learns to synthesize the residual between the current and future MRI voxels. This does not seem common in the literature and raises the question of the advantage of modeling the residual rather than the future voxels themselves, and whether it is integral to the improvements over the baselines shown in the paper.

    • The authors use a segmentation and an adversarial loss as additional signals to train the generative model. Again, their contributions are not isolated by ablations, making it hard to understand what their role is. This also raises a question of whether these losses are crucial for the accurate reconstruction of future states, since the authors predict residuals instead of voxels.

    Currently, the only thing the authors ablate in the main paper is the conditioning, which, although it is their main focus, is supported by a lot more moving parts that they need to explain.

  • The paper presents itself as a longitudinal medical world model, but the evaluations are limited to MRI reconstruction and segmentation quality. By bringing up world models, the expectations are that the proposed method would be evaluated on (long-horizon) dynamics of disease progression. One such evaluation would be to synthesize radiology reports from future states (using an LLM/VLM), decide future interventions, and compare with the real patient progression and treatment. The role of a world model is exactly to simulate these trajectories in time, but the current evaluation scheme relies on single-future, image quality and tumor change metrics.

  • Omitting the related works section from the main text hurts the paper. For a reader who is moderately familiar with the literature, it makes it hard to situate the proposed method in the context of existing works in the field. A related works section is not only a literature review, but mostly serves as a way to clarify what other research has tried before, and exemplifies why the contributions made by the authors are necessary to advance the state-of-the-art.

Questions for authors
  • For the missing data, you mention that you used a modality-synthesis model (line 323). Are you able to measure their contribution somehow? Are you also performing quality control for the synthesized data?

  • Why is the change agreement (cRec, cPrec, cF1) the right metric to use for this task, and what is the issue with simple Dice scores? Could the authors provide some intuition on why they introduce new metrics to evaluate the models? Are these metrics closer to how a radiologist would evaluate these images?

4 comments
AuthorsOpenReview ↗

Response Part I

We thank the reviewer for recognising that patient history beyond the previous scan improves the forecast, and that the dataset, released as code that rebuilds it from the public sources, can support future work. We answer the questions below.

Q1 The flow learns the residual between current and future voxels, which does not seem common. What is the advantage over predicting the future state, and is it integral to the gains over baselines?

Reply: Thank you for the question. Forecasting the change from the current state is common in longitudinal imaging, in both image and latent space (table below).

Predicts the follow-up statePredicts the change from the current state
Image spaceSADM, TaDiff, CardiacAgingTADM
Latent spaceBrLP, GlioPREDICTΔ-LFM, ImageFlowNet, LMWM

LMWM uses the residual in two places. (1) The flow models the change from the current latent to the follow-up latent (Eq. 2). (2) The readout adds the decoded change to the current scan inside the predicted lesion mask (Eq. 7).

The advantage is that the model learns only what changes. Changed regions cover a median 2.3% of brain voxels in glioma (Table S6), and tissue outside the lesion stays equal to the acquired scan.

Table R1. Residual or direct prediction, for the flow target and the readout.

ModelVariantOutputPSNR ↑LPIPS ↓SSIM ↑cF1 ↑cPrec ↑cRec ↑ρ ↑
LMWMresidual targetresidual readout19.383.660.990145.163.839.349.7
LMWMresidual targetdirect readout19.023.430.989339.157.633.442.2
LMWMdirect targetdirect readout19.113.570.989537.459.431.640.8
LMWMdirect targetresidual readout19.243.850.990040.361.134.843.4

Both uses of the residual help. Replacing the residual target or the residual readout lowers cF1 from 45.1% to 40.3% or 39.1%, and replacing both lowers it to 37.4%, below Δ-LFM (40.6%) and MambaControl (41.4%).

Q2 The segmentation and adversarial losses have no ablations. What does each do, and are they needed when the model predicts residuals?

Reply: Thank you for the question. Table R2 ablates the two losses on LMWM°, and Table R3 removes the auxiliary losses from LMWM.

Table R2. Removing the adversarial or Dice loss from LMWM°.

ModelVariantPSNR ↑LPIPS ↓SSIM ↑cF1 ↑cPrec ↑cRec ↑ρ ↑
LMWM°full loss17.635.410.989531.667.422.939.4
LMWM°no adversarial loss17.745.090.990530.464.222.638.3
LMWM°no dice loss17.915.740.987927.652.819.134.2

Both losses help. Removing the Dice loss lowers cF1 from 31.6% to 27.6% and ρ from 39.4% to 34.2%, and removing the adversarial loss lowers cF1 to 30.4% and ρ to 38.3%.

Table R3. Removing the auxiliary losses, the soft-cF1 loss and the asymmetric lesion loss, from LMWM.

ModelVariantPSNR ↑LPIPS ↓SSIM ↑cF1 ↑cPrec ↑cRec ↑ρ ↑
LMWMfull loss19.383.660.990145.163.839.349.7
LMWMno auxiliary loss19.822.430.990839.875.629.043.4

The auxiliary losses also help. Removing them lowers cF1 from 45.1% to 39.8% and ρ from 49.7% to 43.4%, at a cost of 0.44 dB in PSNR.

43.4%, at a cost of 0.44 dB in PSNR.

Q3 A "world model" is claimed, but evaluation is single-future image and tumour metrics, with no long-horizon dynamics and no simulated reports or interventions compared with the real course

Reply: Thank you for the suggestion of a broader world-model evaluation with reports and treatment advice. We agree that this is what a world model is targeting, and it is the direction we plan to take next.

This work focuses on how treatment changes disease progression on imaging. We follow the definition and evaluation of action-conditioned world models in robotics and medicine, which predict the future state under a given action (Ha & Schmidhuber, 2018; Hafner et al., 2025; Yang et al., 2025b).

For long-horizon dynamics, Table R4 reports LMWM forecasts to different future intervals ahead, each scored against the patient's acquired scan at that time.

Table R4. LMWM by forecast horizon. nn: test cases.

HorizonnPSNR ↑LPIPS ↓SSIM ↑cF1 ↑cPrec ↑cRec ↑ρ ↑
Overall27819.383.660.990145.163.839.349.7
0–0.25 years19519.193.590.989445.662.940.750.8
0.25–0.5 years7320.113.410.992043.564.535.845.4
0.5–1 year517.736.450.986836.565.627.845.2
1–2 years217.698.520.991567.698.254.471.2
≥2 years317.836.740.995353.375.841.578.3
AuthorsOpenReview ↗

Response Part II

Q4 Related work is missing from the main text, so readers cannot place the method or see why it is needed

Reply: Thank you for the advice. We moved the related work from Appendix G into the main text, after the contributions.

We also added a closing paragraph that compares LMWM directly with the closest methods. The new paragraph is below.

Existing methods feed treatment or patient history to one network as extra inputs. LMWM instead learns one shared transition that predicts disease change from the current latent, and designs the treatment and history corrections from what each input carries. We observe that surgery, radiotherapy and chemotherapy act at different places and on different time scales, so treatment enters as a response field with a spatial code and a learned time kernel for each class (Section 2.4). A patient's history is informative where the forecast errors of completed intervals recur in the same direction, so history enters as a map that keeps these consistent errors and cancels conflicting ones (Section 2.5). This adapts the forecast to each patient's care and course without per-patient fitting, and shows what each correction contributes (cF1 from 31.6% to 37.2% and 45.1%, Table 1).

Q5 The modality-synthesis model (line 323). Can you measure its contribution, and do you run QC on synthesized data?

Reply: Thank you for the question. The synthesis model fills a single missing sequence so that a visit stays usable. It completes 22% of glioma visits (574 of 2,613), almost all of them LUMIERE visits whose T1ce is synthesized (Table S5). Every visit passes the quality control of Appendix D.6, and synthesized T1ce volumes pass the same T1ce checks as acquired ones.

We measure the contribution by inference in two ways. Table R5 splits the existing results into test cases whose sequences are all acquired and cases with a synthesized sequence. Table R6 replaces the acquired T1ce with its synthesized version on test cases with all four sequences acquired, and reports the gap between the two images and the change in the LMWM forecast.

Table R5. LMWM on test cases with all sequences acquired and with a synthesized sequence.

ModelCasesnPSNR ↑LPIPS ↓SSIM ↑cF1 ↑cPrec ↑cRec ↑ρ ↑
LMWMall27819.383.660.990145.163.839.349.7
LMWMacquired only16819.593.970.990942.660.939.847.8
LMWMsynthesized11019.063.180.988948.968.242.952.7

Table R6. LMWM with the acquired or the synthesized T1ce, on cases with all four sequences acquired. T1ce PSNR compares the two T1ce images.

T1ce inputT1ce PSNRPSNR ↑LPIPS ↓SSIM ↑cF1 ↑cPrec ↑cRec ↑ρ ↑
acquired–19.593.970.990942.660.939.847.8
synthesized17.9317.396.850.989238.544.946.345.8

With the synthesized T1ce, LMWM still forecasts the change, with ρ 45.8% against 47.8% for the acquired T1ce and higher cRec (46.3% against 39.8%).

Q6 Why are cRec, cPrec and cF1 the right metrics, what is wrong with Dice, and are they closer to how a radiologist reads images?

Reply: Thank you for the question. The task is to forecast how the tumour changes, and the change agreement scores (cPrec, cRec, cF1) measure that change directly. They compare the predicted intensity change with the observed change inside the tumour region, so copying the current scan scores 0 and a perfect forecast scores 1 (Appendix B).

Dice has two issues here. It rewards leaving the input unchanged, since copying the current scan reaches follow-up Dice 72.9%, above all twelve baselines (Table S1). It also needs a segmenter between the image and the score, which adds its own error (68.1% Dice against the annotations, Appendix B.3), whereas cF1 is computed on the predicted image itself.

A radiologist compares the follow-up with the prior scan and judges where the lesion grew or shrank, and cF1 likewise scores the signed change from the current scan. The growth and regression scores in Table S1 give the lesion-level reading beside it.

Thank you for your detailed responses.

I believe that the paper has substantially improved with the additional evaluations and the inclusion of the related work in the main text.

Given that most of the Weaknesses and Questions I raised in my initial review have been addressed, I will be raising my score to a 3.

Thank you for your positive feedback and for taking the time to reassess our work. We sincerely appreciate your recognition of the improvements and your willingness to raise the score. Your suggestions are insightful, and we will incorporate them into the revised version.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.