- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Summary: The paper introduces BrainGenFlow, a two-stage framework for forecasting month-resolved trajectories of 145 regional brain volumes from a single baseline scan plus clinical covariates. Stage 1 fits one deep kernel Gaussian process (DKGP) per region of interest (ROI) on the training cohort, and its posterior mean is used to impute dense monthly trajectories over each subject’s observed follow-up. Stage 2 trains an autoregressive flow-matching model that predicts month-to-month residuals rather than absolute next states. It is conditioned on the current state, a GRU embedding of the generated history, and a baseline embedding applied via FiLM. Experiments on nine longitudinal cohorts report improved forecasting accuracy over statistical and neural baselines, generalization under leave-one-study-out evaluation, and associations between predicted atrophy rates and clinical progression groups.
Critical strengths
- The problem is clinically meaningful and well-scoped. Forecasting multi-regional trajectories from a single baseline addresses a useful setting with sparse, irregular longitudinal observations.
- The two-stage framework design is well-motivated. Temporal imputation provides the generative model with dense monthly trajectories to train on, enabling long-horizon forecasting from a single baseline scan at inference time.
- The evaluation is well-designed. The paper includes subject-level cross-validation, seven leave-one-study-out evaluations, persistence and statistical baselines, objective-matched deterministic residual predictors, and component ablations.
Critical weaknesses
-
The improvement over the strongest baseline is not significant. The deterministic Transformer (0.2240) lies within the 95% CI of BrainGenFlow ([0.2079, 0.2255], Table 1). The headline improvements (26.8% over RNN-AD and 69.9% over TFM-ODE) are measured against baselines that perform worse than the no-change persistence baseline (0.2684). This suggests that these baselines may be under-tuned or poorly adapted. The “state-of-the-art” claim is not supported.
-
A direct DKGP forecasting baseline is missing. Because the DKGP takes [x0; c; t] as input (Eq. 4), the trained teacher can in principle forecast a held-out subject directly from the baseline scan and covariates. However, this baseline-only DKGP forecast is not reported. Since Stage 2 is described as learning to reproduce the same dynamics “in an amortized form” (Section 4.1), the teacher is the most natural reference point. Without this comparison, it is unclear whether Stage 2 improves accuracy or only amortizes the GP.
-
The decade-long forecasting claim needs more evidence.
Questions for authors
-
What is the MAE of the DKGP teacher used directly as a single-baseline forecaster on held-out subjects, under the same protocol? Please also report per-fold results for the deterministic Transformer (Table 3 currently omits it) and a paired comparison with BrainGenFlow under comparable capacity and tuning budgets.
-
Please report horizon-stratified MAE (as in Table 10) for persistence, GAM, Det-Transformer, and the endpoint-pair flow, as well as LOSO results for at least the strongest baselines. Please also report an error metric on change from baseline, or a skill score relative to persistence.
-
(Minor) Several abbreviations are used before they are defined, or are never defined:
2 comments
I thank the authors for responding to my concerns. I will discuss this with other reviewers and make a final decision. I dont have further question
Response to review
We thank the reviewer for the careful review, and for recognizing the clinically meaningful, well-scoped problem and the well-motivated two-stage design. All neural results use the corrected evaluation (general comment).
Significance and the state-of-the-art claim
We agree with the reviewer. The Det-Transformer shares our framework (DKGP supervision, residual target) and replaces only the flow head, with roughly 3× the velocity network's parameters and the same training settings and validation split (at matched capacity, the MLP head reaches 0.2047). Per fold results are: Det-Transformer: 0.2019, 0.1995, 0.1997, 0.2074, 0.1989, BrainGenFlow: 0.2122, 0.2076, 0.2005, 0.2137, 0.2028. The Det-Transformer is slightly more accurate under five-fold (fold-paired, +0.0059 [+0.0013, +0.0104]), and the two are tied under LOSO (Table W2). We report the two as comparable in point accuracy, withdraw the 26.8%/69.9% comparisons, and narrow the "state-of-the-art" claim to published methods that forecast all 145 ROIs jointly from a single scan (TFM, RNN-AD, endpoint-pair flow; Table Z1 in the response to Reviewer zJda), all of which BrainGenFlow outperforms. Flow matching adds a calibrated predictive distribution: its 95% intervals cover 95.0–97.3% at every horizon (94.5–97.1% under LOSO), vs. 82.9% at 60–120 months for MC dropout on the Det-Transformer (Table M1 in the response to Reviewer Mswv).
On baselines below no-change: scan noise accounts for ~90% of no-change squared error in years 1–3, where most visits fall. RNN-AD and Deep Regression predict absolute volumes and stay below no-change even with DKGP supervision (0.2591, 0.3091; Table D1 in the response to Reviewer DeHU); TFM's next-state prediction compounds over the rollout (0.608 at best).
Direct DKGP baseline
We agree that the teacher is the natural reference. With the same conditioning, the DKGP (145 per-ROI models) reaches 0.1971, more accurate than BrainGenFlow overall (fold-paired, +0.0103 [+0.0057, +0.0149]). BrainGenFlow is more accurate in the first year (0.166 vs. 0.170); the DKGP leads from two years on. The DKGP uses exact inference conditioned on the full training cohort, which it must keep at prediction time. Stage 2 distills it rather than competing with it, and comes within about 5% of its MAE. What it adds: one joint model, no training data at inference, and uncertainty that grows with horizon. The DKGP's predictive SD is nearly constant (0.36–0.37), so it over-covers early (98.6% at 0–12 months) and is nominal only at 60–120 months. We will add it as a baseline and revise the description of Stage 2.
Decade-long forecasting
Every horizon bin has held-out subjects from all five folds: 1,231 subjects and 2,076 visits at 60-120 months (Table D2 in the response to Reviewer DeHU), and 1,101 subjects under LOSO. There, BrainGenFlow improves on no-change by 20.9%, with 97.3% coverage (97.1% LOSO). We will claim 120-month generation validated within observed follow-up, not uniformly accurate decade-long prediction.
Requested analysis
Horizon-stratified MAE with per-bin counts is in Table D2 (response to Reviewer DeHU).
Table W1. Skill relative to no-change (1 − MAE/MAE_no-change), five-fold, computed per fold and averaged. Endpoint-pair flow: one seed.
Since |(p − x₀) − (y − x₀)| = |p − y|, MAE is already a change-from-baseline error; Table W1 gives the skill score.
Table W2. LOSO MAE by held-out cohort (Det-Transformer: three seeds). Per cohort: 95% subject-bootstrap CI; mean: 95% t-interval across studies.
BrainGenFlow and the Det-Transformer are tied (paired over subjects, +0.0004 [−0.0000, +0.0009]; averaged over studies, +0.0016 [−0.0031, +0.0063]). BrainGenFlow improves on no-change for 79.0% of subjects (paired over subjects, −0.0308 [−0.0324, −0.0292]). GAM and DKGP LOSO results will be added in the revision, and Figure 3a updated.
Abbreviations. We will define all abbreviations at first use.