- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces DevelopmentODE, a structured Neural ODE for predicting a child’s future brain functional connectivity. It represents connectivity in a PCA latent space and combines a population developmental trajectory with individual deviation dynamics. Experiments on a longitudinal cohort report modest improvements over some baselines that include direct predictors, sequence models, and previous Neural ODE methods.
Critical strengths
-
The proposed method seems to be well-motivated. Separating population-level development from individual deviations, while allowing dynamics to vary with age, is a reasonable inductive bias that could be beneficial given the longditudal forcasting task the model is trying to solve.
-
The sensitivity analysis in section 4.6 show that the proposed method is reasonably robust to the different hyperparameters and moving parts within the model. These results suggest that the proposed method is relatively robust to these choices and we could expect it to work similarly no matter the choice we make.
Critical weaknesses
-
The improvement of the proposed method in the headline experiment (table 1) over the baselines is modest. The amount of improvement over the best baseline is relatively small, ranging around 1% ~ 5% across different metrics. Due to this it is difficult to know if the supposed improvement of the proposed method is statstically significant, and even if so, the improvement may be too incremental for this work to be considered a practically useful contribution for the given task.
-
The predictive contribution of the complementary-space projection, which is the geometric structure the paper presents as its central contribution, is not clearly isolated from the experiments provided in the paper. The “w/o Canal” ablation removes the population drift and the projection together, so the reported 1.2% MSE degradation cannot establish the projection’s specific contribution. Appendix C.6 provides a useful sensitivity analysis, but its learned values (0.15–0.20) retain substantial attenuation of the population-aligned component. The small performance changes therefore show robustness to a modest relaxation, rather than establish whether projection improves prediction. A matched variant retaining the canal drift and stage modulation but fixing , thereby removing the state/output projections, would directly test the predictive value of the proposed geometric constraint.
Questions for authors
-
Given that the proposed method show relatively small improvements agasint the baselines for table 1, some within the standard deviation margin, could the authors perform statistical tests to see if the improvements are statstically significant? For example, the authors may perform paired significance tests to see if the proposed method is better than the best baseline method.
-
The task has two trivial solutions (which could act as a performance floor for prediction models), where the prediction is (1) identical to the previous timepoint (), or (2) simply the population average for all subjects. For these two cases, what is the performance?
6 comments
W2. The contribution of complementary-space geometry
We appreciate your observation that the original w/o Canal ablation jointly removes the population developmental reference and its associated geometric constraints. Following your suggestion, we performed a strictly matched fixed α = 1 experiment, retaining the population canal drift, shared deviation field, and ordered age-dependent adaptation while removing both complementary-space projections.
Table 2. Matched sensitivity analysis of the geometric constraint.
| Configuration | α | All MSE (×100) | Adjacent MSE (×100) |
|---|---|---|---|
| Strict complementary | 0 | 2.9403 ± 0.0326 | 2.9451 ± 0.0262 |
| Learnable aligned | Learned | 2.9390 ± 0.0321 | 2.9428 ± 0.0266 |
| Fully aligned | 1 | 2.9395 ± 0.0323 | 2.9444 ± 0.0260 |
Together with the learnable α experiment in the Appendix, the results demonstrate that forecasting performance remains highly stable across substantially different geometric constraints. Moving from strict complementary dynamics (α = 0) to fully retained population-aligned dynamics (α = 1) changes all-forward MSE by only a little bit. This finding further clarifies that the principal value of complementary-space geometry is not an isolated reduction in prediction error, but the explicit organization and interpretability it provides for developmental dynamics.
We highlight two contributions of this geometric formulation.
-
Explicit organization of population and individual dynamics. The population canal defines a local developmental direction through its tangent. Complementary projection provides an explicit mechanism for separating population-aligned progression from deviation motion. This constrains the degrees of freedom available to the individual deviation field and reduces its ability to freely re-explain population-level progression. Consequently, the model incorporates an explicit population–individual organization. Importantly, the α-relaxation experiment demonstrates that this organization can accommodate different assumptions about population-aligned individual motion without substantially affecting forecasting accuracy.
-
Interpretable and empirically analyzable developmental variation. The geometric formulation also enables direct examination of developmental quantities that would otherwise remain entangled within an unconstrained vector field. In the Appendix, we analyze the population canal velocity, individual deviation magnitude, and the population-aligned and complementary components of observed longitudinal changes. For example, the population-aligned fraction of observed displacement varies substantially across developmental intervals, from 53.0% for the neonatal-to-1-year transition to 13.8% for the 6-to-10-year transition.
These analyses demonstrate how the proposed geometry provides a common reference for characterizing age-dependent population progression and individual variation. This modeling principle is consistent with scientific machine learning approaches that incorporate domain-motivated structure into flexible differential-equation models [4], and is biologically motivated by developmental canalization [5].
The structural ablations further support the broader formulation:
- Removing the population-referenced canal formulation increases MSE by 1.22%.
- Removing ordered age-dependent adaptation increases MSE by 1.97%.
We revised the manuscript to distinguish the role of complementary geometry as a structural inductive bias from the isolated predictive effect of strict orthogonality, and to make the corresponding analyses more prominent.
Q2. Identity and population-average prediction baselines
We thank the reviewer for suggesting these important reference baselines. We evaluated the simple alternatives:
- Identity: directly carries forward the observed source FC, Xi(tb)=Xi(ta).
- Population average predicts the mean FC computed from the population average for all subjects.
Table 3. Comparison with Identity and population-mean baselines.
| Comparator | MSE reduction by Ours | Fold t-test p | Subject t-test p | Permutation p | Bootstrap 95% CI |
|---|---|---|---|---|---|
| Identity | 49.79% | 7.15e-9 | 2.32e-301 | 5.00e-6 | [0.028734, 0.029581] |
| population average | 3.73% | 8.22e-5 | 3.78e-100 | 4.00e-6 | [0.0010675, 0.0012088] |
The reported p-values are Holm-corrected; the Identity results use the expanded five-comparator family, whereas the population-mean results use the original four-comparator family. Bootstrap intervals report unscaled paired MSE reductions and are conditional on the evaluated checkpoints. DevelopmentODE substantially outperforms Identity and the population average.
This demonstrates that simply retaining the previous FC observation or population average is insufficient for multi-year developmental forecasting. The population average is a stronger baseline than Identity, consistent with the substantial population-level developmental regularity observed in longitudinal neuroimaging studies [1, 2].
We hope these clarifications and additional findings address your concerns. Please let us know if you have any further questions.
References
[1] Modeling Sparse Longitudinal Data in Early Neurodevelopment.
[2] Longitudinal Developmental Trajectories of Functional Connectivity Reveal Regional Distribution of Distinct Age Effects in Infancy.
[3] A Decade of Test-Retest Reliability of Functional Connectivity: A Systematic Review and Meta-Analysis.
[4] Universal Differential Equations for Scientific Machine Learning.
[5] Waddington's Canalization Revisited: Developmental Stability and Evolution.
Dear Reviewer Bxwh,
We thank you for the detailed and constructive feedback. We have conducted additional experiments addressing the statistical reliability of our forecasting improvements, the contribution of complementary-space geometry, and the performance of simple forecasting baselines.
We agree that improvements over competitive forecasting baselines should be supported by rigorous statistical evidence. We therefore conducted three paired significance tests, together with bootstrap confidence intervals, against two of the strongest learning-based baselines, Time-MLP and MoE-NODE. We additionally examined the consistency of improvements across subjects and prediction horizons.
Table 1. Statistical significance against competitive learning-based baselines.
All comparisons use matched held-out predictions. Fold-level paired t-tests use five matched folds. Subject-level tests aggregate prediction-pair errors, while permutation tests use subject-level paired sign flips. P-values are Holm-corrected over the original four-comparator family. Confidence intervals represent unscaled MSE reductions and are obtained through fold-stratified subject-clustered bootstrap resampling. All three statistical tests confirm significant improvements over both baselines, with bootstrap confidence intervals entirely above zero.
Beyond statistical significance, three additional findings support the relevance of these improvements.
The gains are broadly distributed across subjects. DevelopmentODE achieves lower subject-level MSE than Time-MLP and MoE-NODE for 94.53% and 93.23% of held-out subjects, respectively. Thus, the improvements are not driven by a small number of subjects or prediction pairs.
The improvements extend across forecasting horizons. Relative to Time-MLP, DevelopmentODE reduces MSE by 2.92% on short-term and 3.57% on long-term predictions. Relative to MoE-NODE, the improvements are 4.67% and 1.37%, respectively. All four comparisons are significant under Holm-corrected fold-level paired tests (p < 0.001).
The task addresses a scientifically important and data-limited problem. Longitudinal neuroimaging studies have documented the difficulty of modeling developmental trajectories from sparse observations [1], heterogeneous age-dependent FC changes [2], and measurement variability in functional connectivity [3].
These characteristics motivate approaches that combine broadly supported population developmental information with limited individual longitudinal supervision. DevelopmentODE explicitly organizes population progression, shared deviation dynamics, and age-dependent adaptation within a continuous-time framework.