- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper developed a graph clock conditioning on DNA sequences to improve predicting biological age. The proposed approach enables to jointly model multiple molecular features related to DNA methylation, including individual CpG sites, sequence-specific context, and CpG-CpG relationships. Through matched comparisons, the paper singled out the feature-specific contributions from sequence-specific context and CpG-CpG interactive relationship, while assessing extra gains by including local sequence context in the model.
Critical strengths
This paper demonstrates the added benefit for biological age prediction in the framework of graphic models from the use of embedding information extracted from local sequence-specific context and CpG relationships, along with the use of neural networks in prediction model formulation. The factorial design to evaluate graph contribution is well reasoned as this design allows to isolate the effect of graph-message-passing from the effect of DNA sequence construct. In this way, this paper attempted to provide quantitative evidence on the influence of both sources of information in the building of the prediction model.
Critical weaknesses
- The proposed model shows little improvements over simpler methods such as the Elastic Net Regularized Linear Model (ENReLM), despite its much greater complexity due to the invocation of overparametrization and nonlinearity. It also has shown higher run-to-run variability, which raises concerns about stability. Ensembling helps add extra information to potentially gain predictive power, but it is unclear whether these gains justify the additional costs of obtaining DNA sequence data and the required neural network computing in comparison to the classical ENReLM. Overall, the results reported in this paper do not convincingly show that the proposed method offers a meaningful advantage over existing epigenetic clocks.
- The proposed architecture combines sequence encoding, graph message passing, and an eight-layer MLP. It is unfortunate that the paper does not sufficiently justify, biologically, the need for such a complex combo of signals in the biological age prediction. The factorial ablations examine the contributions of sequence conditioning and graph propagation, but leave other architectural choices largely unexplored. The proposed approach is also harder to interpret biologically than simpler models, especially the ENReLM, which may limit its practical usefulness as seen by practitioners.
Questions for authors
- What’s the biological rationale behind the use of the DNA sequence for predicting biological age, when methylation data are already reported in the literature as primary molecular aging features? Aging, especially aging heterogeneity is heavily influenced by exposures to environmental and contextual factors, and there lacks clear biological basis that links the DNA sequence with such exposures.
- Why neural networks, particularly the MLP component in terms of overparametrization and nonlinearity, should improve your performance? ENReLM is known to be highly adaptive to local data, and the resulting sparse linear prediction model is also known to be robust and generalizable such as Horvath’s clock. It seems that the proposed neural network model does not really help for the prediction power improvement.
- There are hundreds of biological age clocks available in the literature, what’s the reason for picking the two benchmark clocks in the comparison as shown in Table 1? The presented comparison to the two chosen benchmark clocks is not really fair, because typically a transfer learning step is used to calibrate an existing clock to make it more adaptive to the local target data. This step is not considered in this paper, resulting in an unfair comparison.
7 comments
response
Question 2 — Why neural networks?
Reviewer concern. Why should an over-parameterised, non-linear model outperform a sparse linear clock?
Methylation changes non-linearly with age, rapidly in childhood and slowly in adulthood, which is why Horvath’s clock needs a logarithmic age transformation below age 20. A neural network learns such non-linear relations, and interactions among sites, directly from the data instead of through a fixed transformation, and over-parameterisation is controlled by dropout, weight decay and validation-based checkpoint selection. Our results support this: on the original split, the neural models outperform the strongest linear model we tested, an Elastic Net with Horvath’s age transformation, in all five folds (tuned MLP by 0.17 years, DeepMAge by 0.13 and our three-seed ensemble by 0.23). Our ensemble improves on it both for samples aged 20 or younger (0.68 vs 0.88) and for adults (4.15 vs 4.40).
Question 3 — Choice of clocks and transfer learning
Reviewer concern. Why Horvath and AltumAge, and why without local calibration?
Table 1 is not limited to two clocks: it compares our model with six models. Four are trained on our partitions, an Elastic Net, a sequence-agnostic MLP, ResnetAge and a sequence-agnostic PNA, and two are published clocks used as external references, Horvath and AltumAge. Together they cover the main families of chronological-age clocks: penalised linear (Elastic Net, Horvath), deep feed-forward (MLP, AltumAge), convolutional (ResnetAge) and graph-based (sequence-agnostic PNA). We chose models that predict chronological age, as ours does, and that represent distinct model families. Horvath’s clock is the most widely used linear clock, and AltumAge is a deep clock defined on exactly our 20,318 sites. Clocks with other targets, such as PhenoAge, GrimAge and DunedinPACE, are not like-for-like comparisons.
Following the reviewer’s suggestion to adapt the clocks to the local data, and to ensure a fair comparison, we retrained both published clocks on our data while keeping their original design, so that every model in the comparison is trained on our local training data. Both use the same training partitions, validation-based model selection and test set as our model, and retraining also removes any overlap between the published clocks’ training data and our test samples.
| MAE (years), original split | Retrained from scratch |
|---|---|
| Horvath’s clock | 3.949 |
| AltumAge | 3.588 |
| Ours | 3.405 |
The rebuttal addresses some of the concerns in the review reports, with a few major points remaining unresolved. Regarding the performance comparison against Elastic Net, the proposed approach demonstrates an advantage only when a three-seed ensemble model is fitted. Particularly, an improvement of 0.23 years (2-3 months) is not biologically significant. This improvement comes at a substantially higher cost, both in terms of the additional data requirements associated with sequential inputs and the computational resources needed to train and fit a complex neural network architecture. Regarding the responses to the interpretability concerns, the author provided a detailed discussion of the rationale behind the design choices for each module, as well as their discussion of the possibility of interpreting model predictions using standard attribution methods. The concern remains: the paper did not include a concrete example of applying attribution methods for prediction interpretability demonstration. Regarding the modeling of nonlinear relationships using neural networks, there lacks comparison against other classes of nonlinear models with simpler structures, such as Random Forests, XGBoost, or generalized additive models, to further support the benefits of choosing the neural network approach. Despite these clarifications above, the manuscript itself did not incorporate any changes. Given these remaining concerns, the original rating is adequate.
response
We thank the reviewer for the detailed, practice-oriented review and for recognizing the factorial design.
Weakness 1 — Improvement over Elastic Net, variability and cost
Reviewer concern. The gain over Elastic Net is small relative to the added complexity, runs are variable, and the cost may not be justified.
We added an Elastic Net with Horvath’s age transformation, the strongest linear model we tested (MAE 3.360 years on the original split). A single run of our model is on par with it (+0.05 years, not significant), whereas our three-seed ensemble improves on it by 0.23 years [0.07, 0.40] in all five folds, both for samples aged 20 or younger (0.68 vs 0.88) and for adults (4.15 vs 4.40). Ensembling also reduces the run-to-run standard deviation from 0.21 to 0.085 years. The sequence adds no data cost, because each probe’s 122-bp context is listed in the public Illumina manifest, and no inference cost, because the sequence features are fixed per site after training.
Weakness 2 — Architectural complexity and interpretability
Reviewer concern. The combination of sequence encoding, message passing and an eight-layer readout is not justified, and the model is hard to interpret.
As the reviewer notes, the model has three components, and the factorial ablations already isolate the contributions of two of them, sequence conditioning and graph propagation. Each component has a distinct role:
Interpretability. A sparse linear clock offers a list of sites and weights, but this list is unstable: Horvath's and Hannum's clocks share only six CpGs, and in our data the Elastic Net selects 1,731 to 3,147 sites per fold, only 776 of them in all five folds, although the training partitions overlap by three quarters. Our model can be interpreted at two levels. The sequence pathway shows that the clock relies on GC content and GC skew, which mark unmethylated CpG islands, and the contribution of each site to each prediction can be computed with standard attribution methods and summed over co-methylated neighbourhoods.
Question 1 — Biological rationale for DNA sequence
Reviewer concern. What is the biological rationale for using DNA sequence to predict biological age, when methylation is already reported as the primary molecular feature of ageing? Ageing, especially its heterogeneity, is heavily influenced by environmental and contextual exposures, and there is no clear biological basis linking DNA sequence to such exposures.
Sequence does not replace methylation as the ageing feature; it tells the model how to read each site's methylation. A site's local sequence determines its typical methylation level and how it tends to change: GC-skewed CpG-island promoters are protected from methylation, whereas CpGs flanked by A or T lose methylation with age. Our trained models learn this kind of information: their shared sequence signal tracks GC content and GC skew and reflects each site's typical methylation level.
This is useful even though sequence does not reflect exposures. In a methylation clock, exposures are visible only through the methylation changes they cause, and how a site responds depends on its sequence context: age- and exposure-related changes differ systematically between CpG islands and other regions. Sequence therefore provides the reference for reading these changes. The same measured value indicates a gain of methylation at a site whose sequence keeps it unmethylated, but a loss at a site whose sequence favours methylation.