- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper propose RelAge-GNN, a multi-relational graph neural network framework for DNA methylation-based age prediction, which construct three complementary graphs to model relationships among CpG sites, capturing co-methylation patterns, genomic co-localization (i.e., whether two CpG sites are located on the same chromosome), and gene-level associations (i.e., whether two CpG sites are located within the same gene). In this paper, the model was trained on a dataset containing more than 3,000 samples; subsequently, several cohorts of varying sizes (each comprising hundreds of individuals) were selected to validate the model’s predictive error—the association between AA (age acceleration) positivity and three different diseases. However, the corresponding experiments did not support whether the new method he designed—namely, the additional subgraph structure—offered any obvious benefits.
Critical strengths
This paper proposes a multi-relationship graph neural network framework called RelAge-GNN for age prediction based on DNA methylation. This paper proposes an interesting research hypothesis: whether designing graph structures to capture gene-level associations and genomic associations of CpG sites can help improve the predictive capabilities of the aging clock and the prediction of health outcome endpoints.
Critical weaknesses
I could discuss this paper from two perspectives, but I believe there are some fundamental issues with both of them.
-
From a biological perspective, the first problem is that the sample size is too small. Some contemporary studies have utilised larger sample sizes; many recent epigenetic clock studies have already included over 10,000 samples (NCAE-Age 2023 -13,647 samples , GPage-2023 &iTARGET -2024 -11,910 samples, AltumAge-2022-13,505 samples, DeepStrataAge 2026- 29,167 samples ... ...). Furthermore, the experimental scope is limited. This paper examined only three diseases. Compared with other relevant studies, it covers a very small number of diseases and health outcomes, and does not assess mortality. The findings for these three diseases lack sufficient support from statistical significance analyses. The assessment of disease is primarily based on whether age acceleration is greater than zero. The study did not fully examine the relationship between continuous AA values and disease incidence through adjusted regression analysis. This means that the observed differences may reflect actual age differences between the different groups. For example, the higher proportion of positive AA results in the elderly group may have been mistakenly interpreted as a disease-related effect. The paper failed to adequately rule out this confounding factor. Therefore, I believe that the experiments can not support the proposed biomarker demonstrate any biomedical value. Furthermore, the authors did not examine whether additional intra-gene and intra-chromosomal graphs could enhance the health-related value of AA. If these graphs were intended to construct better biomarkers of ageing, the authors should have compared the impact of the resulting AA measurements on disease and health outcomes. The current experiments do not substantiate such added value.
-
Algorithmic contribution and experimental design From an algorithmic perspective, the novelty is very limited. The experiments do not establish that the two additional designs intended to capture CpG interactions—whether sites belong to the same gene or lie on the same chromosome—provide useful additional information or a clear performance benefit. First, the model’s prediction error is worse than that of an existing model in the paper’s own comparison. Second, the ablation study does not establish that the complete model is the best configuration and the extra benefit of using all three graphs remains unclear. The baseline selection is also too narrow. Many relevant methods already have open-source implementations. Biolearn -2024 & ComputAgeBench -2025 contains numerous aging clocks (https://bio-learn.github.io/clocks.html , https://github.com/ComputationalAgingLab/ComputAge/tree/pre-release/computage/models_library/clocks ) that could be considered as baselines, but the paper does not compare against many of them. There are also several recent methods that are not adequately discussed or evaluated. Biolearn provides access to additional cohorts, while other studies already use much larger sample collections. The paper does not sufficiently justify its choices of datasets and baselines. The health-outcome experiments have the problems described above: they do not adequately establish a disease-specific effect, lack statistical support for the main improvements, and do not demonstrate useful additional information. The motivation for the method also appears to rest on an incomplete understanding of previous work. CpG measurements may enter a model as a vector, but the model can still learn interactions between CpG sites. Existing methods already do this. The authors also motivate their approach by referring to the risk of overfitting in deep learning. However, their own method is still a deep-learning model. Adding a more complex architecture does not automatically solve overfitting, improve interpretability, or turn the model into a white-box model.
Therefore, the experimental design does not support the claim put forward in this paper. Moreover, given the significant drawbacks in the experimental design, it falls far short of the publication requirements of ICLR.
Btw,the introduction discusses related work; however, it does not cite many of these works, citing only one piece of related work.
Questions for authors
Firstly, What criteria did the author use to select the baseline? Why were other compatible methods excluded Biolearn -2024 & ComputAgeBench -2025 ? (https://bio-learn.github.io/clocks.html , https://github.com/ComputationalAgingLab/ComputAge/tree/pre-release/computage/models_library/clocks )
Secondly, why did the author choose such a small sample size? Are these few cohorts sufficiently representative? What is the rationale for not comparing this study with other contemporaneous studies that have a larger total sample size?
Furthermore, in the author’s experimental design, why was only ‘age acceleration greater than 0’ assessed, rather than evaluating continuous predictive power—that is, the true ability to predict disease?
Do the age distributions differ across the various AA groups? Does the association with the disease persist after controlling for age, age-related prediction bias and cohort factors?
Given that existing methods are already capable of handling interactions between CPG points (even with vector inputs, the model architecture can handle this), What exactly does the three-graph structure add? Which comparisons and ablation studies demonstrate this incremental benefit?
3 comments
Firstly, thank you for your thorough response; it appears you have carried out a great deal of supplementary experimentation, increasing the sample size and broadening the range of diseases covered in the paper, amongst other things. Of course, I am not certain whether this forms part of your future work or has already been completed.
If you have already completed this work, you may upload a new PDF, and I will consider awarding you additional marks at my discretion. To clarify, I would like to explain why I awarded you one mark. For this particular conference, I believe that to meet at least one of the ICLR acceptance criteria—which can be met in two respects—the paper should either demonstrate scientific value, that is, the observation of a new phenomenon, on a par with or exceeding that of top-tier journals in the field of ageing (excluding Nature Aging and Nature Communications I would not set the bar that high), or that you have made a significant methodological advance—in which case you would need to validate the performance of your method, whether in terms of speed, interpretability, or a substantial improvement in performance or accuracy, etc. As this is a top-tier conference in the field of computer science, I applied very high standards when reviewing your paper, However, at the same time, I would also like to point out that, as things stand, if you have completed the supplementary work you mentioned and discovered some interesting new phenomena that offer fresh insights to the community, I would be willing to raise your score. This is mainly because your core methodology will not change any further, so from the perspective of a methods paper, it does not meet the requirements of ICLR; however, if you wish to present it from the perspective of providing insights or scientific value, you must offer some interesting and broadly applicable insights within your extended results.
Thank you for clarifying your concerns and for your willingness to reconsider the paper based on additional evidence. We apologize that our previous response did not clearly distinguish planned revisions from completed work. We have not yet completed the additional experiments or obtained new results to report. In particular, expansion of the sample collection and disease coverage remains planned rather than completed. We are currently working on the supplementary evaluation and plan to proceed as follows:
Optimizing model performance. We plan to continue optimizing the model architecture to improve performance so that it can outperform all baselines as well as G1.
Continuous and adjusted AA analysis. We will evaluate healthy-reference-calibrated AA rather than relying primarily on AA>0. The analyses will examine age distributions and adjust for age, sex where applicable, and other available covariates, reporting effect sizes and confidence intervals. These will be interpreted as cross-sectional disease associations, not evidence of early detection or future disease incidence.
Health-related value of the additional graphs. Under the same evaluation protocol, we will compare G1-only, G1+G2, G1+G3, and the full model across repeated training seeds. We will assess both age-prediction errors and adjusted disease associations, directly testing whether adding the chromosome and gene relations provides information beyond G1.
We understand that completing additional experiments alone would not establish scientific value. The central question is whether relation-specific modeling reveals a consistent, independently reproducible pattern. We appreciate the opportunity to address your concerns.
W1. Limited sample size, insufficient disease coverage, and insufficient biological validation
We agree that the current sample size and the evaluation of only three diseases are insufficient to establish broad biomedical utility. This study investigates whether explicit CpG relationships can support age prediction, but this architectural motivation does not by itself validate a biological aging biomarker. We will clarify the dataset inclusion criteria and representativeness, extend independent validation where compatible data are available, and explicitly acknowledge that mortality was not assessed. For disease analyses, we will move beyond AA positivity to continuous, healthy-reference-calibrated AA, using age-matched controls or covariate-adjusted models, and report effect sizes and confidence intervals. Given the small cohort size, the osteoporosis-related findings will remain exploratory. We will also compare the full model with G1-only and relationship-removal variants on the same adjusted disease analyses to determine whether the additional graphs contribute health-related information.
W2. Limited methodological novelty and insufficient experimental support
We acknowledge that the proposed contribution is an architectural integration of relationship-specific branches and adaptive fusion, rather than a new message-passing algorithm. The current results do not establish consistent superiority: AltumAge achieves lower MAE and MSE, and G1 alone also has lower MAE than the full model. However, our intended contribution is relationship-specific modeling. Biological age itself does not perfectly coincide with chronological age, so once the error is small enough, comparisons between models cannot be judged on this alone, because a perfect model should not be able to accurately predict chronological age in the first place; rather, it should accurately predict biological age. This is why we present age acceleration, which demonstrates the model's real utility, namely early disease detection. This is a practical task rather than an abstract error value, and it better demonstrates the model's capability, where we do indeed have an advantage. We will correct the corresponding statements and strengthen the evaluation by including compatible clocks from Biolearn and ComputAgeBench, as well as feature-matched non-graph models, merged-graph models, and uniform or fixed-weight fusion. We also agree that vector-based nonlinear models can learn CpG interactions; the intended distinction of this paper lies in the explicit use of relational priors, and their incremental benefit requires empirical validation.
Q1. Baseline selection criteria
The original baselines covered linear, deep feedforward, and graph-based aging clocks, but this coverage is not comprehensive. We will include additional compatible methods from Biolearn and ComputAgeBench, clearly specify requirements such as tissue, platform, CpG coverage, and prediction target, and clearly distinguish evaluation with published weights from retraining.
Q2. Sample size and cohort representativeness
The sample size reflects the public blood methylation cohorts assembled in this study; we have not established that it is sufficiently representative. We will clarify the inclusion criteria, report demographic and platform coverage, and seek additional independent cohorts. Comparisons with larger studies must account for differences in data composition and evaluation protocols.
Q3. Why evaluate AA positivity rather than continuous predictions?
AA positivity was intended as a descriptive metric, but it discards information and cannot establish disease predictive capability. We will evaluate continuous, calibrated AA through adjusted disease-state models, reporting effect sizes, confidence intervals, and discriminative ability. Cross-sectional data do not support claims about future disease incidence.
Q4. Age differences and persistence after adjustment
The current results cannot establish whether disease associations persist after controlling for age, prediction bias, and cohort effects. We will report age distributions for each group and conduct healthy-reference calibration and covariate-adjusted analyses. Any remaining disease associations will be stated only if these analyses support them.
Q5. Incremental benefit of the three-graph structure
We agree that vector-based nonlinear models can learn CpG interactions. The three graphs introduce explicit relational constraints and separated relationship-specific representations, but their incremental benefit remains insufficiently demonstrated. We will compare feature-matched non-graph models, merged-graph models, and fixed or uniform fusion, combined with repeated graph ablations, to isolate the value of this design.