- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes LT-MSGCN, a multi-hop signed graph convolutional network incorporating long-term dependency supervision for neurodevelopmental disorder classification, with a focus on Autism Spectrum Disorder (ASD) and Attention Deficit Hyperactivity Disorder (ADHD).
To address the limitations of conventional GCNs in modeling negative functional connectivity, long-range interactions between brain regions, and over-smoothing, the authors introduce a dual-branch architecture. A Transformer-based autoencoder (AE) learns long-term relation matrices between brain regions, while a multi-hop GCN branch constructs hop-specific adjacency matrices using APPNP and extracts graph representations through Signed GCN. The two branches are jointly optimized with a consistency loss between the learned relation matrices and multi-hop adjacency matrices. The model also incorporates hop-aware and region-aware feature fusion modules.
The authors evaluate the model using 10-fold cross-validation on ABIDE-I and ADHD-200, reporting classification accuracies of 71.53% and 70.20%, respectively. They further investigate biological interpretability through between-group statistical analyses of learned brain-region relationships.
Critical strengths
- The research problem is well motivated.
Modeling both positive and negative functional connectivity, together with long-range interactions between brain regions, is relevant to brain disorder classification. The authors identify limitations of conventional GCNs in local information aggregation and the lack of explicit guidance for multi-hop propagation, providing a reasonable motivation for the proposed approach.
- The proposed framework is relatively comprehensive.
The paper integrates multi-hop graph propagation, relation modeling, and attention-based feature fusion into a unified framework. It also includes comparative experiments, component ablations, and biological analyses on two disorder classification tasks, demonstrating potential practical relevance.
Critical weaknesses
W1. Limited Methodological Novelty and Insufficient Validation of the Core Contribution
The main components of the proposed method, including APPNP-based multi-hop propagation, Signed GCN, Transformer-based relation modeling, and hop-wise attention, are largely based on established techniques. The primary proposed contribution appears to be the use of Transformer AE to learn relation matrices and guide multi-hop propagation through a consistency loss. However, the novelty and necessity of this mechanism are not sufficiently established.
Table 2 shows that removing the AE branch reduces accuracy by only 0.88 and 0.19 percentage points on ABIDE-I and ADHD-200, respectively, whereas removing the multi-hop component leads to decreases of 5.92 and 5.75 percentage points. This suggests that much of the performance gain may come from the existing multi-hop propagation mechanism. Moreover, the paper does not independently ablate the guidance loss or disentangle the contributions of AE reconstruction and relation alignment, leaving the role of long-term dependency supervision insufficiently validated.
W2. Unclear Model Selection Protocol and Limited Performance Improvements
The paper adopts 10-fold cross-validation but does not specify whether a separate validation set is used or how the final model checkpoint is selected. Section 3.3 identifies the optimal number of hops based on classification performance without clarifying which data are used for hyperparameter selection. Consequently, potential test-set selection bias cannot be ruled out.
Despite introducing multiple architectural components, the performance improvements are limited and inconsistent. According to Table 1, the proposed model does not achieve state-of-the-art accuracy or AUC on ABIDE-I, while its accuracy improvement over the strongest competing baseline on ADHD-200 is only 0.60 percentage points. Given the model's complexity, these results do not establish a clear and consistent performance advantage. Furthermore, the absence of standard deviations, confidence intervals, or significance tests, together with insufficient clarification of baseline evaluation protocols, weakens the reliability of the reported comparisons.
W3. Mathematical Inconsistencies and Insufficient Justification of the Propagation Mechanism
Section 2.4 claims that if all elements of the original adjacency matrix lie within (-1, 1), the resulting multi-hop matrices A_i must also lie within this range. However, this claim is mathematically incorrect. For example, consider a three-node adjacency matrix with zero diagonal entries and all off-diagonal entries equal to 0.9. With α = 0.1, Equation (5) yields (A₂)₁₁ = 1.4122, which exceeds 1. This raises concerns about a potential scale mismatch between A_i and the tanh-bounded relation matrices R_i. Moreover, the paper does not adequately clarify whether the normalization assumptions and spectral properties underlying standard APPNP remain applicable to the proposed signed multi-hop propagation.
In addition, the Frobenius-norm loss in Equation (14) imposes a consistency constraint between R_i and A_i, but does not by itself demonstrate that the learned Transformer relations effectively guide multi-hop propagation. The paper also lacks direct theoretical or empirical evidence that the proposed mechanism alleviates over-smoothing. Therefore, the mathematical properties, direction of supervision, and actual contribution of the core propagation mechanism remain insufficiently justified.
W4. Inadequate Biological Validation and Insufficient Support for Interpretability Claims
Section 4 and Appendix C identify disease-associated connections through between-group t-tests on the learned relation matrices R_i. However, the paper does not clearly state whether multiple-comparison correction was applied or how potential confounding factors, such as age, sex, acquisition site, and head motion, were controlled. Moreover, since R_i is obtained from a supervised classification model, the lack of clarity regarding which samples were used for statistical testing raises concerns about potential circular statistical inference.
The authors further interpret the decreasing number of significant connections in higher-order relation matrices as evidence of more stable and disease-specific patterns. However, this interpretation is not supported by cross-fold stability analysis, independent validation, or an assessment of the identified connections' contributions to model predictions. Therefore, the current evidence is insufficient to establish these connections as reliable disease-specific biomarkers.
W5. Inconsistent Reporting and Insufficient Reproducibility Details
The paper contains several reporting inconsistencies and missing implementation details. The Contributions section reports classification accuracies of 70.2% and 71.8%, which differ from the 71.53% and 70.20% reported in the Abstract and Table 1. In addition, important methodological details remain unclear, including the handling of self-loops, constraints on APPNP parameters, aggregation of evaluation metrics across folds, and the mapping from AAL90 ROIs to Yeo7 functional networks. These omissions hinder reproducibility and independent verification of the reported results.
The presentation of biological findings also requires improvement. Figures 3 and 4 do not clearly distinguish statistical significance, learned relation weights, brain-region importance, and functional network assignments. The criteria used to identify and rank important brain regions are insufficiently explained, making the results difficult to interpret. These presentation issues further limit the clarity and reproducibility of the biological analyses.
Questions for authors
Q1. Model Selection and Validation
Could the authors clarify the model selection procedure within the 10-fold cross-validation framework? Was a separate validation set used within each fold? How were the final checkpoints, the optimal number of hops in Section 3.3, and other hyperparameters selected? Were the outer test folds used at any stage of model selection?
Q2. Contribution of Long-Term Guidance
Could the authors provide targeted ablations that independently remove the guidance loss, retain the AE reconstruction objective, or replace the Transformer relation matrices with simpler alternatives? What evidence demonstrates that R_i provides more effective guidance than the existing APPNP propagation matrices?
Q3. Fairness and Reliability of Experimental Comparisons
Were all baselines in Table 1 evaluated using consistent data splits, preprocessing settings, and training protocols? For baselines whose settings or results differ from their original publications, could the authors provide reimplementation details? Could the authors also report variability across runs or confidence intervals to establish the reliability of the relatively modest performance gains?
Q4. Mathematical and Implementation Details
Was the original adjacency matrix normalized before computing its higher-order powers? How is the numerical range of the multi-hop matrices controlled? Was the adjacency matrix symmetrized after per-node top-20% sparsification? How do gradients from Equation (14) propagate through the AE and APPNP branches?
Q5. Reliability of Biological Validation
Which samples and model checkpoints were used to obtain the R_i matrices for the biological analyses? Were multiple-comparison correction and confound control performed? Could the authors provide cross-fold stability analyses or independent validation to support the reliability and disease specificity of the identified connections?
I have carefully revisited my original review and confirmed that the concerns raised remain valid. As the authors have not provided a response addressing these concerns, I will maintain my original rating.