- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper examines whether resting-state functional connectivity (FC) models that distinguish diagnostic groups can also track changes within individuals. The authors separate FC profiles into subject means and within-subject residuals, compare diagnostic and identity-related features, and test several representation-removal methods. On ADNI multi-visit data, diagnostic AUCs are 0.734 for full profiles, 0.728 for subject means, and 0.495 for residuals. Scan-level splitting also yields a much higher AUC than subject-level splitting (0.933 vs. 0.742).
Critical strengths
- Careful evaluation of data leakage. The subject-grouped cross-validation and permutation tests address an important issue in repeated-scan neuroimaging. The large gap between scan-level and subject-level AUCs (0.933 vs. 0.742) clearly shows how improper splitting can inflate diagnostic performance.
- Clear distinction between mathematical cancellation and longitudinal tracking. The paper recognizes that centering removes additive effects of subject-constant labels by construction, so chance-level residual classification alone cannot rule out longitudinal information. The controls support this distinction: centering improves head-motion prediction (0.577 to 0.708) and speech-emotion recognition (0.840 to 0.880), while reducing speaker-sex classification to chance (0.980 to 0.499).
Critical weaknesses
- Limited evidence for the longitudinal conclusion. The main tracking experiment uses only 57 ADNI participants, with high-dimensional FC slopes estimated from varying numbers of visits. The negative result (ρ = −0.186, 95% CI [−0.514, 0.046]) shows that this pipeline fails to predict MMSE change, but does not rule out longitudinal information detectable by other approaches. More importantly, Appendix E.3 reports that simulated temporal trends were not detected at the tested strengths, leaving the method's sensitivity to longitudinal changes unclear. The reported fivefold detection threshold concerns variance differences, not temporal trends. The structural MRI comparison does not fully address this issue either: while structural rates predict diagnosis (AUC = 0.807), they also fail to predict MMSE change (ρ = −0.084, n = 55; Table 13). This raises concerns about whether the evaluation can reliably detect cognitive decline. The longitudinal conclusions should therefore be more cautious.
- Unclear distinction between subject identity and disease-related information. Centering removes additive effects of subject-constant labels by construction, but this does not imply that disease-related longitudinal changes are absent. The seven removal methods also lack a consistent evaluation of identity suppression. For instance, pulling visits from the same subject together through contrastive learning may preserve identity information rather than remove it. Appendix E.3 further shows that residuals retain substantial subject-identification accuracy (0.245 vs. 1/61 chance). The single-visit subspace analyses cannot establish temporal stability, and the ADNI subspace-angle result is not significant (p = 0.312; Table 15). Finally, Proposition 4 assumes an unquantified bound on residual identity information, and its data-processing argument needs clarification since centering uses all visits rather than a single observation. These issues weaken the proposed interpretation of where disease information resides.
Questions for authors
- Reliability of longitudinal estimates. How reliable are the FC and MMSE slopes across different numbers of visits and follow-up durations? How sensitive are the results to age, sex, acquisition site, and motion? Also, why do structural MRI change rates predict diagnosis (AUC = 0.807) but fail to predict MMSE change?
- Sensitivity to temporal trends. Appendix E.3 reports that injected temporal trends were not detected at the tested strengths. What is the minimum temporal effect size the slope-based analysis can reliably detect? A simulation or power analysis accounting for uncertainty in individual slope estimates would be more informative than the Fisher-z threshold alone.
- Identity removal and theoretical assumptions. Can the authors measure identity predictability consistently before and after each of the seven removal methods? Are held-out subject means computed using all visits, including future observations relative to the prediction time? Finally, can the authors clarify the data-processing argument in Proposition 4 and justify the assumed bound on residual identity information?