acceptodds
Under review as a conference paper at ICLR 2027

Gene2Brain-AE: Self-Supervised Representation Learning for Genetically-Informed Patterns of Brain Atrophy

Abstract

Characterizing heterogeneity in neurodegenerative diseases from imaging alone is limited, as similar patterns of brain change can arise from distinct underlying mechanisms. Integrating genetic information offers a way to disentangle these processes, but genetic effects are indirect, partially mediated, and entangled with non-genetic sources of variation. We propose Gene2Brain-AE (G2B-AE), a self-supervised multimodal framework that combines an interpretable NMF-style imaging autoencoder with a genetic variational autoencoder, coupled through a directional predictor that maps genetic embeddings into a subset of imaging latent factors. This directional, predictive coupling reflects the asymmetry of the underlying biological pathway and decomposes imaging variation into genetically predictable and residual components, without requiring genetic input at inference time. On semi-synthetic data, G2B-AE recovers genetic and residual patterns that symmetric alignment-based variants entangle or absorb into the wrong subspace. Applied to ADNI, the model identifies multiple genetically-guided atrophy axes, including distinct APOE-associated patterns, alongside a residual axis linked to vascular and metabolic variables; despite using no genetic input at test time, these components' clinical, cognitive, and neuropathological signatures replicate across eight independent external cohorts, indicating generalizable biology rather than dataset-specific correlations. These results show that directional, partially shared latent representations offer a more biologically-informed and interpretable characterization of disease heterogeneity. Code will be released upon acceptance.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Multimodal representation learning, Brain imaging genetics, Self-supervised learning, Alzheimer's disease, Interpretability, NMF

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cYd3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Gene2Brain-AE, a two-stage self-supervised framework for imaging genetics. A frozen genetic VAE embeds SNPs. An imaging autoencoder with an NMF-style non-negative decoder splits the latent space into a genetically guided block and a residual block. A predictor and a bidirectional InfoNCE loss link the two blocks, and a cross-covariance penalty separates them. Inference uses imaging only. The authors evaluate on three semi-synthetic UK Biobank scenarios, ADNI with 1,045 genotyped MCI/AD subjects, and eight imaging-only external cohorts. On ADNI the model yields four genetic axes and one residual axis. They report SNP associations, clinical and biomarker correlations, and external replication of clinical and neuropathological signatures, and they extend the model to clinical risk factors. The stated contributions are an interpretable genetics-guided decomposition, directional partial coupling, and empirical validation.

Critical strengths

The strongest reason for giving this rating is the external validation: eight independent cohorts, including autopsy data, with the imaging encoder applied without genotypes. That is more out-of-distribution testing than is usual in this area. I also think the design is right: an NMF-style non-negative decomposition with a predictor that maps genetics into only part of the latent space. It keeps the components interpretable and makes the genetic guidance partial rather than symmetric, which is a sensible structure for this problem.

Critical weaknesses

(1) The evidence for distinct genetic mechanisms is insufficient. Three of the four genetically guided components (L0, L2, and L3) are associated with the APOE region, yet the paper does not report APOE-conditional, LD-clumped, chr19-excluded, or APOE-e4-stratified analyses that directly assess whether these components capture distinct genetic effects beyond shared APOE-related signals. The genome-wide analysis uses the same subjects and is exploratory, and the external SNP validation in NACC and WRAP is limited to the APOE region. These results do not rule out that the four axes reflect different aspects of a shared APOE-related signal associated with disease severity. Interpreting them as distinct biological mechanisms therefore seems premature, especially since this is the paper's central biological claim. (2) The reconstruction objective appears inconsistent with the imaging preprocessing. Appendix A.6.2 states that imaging maps are z-scored against the CN reference, with the sign inverted, so the resulting inputs can contain negative values. However, the bias-free decoder uses nonnegative weights and sigmoid-bounded latent representations, so the reconstruction is nonnegative, while the loss is defined as MSE on the input maps. The paper does not explain how negative input values are handled. If they are passed to the model unchanged, the decoder cannot reproduce them, potentially affecting reconstruction fidelity and the interpretation of the learned components as NMF-style factors. The authors should clarify the exact preprocessing and decoder implementation, including whether any shift, clipping, or signed reconstruction is used, and report the fraction of negative input values.

Questions for authors
  1. What preprocessing is applied to the voxel-wise z-scored maps before they enter the imaging autoencoder? Please report the range of values and the proportion of negative entries. If negative values are retained, how does the nonnegative, bias-free decoder reconstruct them, and would that change the interpretation of the learned components as NMF-style spatial bases?
  2. What evidence establishes that L0-L3 capture distinct genetic effects rather than different aspects of a shared APOE-related signal? APOE-conditioned association tests, chromosome 19 exclusion, or APOE e4 stratification would help clarify this. How should the near-zero correlation between L3 and e4 count be reconciled with the 118 significant SNP associations reported for this axis?
  3. What subject-level split was used to calculate the held-out SNP-panel R2? Were the encoder and predictor trained without the held-out subjects, and which SNP-latent associations and GWAS results are based on held-out data? Since Appendix A.8 uses the training or selection subjects, would it be more accurate to restrict "generalizable biology" to the external validation of the imaging components?
  4. The Contr K=2 and K=3 baselines differ from the proposed model in several respects, including the predictor, latent dimensionality, loss direction, and encoder training. How much of the observed difference can be attributed specifically to directional coupling? A projection-matched symmetric baseline would help isolate this effect. What motivated the choice of the lambda_link search range, which ends at 0.3?
  5. Could the authors clarify the configuration used for Fig. 4 and Tables 8-9 by reporting K, Kg, Kr, and C? Section 5.2.1 describes four genetic axes and one residual component (C = 5), whereas Appendix A.7 reports a selected configuration of (K, C) = (8, 4) and near-zero variance beyond C = 4. Which configuration produced the reported results?
  6. The ADNI training cohort includes only MCI and AD cases. Could the authors explain this selection and discuss whether conditioning on diagnosis may induce or distort genotype-imaging associations? If suitable cognitively normal participants with genotype data are available, would including them or running a sensitivity analysis change the conclusions?
5 comments
AuthorsOpenReview ↗

Response to Reviewer cYd3 (1/4): distinct genetic effects vs a shared APOE signal

We thank the reviewer for a thorough review. We performed all requested analyses: APOE-conditional, chr19-excluded, ε4-stratified and LD-clumped tests, severity controls, longitudinal NACC analyses, retraining with CN participants and a controlled coupling comparison. We also correct the preprocessing description and the A.7 error. Details follow point by point (summary in the general comment).

W1/Q2: Distinct genetic effects vs a shared APOE signal

We performed all four requested analyses. They show that L0, L2 and L3 do not share one APOE signal, and that every axis retains distinct loci outside the APOE region. All SNP-level results are in ADNI (in-sample). Out-of-sample evidence follows in point 2 (next comment).

1. APOE controls (ADNI)

AxisVariance explained by ε4/ε2Non-APOE loci (APOE ±1 Mb excluded, ε4/ε2 conditioned)Independent signals: all / strictSignificant SNPs: ε4 non-carriers / carriers
L036.9%PICALM42 / 112 / 3
L1≤1.3%MS4A cluster4 / 413 / 17
L20.5%CLU17 / 113 / 9
L31.3%MS4A, BIN1, PICALM33 / 733 / 31

Independent signals: LD-clumped at r2>0.1r^2 > 0.1 within 1 Mb. "Strict" = APOE region excluded and ε4/ε2 conditioned. Strata: N = 481 non-carriers, 564 carriers.

  • L0 is the ε4 axis. Conditioning on ε4/ε2 removes most of its APOE-region signal (in-sample panel R2R^2 0.424 → 0.099).
  • L2 and L3 carry non-ε4 APOE-locus variation. Their associations persist after conditioning, and their R2R^2 is unchanged (0.130 → 0.128 and 0.395 → 0.389).
  • Whole chr19 excluded: same result. Only 20 panel SNPs lie on chr19 outside the APOE region, and none is significant.
  • Genome-wide (N = 937, ~18.4M variants, MAF ≥ 1%, chr19:44.4–46.5 Mb and all 983 panel SNPs removed): each genetically guided axis retains genome-wide significant loci (p<5×10−8p < 5 \times 10^{-8}) (Fig. 19). The residual shows one locus (chr11p15.5, p=1.8×10−10p = 1.8 \times 10^{-10}).
  • Overlap: in the strict setting, L0, L1 and L2 share no significant SNPs. L3 is the least specific axis: it shares PICALM with L0 and MS4A with L1 (opposite effect direction). We state this in the revision.

L3 vs ε4 count (Q2). The near-zero correlation and the 118 SNPs are consistent:

  • 73 of L3's significant SNPs lie in the APOE ±1 Mb region, but their median LD with ε4 is low (r2=0.03r^2 = 0.03). The strongest (rs439401, rs584007, rs157595, rs440446, rs157580, in TOMM40/APOC1/NECTIN2) have r2≈0.21r^2 \approx 0.21–0.24 with ε4, yet their association is unchanged by ε4/ε2 conditioning (lead p=4×10−47p = 4 \times 10^{-47} before, 6×10−486 \times 10^{-48} after, with 61/73 remaining significant). L3 thus tracks a non-ε4 haplotype, consistent with its null ε4 association in NACC (p=0.18p = 0.18).
  • The rest are non-APOE loci (mainly MS4A, PICALM, BIN1), which persist after APOE exclusion and conditioning (38 SNPs) and in non-carriers (33).
  • In LD-independent terms, the 118 SNPs are 33 signals (8 with the APOE region excluded).

Note on ancestry PCs: we updated the main results to adjust for 20 ancestry PCs (previously 5). Panel counts become 139/19/74/122 (was 140/17/79/118) for L0–L3, conclusions are unchanged, and Section 5.2.1 and Fig. 5b will be updated. The L3 breakdown above refers to the 118 SNPs in the submitted version.

AuthorsOpenReview ↗

Response to Reviewer cYd3 (2/4): out-of-sample evidence and preprocessing

2. Out-of-sample and non-genetic evidence

We agree that the genome-wide analysis is exploratory and that external SNP validation is limited to APOE, and now present the non-APOE loci as hypothesis-generating in the main text. The distinctness of the axes does not rest on them alone:

  1. Different APOE relationships out-of-sample. In NACC (never used for training, 4,467 participants), ε4 count is associated strongly with L0 (0.21 SD/allele, p=10−15p = 10^{-15}), weakly with L1 and L2 (0.09, 0.08 SD), not with L3 (p=0.18p = 0.18), and negatively with the residual.
  2. Different pathology. In NACC autopsy data, L0 and L2 replicate amyloid/tau neuropathology, whereas L3 shows a TDP-43 signature.
  3. Individual, stable profiles. In NACC (891 with repeat MRI, median 2.5 years): ICC 0.73–0.91, and profile similarity 0.77 within vs 0.24 between same-diagnosis participants (permutation p<0.001p < 0.001). ADNI follow-up scans not used for training agree (ICC 0.66–0.82, profile similarity 0.68 vs 0.10).
  4. Differences at comparable severity. Severity (diagnosis, MMSE, CDR-SB) explains 14–20% of L0–L2 and 2% of L3. Adjusting for it leaves the non-APOE associations intact. Biomarker profiles differ formally (Steiger, 27/42 comparisons at FDR < 0.05, e.g. CSF Aβ ρ = −0.45 for L0 vs +0.04 for L3).
  5. No staging between axes. Cross-lagged mixed models in NACC test whether one axis is a later stage of another: if it were, a high baseline value on the first axis would predict faster later growth of the second. In 854 participants with repeat MRI, we modelled each axis's annual rate of change with a linear mixed model, with all four baseline axes as predictors and baseline diagnosis, age and sex as covariates. Across all 12 directed pairs (FDR-corrected), no baseline axis significantly predicted change in another. For L0↔L2, the least separable pair, 0.006 SD/year per SD each way (95% CI −0.018 to 0.030 and −0.033 to 0.045). Only L1→L3 was nominal (0.034, p=0.004p = 0.004, q=0.05q = 0.05).

We treat points 3 and 5 as supporting rather than decisive: stability does not exclude a shared factor with stable individual offsets, and in this mostly cognitively normal cohort small staging effects cannot be excluded. The APOE-related axes change fastest, so profiles shift towards them over time while remaining individually distinguishable.

3. Interpretation. These results support L0–L3 as distinct disease axes with different relationships to APOE, pathology and clinical course: different combinations of APOE-related and non-APOE risk may shape different atrophy patterns, consistent with known locus biology (MS4A and microglial/TREM2 biology, Deming et al., Sci. Transl. Med. 2019; CLU and amyloid-β clearance, Nelson et al., PNAS 2017). We present this as a hypothesis. Testing it requires external genotyped replication, Mendelian randomization and colocalization, now listed as future work.

W2 / Q1: Negative inputs and the nonnegative decoder

The input is nonnegative by construction. After voxel-wise z-scoring against the CN reference and sign inversion (positive = atrophy), negative values are set to zero before the autoencoder. We apologise for omitting this step and have added it to Section 3.2 and Appendix A.6.2.

  • No shift, rescaling or signed reconstruction. Input = z-scores truncated at zero, range [0, 6.78].
  • The objective is consistent. Input, decoder (W≥0W \geq 0, bias-free) and latents (sigmoid, [0, 1]) are all nonnegative, so x^=Wh\hat{x} = Wh lies in the input's domain and the MSE is well-posed. This is the standard NMF setting. Components keep their interpretation as additive, parts-based atrophy patterns with nonnegative loadings.
  • Why atrophy only. The study targets tissue loss. Values above the CN reference mostly reflect anatomical variability and registration noise. Nonnegativity is what makes NMF components interpretable as regional patterns (Sotiras et al., 2015).
  • Same pipeline everywhere: semi-synthetic data and all eight external cohorts, so training and inference inputs share one scale, and baselines see the same representation. Extensions to CSF maps (where disease appears as an increase) or a mixed-sign semi-NMF decoder are natural future work.
AuthorsOpenReview ↗

Response to Reviewer cYd3 (3/4): held-out R², directional coupling and λ_link

Q4: Directional coupling and the λ_link range

In the original baselines, the genetic VAE was also pretrained and frozen (Appendix A.4.2). Contr K=3K=3 used the same VAE as our model (K=3K = 3), and Contr K=2K=2 a VAE retrained at K=2K = 2 to match KgK_g. The remaining differences were the predictor, the loss direction and the imaging-side dimensionality. The new requested comparison removes these. All variants share the frozen genetic VAE, imaging autoencoder, loss terms, temperature, coupling weight and optimizer, and differ only in the coupling: two-tower MLP or linear (CLIP) projection heads with symmetric Bi-InfoNCE, one-way InfoNCE (gen→img or img→gen), and a reversed predictor (hgen→z^genh_{\text{gen}} \to \hat{z}_{\text{gen}}, aligned with the frozen genetic embedding).

Basic scenario (full table in Appendix A.14). The two-tower MLP reproduces Contr's profile: genetic/environmental similarity 0.934/0.535 vs Contr K=2K=2 0.939/0.540 and ours 0.905/0.738. Both two-tower variants carry genetic signal in the residual (0.4/2 vs 0 for ours) and lose causal-SNP association (1.6/2 MLP, 1.2/2 linear, vs 2/2). In 4/5 seeds, a genetic component best matches the environmental pattern. One-way variants match ours. The reversed predictor matches or exceeds ours on cosine metrics (0.912/0.756), but its residual spans nearly the whole brain (A.14, Fig. 16c), so its higher environmental cosine reflects a diffuse map rather than recovery of AenvA_{\text{env}}.

Weak-effect scenario (5 seeds; large-overlap results, per-seed results and SDs in Appendix A.14):

ModelGen. spatialEnv. spatialLatent–sev. rrGen. assoc. /2
Ours0.6490.6250.2731.8
One-way (img→gen)0.6460.5600.2891.4
One-way (gen→img)0.6520.6340.2411.0
Reversed predictor0.5570.5290.1040.2
Two-tower MLP0.6430.4880.1451.0
Two-tower linear0.6110.4670.2050.8
  • Capacity does not explain the benefit. The two-tower MLP (more capacity) leaks into the residual, loses the environmental pattern and carries fewer causal SNPs. The linear two-tower (less capacity) fails the same way.
  • Anchoring hgenh_{\text{gen}} directly is what matters. The matched-capacity reversed predictor is competitive under strong effects but collapses under weak ones (no meaningful coupling in 4/5 seeds): an imaging-side head can fit weak genetic variation from noise, and weak polygenic effects are the realistic regime.
  • Both InfoNCE terms contribute under weak effects, based on the one-way ablations' genetic associations.

λ_link range. A coarse range-finding step with λ_link ∈ {0.1, 0.5, 1.0} preceded the systematic search: 0.1 gave too weak coupling and 1.0 let the link loss dominate reconstruction. 0.5 was better than both, so the stability-based grid {0.3, 0.5, 0.8} was centred on it.

Q3: Held-out R2R^2 and "generalizable biology"

The R2R^2 was computed with 5-fold cross-validation over subjects for the ridge regression only. The genetic VAE, encoder and predictor were trained on all ADNI training subjects. These are therefore cross-validated in-sample estimates and all SNP-level analyses, including Appendix A.8, are in-sample. With ~1,000 genotyped ADNI participants, a held-out set large enough for SNP-level replication would leave both training and testing underpowered, and no external cohort offered genotypes beyond APOE. Out-of-sample genetic evidence is therefore limited to ε4 on the APOE axis in NACC and WRAP.

We agree with the suggested scope. "Generalizable" will refers only to the imaging axes and their external validation (biomarkers, pathology, longitudinal stability, ε4 on the APOE axis). External genotyped replication and biological causal analyses will be listed as future work.

Reviewer_7z5WOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Gene2Brain-AE (G2B-AE), a self-supervised framework combining a genetic-to-imaging coupling strategy and an interpretable NMF-style imaging autoencoder, to learn genetically informed brain atrophy patterns. Experiments on semi-synthetic data demonstrate its ability to recover distinct atrophy patterns. Extensive clinical analyses across ADNI and eight external cohorts were conducted to verify the biological relevance of the learned representations.

Critical strengths
  • The authors try to address an important problem of identifying genetically informed patterns of brain atrophy. The extensive clinical and biological validation across multiple external cohorts strengthens the practical significance of the study.
  • The proposed genetic-to-imaging coupling provides an interesting approach to guiding imaging representation learning with genetic information.
Critical weaknesses
  1. Limited clarity of the methodological novelty. The proposed framework mainly combines existing components, including a genetic VAE, an NMF-style autoencoder, a predictor, and InfoNCE, and shares similarities with existing multimodal shared/private representation learning methods. Although incorporating genetic information into interpretable brain atrophy decomposition is valuable, the methodological novelty over existing approaches is not sufficiently clear. In particular, genetic-to-imaging prediction appears to be the main technical contribution, but its unique advantages are not adequately established.

  2. Insufficient validation of the proposed genetic-to-imaging coupling. The authors mainly compare G2B-AE with Contr K=2/K=3 to demonstrate the advantages of the proposed coupling strategy. However, these baselines differ in several aspects, including the predictor, latent dimensionality, and InfoNCE objective, making it difficult to identify the source of the improvements. Moreover, Contr K=2 outperforms G2B-AE in genetic spatial similarity in the basic scenario (0.939 vs. 0.905, Appendix Table 2). More controlled comparisons, particularly with an imaging-to-frozen-genetics prediction baseline, would better justify the necessity and effectiveness of the proposed design.

  3. Limited consideration of genetic–environmental interactions. The semi-synthetic experiments consider spatial overlap between genetic and environmental effects but do not sufficiently examine their interactions. In practice, genetic and environmental factors may jointly influence the same atrophy patterns, whereas the proposed model encourages the corresponding representations to be decorrelated. This may lead to an artificial separation of interacting effects, and the model's robustness under such conditions remains unclear.

  4. Some figures and methodological descriptions could be clearer. The text in Figure 1 is relatively small, making some details difficult to read. In addition, several mathematical symbols are not clearly explained when first introduced, such as (N) in Section 3 and (C) in Section 3.2.

Questions for authors
  1. Could the authors clarify the main methodological novelty of G2B-AE compared with existing shared/private crossmodal representation learning methods?
  2. Why is genetic-to-imaging prediction preferred over imaging-to-genetics prediction? Could the authors provide a controlled comparison with the reverse direction?
  3. Could the authors explain whether the improvements over Contr K=2/K=3 come from genetic-to-image coupling or other architectural differences?
  4. How would the proposed disentanglement perform when genetic and environmental factors interact rather than simply overlap spatially?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 7z5W (1/3): novelty and clarity

We thank the reviewer for the constructive review. We added (i) single-factor ablations of the coupling (predictor vs. symmetric heads, loss direction, prediction direction), including (ii) the requested imaging-to-frozen-genetics baseline, and (iii) five scenarios with gene–environment correlation and interaction.

W1 / Q1: Novelty. The building blocks are mostly established. The contribution is a composition that fills a gap in the landscape. Three families of imaging-genetics methods are relevant, and none gives an interpretable, genetically shaped decomposition of brain structure. Association models encode the right direction (SNPs→imaging) but work on raw features or ROIs, capturing feature-level links rather than latent patterns. Joint matrix factorization is interpretable but shares loadings symmetrically, so its loadings must also explain genetic variation with no imaging correlate, which hurts under weak effects (jNMF: association 1/2, coeff. 0.12). Latent fusion, including shared/private variants (DMAAN, Wang et al., 2023, and Gene-SGAN), stays symmetric and is mostly trained with diagnostic labels on ROI features, biasing the shared subspace toward discriminative signal. Hence, beyond Gene-SGAN, we do not benchmark these supervised models directly, and instead represent symmetric latent fusion with the contrastive variants (Section 4.1.3).

G2B-AE combines directionality (association), interpretability (NMF: a non-negative linear decoder with voxel-wise spatial columns) and learned embeddings (latent fusion: a genetic VAE and a nonlinear predictor). Four properties follow:

  1. Predictability, not agreement, defines the shared subspace. hgenh_{\text{gen}} is the imaging variation a frozen genetic embedding can predict. Symmetric methods use mutual alignment, which spurious correspondences in either direction can satisfy, including chance correlations, rather than only variation that genetics predicts.
  2. The split lives in an interpretable dictionary. One non-negative, voxel-wise decoder reads out hgenh_{\text{gen}} and hresh_{\text{res}} as additive spatial maps, with no ROI parcellation fixing the scale and no reliance on post-hoc attribution, which shows what a network uses rather than a testable pattern.
  3. No genetic-private subspace is needed. The K→KgK \to K_g predictor discards genetic variation with no imaging correlate, so no private-subspace dimension has to be chosen. Most variation in real panels has no imaging correlate. Leaked into the shared subspace, it yields components that look genetic but are not.
  4. Self-supervised, imaging-only at inference. No labels bias the components, and genetics guides training only. Since imaging-only cohorts far outnumber genotyped ones, this enables replication in eight external cohorts without genotypes. On directionality, Section 3.3 now clarifies that the gene→brain asymmetry comes from the frozen genetic encoder, the stop-gradient and the predictor.

W4: Clarity. Figure 1 will be enlarged with larger text, and NN and CC are now defined at first use (Sections 3 and 3.2).

AuthorsOpenReview ↗

Response to Reviewer 7z5W (2/3): validating the coupling

W2 / Q2 / Q3:

Why this coupling. The key mechanism is a bottleneck. With gen→img, the target P(zgen)P(z_{\text{gen}}) depends only on the frozen genetic embedding, so hgenh_{\text{gen}} can match it only by carrying genetically predictable variation. When an imaging-side mapping can carry the alignment instead, as in the reversed predictor (img→gen) and the symmetric two-tower heads, hgenh_{\text{gen}} is only loosely constrained. With weak genetic signal, reconstruction then dominates it.

Controlled variants. We ran the requested controlled comparison. All variants share the frozen genetic VAE, imaging autoencoder, loss terms, temperature, coupling weight and optimizer, and differ only in the coupling: two-tower MLP or linear (CLIP) projection heads with symmetric Bi-InfoNCE, one-way InfoNCE (gen→img or img→gen), and a reversed predictor (hgen→z^genh_{\text{gen}} \to \hat{z}_{\text{gen}}, aligned with the frozen genetic embedding).

Contr vs. ours. Contr couples hgenh_{\text{gen}} to the genetic latent directly, without a predictor, through one-directional InfoNCE. Without projection heads, symmetric alignment requires matched dimensions, so we tested both matchings: Contr K=2K=2 matches our imaging side (Kg=2K_g = 2) and Contr K=3K=3 matches our genetic embedding (K=3K = 3). Neither closes the gap. The two-tower variants add projection heads, so both sides keep their native dimensionality, and they reproduce Contr's profile.

Basic scenario. Two-tower: genetic/environmental similarity 0.934/0.535, vs. Contr K=2K=2 0.939/0.540 and ours 0.905/0.738. In 4/5 two-tower seeds, the component best matching the environmental pattern is a genetic one. Both two-tower variants also leak genetic signal into the residual (0.4/2) and lose genetic association (1.6/2 MLP, 1.2/2 linear, vs. 2.0/2), which shows the disentanglement failure directly.

Prediction direction (5 seeds, mean ± SD):

ScenarioModelGen. simEnv. simCoeff.Assoc.
BasicOurs.905±.006.738±.005.502±.0172.0±0.0
BasicReverse.912±.010.756±.075.543±.0122.0±0.0
Large overlapOurs.947±.002.569±.166.445±.0541.8±0.4
Large overlapReverse.930±.015.671±.040.440±.0072.0±0.0

All variants, weak-effect scenario (5 seeds):

ModelGen. simEnv. simCoeff.Assoc.
G2B-AE (ours).649±.048.625±.023.273±.0411.8±0.4
One-way gen→img.652±.032.634±.013.241±.1001.0±0.6
One-way img→gen.646±.050.560±.049.289±.1291.4±0.8
Reversed predictor.557±.049.529±.050.104±.1330.2±0.4
Two-tower, MLP.643±.034.488±.068.145±.0561.0±0.7
Two-tower, linear.611±.071.467±.165.205±.0860.8±0.4
Contr K=2.631±.024.648±.006.167±.0671.0±0.6
Contr K=3.618±.041.625±.039.122±.0581.4±0.8
  • Direction: with strong effects the reverse direction matches ours on cosine metrics, but in the basic scenario its residual spans nearly the whole brain (Appendix A.14, Fig. 16c), so its higher environmental cosine reflects a diffuse map, the same failure seen under permutation. With weak effects it loses all genetic association in 4/5 seeds, while ours keeps at least one causal-SNP association in all 5.
  • Symmetric heads lose the residual (env. 0.47–0.49 vs. 0.63) and association (0.8–1.0).
  • Loss direction: both one-way variants keep good maps but lose association (1.0 and 1.4 vs. 1.8).
  • Overall: under weak effects, every coupling variant has lower mean genetic association than the full model (≤1.4 vs. 1.8/2). Full tables for all scenarios are in Appendix A.14.
AuthorsOpenReview ↗

Response to Reviewer 7z5W (3/3): gene–environment interaction

W3 / Q4:

Under correlated and interacting gene–environment effects (apart from spatial overlap), genetic patterns, severities and SNP associations are recovered at levels comparable to the independent case, and the decorrelation term does not remove interacting effects.

Design. Five scenarios keep the subjects, SNPs, genetic latents, atrophy patterns and effect sizes of the basic scenario. Only the gene–environment relationship changes:

  • ρ=0\rho = 0 (reference): senvs_{\text{env}} independent of sgen1s_{\text{gen1}}, additive effects. Unlike the basic scenario, a subtyping design, subjects can express more than one dominant factors at once. Expected: clean separation, no causal-SNP association in the residual. Observed: genetic similarity 0.92, residual association 0/2.
  • ρ=0.3/0.6\rho = 0.3 / 0.6 (rGE): senvs_{\text{env}} drawn via a Gaussian copula on the ranks of sgen1s_{\text{gen1}}, so Pearson(sgen1,senv)=ρ\mathrm{Pearson}(s_{\text{gen1}}, s_{\text{env}}) = \rho with marginals unchanged. Expected: patterns stay separated, with a weak residual association with the causal SNPs, since senvs_{\text{env}} is correlated with sgen1s_{\text{gen1}}. Observed: residual association 0.3/2, genetic similarity 0.91–0.92, and the genetic pattern stays in a genetic component in 7/9 runs (0/9 without the decorrelation loss).
  • G×E, ρ=0/0.3\rho = 0 / 0.3: environmental atrophy c⋅senv(1+γ sgen1)c \cdot s_{\text{env}} (1 + \gamma\, s_{\text{gen1}}), γ=1\gamma = 1. Expected: patterns stay separated, with a weak residual association with the interacting variants. Observed: residual association 1/2 in every seed, genetic similarity 0.87, environmental similarity 0.86–0.90 (maps in Appendix A.15). Each is run with and without the decorrelation loss (3 seeds, means; SDs in the Appendix):
ScenarioDecorr.Gen. simEnv. simCoeff.Assoc. (/2)Gen. signal in residual (/2)
ρ=0\rho = 0on / off0.92 / 0.920.56 / 0.610.47 / 0.522.0 / 1.30.0 / 1.0
ρ=0.3\rho = 0.3on / off0.92 / 0.920.59 / 0.660.49 / 0.561.7 / 1.00.3 / 1.0
ρ=0.6\rho = 0.6on / off0.91 / 0.910.57 / 0.670.53 / 0.571.7 / 1.30.3 / 1.0
G×E, ρ=0\rho = 0on / off0.87 / 0.900.86 / 0.830.51 / 0.521.7 / 2.01.0 / 1.0
G×E, ρ=0.3\rho = 0.3on / off0.87 / 0.900.90 / 0.870.54 / 0.561.7 / 1.31.0 / 1.0
  • Genetic recovery is stable: similarity 0.87–0.92, coeff. 0.47–0.54, association 1.7–2.0, with no degradation from ρ=0\rho = 0 to 0.6.
  • Decorrelation regularizer keeps interactions without breaking separation: the residual carries the genetic modulation (1/2) with and without it. Without it, however, the maps fail under both G×E settings: the residual takes subcortical and temporal territory from Agen1A_{\text{gen1}}, a genetic component spreads to subcortex and cerebellum, and the parietal part of AenvA_{\text{env}} is weakly expressed (Fig. 17). Cosine metrics miss this because the temporal region is shared.
  • Decorrelation regularizer prevents leakage: without it, the residual already carries genetic signal at ρ=0\rho = 0, and across ρ=0,0.3,0.6\rho = 0, 0.3, 0.6 the genetic pattern moves into the residual in 9/9 runs.
  • Why interactions survive: Lh-ortho\mathcal{L}_{h\text{-ortho}} acts on latents, is linear and soft, and is weighed against reconstruction, so variation needed to reconstruct the images is kept.
  • Effect size: G×E also raises environmental amplitude, so part of the gain in environmental recovery (0.56→0.86) likely reflects effect size rather than the interaction itself.

We hope the clarified positioning against shared/private and joint-factorization methods, together with the controlled comparison including the imaging-to-frozen-genetics baseline you suggested, makes clear both what is new and where the benefit comes from. If these additions address your concerns, we would be grateful if you would consider updating your score, and we are happy to clarify anything further before the discussion closes.

Reviewer_a6y4OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper proposes an interesting framework for genetically informed representation learning, supported by extensive empirical validation across multiple cohorts. However, my main concern is the identifiability of the learned latent factors. Since distinct biological mechanisms may produce similar or identical imaging patterns, recovering the underlying mechanisms is not necessarily possible. The paper provides no theoretical guarantees addressing this issue. I would encourage the authors to clarify the assumptions required for disentanglement and moderate their claims about recovering distinct biological mechanisms.

Critical strengths

Biologically motivated architecture: Directional genetic-to-imaging coupling is a sensible approach that accounts for partial correspondence between modalities.

Interpretability: The NMF-style decoder produces spatially interpretable patterns of brain atrophy.

Extensive empirical evaluation: Semi-synthetic experiments, baseline comparisons, and ablations provide useful evidence for the proposed design.

Strong external validation: Evaluation across eight independent cohorts demonstrates promising generalizability and clinical relevance without requiring genetic data at inference.

Critical weaknesses

Lack of identifiability guarantees: Distinct biological mechanisms may produce identical imaging patterns. The paper provides no theoretical analysis establishing when the underlying latent factors can be recovered.

Overinterpretation of latent factors: Statistical associations and contrastive alignment do not establish that the learned components correspond to distinct biological mechanisms.

Limited disentanglement guarantees: The covariance-based orthogonality constraint does not ensure independence or separation of genetic and non-genetic influences.

Simplified synthetic experiments: The simulations do not adequately examine fundamentally non-identifiable scenarios, particularly many-to-one mappings from biological mechanisms to imaging phenotypes.

Incomplete baseline comparisons: Additional controlled experiments are needed to isolate the benefits of directional coupling from those of predictor capacity and other architectural choices.

Questions for authors

Identifiability: Under what assumptions can the proposed framework recover distinct genetic and residual latent factors, particularly when different biological mechanisms produce identical or highly overlapping imaging patterns?

Imaging-only inference: If two individuals have identical imaging phenotypes but different underlying genetic mechanisms, how can the imaging encoder distinguish them at inference time?

Disentanglement: Does the covariance-based orthogonality constraint provide sufficient separation between genetic and residual factors? Could the authors offer theoretical justification or stronger empirical evidence?

Synthetic experiments: Could the authors evaluate a genuinely many-to-one generative setting in which distinct latent mechanisms produce indistinguishable imaging patterns?

Directional coupling: Can the authors isolate the contribution of directional coupling through a controlled comparison with symmetric alignment using matched predictor capacity and contrastive objectives?

5 comments
AuthorsOpenReview ↗

Response to Reviewer a6y4 (1/4): Identifiability

We thank the reviewer for a careful review. We ran every requested analysis. In brief, we now state the assumptions under which genetic and residual factors are separable, test the model's robustness in new simulated settings beyond subtyping (correlated and interacting factors, complete overlap), add a controlled coupling comparison, and examine the scope of our ADNI claims given the available data (see also the general comment).

W1 / Q1: Identifiability

Generative model. For subject ii, imaging xi∈RVx_i \in \mathbb{R}^V and genotype gig_i follow:

xi=Ag sig+Ar sir+εi,sig=f(gi)+ηix_i = A_g\, s_i^g + A_r\, s_i^r + \varepsilon_i, \qquad s_i^g = f(g_i) + \eta_i

AgA_g and ArA_r are nonnegative genetic and residual spatial patterns, sg,sr≥0s^g, s^r \geq 0 are subject loadings, and ff maps genotype to genetic loadings. The noise terms ε\varepsilon and η\eta have mean zero and are independent of gg. We observe only xx and gg.

Assumptions.

  • (A1) Residual exogeneity: E[sr∣g]=E[sr]\mathbb{E}[s^r \mid g] = \mathbb{E}[s^r]. On average, residual loadings do not vary with genotype. Our ρ>0\rho > 0 simulations deliberately violate this (Q3).
  • (A2) Genetic relevance: Cov(f(g))\mathrm{Cov}(f(g)) has full rank KgK_g. Each genetic factor carries genetic signal not spanned by the others.
  • (A3) Spatial distinctness: [Ag,Ar][A_g, A_r] has full column rank. Patterns may overlap, but none is a linear combination of the others.

What is identifiable. Under (A1),

E[x∣g]−E[x]=Ag(f(g)−E[f(g)]).\mathbb{E}[x \mid g] - \mathbb{E}[x] = A_g\big(f(g) - \mathbb{E}[f(g)]\big).

Under (A2), these vectors span span(Ag)\mathrm{span}(A_g), so the genetic spatial subspace is identified from the data without knowing ff. Within it, individual patterns are identified only up to an invertible Kg×KgK_g \times K_g transform. Nonnegativity resolves this up to permutation and scale when standard NMF uniqueness conditions hold (Donoho & Stodden, 2004; Fu et al., 2019). These conditions require more than (A3), for example that each pattern has voxels the others do not cover. Under (A3), and assuming non-degenerate loadings (Cov(s)\mathrm{Cov}(s) of full rank) and noise separable from the signal (e.g. isotropic), span([Ag,Ar])\mathrm{span}([A_g, A_r]) is identified from Cov(x)\mathrm{Cov}(x). The residual patterns are then identified up to adding combinations of the genetic patterns and invertible mixing among themselves. Nonnegativity removes both ambiguities under the same NMF conditions. With Kr=1K_r = 1, as in all our experiments, the mixing reduces to scale.

Scope. This analysis concerns the generative model, not our estimator. We do not prove that the minimizer of Eq. 5 recovers these quantities. We test this empirically, including when (A1) is violated (Q3) and when (A3) fails through complete overlap (Q4). The coupling is designed to capture the genetically predictable part of imaging variation, which is the quantity identified above.

Partial overlap is the target regime. Distinct pathologies can produce similar atrophy: medial temporal atrophy arises in AD, LATE and PART (Nelson et al., 2019), and frontotemporal atrophy from tau or TDP-43. Yet group-level studies typically still find distinguishable patterns, so we target partial, not complete, overlap. This is supported by distinct but overlapping syndrome networks (Seeley et al., 2009), AD atrophy subtypes (Young et al., 2018; Vogel et al., 2021), regional genetic architectures (Zhao et al., 2019; Grasby et al., 2020) and vascular and ageing patterns (Habes et al., 2016). (A3) alone does not guarantee recovery in this regime, so we test it. Factors are recovered under partial and large overlap in simulation and in ADNI, where L0 and L1 share a medial temporal peak yet have nearly uncorrelated loadings (r=0.14r = 0.14) and different ε4 relationships (W2). Complete overlap is tested in Q4.

References (others refer to the manuscript bibliography)

  • Donoho & Stodden (2004). When does non-negative matrix factorization give a correct decomposition into parts? NeurIPS.
  • Fu et al. (2019). Nonnegative matrix factorization for signal and data analytics. IEEE Signal Process. Mag.
  • Grasby et al. (2020). The genetic architecture of the human cerebral cortex. Science.
  • Habes et al. (2016). White matter hyperintensities and imaging patterns of brain ageing in the general population. Brain.
  • Nelson et al. (2019). Limbic-predominant age-related TDP-43 encephalopathy (LATE): consensus working group report. Brain.
  • Seeley et al. (2009). Neurodegenerative diseases target large-scale human brain networks. Neuron.
  • Vogel et al. (2021). Four distinct trajectories of tau deposition identified in Alzheimer's disease. Nat. Med.
  • Young et al. (2018). Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with Subtype and Stage Inference. Nat. Commun.
  • Zhao et al. (2019). Genome-wide association analysis of 19,629 individuals identifies variants influencing regional brain volumes. Nat. Genet.
AuthorsOpenReview ↗

Response to Reviewer a6y4 (2/4): imaging-only inference and disentanglement

Q2: Imaging-only inference

Since the encoder is a function h=φ(x)h = \varphi(x), identical images give identical latents (x1=x2⇒h1=h2x_1 = x_2 \Rightarrow h_1 = h_2), so no imaging-only model, ours included, can separate genetic pathways that produce completely overlapping atrophy. We target partial overlap (A3, Q1): training the imaging latents to be predictable from genotype separates atrophy that imaging-only decompositions merge, e.g. L0 and L1, which share a medial temporal peak but differ in ε4 association and amyloid profile (W2).

Genotype is used only in training, to decide which directions of imaging variation the genetic latents capture. At inference, the latents are genetically informed imaging phenotypes, not inferred genotypes. A subject's loadings give both an imaging subtype and a hypothesis about the genetic pathway behind it: a high L0 loading suggests ε4-related, amyloid-associated atrophy, whereas a high L1 loading in the same region suggests a largely amyloid-independent process linked to non-APOE loci. Different variants converging on the same pattern intentionally receive the same value.

With genotype at inference, predisposition and manifestation can be compared. The predictor gives the genetic predisposition h^gen(g)\hat{h}{\text{gen}}(g), the encoder the current manifestation hgen(x)h{\text{gen}}(x). Their difference shows whether a subject has more or less axis-specific atrophy than their genetics predicts, possibly pointing to resilience or to additional non-genetic drivers. We add this to the discussion as future work.

If overlapping patterns were separated unreliably, a subject's loadings would shift between scans. In NACC (not used for training, no genotypes given), each axis is stable within individuals over a median of 2.5 years (ICC 0.73–0.91; ADNI follow-up scans: 0.66–0.82), and individual profiles remain distinct from others with the same diagnosis (similarity 0.77 within vs 0.24 between, permutation p<0.001p < 0.001). This rules out arbitrary assignment, though stability alone does not show that overlapping mechanisms are separated. That evidence comes from the distinct ε4 and pathology profiles (W2).

W3 / Q3: Disentanglement

Lh-ortho\mathcal{L}_{h\text{-ortho}} enforces linear decorrelation, not independence, which coincide only in the linear-Gaussian case, as we now state. It is the narrowest of three separation mechanisms. The genetics-to-imaging prediction pulls hgenh_{\text{gen}} toward genetically predictable variation, similar in spirit to identifiability through an observed auxiliary variable (iVAE, Khemakhem et al., AISTATS 2020). Decoder orthogonality keeps spatial components apart, and Lh-ortho\mathcal{L}_{h\text{-ortho}} discourages the residual from duplicating genetic signal.

New experiments (motivated by 7z5W). Environmental severity is drawn through a Gaussian copula so that Pearson(sgen1,senv)=ρ\mathrm{Pearson}(s_{\text{gen1}}, s_{\text{env}}) = \rho, with marginals unchanged. Unlike the basic scenario, a subtyping design, a subject can express genetic and environmental patterns together. For ρ>0\rho > 0 these scenarios deliberately violate (A1), so some residual genetic association reflects the true correlation. Leakage = causal SNP groups (of 2) significantly associated with the residual. Each cell: with / without Lh-ortho\mathcal{L}_{h\text{-ortho}}.

ScenarioGen. spatialEnv. spatialGen. assoc. /2Leakage /2
Basic0.905 / 0.9040.738 / 0.7412.0 / 2.00.0 / 0.0
Large overlap0.947 / 0.8410.569 / 0.3511.8 / 2.00.2 / 0.0
ρ=0\rho = 00.920 / 0.9240.557 / 0.6142.0 / 1.30.0 / 1.0
ρ=0.3\rho = 0.30.917 / 0.9230.590 / 0.6571.7 / 1.00.3 / 1.0
ρ=0.6\rho = 0.60.907 / 0.9110.565 / 0.6701.7 / 1.30.3 / 1.0
  • No effect without overlap or correlation: basic-scenario metrics change by at most 0.003.
  • The penalty prevents genetic and residual factors from exchanging components. Without it, in every ρ\rho run, the residual takes the first genetic pattern with its causal SNPs (leakage 1/2) and a genetic component takes the environmental one. This occurs even at ρ=0\rho = 0, so the trigger is co-expression of overlapping patterns, not correlation. With the penalty, patterns stay in their own components in 7/9 runs. Cosine similarity misses this swap, so we also inspect the maps (A.15). In ADNI, the residual L4 is associated with 1 of 983 SNPs, and its cross-validated in-sample predictability from the panel is at the permutation null (R2=−0.007R^2 = -0.007, p=0.754p = 0.754).

References

  • Khemakhem et al. (2020). Variational autoencoders and nonlinear ICA: A unifying framework. AISTATS.
AuthorsOpenReview ↗

Response to Reviewer a6y4 (3/4): many-to-one settings and directional coupling

W4 / Q4: Many-to-one settings

We added three scenarios in which two mechanisms produce an identical imaging pattern, with no correlation or interaction (ρ=0\rho = 0, γ=0\gamma = 0). In gen = gen (temporal–subcortical), both genetic factors share genetic factor 1's map, each driven by its own causal SNPs. gen = gen (occipital–frontal) does the same for factor 2's map. In gen = env, the environmental factor shares genetic factor 1's map. These deliberately violate (A3): imaging sees only the summed loading, so the shared map should be recovered, but which component carries it is not determined by imaging. 3 runs each.

ScenarioShared map recovered (cos)Genetic association (/2)Residual genetic association
gen = gen (temporal–subcortical)0.8881.000.00
gen = gen (occipital–frontal)0.9471.000.00
gen = env0.9182.000.00

The shared map is always recovered, with no genetic signal in the residual. For gen = env, both genetic factors recover their causal SNPs (2/2), but the residual does not duplicate the shared map (cosine with the environmental map 0.32). For gen = gen, one component carries the shared map and its SNPs (1/2), the most any imaging-based decomposition can recover when imaging observes only s1+s2s_1 + s_2.

Decoder orthogonality encodes our partial-overlap assumption. Without it, the residual becomes less specific in the basic scenario, spreading into the second genetic pattern (cosine with that map 0.63–0.67 vs 0.48–0.56 with the constraint, Fig. 10f). Since partial overlap is the biologically supported regime (Q1) and a spatially specific residual is essential for interpreting non-genetic patterns, we keep soft decoder orthogonality and accept that fully overlapping mechanisms are not separated by default. Where complete overlap is plausible, we recommend relaxing it and retuning the coupling strength, and the released code will expose these settings. The regime can be checked without ground truth by testing the residual for genetic association, which in our simulations appeared whenever the residual absorbed a genetic pattern. In ADNI it is minimal (Q3), consistent with partial overlap.

W5 / Q5: Directional coupling

We ran the requested controlled comparison. All variants share the frozen genetic VAE, imaging autoencoder, loss terms, temperature, coupling weight and optimizer, and differ only in the coupling: two-tower MLP or linear (CLIP) projection heads with symmetric Bi-InfoNCE, one-way InfoNCE (gen→img or img→gen), and a reversed predictor (hgen→z^genh_{\text{gen}} \to \hat{z}_{\text{gen}}, aligned with the frozen genetic embedding).

In our model the imaging latent itself must match the genetic prediction, InfoNCE(hgen,P(zgen))\mathrm{InfoNCE}(h_{\text{gen}}, P(z_{\text{gen}})), so genetic information is expressed along the axes the decoder uses. In symmetric alignment only projections of the two modalities must agree, so hgenh_{\text{gen}} is never constrained directly.

Basic scenario. The two-tower reproduces Contr's profile: genetic/environmental similarity 0.934/0.535, vs. Contr K=2K=2 0.939/0.540 and ours 0.905/0.738. In 4/5 two-tower seeds, a genetic component best matches the environmental pattern, and genetic signal leaks into the residual (0.4/2, with genetic association 1.6/2 and 1.2/2 vs. 2.0/2 for ours).

Weak effect (5 seeds, SDs and all scenarios in Appendix A.14):

ModelGen. spatialEnv. spatialLatent–sev. rrGen. assoc. /2
Ours0.6490.6250.2731.8
One-way (img→gen)0.6460.5600.2891.4
One-way (gen→img)0.6520.6340.2411.0
Reversed predictor0.5570.5290.1040.2
Two-tower MLP0.6430.4880.1451.0
Two-tower linear0.6110.4670.2050.8
  • Capacity does not explain the benefit. The MLP two-tower (more capacity) and the linear two-tower (less) both lose the environmental pattern and genetic association.
  • Anchoring hgenh_{\text{gen}} directly is what matters. The reversed predictor is competitive under strong effects but collapses under weak ones (no meaningful coupling in 4/5 seeds), because an imaging-side head can fit weak genetic variation from noise. Weak, polygenic effects are the realistic regime.
  • Both InfoNCE terms contribute under weak effects (one-way variants: 1.0–1.4 vs 1.8).

Comments on the paper

3 comments
AuthorsOpenReview ↗

Rebuttal and Response to Reviewers

We thank all three reviewers for careful and constructive reviews. We were encouraged that they agreed on the main strengths of the work:

  • Directional, partial coupling design (all three reviewers): "a sensible approach that accounts for partial correspondence between modalities" (a6y4), "an interesting approach to guiding imaging representation learning with genetic information" (7z5W), and "I also think the design is right" (cYd3).
  • External validation across eight independent cohorts, without genotypes at inference (all three reviewers): "promising generalizability and clinical relevance" (a6y4), "strengthens the practical significance of the study" (7z5W), and "more out-of-distribution testing than is usual in this area" (cYd3).
  • Interpretability of the NMF-style decoder (a6y4, cYd3): "spatially interpretable patterns of brain atrophy" (a6y4), and a structure that "keeps the components interpretable" (cYd3).

The reviews raised three shared concerns. We address each with new analyses, all of which will be in the revised PDF (new Appendices A.13–A.18). Main-text changes are limited to targeted edits and the additional page.

  1. Is directional coupling what helps? (7z5W, cYd3, a6y4). We ran a controlled comparison inside our model, changing only the coupling: symmetric two-tower heads (more and less capacity), one-way InfoNCE in each direction, and the requested reversed, imaging-to-frozen-genetics predictor. The advantage of anchoring the imaging latent to the genetic prediction is robustness under weak genetic effects. Under strong effects the reversed predictor is competitive on cosine metrics, but its residual is spatially diffuse (A.14).
  2. Are the axes distinct, or one APOE/severity signal? (cYd3, a6y4). APOE-conditional, chr19-excluded, $\varepsilon$4-stratified and LD-clumped analyses, severity adjustment, and longitudinal and cross-lagged analyses in NACC (A.16, A.17).
  3. Identifiability and disentanglement (a6y4, 7z5W). The assumptions under which genetic and residual factors are separable, and new simulations where they are violated: gene–environment correlation, gene–environment interaction, and complete spatial overlap (A.13, A.15).

Point-by-point replies follow under each review. We believe these additions strengthen the soundness of the work, and we thank the reviewers again for their time. We welcome further questions during the discussion.

AuthorsOpenReview ↗

Updated Manuscript and Appendix

Revision uploaded: summary of changes

We have uploaded the revised PDF. We used the additional page for the two concerns shared across reviewers: the controlled coupling comparison (§5.1) and the analyses of distinct axes beyond APOE and severity in ADNI (§5.2.1). Other main-text changes are targeted edits, and all new analyses are detailed in Appendices A.13–A.18. Summary by concern:

ConcernRaised byWhere addressedNew figures / tables
Contribution of directional coupling, imaging-to-genetics baseline7z5W, cYd3, a6y4§4.1.3, §5.1, A.14Fig. 4 (main text), Fig. 16, Table 10
Distinct axes vs shared APOE/severity signalcYd3, a6y4§4.2.2, §5.2.1, A.16, A.17Tables 13–18, Fig. 19
Identifiability assumptions, many-to-one settingsa6y4A.13, A.15Table 12
Gene–environment correlation and interaction7z5W, a6y4A.15Fig. 17, Table 11
Scope of claims: genetically informed disease axesa6y4Throughout (§5.1, §5.2, §6)
Negative inputs / NMF consistencycYd3§3.2, A.6.2
Model configuration (K, K_g, K_r)cYd3A.7Fig. 11 (corrected)
MCI/AD-only training, selection biascYd3§4.2.2, A.18Fig. 18, Table 19
Notation7z5W§3, §3.2

We thank the reviewers again. Their questions helped us further test the robustness of our model and the support for our claims.

AuthorsOpenReview ↗

Final remarks

Dear Area Chair and Reviewers,

We thank all reviewers for their engagement and would like to briefly summarize the broader contribution of the work.

The problem. Brain atrophy reflects several overlapping processes, including genetic, vascular, ageing and environmental factors. Imaging-only decompositions cannot tell which patterns are driven by a given factor. Existing imaging-genetics methods are either association models on ROIs, symmetric joint factorizations, or supervised, label-driven fusion. We address this as a representation learning question: how can a weak, partial auxiliary modality shape an imaging representation without being required to explain all of it? (7z5W)

What the paper contributes.

  • To our knowledge, the first self-supervised imaging-genetics model to learn an interpretable, voxel-level decomposition of brain structure.
  • Partial, directional guidance. A frozen genetic embedding guides only part of the latent space through a predictor, so the guided latents carry only the variation the guiding modality can predict. A residual block captures the rest. (noted by all reviewers)
  • Interpretable by construction. An NMF-style nonnegative decoder gives additive spatial maps at the voxel level, building on an established interpretable framework (Sotiras et al., 2015). It needs no diagnostic labels, ROI parcellation or post-hoc attribution (a6y4, cYd3).
  • Modular. Genetics is one instance. Replacing the genetic inputs with clinical risk factors (blood pressure, diabetes, BMI and others) yields imaging patterns distinct from the genetic axes (Section 5.2.2, Fig. 15), showing that the same framework applies to other guiding modalities and clinical questions.
  • Flexible at inference. With imaging alone, the model provides guided axes for cohorts without the guiding modality (a6y4, cYd3). With both modalities, it allows predisposition (predicted from genetics or risk factors) to be compared with manifestation (observed atrophy), which can point to preclinical risk or other drivers and offers a natural route to clinical translation. In genotyped external cohorts, it opens the way to causal analyses such as Mendelian randomization.
  • Controlled evaluation. Semi-synthetic data with known ground truth across three regimes (partial overlap, large overlap, weak effects), compared against representative families of existing methods: association models, joint factorization and weakly-supervised latent fusion (a6y4).

Evidence strengthened during the discussion.

  • Why the predictor (raised by all reviewers). A controlled study of each coupling design choice (Section 5.1, Fig. 4; Appendix A.14) shows that variants that remove or reverse the predictor absorb non-genetic patterns into genetic components or lose the genetic signal under weak effects, the realistic regime for imaging genetics. The full bidirectional objective gives the most consistent genetic recovery.
  • Identifiability. We state the conditions under which genetic and residual factors can be separated (A.13) and test the model when they are violated: correlated, interacting and completely overlapping factors (A.15).
  • Distinct axes. APOE-conditional, chr19-excluded, ε4-stratified and severity-controlled analyses (A.16).
  • External longitudinal validation. In NACC, individual profiles are stable over a median of 2.5 years, and cross-lagged models show that no axis is a later stage of another (A.17).
  • Training population. Results hold when cognitively normal participants are included (A.18).

Clinical value.

  • External validation across eight independent cohorts, including autopsy-confirmed neuropathology, with distinct amyloid/tau, TDP-43 and cerebrovascular profiles (noted by all reviewers).
  • Associations with diagnosis, cognition and cognitive decline in external cohorts.
  • Applicable wherever MRI is available, and extensible to other guiding modalities.

We believe G2B-AE offers both the representation learning and neuroimaging communities a general, interpretable way for a partial auxiliary signal to shape a learned representation, demonstrated on a clinically important problem.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.