acceptodds
Under review as a conference paper at ICLR 2027

Deep Independent Vector Analysis

Abstract

We introduce Deep Independent Vector Analysis (DeepIVA), a multivariate latent variable model for learning linked and identifiable latent sources from multimodal data. DeepIVA jointly identifies sources within each modality and aligns sources across modalities. We propose a rigorous set of performance metrics measuring unimodal identifiability, cross-modal linkage, and cross-segment consistency. We validate DeepIVA using synthetic data and then apply it to a large multimodal neuroimaging dataset with structural and functional magnetic resonance imaging. In the synthetic data experiment, DeepIVA successfully recovers nonlinearly mixed multimodal sources, outperforming baseline methods. In the neuroimaging experiment, DeepIVA reveals linked imaging biomarkers associated with sex and age, shedding light on coupled structural-functional brain relationships.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Independent Vector Analysis, Latent Variable Modeling, Multimodal Fusion, Neuroimaging

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Pd7DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce DeepIVA, a latent variable model with the purpose of finding independent vectors that remain identifiable and are linked across multiple modalities. To do so, a two-step architecture is introduced. Step 1 trains individual iVAEs per modality given data x\bm{x} and auxiliary variable u\bm{u}. Step 2 freezes the part of the encoder that takes in u\bm{u}, and uses the MISA loss to align the independent vectors from each modality. This leads to a blind source separation model that is identifiable, multimodal, nonlinear, and links the multiple modalities. Synthetic results show a better mean correlation coefficient (MCC) and minimum distance (MD) than available baselines. Results on neuroimaging find sources that identify auxiliary variables.

Critical strengths
  • Focus on Identifiability: The paper does a good job at focusing on maintaining identifiability of the independent vectors found while introducing non-linearities. Connecting the iVAE to MISA via Sklar’s Copula Theorem is clever conceptual framing and aims to tackle a gap in the BSS literature by aligning the representations across modalities.

  • Promising Results: Positive results on synthetic data show successful recovery of unimodal sources with proper cross-modal linkage. The neuroimaging experiments show an interesting application of recovering SCVs using DeepIVA.

Critical weaknesses
  • Unclear Novelty and Practical Implementation: The methods section is unclear and fails to separate previously-existing methods and architecture from new insights and novelty. At its core, DeepIVA is combining iVAE with a MISA objective via an alternating optimization approach. While this is interesting in concept, the main text focuses on the math and conceptual nature of iVAEs and the MISA loss, and only introduces a sketch of Theorem 2.5 in the main text (which would be considered the main contribution). I would instead like to see more details regarding the alternating optimization, as some things remain unclear. I do not understand how model parameters with respect to u\bm{u} remain unchanged in hidden layers given shared downstream weights, or how setting u\bm{u} to 0 does not distort encoder features through an out-of-distribution auxiliary vector. The authors should clarify how the layer-freezing scheme preserves the iVAE identifiability.

  • Incomplete Identifiability Claims: There is a critical gap in the identifiability proof of DeepIVA (Theorem 2.5). While the authors show that an identifiable cross-modal solution exists, there is no guarantee that this solution is found via backpropagation. Because the MISA loss changes iVAE encoder parameters in Step 2 without using uu as a structural anchor, the network can minimize this loss in ways that completely destroy the independent, identifiable structure in Step 1. To complete the identifiability claims, authors need to show that the backpropagation step preserves the iVAE guarantees, not just that the optimal aligned version is identifiable.

Questions for authors
  1. Given that the weights of the encoder are changed during the cross-modal alignment step, can you provide evidence that this doesn’t break the identifiability claims presented in Theorem 2.5?

  2. Can you provide additional details on the freezing of the encoder for uu in the MISA training step? It is unclear if only the input layer remains frozen, and how parameters are updated during the MISA step. Besides, setting uu to 0 during the MISA training step does not guarantee no influence unless strict assumptions are made.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Pd7D

Thank you for your valuable feedback. We respond to each point below (W: Weakness; Q: Question). W1 is addressed together with Q2, and W2 together with Q1. All section, figure, and appendix numbers below refer to the revised manuscript, where changes are highlighted in blue.

W1 novelty.

We agree that the original Methods section did not clearly separate existing methods from our contributions. DeepIVA builds on two existing methods: iVAE [Khemakhem et al., 2020] for unimodal source identification and MISA [Silva et al., 2020] for cross-modal alignment. Our contributions are (i) a framework that combines them through alternating optimization with segment-wise MISA updates, (ii) an identifiability result for linked nonlinear sources across modalities (Theorem 2.5), and (iii) an evaluation on synthetic data and a large multimodal neuroimaging dataset. We have added this statement to the Discussion (Section 4) and expanded the description of the alternating optimization in Section 2.1.5, as detailed below.

W1/Q2 implementation of the MISA step.

Thank you for this careful question. We apologize that our original description of the MISA step was misleading; we have rewritten it in Section 2.1.5 of the revised manuscript. For each modality, the iVAE consists of four separate networks: the encoder mean network g(x)g(x), the encoder variance network, the decoder f(z)f(z), and the prior network λ(u)\lambda(u), which parameterizes the conditional prior p(z∣u)p(z \mid u). In our implementation, the auxiliary variable uu enters only the prior network; the encoder takes only xx as input. Consequently, no uu-specific weights are shared with the encoder, uu is not set to zero, and the encoder never receives an out-of-distribution auxiliary input. The alternating optimization proceeds as follows. (1) iVAE step: all four networks are updated by maximizing the ELBO on (x,u)(x, u). (2) MISA step: the MISA loss depends only on the encoder output g(x)g(x), so only the encoder (all of its layers) is updated, while the decoder, prior network, and variance network remain unchanged. The MISA loss is computed separately for each segment, and each epoch cycles over all segments before the next iVAE step.

W2/Q1 identifiability after the MISA step.

Thank you for this important question. We agree that Theorem 2.5 characterizes the optimum and does not by itself guarantee that alternating gradient-based optimization reaches it. This limitation is shared by identifiability results for deep latent variable models in general, including iVAE [Khemakhem et al., 2020], whose guarantee also holds at the optimum of the ELBO rather than for a particular optimization trajectory. We now state this explicitly in a remark following Theorem 2.5 (Section 2.1.4). We address the concern in three ways.

(i) The auxiliary variable is used in the MISA step. The MISA loss is computed separately for each segment, i.e., for each value of uu, after removing the segment mean. The segment structure defined by uu therefore anchors both the iVAE and the MISA steps.

(ii) The MISA step is compatible with iVAE identifiability. The iVAE identifies the sources of each modality only up to permutation and component-wise invertible transformations, so the marginal distribution of each recovered source is not determined by the iVAE. Imposing the Kotz distribution of the MISA objective on the source component vectors therefore does not conflict with the iVAE solution; the MISA step uses this remaining freedom to align sources across modalities. In addition, the MISA learning rate is one to two orders of magnitude smaller than the iVAE learning rate (3.2×10−53.2\times10^{-5} to 8×10−58\times10^{-5} vs. 10−310^{-3}), and every round of MISA updates is followed by iVAE updates that restore the ELBO.

(iii) Empirical evidence. If the MISA updates destroyed the identifiable iVAE structure, the unimodal recovery would degrade during joint training. To test this directly, we tracked the MCC on the training set every 5 epochs throughout joint training on synthetic data with known ground truth (8 seeds, dynamic MISA learning-rate schedule; Figure A1). The unimodal MCC of both modalities increases from about 0.69 at initialization to above 0.95 by epoch 300 and then remains at 0.97 (s.d. ≤0.003\leq 0.003) until epoch 1000, while the cross-modal MCC between the recovered sources of the two modalities increases from 0.56 to 0.73. The unimodal MCC never drops by more than 0.014 between consecutive evaluations, and at the epoch with the highest cross-modal MCC it ranges from 0.96 to 0.97 across seeds. Results are consistent with a fixed MISA learning rate (unimodal MCC 0.94–0.97, cross-modal MCC 0.71–0.74; Figure A6). The MISA updates therefore align sources across modalities without degrading unimodal identifiability. We have added this analysis to Appendix B.2 (Figure A1) of the revised manuscript.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer Pd7D,

Thank you again for your careful reading of our method. As the discussion period is ending soon, we wanted to kindly follow up on our response. In the revised manuscript, we rewrote the description of the alternating optimization to clarify that the auxiliary variable enters the prior network and that the MISA step updates the encoder (Section 2.1.5). We also added a remark after Theorem 2.5 on the relationship between identifiability and optimization (Section 2.1.4), and training-dynamics results showing that unimodal MCC remains at 0.97 while cross-modal MCC increases during joint training (Appendix B.2). In addition, we now separate the existing methods from our contributions (Section 4).

We would greatly appreciate hearing whether these revisions address your concerns, and we are happy to clarify anything further. If they do, we would be grateful if you would consider updating your score.

Reviewer_x7oZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a multimodal latent variable model to recover sources that are identifiable within modalities and linked across modalities. It also introduces metrics to evaluate these properties. DeepIVA improves source recovery under nonlinear mixing in synthetic experiments and identifies linked imaging biomarkers associated with sex and age.

Critical strengths
  1. The motivation is clear, combining joint nonlinear identifiability with cross-modal linkage.
  2. The paper provides extensive evaluation metrics and a large real-world application.
Critical weaknesses
  1. The "deep" part is never tested on real data. In the neuroimaging experiment, the nonlinear activation is skipped, so DeepIVA becomes a linear model. Its MCC is also similar to the mvICA baseline.
  2. The comparison on synthetic data is not fair. Only the proposed method's hyperparameters are tuned on the synthetic data, while the baselines use default settings.
Questions for authors
  1. From my perspective, linear dimension reduction does not make the underlying mixing linear. Could the authors justify skipping the nonlinearity, or report results with it enabled?
  2. Since age and sex are used as auxiliary variables, is it circular to then identify sources associated with age and sex?
3 comments
AuthorsOpenReview ↗

Response to Reviewer x7oZ

Thank you for your feedback. We respond to each point below (W: Weakness; Q: Question). All section, figure, and appendix numbers below refer to the revised manuscript, where changes are highlighted in blue.

W1/Q1 nonlinear DeepIVA on real data.

We agree that linear dimensionality reduction does not make the underlying mixing linear. Using a linear encoder on real data was a modeling simplification, which we listed as a limitation. We now report results with the nonlinearity enabled. Nonlinear DeepIVA (two-layer leaky ReLU MLP, initialized from the converged linear model) achieves a cross-modal MCC of 0.398±0.0110.398 \pm 0.011 on the training set and 0.418±0.0130.418 \pm 0.013 on the test set across 8 seeds (Figure A9). Regarding the comparison with mvICA, both linear and nonlinear DeepIVA significantly outperform mvICA on the training set (p<0.01p < 0.01 and p<0.001p < 0.001, Welch's tt-test over 8 seeds), whereas the differences on the validation and test sets are not significant (Figure A11). Please see our response to Reviewer giMD (W1/Q1, W2/Q4) for details. We have revised Sections 2.4, 3.2, and 4 and added Appendices D.3 and D.5 to reflect these results.

W2 hyperparameter tuning on synthetic data.

We appreciate this point. We believe that the gap between DeepIVA and the baselines on synthetic data stems mainly from their modeling assumptions rather than from tuning: the linear methods (ICA, MISA, mvICA) cannot model nonlinear mixing, unimodal iVAE does not link sources across modalities, and group iVAE assumes identical sources across modalities. These limitations persist regardless of hyperparameter values. To reduce the dependence of DeepIVA on tuning, we also ran DeepIVA with a fixed MISA learning rate set by a simple rule (the iVAE learning rate divided by the number of segments) instead of the tuned dynamic schedule. It still achieves unimodal MCC of 0.94–0.97 and cross-modal MCC of 0.71–0.74 across 8 seeds (Figure A6; Section 2.3 and Appendix C.3).

Q2 circularity of age and sex.

We evaluated the age and sex associations on a held-out test set (Figure A15; Appendix D.8), and we additionally found age- and sex-adjusted associations with phenotypes not used in training (most notably time spent watching television) that are significant on the held-out test set after FDR correction (Section 3.2 and Appendix D.7). Moreover, when sex was removed from the segmentation, almost all sources became associated with sex, indicating that the auxiliary variable conditions age and sex effects out of the sources rather than building them in (Appendix D.6). Please see our response to Reviewer giMD (Q2, Q3) for details.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer x7oZ,

Thank you again for your helpful feedback. As the discussion period is ending soon, we wanted to kindly follow up on our response. In the revised manuscript, we report nonlinear DeepIVA results on the neuroimaging data with variability across 8 seeds (Appendices D.3 and D.5), show that a fixed MISA learning rate set by a simple rule performs comparably to the tuned schedule (Appendix C.3), and address the circularity of age and sex with held-out phenotype associations and a segmentation sensitivity analysis (Appendices D.6 and D.7).

We would greatly appreciate hearing whether these revisions address your concerns, and we are happy to clarify anything further. If they do, we would be grateful if you would consider updating your score.

Thank the authors for the response and additional experiments.

  1. W1/Q1: The added nonlinear real data experiments have addressed part of my concerns. However, as the authors stated, the proposed method does not perform much better than the baseline on validation and test sets.

  2. W2: The argument made by the authors can only show the robustness of the proposed method in the given setting. It does not address the fairness issue of hyperparameter tuning for baselines.

  3. Q2: The added ablation has addressed my concern. Thank you!

Overall, the rebuttal has addressed some of the questions, but the real data improvement is not significant, and the fairness concern is still there. I will keep my rating.

Reviewer_giMDOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Deep Independent Vector Analysis (DeepIVA), a multimodal framework that combines identifiable variational autoencoders (iVAEs) for identifying latent sources within each modality and multidataset independent subspace analysis (MISA) for aligning sources across modalities. The authors provide a theoretical analysis of identifiability and evaluate the method using synthetic data and UK Biobank structural and functional MRI data. DeepIVA achieves improved source recovery and cross-modal alignment compared with several baseline methods on synthetic data. The neuroimaging analysis also identifies linked components associated with age and sex.

Critical strengths
  1. The paper addresses an important problem in multimodal representation learning by considering both source identifiability within each modality and alignment across modalities. The integration of iVAE and MISA provides a reasonable framework for addressing these two objectives.

  2. The synthetic experiments include comparisons with several baseline methods and evaluate both source recovery and cross-modal alignment. The application to UK Biobank neuroimaging data demonstrates the potential use of the method on real multimodal datasets.

Critical weaknesses
  1. The real-data experiments do not fully support the proposed nonlinear framework. In Section 2.4 (lines 365–367), the authors assume linear mixing after dimensionality reduction and remove the nonlinear activation functions. Therefore, the neuroimaging experiments do not demonstrate whether nonlinear modeling provides additional benefits compared with existing linear methods. This limits the experimental support for one of the main contributions of the paper.

  2. The contribution of individual model components is not sufficiently evaluated. DeepIVA combines iVAE and MISA through alternating optimization, but there is limited analysis of how different components and training strategies affect performance. Additional ablation experiments would help clarify the source of the improvements. Furthermore, the improvement over mvICA in the neuroimaging experiment is relatively small (cross-modal MCC of 0.38 versus 0.36 on the training set). Results across multiple random seeds would help determine whether this improvement is consistent and significant.

Questions for authors
  1. Can the authors evaluate the nonlinear version of DeepIVA on real data? This would help demonstrate whether nonlinear modeling provides benefits beyond existing linear methods.

  2. Age and sex are used as auxiliary variables during training and are also used to evaluate the recovered components. Is there any circularity issue here? Can the authors provide additional validation using phenotypes that are not included as auxiliary variables?

  3. How sensitive is DeepIVA to the choice of auxiliary variables and the definition of the 22 age/sex segments?

  4. Can the authors provide ablation experiments to evaluate the contributions of the MISA alignment objective, alternating optimization, and learning-rate schedule? It would also be helpful to report the variability in performance across different random seeds.

4 comments
AuthorsOpenReview ↗

Response to Reviewer giMD (1/2)

Thank you for your constructive feedback. We respond to each point below (W: Weakness; Q: Question). All section, figure, and appendix numbers below refer to the revised manuscript, where changes are highlighted in blue.

W1/Q1 nonlinear DeepIVA on real data.

We agree with this point. The neuroimaging experiment in the original submission used a linear encoder, which we acknowledged as a limitation. To address it, we trained a nonlinear DeepIVA on the same MGPCA-reduced sMRI and fMRI data. Each iVAE network is a two-layer MLP with leaky ReLU activations plus an affine skip connection. The skip connection is initialized from the converged linear DeepIVA, and the output layer of the nonlinear branch is initialized to zero. Training therefore starts exactly at the linear solution, and nonlinearity is added only where it improves the objective. We trained the model with 8 random seeds for 300 epochs and selected the epoch with the highest validation cross-modal MCC. As shown in Figure A9, nonlinear DeepIVA achieves a cross-modal MCC of 0.398±0.0110.398 \pm 0.011 on the training set and 0.418±0.0130.418 \pm 0.013 on the held-out test set (mean ±\pm s.d. over 8 seeds). For comparison, linear DeepIVA achieves 0.3740.374 and 0.4150.415, and mvICA achieves 0.3610.361 and 0.4090.409 (Figure A11). These results show that DeepIVA with nonlinear activations can be trained stably on real data when initialized from the linear solution. We have added this experiment to the revised manuscript (Sections 2.4 and 3.2, Appendix D.3, Figure A9).

W2/Q4 ablations and variability across seeds.

Model components. The original manuscript compares DeepIVA with each of its two components alone: iVAE without MISA, and MISA without iVAE (Figures 3a, 3b, and 4a). These comparisons show that iVAE recovers nonlinearly mixed sources within each modality but cannot align them across modalities, whereas MISA aligns sources across modalities only weakly and cannot fully recover the unimodal sources. DeepIVA combines both strengths.

Learning-rate schedule. We replaced the dynamic MISA learning-rate schedule with a fixed learning rate of 4.55×10−54.55\times10^{-5}, i.e., the iVAE learning rate (10−310^{-3}) divided by the number of segments (2222). This experiment uses synthetic data with two modalities, 10 sources, 22 segments, 130 samples per segment, and a two-layer leaky ReLU mixing function. As shown in Figure A6, across 8 seeds the fixed learning rate yields unimodal MCC between 0.94 and 0.97 and cross-modal MCC between 0.71 and 0.74. For reference, the cross-modal MCC of the ground-truth sources themselves is 0.76–0.80, so DeepIVA recovers most of the true cross-modal dependence. In earlier experiments, however, a fixed learning rate did not reach the best solution in every setting, which is why we proposed the dynamic schedule. In practice, we recommend starting with this fixed learning rate and tuning the dynamic schedule only if needed. We have added this experiment and guidance to the revised manuscript (Section 2.3 and Appendix C.3, Figure A6).

Variability across seeds. We repeated the neuroimaging experiment with 8 seeds for all methods (Figure A11). On the training set, both linear DeepIVA (0.3740.374 vs. 0.3610.361, p<0.01p < 0.01) and nonlinear DeepIVA (0.3980.398 vs. 0.3610.361, p<0.001p < 0.001) achieve significantly higher cross-modal MCC than mvICA (two-sided Welch's tt-test). On the validation and test sets, the differences are not statistically significant (test set: 0.4150.415 and 0.4180.418 vs. 0.4090.409). We have added these results to the revised manuscript and revised the comparison with mvICA accordingly (Section 3.2 and Appendix D.5, Figure A11).

AuthorsOpenReview ↗

Response to Reviewer giMD (2/2)

Q2 circularity of age and sex.

Thank you for raising this point. Age and sex enter DeepIVA only through the conditional prior: the age–sex bins (a single categorical variable, not the actual age and sex values) define segments within which the source variances may differ. The model uses these bins to identify sources, but it is not trained to predict age or sex. In addition, we evaluated the associations with age and sex on the held-out test set, which was not used for training, and found consistent spatial patterns (Figure A15; Appendix D.8).

We agree that validation with phenotypes not used during training is valuable. We therefore tested associations between the recovered sources and 25 UK Biobank phenotypes that were not used in training. To remove age and sex effects, we regressed age and sex out of both the sources and the phenotypes and computed partial correlations, with Benjamini–Hochberg FDR correction. On the training set, 39 associations were significant after FDR correction (q<0.05q < 0.05). On the held-out test set, 8 associations were significant after FDR correction (Figure A14): seven involving sMRI sources, most notably five sources associated with time spent watching television, and one involving an fMRI source. We have added this analysis to the revised manuscript (Section 3.2 and Appendix D.7, Figure A14).

Our segmentation sensitivity analysis (Q3) further suggests that using age and sex as the auxiliary variable conditions their effects out of the sources rather than building them in: when sex was removed from the segmentation, almost all sources became associated with sex (9.9 of 10 sMRI and 10 of 10 fMRI sources, vs. 5.9 and 6.1 with the age–sex segments; Figure A13 and Appendix D.6).

Q3 choice of auxiliary variable and segments.

We chose age and sex as the auxiliary variable because both are major, well-established sources of variation in brain structure and function. The number of segments follows from Theorem 2.3 (iv): LL is invertible only with at least kNs+1kN_s + 1 segments, i.e., 21 for k=2k=2 sufficient statistics and Ns=10N_s=10 sources. We therefore used 22 segments, 11 age bins for each sex (94–171 subjects per segment), and have added this rationale to Section 2.4.

To test the sensitivity to these choices, we reran the neuroimaging experiment with (i) 28 segments (14 age bins per sex) and (ii) 22 age bins with the sexes pooled, keeping the subjects, data split, inputs, seeds, and all other settings fixed. We measured agreement with the reference as the mean absolute correlation between test-set sources after jointly matching the linked sMRI–fMRI pairs; the agreement between reference seeds is the baseline (0.675±0.0080.675 \pm 0.008), and random rotations of the inputs give chance (0.4630.463).

DeepIVA is robust to the number of segments but not to the variables that define them (Figure A12). With 28 segments, the agreement (0.645±0.0130.645 \pm 0.013) recovers 86% of the gap between chance and the baseline, the cross-modal MCC is similar (0.3970.397 vs. 0.4150.415), and the MD is slightly better (0.4100.410 vs. 0.4420.442). With age alone, the agreement drops to 0.568±0.0210.568 \pm 0.021 (50% of the gap), and the MCC increases (0.4470.447) but so does the MD (0.5360.536): almost all sources become associated with sex (Figure A13; on average 9.9 of 10 sMRI and 10 of 10 fMRI sources, vs. 5.9 and 6.1 for the reference), so the sources share a common sex effect and become less independent. Because the MISA loss is computed after removing each segment's mean, age–sex segments condition the mean age and sex effects out of the sources, whereas without sex the strong shared sex effect is absorbed into them.

This also bears on the circularity concern (Q2): using age and sex as the auxiliary variable conditions their effects out rather than building them in. Consistent with this, age- and sex-adjusted phenotype associations that are significant on the held-out test set after FDR correction (as in Appendix D.7) are more numerous with age–sex segments (8.8±2.18.8 \pm 2.1 for the reference and 4.4±1.64.4 \pm 1.6 for 28 segments) than with age only (3.4±2.53.4 \pm 2.5). We have added this analysis to Section 3.2 and Appendix D.6 (Figures A12 and A13) and revised the limitations in Section 4 accordingly.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer giMD,

Thank you again for your detailed and constructive review. As the discussion period is ending soon, we wanted to kindly follow up on our response. Following your suggestions, we added (i) nonlinear DeepIVA on the neuroimaging data (Appendix D.3), (ii) results across 8 random seeds with a revised comparison to mvICA (Appendix D.5), (iii) a fixed MISA learning-rate experiment (Appendix C.3), (iv) a sensitivity analysis of the number of segments and the variables that define them (Appendix D.6), and (v) age- and sex-adjusted associations with phenotypes not used in training to address the circularity concern (Appendix D.7).

We would greatly appreciate hearing whether these revisions address your concerns, and we are happy to clarify anything further. If they do, we would be grateful if you would consider updating your score.

I appreciate the authors’ efforts to address the concerns and have raised my rating to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We thank all reviewers for their constructive comments. The main changes in the revised manuscript (highlighted in blue) are:

  • Remark on identifiability and optimization (Section 2.1.4, after Theorem 2.5; Reviewer Pd7D): we state that, as for iVAE, Theorem 2.5 characterizes the optimum rather than the optimization trajectory, and explain why the segment-wise MISA step remains consistent with the unimodal iVAE solution.
  • Training dynamics (Appendix B.2, Figure A1; Reviewer Pd7D): we track MCC during joint training on synthetic data; unimodal MCC rises to and stays at 0.97 while cross-modal MCC increases to 0.73, showing that the MISA updates do not degrade unimodal identifiability.
  • Implementation of the alternating optimization (Section 2.1.5; Reviewer Pd7D): the encoder takes only the imaging data as input, the auxiliary variable enters only the prior network, and the MISA step updates only the encoder mean network.
  • Fixed MISA learning rate (Section 2.3 and Appendix C.3, Figure A6; Reviewers giMD and x7oZ): replacing the tuned dynamic schedule with a fixed rate (the iVAE learning rate divided by the number of segments) still yields unimodal MCC of 0.94–0.97 and cross-modal MCC of 0.71–0.74; we add practical guidance on when to tune the schedule.
  • Rationale for the 22 segments (Section 2.4; Reviewer giMD): the number of segments follows from the segment requirement of Theorem 2.3 (iv); we use 11 age bins for each sex.
  • Nonlinear DeepIVA on neuroimaging data (Sections 2.4 and 3.2, Appendix D.3, Figure A9; Reviewers giMD and x7oZ): a leaky ReLU DeepIVA initialized from the linear solution achieves a cross-modal MCC of 0.398 (training) and 0.418 (test) across 8 seeds.
  • Variability across 8 random seeds (Section 3.2 and Appendix D.5, Figure A11; Reviewers giMD and x7oZ): linear and nonlinear DeepIVA significantly outperform mvICA on the training set, whereas differences on the validation and test sets are not significant; the comparison with mvICA is revised accordingly.
  • Sensitivity to the auxiliary variable and segmentation (Sections 3.2 and 4, Appendix D.6, Figures A12 and A13; Reviewer giMD): with 28 instead of 22 age–sex segments, the recovered sources remain close to the reference solution, whereas age-only segments substantially reduce agreement with it and spread the sex effect to almost all sources.
  • Associations with phenotypes not used in training (Section 3.2 and Appendix D.7, Figure A14; Reviewers giMD and x7oZ): to address circularity, we test age- and sex-adjusted associations with 25 UK Biobank phenotypes; 39 associations are significant on the training set and 8 on the held-out test set after FDR correction, mostly involving sMRI sources and time spent watching television.
  • Contributions and limitations (Section 4; Reviewers Pd7D and giMD): we separate the existing methods (iVAE and MISA) from our contributions, and extend the limitations to the linear encoder in the main neuroimaging analysis and the sensitivity to the variables that define the segments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.