- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies source recovery in nonlinear multi-view CCA, focusing on the conditions under which optimizing SUMCOR leads to recovery. It considers three cases: 1) recovery under global first-order dominance, 2) recovery using rank-constrained objectives under pairwise first-order dominance and known overlap ranks, and 3) recovery using finite overcomplete representations under joint spectral compatibility. These results highlight the role of first-order dominance in source recovery.
Critical strengths
-
The paper analyzes the global optimality properties of SUMCOR for source recovery in nonlinear multi-view CCA.
-
It discusses three recovery regimes under different conditions and provides experiments supporting the theoretical results.
Critical weaknesses
-
The recovery guarantees seem to rely heavily on the first-order dominance assumptions. However, it is not clearly explained why source recovery requires the stated assumptions (Assumptions 2, 4, and 5) or what role each assumption plays in establishing the corresponding theorems. The authors should provide more insights or high-level interpretations.
-
The criterion for successful recovery needs clearer explanation. How should readers determine whether optimizing Equation (2) successfully recovers ? In particular, what is meant by "up to an affine transformation"? Could the authors state an explicit mathematical definition or criterion for correct recovery of in this sense? In the experiments, how is recovered from the learned representation ? Please explain how the affine readout is fitted and how the recovery is evaluated.
-
Many equations are presented with insufficient intuition or explanation. In particular, Section 3.2 is difficult to follow, and the motivation for its mathematical constructions needs to be clearer.
-
Is Proposition 2 intended to state that, under Assumptions 1–3, if there exist matrices and functions satisfying the stated requirements and Equation (3), then any encoder collection that fully recovers the sources in all views is strictly suboptimal for SUMCOR. If so, do such and necessarily exist under Assumptions 1–3, and how can their existence be proved? If no such and exist, would the conclusion of Proposition 2 change?
-
The representation dimension required by Theorem 4.3 depends on , but how to calculate this quantity in practice needs clearer explanation. In addition, please clarify the distribution with respect to which the expectation in is taken. Is there any assumptions on the distributions of and ? In particular, it would be helpful to show how to calculate for the Gaussian model in Equation (1) under Assumption 2.
-
The function class and practical implementation of in Equation (2) need clearer explanation. Is restricted to linear or affine maps, or can it be nonlinear? Is the optimization in Equation (2) performed over the function space ? If so, how can a solution be obtained in practice? Although Section 5.1 mentions residual encoders and Adam, please explain more explicitly how is parameterized in the experiments and how the optimization problem is solved.
Questions for authors
see the weaknesses.
5 comments
W2. What counts as recovery, and how is it evaluated?
We now state an explicit recovery criterion in Definition 2 and explain how the affine readout is fitted and scored. A high SUMCOR value alone does not establish recovery.
Recovery criterion. An encoder recovers the noisy source if
\quad\text{almost surely}$$ for fixed $\mathbf L_i\in\mathbb R^{m_i\times d_{\mathcal Z_i}}$ and $\mathbf b_i\in\mathbb R^{m_i}$. Equivalently, every source coordinate, and hence every nonzero linear source function, has population $R^2=1$ under its best affine readout. Our theorems ensure this at **every population global optimum** under their stated assumptions and widths. **Affine ambiguity.** At matched width $d_{\mathcal Z_i}=m_i$, $\mathbf L_i$ is invertible because the source covariance is nonsingular under A1. Thus $\mathbf z_i$ is an invertible affine transformation of $\mathbf s_i$. SUMCOR cannot remove this ambiguity because Equation (2) is invariant to such transformations. At larger width, recovery requires only an affine source readout, and the remaining coordinates may be nonlinear. **Readout evaluation.** Sources do not enter the encoder training objective. Evaluation uses ten batches of 4,096 samples from a random stream independent of the training samples. On each batch and view, we fit an affine least-squares readout, including an intercept, from the complete representation to the noisy source and score it on the same batch. We average coordinate-wise $R^2$ over source coordinates, views, and batches. Appendix F.3 estimates the optimism from same-batch fitting under a homoscedastic approximation. This is not a finite-sample bound and does not account for initialization selection on these observations. **Detecting missing directions.** Worst-direction $R^2$ is the smallest same-batch score over all nonzero linear source functions $\mathbf a^\top\mathbf s_i$. It equals the squared smallest source–representation canonical correlation on each batch. Its population value is one if and only if Definition 2 holds. For Setups B/C, Table 3 reports the batch mean of this correlation, **unsquared**, in the worst view as $\gamma_{\min}$. The Setup B results at width three show why the worst-direction diagnostic matters (Tables 1 and 3): | Method | Average $R^2$ (%) | $\gamma_{\min}$ (%) | |---|---|---| | SUMCOR | 68.47 ± 0.21 | 1.06 ± 0.70 | | tSUMCOR | 99.93 ± 0.00 | 99.95 ± 0.01 | | mSUMCOR | 99.94 ± 0.00 | 99.94 ± 0.01 | Here tSUMCOR and mSUMCOR denote Top-$2$ and masked SUMCOR. The standard deviation is over ten evaluation batches of one trained model. SUMCOR's average $R^2$ indicates incomplete recovery, while $\gamma_{\min}$ reveals an almost entirely lost source direction. The corresponding batch-averaged worst-direction $R^2$ is below 0.1% for SUMCOR and above 99.8% for both rank-constrained methods. The five-seed results show the same separation (Tables 4 and 12). Their worst-direction $R^2$ is averaged over batches, then minimized over views. Initialization selection uses the objective on evaluation observations without sources (Appendix F.6). Setup A reports average $R^2$ only.W3. What is the intuition behind Section 3.2?
Section 3.2 introduces the spectral tools and pairwise first-order dominance condition used in our later theorems, connecting our analysis to classic spectral theory and prior nonlinear CCA results. The constructions serve three purposes:
-
Whitening and canonical coordinates. Whitening the representations removes arbitrary scale. Under A1, whitening the sources and applying the SVD of the normalized source cross-covariance decomposes their joint law into mutually independent Gaussian pairs and uncoupled coordinates, while is not required to be full rank. Within each pair, the two variables retain correlation . This factorization makes the cross-view dependence explicit.
-
Hermite expansion. Under the isotropic Gaussian model (A1), the orthonormal Hermite basis separates polynomial degrees, isolating first-order source functions from higher-order components. Hermite expansion also gives a closed-form spectral description: for a canonical Gaussian pair with correlation , matching degree- features have correlation for , and different degrees are uncorrelated. For multiple canonical pairs, a nonconstant mode with degrees has strength . These are the nonzero singular values of the cross-view conditional-expectation operator on centered functions. Whitening the representations makes their Hermite coefficient rows orthonormal, allowing CCA to compare subspaces of these modes. The scalar Gaussian case is classical (Lancaster, 1958), and Han et al. (2026a; 2026b) used it with whitened encoders for nonlinear CCA. We do not claim it as our contribution. Appendix B.3 states the bound we use (Equation (18)).
-
Connection to prior theory. The weakest nonzero first-order canonical correlation is , while the strongest higher-order one is . Pairwise first-order dominance (A2) places every correlated first-order mode ahead of all higher-order modes. Thus, under A1 and A2, every population pairwise CCA maximizer with representation dimension recovers the pairwise first-order targets, as recalled from Han et al. (2026b) in Proposition 1. Appendix B.3 reuses this Hermite–Mehler spectral bound to control higher-order correlations, then compares their aggregate gains with first-order losses when one encoder serves several view pairs.
W4. Is the existence condition in Proposition 2 guaranteed?
Proposition 2 is conditional. If there exist matrices and functions satisfying the stated requirements and Equation (3), then every encoder collection permitting full affine source recovery in all views at minimum-sufficient capacity is strictly suboptimal for SUMCOR. A1–A3 alone do not guarantee that such matrices and functions exist.
Explicit construction. Appendix B.2 gives a three-view example with three-dimensional sources and full cross-view latent coverage. In its aligned, whitened source coordinates, let denote the standardized globally shared coordinate and choose
\boldsymbol{\psi}_i(\mathbf{s}_i) =\bigl(H_2(\zeta_i),H_3(\zeta_i)\bigr)^{\top},$$ where $H_2$ and $H_3$ are normalized Hermite polynomials. These functions are centered, whitened, and orthogonal to all first-order source coordinates, so concatenating them with $\zeta_i$ gives a feasible three-dimensional representation. For the Setup B correlations $\rho=0.6$ (globally shared) and $\tau=0.45$ (pair-specific), A2 holds because $0.45 > 0.6^2=0.36$, yet this construction gives $$J_{\mathrm{drift}}=3(\rho+\rho^2+\rho^3)=3.528 \gt 3.15=3(\rho+\tau)=J_{\mathrm{aff}}.$$ Here $J_{\mathrm{aff}}$ is the value of every collection permitting full affine source recovery at minimum-sufficient capacity. A feasible collection with a larger value proves strict suboptimality. It need not itself be a global maximizer. Appendix B.2 verifies Equation (3) explicitly. **If the existence condition fails.** Proposition 2 implies neither recovery nor failure. For example, when A4 also holds, Theorem 4.1 makes full affine source recovery optimal, ruling out the strict inequality in Equation (3). Among A1–A3, the proof uses only A1. A2 and A3 show that failure can occur despite pairwise subspace recovery guarantees and full cross-view latent coverage.
We thank the reviewer for the careful reading and constructive questions. We agree that the recovery criterion and the roles of the assumptions needed clearer explanation. We have revised these passages and address each concern below. All locations refer to the revised PDF identified in our general comment. A1–A5 denote Assumptions 1–5.
W1. Why are A2, A4, and A5 needed?
These assumptions are the key conditions of three alternative sufficient routes to full affine source recovery. They are not simultaneously required, e.g., Theorem 4.3 requires neither A2 nor A4.
One major difficulty of source recovery in multi-view setting is that one view-specific encoder must serve several view pairs simultaneously. A higher-order mode of a widely shared coordinate can earn correlation on several pairs, while a pair-specific first-order source direction earns it once. Pair-specific first-order directions are even harder to recover when they have low intra-pair correlation. At minimum-sufficient capacity, namely dZi=mi, SUMCOR can therefore prefer dominant higher-order modes over the desired first-order directions, causing representation drift. The three routes prevent this in different ways:
Role of A1. Under the isotropic Gaussian model (A1), Hermite polynomials separate first-order source functions from higher-order ones, and every higher-order canonical correlation on edge {i,j} is at most tij,12. A2 and A4 compare against this bound. A1 also makes Di(γ) finite.
Role of A3. Full cross-view latent coverage (A3) requires every latent coordinate visible in a view to be shared with a neighbor. Under A1, the pairwise first-order targets then span the full first-order source space, so every nonzero linear source direction carries cross-view signal.
Failure outside guarantees. In Setup B, A2 and A3 hold but A4 fails, and full affine source recovery is strictly suboptimal for SUMCOR at dZi=mi=3 by Proposition 2 applied to the construction in Appendix B.2. In Setup C, both first-order dominance conditions fail. Even with the true overlap ranks rij=2, the rank-constrained methods show incomplete source recovery at the tested widths (Tables 1 and 3), whereas SUMCOR at dimension six achieves near-perfect affine source recovery (Table 1), consistent with Theorem 4.3 at Di(γ)=6, as calculated in Appendix D.4.