acceptodds
Withdrawn from ICLR 2027

Learning a Forward Operator from Cross-Modal Second-Order Statistics

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We consider the linear forward model , , , with the high-dimensional source unobserved during training; we learn from matching the -covariance together with a given covariance for , where aggregates into dimensions. The two covariances of different dimensions are linked through the pseudo-inverse of : With , can be learned under the covariance-matching objective . We parameterise in low-rank (LoRA) form: , with a known baseline. Our theoretical analysis addresses three questions relevant to training under this objective: identifiability of the matching set in -space, the size of the derivative of through which the gradient passes, and conditions under which training keeps close to a warm-start . Identifiability tells us how the set of that best fits the references sits in . A single reference leaves a smooth manifold whose dimension depends on the regime. Several references, under simultaneous-diagonalisation conditions on the reference covariances, reduce the matching set , everywhere on , to isolated solutions, mutually Frobenius-separated, which differ from one another only by sign flips of the rows in the whitened coordinates of . The gradient bound tells us flow through stays controlled while is small relative to ; the controlled scale tightens as rank grows, so rank has a usable ceiling. The trajectory result tells us that, starting from , gradient flow stays nearest the same solution as . The other solutions are loss-equivalent and Frobenius-far from , so confinement keeps a small correction to throughout training, consistent with the LoRA premise (low-rank adaptation of pre-trained networks, Hu et al. (2022)).

Primary area:
learning theory
Keywords:
covariance matching, second-order statistics, low-rank adaptation (LoRA)

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4NyEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors consider learning linear forward operators from covariance matching. They theoretically analyze the identifiability of solutions, the stability of gradients, and the behavior of gradient flow.

Critical strengths

The strength of this paper is to theoretically show that multiple reference covariances can reduce the set of solutions to finitely many isolated solution under some assumption. Also, the authors show that gradient flow with a warm start stays close to the initial model, which support the stability of low-rank learning. I think such an idea of using multiple reference covariances is very relevant and important for inverse problems to mitigate the ill-posedness.

Critical weaknesses

The theoretical analysis require several assumptions, such a suitable warm start A0A_0, and simultaneous diagonalization conditions on the reference covariances Σref,(k)\Sigma^{{\rm ref}, (k)}. However, I am not sure how these assumptions can be verified in practice, since a warm start A0A_0 and reference covariances Σref,(k)\Sigma^{{\rm ref}, (k)} may be unknown in practice. I might have misunderstood points, though.

Questions for authors
  1. Does the warm start A0A_0 need to be close to the true forward operator AA ? If so, is there any practical way to check if A0A_0 is close to AA ?

  2. How can we verify the simultaneous diagonalization conditions on the reference covariances Σref,(k)\Sigma^{{\rm ref}, (k)} in practice ? Could the authors provide some insight on how to obtain the reference covariances Σref,(k)\Sigma^{{\rm ref}, (k)} ?

  3. This is a question out of interest. Could the author discuss the possible extension of the argument to the nonlinear forward models ? I think this paper is very relevent to ill-posded inverse problem, and inverse problem often involve nonlinear setting. I do not ask the authors to add details, but just a brief discussion on this possibility.

Reviewer_LrjnOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies learning a linear forward operator from cross-modal second-order statistics without sample-level supervision of the underlying source. The authors parameterize the operator as a low-rank correction to a known baseline and optimize a covariance-matching objective involving the pseudo-inverse. The theoretical analysis focuses on three aspects: the identifiability of the covariance-matching solutions, the sensitivity of the pseudo-inverse and its dependence on the adaptation rank, and the stability of training trajectories near the warm start. Synthetic experiments further demonstrate that low-rank adaptation can achieve better operator recovery and reconstruction quality than unconstrained optimization.

Critical strengths
  1. The training trajectory analysis is meaningful and well-motivated within the proposed framework. Given the existence of multiple loss-equivalent solutions, analyzing whether training can remain near the solution selected by the warm start is important. The paper provides a theoretical characterization of this behavior under explicit conditions.
  2. The paper provides a systematic theoretical analysis of the proposed framework, covering identifiability, pseudo-inverse sensitivity, and training stability. These results offer useful insights into learning forward operators under covariance-based supervision.
Critical weaknesses
  1. The motivation and practical significance of the problem are not sufficiently established. Although the paper introduces cross-modal covariance matching as a way to learn forward operators, it does not clearly demonstrate the limitations of existing approaches or why addressing this particular problem is important. The practical relevance of the proposed setting therefore remains somewhat unclear.
  2. The novelty of the theoretical contributions are not sufficiently clarified. Several results build upon established tools from joint diagonalization, pseudo-inverse perturbation theory, and gradient-flow analysis. While the paper derives results for the considered setting, it is difficult to identify the key advances over existing work and understand what new insights these results provide for forward-operator learning.
Questions for authors
  1. Could the authors further clarify the main technical novelty of this work compared with existing studies on covariance-based identifiability and forward-operator learning? In particular, what are the key insights that could not be obtained from existing theoretical results?
  2. The training trajectory analysis is an important part of the paper. How restrictive are its assumptions in practical applications, and to what extent do the guarantees extend to the discrete optimization of LoRA factors used in the experiments?
  3. The experiments are limited to synthetic data and mainly compare LoRA with an unconstrained baseline. Could the authors provide comparisons with more methods and additional evidence supporting the practical benefits of the proposed framework?
Reviewer_Vnp5OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies calibration of a linear forward operator from cross-modal covariance statistics, without access to GT sources. It uses low-rank updates to a known baseline and analyzes identifiability of the induced inverse mapping, sensitivity to operator changes, and conditions for remaining near the same matching solution during training. Synthetic experiments compare low-rank and unrestricted updates.

Critical strengths

I find that identifiability analysis, as a problem, is fundamentally interesting. Especially the idea, how multiple covariance references can narrow down the possible solutions. The explicit assumptions also help clarify when this works.

Critical weaknesses
  1. My main concern is the gap between identifying the inverse mapping and recovering the forward operator. Different forward operators can give the same inverse mapping, even with low-rank updates. So, it is still unclear what explains the recovery results. A full-operator baseline regularized toward the same starting point would help assess whether low rank provides an additional benefit.

  2. I’m also not clear on what the main training guarantee adds. It assumes the updates stay small, and Proposition 7 already gives confinement under the bounds used later. Also, the reported update bounds in Appendix D.1 and Table 4 do not satisfy the condition claimed before, so those experiments do not validate this guarantee.

Questions for authors

To be honest, the weakness above is the main technical blocker in front of me. Without proper address of them, I don't think any question can help to support this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.