acceptodds
Withdrawn from ICLR 2027

Quasi-SVD: Learning a Lie-constrained matrix factorisation for real-time imaging

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Singular Value Decomposition (SVD) underlies matrix factorisation tasks across many fields, with imaging applications demanding real-time processing. Yet SVD algorithms are inherently sequential, constraining real-time GPU throughput and limit online deployment in imaging pipelines. This study introduces a fully parallelized matrix factorization framework for GPUs by enforcing matrix orthogonality on left singular vectors via Lie-parametrised algebra and recovering the remaining components through soft constraints. This asymmetric constraint design enables an efficient parallel and provably valid decomposition, achieves high reconstruction fidelity and substantially accelerates computation relative to the exact SVD, with real-time throughput exceeding standard video frame rates. Performance is evaluated on multiple imaging tasks spanning complementary computational regimes: (1) spatio-temporal background subtraction for ultrasound localisation microscopy, requiring high-dimensional matrix separation, (2) Mueller matrix polarimetry for neurosurgical tissue characterisation, requiring massive batch processing of small matrices, and (3) an MNIST denoising benchmark at an intermediate scale with known ground truth. Across regimes and instruments, the proposed framework demonstrates robust domain transfer at various matrix scales, sufficient for live image-guided workflows that classical solvers cannot currently support in these settings. By prioritising downstream reconstruction fidelity over exact spectral recovery, the proposed SVD framework makes structured matrix factorisation practical for real-time processing.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Singular value decomposition, Lie algebra, GPU parallelization, Real-time imaging

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DugeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Quasi-SVD, a learned matrix-factorization framework intended to reduce SVD latency in imaging applications. A network predicts a skew-symmetric generator, which is mapped to a left factor through a matrix exponential or approximation, while the remaining factors are recovered in closed form. Training uses singular-value, off-diagonal, and right-factor orthogonality losses. Experiments cover batched 3 x 3 Mueller matrices, 280 x 280 MNIST denoising, and 500 x 500 ultrasound matrices, reporting downstream image quality, factor errors, runtime, and architectural ablations.

Critical strengths
  1. The application motivation is clear and relevant. Evaluating large batches of small polarimetric matrices alongside larger denoising and ultrasound matrices addresses distinct workloads encountered in imaging systems. The explicit-factor design offers a potentially useful alternative to direct output prediction when downstream processing requires interpretable matrix factors.

  2. The reported results demonstrate practical predictive value. LieNN+RNN achieves 17.90 dB on MNIST versus 17.65 dB for exact truncated SVD at lower reported runtime, and its ULM output reaches SSIM 0.89. Reporting factor-level errors as well as image quality allows the trade-off between structural fidelity and application performance to be examined rather than relying on visual examples alone.

Critical weaknesses
  1. The structural and spectral guarantees are not supported by the evaluated general-size implementation. The exact exponential of a skew-symmetric matrix is orthogonal, but Eq.3 uses a finite Taylor approximation, and p.16, lines 862-863, identifies a nine-term expansion as the default for general matrices. Such truncation does not generally preserve orthogonality. Consistently, Table 3 reports left-factor orthogonality errors of 0.21 and 0.15 for LieNN and LieNN+RNN. Moreover, exact full reconstruction alone is insufficient evidence of an approximate SVD: for square matrices, Eq.4 gives X D Y^T=X X^T A=A whenever X is orthogonal and D is invertible. Table 4 reports a ULM singular-value relative error of 0.97 and right-factor orthogonality error of 20.80 +/- 1.25. These results leave the central claims of exact structural constraints and maintained decomposition accuracy unestablished, even if the learned filter produces useful images.

  2. The experiments do not establish a compelling quality-latency advantage against appropriate alternatives. The ULM input A=B_c^T B_c is symmetric positive semidefinite, but there is no comparison with a symmetric eigensolver or a method directly targeting the retained subspace. Existing results already show relevant trade-offs: ULM rSVD takes 1.98 ms versus 3.28 ms for LieNN+RNN, while MMP E2E+RNN is both faster and more accurate by the reported SSIM than LieNN+RNN (4.91 ms/0.98 versus 14.19 ms/0.94). Explicit factors may justify additional cost, but their necessary downstream value and retained-subspace fidelity are not demonstrated sufficiently. The headline speed-up over general exact SVD therefore does not establish the claimed practical advantage.

Questions for authors
  1. Implementation clarification: Which mapping and numerical precision were used in each main table and in Figs.5/6? How do you reconcile the reported 0.15-0.21 left-factor orthogonality errors with the exact-orthogonality claim? Please report quality and runtime together for the implementation that satisfies the stated constraint.

  2. Spectral interpretation: Given the ULM factor errors in Table 4, which mathematical property must be preserved for the downstream filter? Please specify the filtering and truncation equations and provide direct evidence of retained-subspace or target-signal preservation. This would determine whether the method supports an approximate-SVD claim or a narrower learned-filter contribution.

  3. Validation of the existing efficiency claim: Can you compare with a numerical baseline exploiting ULM's symmetric structure under the same output requirements and timing boundaries? Please also reconcile the exact-SVD runtime in Table 4 with the nearby-size result in Fig.6a. I am asking for a fair comparison within the submitted workload, not a new application.

1 comment

Thank you for a precise and constructive report. Every point led to a concrete change in the revised manuscript.

1. Orthogonality claim vs. truncated Taylor map: We have removed the claims of "exact" orthogonality and of a "provably valid" decomposition. The revision states which map is used where: Rodrigues' closed form for 3×33\times3 (MMP), which is exactly orthogonal, and the 9-term Taylor series for MNIST and ULM, which is orthogonal only up to its truncation remainder. All experiments run in float32. To separate the constraint from its approximation, we re-evaluated the MNIST models with an exact matrix exponential at unchanged weights: the left-factor orthogonality error falls from 0.15--0.21 to 10−510^{-5}, while PSNR is unchanged (17.79 / 17.91 dB against 17.78 / 17.93 dB). The reported quality therefore does not depend on the truncation.

2. Reconstruction is not evidence of an approximate SVD: Agreed, and the revision says so explicitly: for square A\mathbf{A}, XDY⊤=A\mathbf{X}\mathbf{D}\mathbf{Y}^\top=\mathbf{A} holds for any orthogonal X. We no longer present reconstruction as validation. The property the ULM filter needs is the clutter subspace span(X1:4)\mathrm{span}(\mathbf{X}_{1:4}), since the filter is BXSXS⊤\mathbf{B}\mathbf{X}_S\mathbf{X}_S^\top and uses neither D\mathbf{D} nor Y\mathbf{Y}; the filter equations are now given in the main text. We added principal angles to the exact clutter subspace: the dominant tissue component (98.9% of the energy) is recovered to within 0.4$^\circ$, whereas the weaker clutter components deviate by up to 47$^\circ$. We report this openly and name a subspace-aware loss as future work.

3. Symmetric eigensolver and fair timing: We added eigh on the same Gram matrices, timed identically (complete filter, same GPU, same batch). After redoing the ULM experiment with corrected baselines and longer training (250-frame stacks, RTX 5090), the per-stack times are: exact SVD 60.6 ms, eigh 31.6 ms, rSVD 4.1 ms, LieNN+RNN 0.91 ms, at SSIM 0.95 for LieNN+RNN (UNN 0.92, E2E 0.90). The advantage over rSVD holds for jointly filtered stacks.

4. Table 4 vs. Fig. 6a: Table 4 times the complete filter for one stack, i.e. eleven Gram matrices plus reshaping and projection; Fig. 6a times the decomposition alone on a different GPU. The revision states the unit and device in each caption.

5. MMP: E2E is faster and more accurate: Correct, and we say so. E2E returns no factors, however, and it is the least accurate learned model on ULM, where a data-dependent 250×250250\times250 projection must be learned.

Reviewer_ohj7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Quasi-SVD, an approximate factorisation A = X D Y^T meant to replace exact SVD in GPU imaging pipelines. X is constrained to SO(m) through the matrix exponential of a skew-symmetric matrix, D and Y follow in closed form from X, and Y is only softly pushed toward orthogonality. There are four variants (Analytic, UNN, LieNN, E2E), each optionally with a small GRU refinement. Experiments cover 3x3 Lu–Chipman decomposition for Mueller polarimetry, MNIST patch denoising, and ULM clutter filtering on PALA, with reported 3–20x speed-ups over cuSOLVER at similar downstream quality.

Critical strengths

It addresses a real bottleneck with a GPU-friendly idea. Batched small SVDs and Casorati SVDs really are bottlenecks in imaging pipelines. Predicting only the generator of X and getting D and Y in closed form is simple and suits GPUs. The ULM result (SSIM 0.89 against exact-SVD output at 3.28 vs. 75.16 ms) shows the idea could be practically useful, and it is the main reason I see promise in the work. The variant ablations are systematic, and the appendix reports what did not work (two-sided constraints, LBFGS, learned optimisers) and where cuSOLVER is still faster. This makes the paper's limits easy to see, which I weigh in its favour.

Critical weaknesses
  1. The paper's main selling point is a "provably valid" decomposition with exact orthogonality. But for square A, X D Y^T = X X^T A = A holds for any orthogonal X, so exact reconstruction says nothing about being close to the SVD. The properties that would make it SVD-like (orthogonal Y, correct singular values) are only encouraged by soft penalties and are often far off: in ULM, LieNN+RNN has ||Y||_ort = 20.80 and ||D||_rel = 0.97 (Table 4). Even X is not orthogonal in practice, because a truncated Taylor series for the matrix exponential blows up for large inputs (||X||_ort = 3.57 × 10^8 for MNIST Analytic, Table 3). For the symmetric ULM input, the Analytic variant reduces to X = I. What the method actually does in practice is learn an orthogonal subspace filter that imitates SVD output. That could still be useful, but it is a different and weaker claim than the title and abstract make.
  2. The paper doesn't show a real need for the speed-up or a downstream benefit. The motivation is real-time imaging, but the paper's own exact-SVD timings already meet the targets: about 27 FPS for MMP against a 25 FPS goal, and 75 ms per 0.5 s of ULM data. The fast baselines are weak or look broken (rSVD takes ~21 s per frame on MMP and gets SSIM 0.15 despite exact factors), and standard fast options such as closed-form 3x3 SVD, polar iteration, eigh or low-rank randomized SVD are missing. Quality is mostly measured as similarity to the exact-SVD output rather than imaging outcomes: there are no PALA localisation or velocity metrics for ULM, and train and test come from the same animal and session. So it is unclear when someone would choose this method over existing solvers.
Questions for authors
  1. Beyond exact reconstruction, how close are your factors to the true SVD (e.g., principal angles between X and U), and how should the method be described if they are not close?
  2. Which deployment scenario actually needs the speed-up, given that exact SVD already meets your stated real-time targets?
  3. How does the method compare with strong fast baselines (closed-form 3x3 SVD/polar decomposition, eigh, low-rank rSVD) on the same GPU, and why does rSVD fail on MMP?
  4. Does the ULM result hold on PALA localisation metrics and on a held-out animal?
2 comments

Thank you for the thorough and constructive report, which significantly helped improve the manuscript. The questions are answered in order.

1. How close are the factors to the true SVD, and how should the method be described? The revision contains principal angles between span(X1:4)\mathrm{span}(\mathbf{X}_{1:4}) and the exact ULM clutter subspace: the dominant tissue component (98.9% of the energy) is recovered to within 0.4$^\circ$, the weaker clutter components deviate by up to 47$^\circ$ (rSVD: 1.2$^\circ$, eigh: 0.09$^\circ$). The revision removes "provably valid" and "exact orthogonality", states that for square A\mathbf{A} reconstruction holds for any orthogonal X\mathbf{X}, and describes the method as a learned, Lie-constrained factorisation whose accuracy rests on the leading columns of X\mathbf{X}. The Taylor map is now stated to be orthogonal only up to its remainder. With an exact exponential at unchanged weights, the MNIST orthogonality error falls to 10−510^{-5} at identical PSNR and the divergence of the analytic variant (∥S∥2≈14\lVert\mathbf{S}\rVert_2\approx14) is explained.

2. Which scenario needs the speed-up? The exact SVD meets 25 FPS only at the smallest frame size on a high-end GPU (37 ms for 388×516388\times516 pixels on an RTX 4090). At 600×700600\times700 pixels it takes 99 ms, about 10 FPS, and this is the decomposition alone, before the rest of the pipeline. Exact solvers also do not benefit from batching, whereas LieNN does: with jointly filtered ULM stacks it takes 0.91 ms per stack against 60.6 ms. Since computational performance directly translates to hardware requirements, this speed-up enables real-time deployment on cost-effective, low-power edge hardware.

3. Strong fast baselines on the same GPU. Why did rSVD fail? All added, on one RTX 5090:

  • eigh (ULM): 31.6 ms per stack; LieNN+RNN 0.91 ms, rSVD 4.1 ms.
  • \emph{Closed-form 3×33\times3 (MMP):} 1.24 ms per frame against 6.66 ms for LieNN+RNN, at near-exact accuracy. The revision states plainly that a dedicated solver is preferable for m=3m=3, though it does not extend to larger matrices.
  • rSVD: the SSIM of 0.15 was an error (V\mathbf{V} passed where V⊤\mathbf{V}^\top was expected) and is now rectified reproducing the exact output. Its runtime is inherent to the regime: with q=min⁡(m,n)=3q=\min(m,n)=3 the sketch spans the full space and the library forms the QR factors matrix by matrix. For ULM, rSVD now estimates the low-rank clutter subspace, as the reviewer suggests, and matches the exact filter.

4. Localisation metrics and held-out animal: The principal angles show that the weaker clutter components are captured only approximately potentially affecting localisation agreement in the full pipeline. The revision therefore no longer claims that fidelity is uncompromised, limits the ULM claim to filtered-image similarity and runtime (SSIM 0.95 for LieNN after retraining; UNN 0.92, E2E 0.90), and names a subspace-aware training loss as future work. The evaluation remains a temporal hold-out within one animal stating this as a limitation.

Thank you for the author's effort to respond and revise. I believe that description is more accurate now. I saw that the authors withdrew the paper. Here are some suggestions that might help for future submissions: a downstream imaging metric showing that the accuracy loss does not matter, or a measured deployment where rSVD misses the target and LieNN meets it, would make the work more solid.

Reviewer_mUaeOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Quasi-SVD, a learned stand-in for SVD aimed at real-time imaging on GPUs. The left factor is the matrix exponential of a predicted skew-symmetric matrix, so it is orthogonal by construction, and the singular values and right factor are computed from it with soft losses pushing them toward the true ones. Four variants (analytic, unconstrained, Lie-based, end-to-end) are compared on Mueller matrix polarimetry (3x3), MNIST tile denoising (280x280), and ultrasound ULM clutter filtering (500x500), with large reported speedups over cuSOLVER.

Critical strengths

The problem is real. SVD is the slow step in ULM and polarimetry pipelines, and the paper tests on actual clinical-style data at very different matrix sizes, plus a second instrument for domain transfer. The ablation across the four variants is useful.

Critical weaknesses

The paper is upfront that it trades exact spectral recovery for reconstruction, which I appreciate. My concern is that "provably valid decomposition" then means very little. With Y = A^T X_n D^dagger (Eq. 4), X D Y^T = X_n X_n^T A, which equals A for any orthogonal X when m = n, and all experiments are square, so the validity check in Supp. C.1 holds automatically. What matters is whether D and Y are close to the real thing, and on ULM they are not (||D||_rel = 0.97, ||Y||_ort = 20.8, Table 4). There is no singular-vector accuracy metric anywhere. In practice this looks like a learned filter, and I think the title and claims should be toned down. It is also not clear what the explicit factors buy, since on MMP the end-to-end net beats LieNN on both SSIM (0.98 vs 0.94) and runtime.

The baselines also concern me somewhat. rSVD taking 21 seconds on batched 3x3 matrices (Table 1) looks like a per-matrix loop, and for ULM A = B^T B is symmetric PSD, so a symmetric eigensolver would be the natural comparison. And for the LieNN+RNN configuration behind the 0.94 SSIM on MMP, the speedup is about 2.6x, while contribution (ii) says "> 3-20x."

Questions for authors
  1. How does the method compare to a GPU symmetric eigensolver (for ULM) and a closed-form 3x3 solver (for MMP), timed the same way? I would appreciate this comparison in the rebuttal.
  2. Is there anything in the paper that actually needs the explicit factors rather than the filtered output?
  3. On MNIST, LieNN gives ||X||_ort = 0.15 (+RNN) and 0.21 (no RNN), but on ULM (500x500, 9-term Taylor) it is 0.00. How? Also, B.1 says Taylor is the default while Prop. 9 says Cayley. Which was used?
  4. Why does the end-to-end model do so poorly on ULM but win on MMP?
1 comment

Thank you for the fair assessment and the specific requests, which have all been carried out.

Weakness discussion

1. "Provably valid" and toned-down claims: For square A\mathbf{A}, XDY⊤=A\mathbf{X}\mathbf{D}\mathbf{Y}^\top=\mathbf{A} holds for any orthogonal X\mathbf{X}. The revision states this and removes "provably valid" and "exact orthogonality" from the abstract and contributions. The method is now described as a learned, Lie-constrained factorisation whose accuracy rests on the leading columns of X\mathbf{X}. The revision contains the missing singular-vector metric: principal angles to the exact ULM clutter subspace (dominant component within 0.4$^\circ$, weaker components up to 47$^\circ$).

2. rSVD at 21 s: The reviewer's suspicion was right on both counts. First, the baseline passed V\mathbf{V} where V⊤\mathbf{V}^\top was expected, which has been rectified so that rSVD reproduces the exact output (SSIM 1.00). Second, the runtime is indeed per-matrix: PyTorch forms the orthogonal QR factor one matrix at a time. Beyond the implementation, with q=min⁡(m,n)=3q=\min(m,n)=3 the sketch spans the full space, so rSVD ends in an exact SVD of the same size and offers no saving in this regime.

3. Speed-up claim: Corrected to the measured range, 2.6$\times$ (MMP, LieNN+RNN) to 67$\times$ (ULM).

Questions answered

1. Requested baselines, timed the same way:

  • Symmetric eigensolver (ULM): per stack on one RTX 5090, exact SVD 60.6 ms, eigh 31.6 ms, rSVD 4.1 ms, LieNN+RNN 0.91 ms.
  • Closed-form 3×33\times3 solver (MMP): we implemented McAdams et al. as a CUDA kernel. It is faster than LieNN+RNN (1.24 vs. 6.66 ms per frame, exact SVD 20.66 ms) at near-exact accuracy. The revision reports that a dedicated solver is preferable for m=3m=3. It does not extend beyond 3×33\times3, whereas the same LieNN architecture covers 3×33\times3 to 280×280280\times280.

2. What needs explicit factors? The ULM filter projects onto a selectable band of X\mathbf{X}, and MNIST truncation selects the rank kk after training. An end-to-end network fixes both at training time.

3. ∥X∥Fort\lVert\mathbf{X}\rVert_F^{\mathrm{ort}} on MNIST vs. ULM; Taylor vs. Cayley: Taylor (9 terms) was used throughout for m>3m>3; the Cayley label was an error and is fixed. The orthogonality error is the truncation remainder, which grows with the generator norm: MNIST models predict ∥S∥2≈4.3\lVert\mathbf{S}\rVert_2\approx4.3, ULM models smaller generators. With an exact exponential at unchanged weights, the MNIST error drops to 10−510^{-5} at identical PSNR.

4. E2E on ULM vs. MMP: In MMP, E2E learns a fixed low-dimensional per-pixel map, which is fast and accurate due to the small matrix size. In ULM, it must learn a data-dependent large matrix projection and the absence of any factorisation constraint explains why it is the least accurate learned model there.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.