acceptodds
Under review as a conference paper at ICLR 2027

Efficient Neural Operator Decomposition via Tensor Product Quadrature

Abstract

Recent studies have shown that the operator eigenvalue/singular value decomposition (EVD/SVD) can be efficiently achieved via neural network-based optimization; however, the function quadrature within the objective is computationally intensive, especially in high dimensions. Thus, mini-batch Monte Carlo sampling is used to obtain unbiased quadrature estimates, yet higher accuracy relies on larger batch size, imposing a computation-accuracy trade-off that restricts the efficiency of current methods in dimensions. We propose a tensor product quadrature (TPQ) for operator decomposition, where each spectral function is parameterized by a variable-separable neural network, and the -D quadrature is decomposed into high-accuracy and low-cost 1D quadratures followed by tensor product assembly, which largely bypasses the computation-accuracy trade-off. We further develop TPQ-compatible treatments for sampling, gradient mask, boundary constraints, and non-separable potentials using explicit decomposition or parametric pre-training. Numerical experiments on the Schr\"odinger equation (2D harmonic oscillator, 2D infinite well, 2D and 3D hydrogen atom), neural tangent kernel (NTK) decomposition, and up to 5D integral kernel operator SVD validate the TPQ method. Notably, the TPQ consistently improves the efficacy of neural operator EVD/SVD baselines, including spectral inference network (SpIN), neural eigenfunctions (NeuralEF), and nested low-rank approximation (NeuralSVD).

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Neural operator decomposition, tensor product quadrature, Schr\"odinger equation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9SxiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tries to make neural eigenfunction solvers (SpIN, NeuralEF and NeuralSVD) cheaper and more accurate in more than one dimension. These methods learn eigenfunctions by optimizing objectives full of integrals, estimated with random mini-batches, which get noisy unless the batch is very large. The authors write each eigenfunction as a sum of products of 1D networks, so every high-dimensional integral splits into a few 1D integrals. Sampling a few hundred points per axis then acts like a full grid, and the cost grows linearly with dimension instead of exponentially. They also approximate the Coulomb potential with a sum of Gaussians to keep it separable. Tests cover the 2D infinite well, harmonic oscillator, 2D and 3D hydrogen, kernel SVD up to 5D and a 2D NTK.

Critical strengths
  1. The method is simple and general because the same trick plugs into all three existing methods.
  2. The performance gains are consistent. Errors are lower on every benchmark, especially for excited states, and the learned eigenfunctions look much closer to the true ones. The scaling is pretty good. Memory and speed stay almost flat as dimension grows, while the original NeuralSVD runs out of memory.
Critical weaknesses
  1. The variance argument only covers part of the objective. The paper's own analysis shows TPQ behaves like a B1DNB_{1D}^N batch only when the 1D factors average close to zero, as for cross-terms between different eigenfunctions. The norms and Rayleigh quotients that set the eigenvalues have positive integrands, and there the same formula gives an error closer to an ordinary batch. So the gains may come mostly from the separable network design rather than the quadrature.
  2. It depends on the coordinate axes. Separability is defined along the Cartesian axes, so rotating the same problem can make it much harder. Most benchmarks are product functions aligned with the axes, and the kernel test is built exactly that way. Hydrogen is the only real non-separable case, and only up to 3D. Also, the physics extension is limited. The Gaussian trick works for one fixed nucleus but not for electron–electron repulsion, which couples coordinates of two particles. The many-body example only works because its coupling is quadratic. Boundary masks also restrict the method to box-shaped domains.
Questions for authors
  1. At the same number of network evaluations, how accurate is TPQ on the norm and Rayleigh-quotient terms compared with ordinary sampling?
  2. What happens if you rotate a benchmark such as the harmonic oscillator by 45°, and how much higher does the rank need to be?
  3. How would TPQ handle pairwise Coulomb interactions between electrons?
  4. Does SpIN-TPQ still recover eigenfunctions in the correct order, especially within degenerate levels?
  5. Could you add results over several seeds for the main experiments?
7 comments
AuthorsOpenReview ↗

Response To Reviewer 9Sxi

Response To Reviewer 9Sxi

We sincerely thank the reviewer for the thoughtful comments and questions. We have expanded the analysis and clarified the limitations and applicability of our method. The revised part is highlighted in blue in the uploaded revised manuscript. Our point-by-point responses are provided below.

W1: The variance argument only covers part of the objective. The paper's own analysis shows TPQ behaves like a B1DNB_{1D}^N batch only when the 1D factors average close to zero, as for cross-terms between different eigenfunctions. The norms and Rayleigh quotients that set the eigenvalues have positive integrands, and there the same formula gives an error closer to an ordinary batch. So the gains may come mostly from the separable network design rather than the quadrature.

Response: Thank you for this comment. We agree that the variance advantage of TPQ depends on the properties of the integrand. We have clarified this limitation in Appendix Section E.6. Additionally, we added two experiments to evaluate the practical contribution of TPQ.

(1) Integration accuracy of norm and Rayleigh-quotient (RQ) terms. To evaluate these terms, we added an experiment using the same frozen CP network trained on 2D hydrogen, comparing TPQ with ordinary Monte Carlo (MC) at matched network evaluation budgets. Each setting is repeated 100 times. Table 9 of the revised manuscript reports the integration errors. We report the results against deterministic reference integrals below. TPQ achieves lower errors for both the squared norm and Rayleigh quotient at all tested budgets.

Network evaluationsTPQ norm error (%)MC norm error (%)TPQ RQ error (%)MC RQ error (%)
19220.62934.63210.25610.786
38413.69427.7333.7677.828
76810.04520.1563.1365.066
1,5367.49215.5992.4914.861
3,0725.64610.0872.2285.584
6,1443.7988.0871.1783.634

(2) Ablation separating CP parameterization from TPQ. To isolate the contribution of quadrature, we additionally compare CP-TPQ and CP-MC using the same CP architecture and Gaussian-mixture potential, with comparable training speeds. We compare four settings: (A) CP-TPQ with Gaussian-mixture model (GMM) potential, (B) CP-Monte Carlo (CP-MC) with GMM potential, (C) CP-MC with the original Coulomb potential, and (D) MLP-MC with the original Coulomb potential. All methods are trained for 30,000 iterations over three seeds. The eigenvalue errors are evaluated under the original Coulomb Hamiltonian. The results are reported in Table 4 of the revised manuscript, and we paste partial results here:

MethodTraining potentialFirst 9 modes error (%)First 16 modes error (%)Iterations/s
A: CP-TPQGaussian sum4.53 ± 1.863.48 ± 1.0617.57 ± 1.45
B: CP-MCGaussian sum7.62 ± 2.86113.22 ± 50.0317.78 ± 0.06
C: CP-MCCoulomb8.56 ± 1.3040.89 ± 60.5717.35 ± 0.05
D: MLP-MCCoulomb6.87 ± 1.714.77 ± 0.7317.58 ± 0.10

Under the same CP architecture and potential approximation (A vs. B), TPQ reduces the mean relative eigenvalue error from 7.62% to 4.53% over the first 9 modes. The difference is more pronounced over 16 modes, where CP-MC fails to recover the highest mode. These additional experiments show that TPQ improves both integration accuracy and spectral learning, with benefits beyond the separable network architecture alone.

AuthorsOpenReview ↗

Response To Reviewer 9Sxi

W2: It depends on the coordinate axes. Separability is defined along the Cartesian axes, so rotating the same problem can make it much harder. Most benchmarks are product functions aligned with the axes, and the kernel test is built exactly that way. Hydrogen is the only real non-separable case, and only up to 3D. Also, the physics extension is limited. The Gaussian trick works for one fixed nucleus but not for electron–electron repulsion, which couples coordinates of two particles. The many-body example only works because its coupling is quadratic. Boundary masks also restrict the method to box-shaped domains.

Response: Thank you for this comment. We agree that the efficiency of TPQ depends on the coordinate system and the separability of the operator. To address these concerns, we have added two experiments: a 45° coordinate rotation experiment and a 6D two-electron Schrödinger experiment with Coulomb interaction.

(1) Sensitivity to coordinate rotation. We added a 45° rotation experiment using a 2D anisotropic harmonic oscillator (Section I in the revised manuscript). Unlike the isotropic oscillator, the rotated potential contains a cross-coordinate term −x1x2-x_1x_2, making the eigenfunctions non-separable in the original Cartesian coordinates. We trained NeuralEF-TPQ with CP ranks R∈{17,21,25,29,33}R\in\{17,21,25,29,33\} for both 0° and 45°, using the same random seed and 20,000 training iterations. Table 8 of the revised manuscript reports the energy errors for the first 22 outputs:

CP rank RR0° energy error45° energy error
170.0178180.021846
210.0157180.020992
250.0158060.020370
290.0155960.015937
330.0153080.015589

The rotation increases the error at smaller ranks. However, increasing the rank to R=29R=29 reduces the rotated error to 0.015937, close to the unrotated error of 0.015596. At R=33R=33, the errors are also comparable (0.015589 versus 0.015308). Furthermore, Fig. 10 of the revised manuscript shows that the learned eigenfunctions reproduce the rotated spatial structures. These results indicate that moderately increasing the CP rank can compensate for the additional coordinate coupling in this example.

(2) Extension to electron–electron Coulomb interactions. To address the limitation of the previous quadratic-coupling example, we added a 6D two-electron Schrödinger experiment with the non-quadratic Coulomb interaction 1/∣r1−r2∣21/|r_1-r_2|_2 (Section L.1 in the revised manuscript). The experiment considers two same-spin electrons in a 3D harmonic trap (resulting in a 6D problem) and learns the lowest 12 antisymmetric states. We extend the Gaussian-mixture approximation to the electron–electron interaction using K=30K=30 Gaussian terms. Each Gaussian term is factorized across the three spatial directions, and the coupled coordinates of the two electrons are integrated through three 2D coordinate-pair quadratures, avoiding the full 6D tensor grid. We compare NeuralEF-TPQ (R=15R=15) with a NeuralEF-variational Monte Carlo (NeuralEF-VMC) baseline. Both methods optimize the NeuralEF objective for 10,000 iterations with comparable training iterations per second. Table 11 of the revised manuscript reports the physical energies and relative eigenvalue errors, and we paste partial results here:

Relative eigenvalue error (12 modes)NeuralEF-TPQNeuralEF-VMC
Mean3.4807×10⁻⁴9.3301×10⁻²
Maximum8.0194×10⁻⁴2.2897×10⁻¹

NeuralEF-TPQ achieves lower relative eigenvalue errors. Additionally, Fig. 12 of the revised manuscript presents the energy convergence curves for all 12 states, while Fig. 13 compares the learned eigenfunction slices against the references. These results demonstrate that our extended TPQ approach can handle pairwise Coulomb interactions in this 6D system, beyond the previously studied quadratic interactions.

Moreover, the current separable boundary masks are naturally suited to box-shaped domains. More general geometries would require additional treatments that preserve or approximate separability such as coordinate transformation, and we view this as an interesting future direction.

AuthorsOpenReview ↗

Response To Reviewer 9Sxi

Q1: At the same number of network evaluations, how accurate is TPQ on the norm and Rayleigh-quotient terms compared with ordinary sampling?

Response: Thank you for this question. To compare the integration accuracy of norm and Rayleigh-quotient terms, we added an experiment using the same frozen CP network trained on 2D hydrogen, comparing TPQ with ordinary MC at matched network evaluation budgets. Each setting is repeated 100 times. Table 9 of the revised manuscript reports the mean absolute relative integration errors against deterministic reference integrals of the frozen network:

Network evaluationsTPQ norm error (%)MC norm error (%)TPQ RQ error (%)MC RQ error (%)
19220.62934.63210.25610.786
38413.69427.7333.7677.828
76810.04520.1563.1365.066
1,5367.49215.5992.4914.861
3,0725.64610.0872.2285.584
6,1443.7988.0871.1783.634

TPQ achieves lower errors for both the squared norm and Rayleigh quotient at all six evaluation budgets. For example, at 6,144 evaluations, TPQ reduces the norm error from 8.087% to 3.798% and the RQ error from 3.634% to 1.178%. These results show that TPQ can improve the integration accuracy of both terms at the same network evaluation budget.

Q2: What happens if you rotate a benchmark such as the harmonic oscillator by 45 degrees, and how much higher does the rank need to be?

Response: Thank you for this question. We added a 45° rotation experiment using a 2D anisotropic harmonic oscillator (Section I in the revised manuscript). We trained NeuralEF-TPQ with CP ranks R∈{17,21,25,29,33}R\in\{17,21,25,29,33\} for both the unrotated case (0°) and the rotated case (45°), using the same random seed and 20,000 training iterations. Table 8 of the revised manuscript reports the energy errors for the first 22 outputs:

CP rank RR0° energy error45° energy error
170.0178180.021846
210.0157180.020992
250.0158060.020370
290.0155960.015937
330.0153080.015589

The rotation increases the error at smaller ranks. However, increasing the rank to R=29R=29 reduces the rotated error to 0.015937 for 45°, close to the unrotated error of 0.015596. At R=33R=33, the two errors are also very similar (0.015589 versus 0.015308). Therefore, the rotated benchmark requires only a moderate increase in CP rank to recover accuracy comparable to the axis-aligned case. In addition, Fig. 10 of the revised manuscript shows that the learned eigenfunctions reproduce the rotated spatial structures well. These results indicate that TPQ remains effective under coordinate rotation.

Reviewer_57JKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes tensor product quadrature for neural operator eigendecomposition and singular value decomposition. Spectral functions are represented by canonical polyadic networks, allowing mass and operator integrals to be assembled from one-dimensional integrals. The framework integrates this construction into SpIN, NeuralEF, and Neuralsvd, with compatible normalization, boundary constraints, gradient masking, and separable approximations of non-separable potentials. Validation includes Schrodinger eigenproblems, integral operator svd, and ntk decomposition, reporting improved convergence, computational efficiency, and downstream kernel regression performance.

Critical strengths
  • The integration across several spectral objectives is useful and technically coherent. TPQ provides a common mechanism for assembling the quantities required by three established frameworks, including singular-function learning for non-self-adjoint operators. Preserving their normalization and ordering mechanisms within a separable representation makes the contribution practically reusable.

  • The experiments provide credible evidence of benefits within the tested regime. Improvements appear across multiple neural baselines and include convergence against wall-clock time. The NTK regression results also connect spectral approximation to downstream performance. These findings support the practical value of exploiting separability, even though they establish a narrower conclusion than the general scalability motivation.

Critical weaknesses
  • The methodological advance over existing tensor neural methods appears incremental. Wang and Xie (2024) already use tensor neural networks and accurate integration to compute multiple eigenpairs, directly overlapping with the central motivation and computational mechanism. Separable PINNs likewise establish the efficiency of coordinate-wise neural representations. ScienceDirect The distinctive contribution here is primarily the integration of this machinery into existing spectral objectives, together with the associated implementation adaptations. Supporting asymmetric operator SVD is a useful extension, but the decomposition objective itself is inherited from NeuralSVD. Although the comparison with the earlier TNN solver provides supporting evidence, it does not sufficiently establish a substantial methodological advantage across the broader problem class claimed.

  • The validation does not adequately establish scalability at a prescribed spectral accuracy. The main Schrodinger examples are two or three-dimensional, the higher dimensional kernel benchmark has explicitly separable singular functions, and the many-body demonstration involves three distinguishable particles with harmonic coupling. These are useful controlled tests, but they provide limited evidence about the representation bottleneck in more strongly coupled problems. Linear dependence on dimension at fixed CP rank does not establish comparable scaling when the rank must increase to maintain accuracy. Similarly, comparing TPQ with an explicitly evaluated tensor grid does not by itself establish an exponential advantage over Monte Carlo baselines, which do not enumerate that grid. For hydrogen, the Gaussian-mixture potential introduces another approximation and smooths the Coulomb singularity; the resulting gains therefore require a clearer separation of quadrature benefits from changes to the operator. Together, these limitations leave the practical scope of the claimed advantage insufficiently established.

Questions for authors
  • Do the improvements persist when direct Monte Carlo integration and TPQ use the same CP architecture, potential representation, and comparable wall-clock budgets? This would help determine how much benefit comes specifically from quadrature assembly versus the representation and smoothing.

  • Since hydrogen eigenvalues are compared with analytical Coulomb energies, how much of the final spectral error is attributable to the Gaussian-mixture approximation? Are the learned functions also evaluated through residuals or Rayleigh quotients under the original Coulomb Hamiltonian?

  • What do the existing results establish about computational cost at fixed spectral accuracy when dimension or coupling increases, allowing the required CP rank and operator approximation complexity to vary? This concerns the scope of the current scalability claim, rather than a request for a particular future application.

7 comments
AuthorsOpenReview ↗

Response To Reviewer 57JK

Response To Reviewer 57JK

We sincerely thank the reviewer for the thoughtful comments and questions. We have clarified the contribution and scope of our method and added discussion and experiments to address the concerns. The revised part is highlighted in blue in the uploaded revised manuscript. Our point-by-point responses are provided below.

W1: The methodological advance over existing tensor neural methods appears incremental. Wang and Xie (2024) already use tensor neural networks and accurate integration to compute multiple eigenpairs, directly overlapping with the central motivation and computational mechanism. Separable PINNs likewise establish the efficiency of coordinate-wise neural representations. The distinctive contribution here is primarily the integration of this machinery into existing spectral objectives, together with the associated implementation adaptations. Supporting asymmetric operator SVD is a useful extension, but the decomposition objective itself is inherited from NeuralSVD. Although the comparison with the earlier TNN solver provides supporting evidence, it does not sufficiently establish a substantial methodological advantage across the broader problem class claimed.

Response: Thank you for this comment. We agree that variable-separable neural representations have been used in separable PINNs, and that the underlying TPQ principle has been used in the tensor neural network (TNN) literature. We have clarified these points in the revised Section 2 and Appendix F. Our contribution lies in adapting and extending TPQ to different neural spectral objectives and developing TPQ-compatible formulations and techniques, as detailed below.

(1) Technical contributions beyond TNN. Existing TNN-based Rayleigh–Ritz methods learn a trial subspace and recover eigenfunctions through a projected generalized eigenvalue problem. In contrast, our framework incorporates TPQ into different neural spectral optimization objectives (SpIN, NeuralEF, and NeuralSVD), enabling direct and mode-resolved spectral learning without solving a generalized eigenvalue problem. This requires several specific technical developments:

  • A pre-trained Gaussian-mixture surrogate for handling non-separable potentials.
  • A boundary-compatible masked gradient construction for SpIN-TPQ.
  • A factorwise normalization technique for NeuralEF-TPQ to avoid full-dimensional grid evaluations.
  • A two-branch nested low-rank assembly for NeuralSVD-TPQ that enables operator SVD.

These developments are detailed in Sections 2.2–2.5 of the revised manuscript. In particular, NeuralEF-TPQ and NeuralSVD-TPQ avoid the mass-matrix inversion and generalized eigensolve required by the TNN-based Rayleigh–Ritz formulation, offering a potentially more stable optimization approach as the number of target modes increases. As shown in Fig. 8 of the revised manuscript (Appendix F), NeuralEF-TPQ achieves more stable convergence than TNN for 12, 15, and 18 target eigenpairs, with a more pronounced advantage as the number of eigenpairs increases. This provides experimental evidence that directly optimizing individual spectral functions can improve numerical stability compared with the tested TNN Rayleigh–Ritz solver.

(2) Extension to more general spectral problems. Beyond the multi-eigenpair setting, our framework supports non-self-adjoint operator SVD and interacting quantum systems. In particular, we added a 6D two-electron Schrödinger experiment with pairwise Coulomb interaction in Appendix L.1. The interaction is handled through a Gaussian expansion and three two-dimensional coordinate-pair quadratures. We compare NeuralEF-TPQ with a NeuralEF-variational Monte Carlo (NeuralEF-VMC) baseline under comparable training speeds. The following results are extracted from Table 11 of the revised manuscript, after 10,000 iterations:

Relative eigenvalue error (12 modes)NeuralEF-TPQNeuralEF-VMC
Mean3.4807×10⁻⁴9.3301×10⁻²
Maximum8.0194×10⁻⁴2.2897×10⁻¹

These results demonstrate the applicability of our TPQ-based framework to a six-dimensional interacting quantum system. We have also revised the manuscript to distinguish the established TPQ principle from our new algorithmic developments. Our main contribution is a unified framework that extends TPQ to different neural spectral optimization objectives, supporting direct eigenfunction learning, non-self-adjoint operator SVD, and preliminary applications to interacting quantum systems.

AuthorsOpenReview ↗

Response To Reviewer 57JK

W2(1): The validation does not adequately establish scalability at a prescribed spectral accuracy. The main Schrodinger examples are two or three-dimensional, the higher dimensional kernel benchmark has explicitly separable singular functions, and the many-body demonstration involves three distinguishable particles with harmonic coupling. These are useful controlled tests, but they provide limited evidence about the representation bottleneck in more strongly coupled problems. Linear dependence on dimension at fixed CP rank does not establish comparable scaling when the rank must increase to maintain accuracy.

Response: Thank you for this comment. We agree that the computational cost at a fixed CP rank does not fully reflect the cost of achieving a prescribed spectral accuracy, especially when the required rank increases with dimension or coupling strength. To address these concerns, we have added two experiments in the revised manuscript.

(1) Computational cost at prescribed spectral accuracy (Table 10). We added a CP-rank sensitivity experiment on the 2D and 3D hydrogen problems. We vary the CP rank from R=5R=5 to R=40R=40 and record the time required to reach a prescribed mean relative eigenvalue error of 5%, with a maximum of 20,000 training iterations. We compare NeuralSVD-TPQ with the original NeuralSVD baseline. The results are reported in Table 10 of the revised manuscript, reproduced below.

Dimension NNCP rank RRTime to 5% (s)Minimum error (%)Final error (%)
25112.214.21734.9734
210110.873.34314.3769
215151.022.41493.4048
220137.513.28254.2452
230154.172.49043.6304
240167.422.60903.3486
2NeuralSVD1659.223.23393.3862
35Not reached142.3969677.5184
310Not reached17.9857129.0455
315175.230.91581.6463
320159.280.79580.8692
330184.031.38161.8929
340176.311.02401.2633
3NeuralSVD1360.181.09911.4843

It is shown that NeuralSVD-TPQ reaches the target in 110.87,s in 2D (R=10R=10) and 159.28,s in 3D (R=20R=20), giving respective speedups of 15.0×15.0\times and 8.5×8.5\times over NeuralSVD. All tested ranks attain the target accuracy in 2D, whereas in 3D it is reached at the tested ranks R≥15R\ge15. These results suggest that a moderately larger CP rank may be needed as dimension increases, while TPQ remains effective in 3D with a suitable rank. Further rank increases do not consistently improve time to the prescribed target. These results demonstrate reduced computational cost brought by TPQ at matched spectral accuracy for the tested configurations.

(2) Extension to a 6D interacting two-electron Schrödinger problem (Table 11). To further examine TPQ beyond separable or quadratically coupled systems, we added a six-dimensional two-electron Schrödinger experiment with electron–electron Coulomb interaction, described in Section L.1 of the revised manuscript. We consider two same-spin electrons in a three-dimensional harmonic trap, resulting in a 6D configuration space. The Hamiltonian contains a non-separable Coulomb interaction 1/∣r1−r2∣21/|r_1-r_2|_2. We propose to learn a Gaussian expansion with K=30K=30 for this Coulomb interaction, and the six-dimensional potential integral is assembled from three two-dimensional coordinate-pair quadratures. NeuralEF-TPQ uses a CP rank of R=15R=15 and learns the lowest 12 antisymmetric states. We compare it with NeuralEF-VMC with comparable training iterations per second. Both methods are trained for 10,000 iterations. Table 11 of the revised manuscript reports the physical energies and relative eigenvalue errors, with partial results reported below:

Relative eigenvalue error (12 modes)NeuralEF-TPQNeuralEF-VMC
Mean3.4807×10⁻⁴9.3301×10⁻²
Maximum8.0194×10⁻⁴2.2897×10⁻¹

NeuralEF-TPQ achieves smaller relative eigenvalue errors than NeuralEF-VMC for all 12 states. Its mean relative error is 3.4807×10⁻⁴, compared with 9.3301×10⁻² for NeuralEF-VMC. Moreover, Fig. 12 of the revised manuscript shows the energy convergence of all 12 states, while Fig. 13 compares the learned eigenfunction slices at 5,000 and 10,000 iterations. This experiment demonstrates that TPQ can also be applied to a six-dimensional system with non-quadratic electron-electron interactions, extending the validation beyond the previous experiments.

Nevertheless, the computational advantage depends on the required CP rank and the availability of an efficient separable operator representation. We have clarified this limitation in Section 2.1 (Scope of TPQ paragraph). The new experiments provide additional evidence of efficiency at prescribed accuracy in the tested hydrogen problems and applicability to a six-dimensional Coulomb-interacting system.

AuthorsOpenReview ↗

Response To Reviewer 57JK

W2(2): Similarly, comparing TPQ with an explicitly evaluated tensor grid does not by itself establish an exponential advantage over Monte Carlo baselines, which do not enumerate that grid. For hydrogen, the Gaussian-mixture potential introduces another approximation and smooths the Coulomb singularity; the resulting gains therefore require a clearer separation of quadrature benefits from changes to the operator. Together, these limitations leave the practical scope of the claimed advantage insufficiently established.

Response: Thank you for this comment. To better isolate the contributions of quadrature and potential approximation, we have added a controlled ablation study on 2D hydrogen in Appendix D (Table 4) of the revised manuscript, detailed below.

Separating the effects of TPQ, CP parameterization, and potential approximation. We compare four settings: (A) CP-TPQ with Gaussian-mixture model (GMM) potential, (B) CP-Monte Carlo (CP-MC) with GMM potential, (C) CP-MC with the original Coulomb potential, and (D) MLP-MC with the original Coulomb potential. All methods train 16 eigenpairs for 30,000 iterations over three random seeds, with comparable training speeds. Table 4 of the revised manuscript reports the mean relative eigenvalue errors (%) and training speeds (All errors are calculated against analytical Coulomb energies):

MethodTraining potentialFirst 9 modesFirst 16 modesIterations/s
Evaluation with original Coulomb Hamiltonian H\mathcal H
A: CP-TPQGaussian sum4.53 ± 1.863.48 ± 1.0617.57 ± 1.45
B: CP-MCGaussian sum7.62 ± 2.86113.22 ± 50.0317.78 ± 0.06
C: CP-MCCoulomb8.56 ± 1.3040.89 ± 60.5717.35 ± 0.05
D: MLP-MCCoulomb6.87 ± 1.714.77 ± 0.7317.58 ± 0.10
Evaluation with GMM Hamiltonian HK\mathcal H_K
A: CP-TPQGaussian sum4.68 ± 1.863.55 ± 1.0517.57 ± 1.45
B: CP-MCGaussian sum7.68 ± 2.87113.79 ± 50.1917.78 ± 0.06

Comparing A and B, which use the same CP architecture and GMM potential, TPQ reduces the mean error from 7.62% to 4.53% for the first 9 modes, and from 113.22% to 3.48% for the first 16 modes (B and C fail on the 16-th eigenmode), at comparable training speeds. This supports the benefit of TPQ beyond the network parameterization and potential approximation. We further evaluate the same learned functions using both the original Hamiltonian H\mathcal H and its GMM approximation HK\mathcal H_K. For CP-TPQ, the errors change only from 4.53% to 4.68% for the first 9 modes and from 3.48% to 3.55% for the first 16 modes. CP-MC also shows similar errors under the two evaluation operators. These results indicate that the improved accuracy of CP-TPQ is not simply caused by evaluating a smoothed Coulomb operator. The advantage of CP-TPQ over CP-MC remains when both methods are evaluated using the original Coulomb Hamiltonian. The new experiments could better deliver improved spectral accuracy at comparable computational cost under the tested conditions. We have also clarified that TPQ's computational advantage relies on a suitable separable representation of the spectral functions and operator in Section 2.1 of the revised manuscript.

Q1: Do the improvements persist when direct Monte Carlo integration and TPQ use the same CP architecture, potential representation, and comparable wall-clock budgets? This would help determine how much benefit comes specifically from quadrature assembly versus the representation and smoothing.

Response: Yes. We have added a controlled ablation study on 2D hydrogen in Appendix D (Table 4) of the revised manuscript, as detailed in response to your comment W2(2). We compare CP-TPQ and CP-MC using the same CP architecture, GMM potential, and 30,000 training iterations, with comparable training speeds. Both methods are evaluated using the original Coulomb Hamiltonian. For the first 9 modes, TPQ reduces the mean relative eigenvalue error from 7.62% to 4.53%. For the first 16 modes, TPQ reduces the error from 113.22% to 3.48%. Since the architecture and potential representation are fixed, the improvement can be attributed to the quadrature method rather than differences in network representation or potential smoothing. Thus, TPQ provides better spectral accuracy than direct MC under comparable computational budgets in this experiment.

Reviewer_aXxYOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes tensor product quadrature (TPQ) for neural operator eigendecomposition and singular value decomposition. The main idea is to parameterize each spectral function using a variable-separable CP representation. With this representation, quantities such as the mass matrix ⟨uℓ,um⟩\langle u_\ell,u_m\rangle, and operator quadratic forms ⟨uℓ,Tum⟩\langle u_\ell,T u_m\rangle when the operator has suitable separable structure, can be computed through one-dimensional quadratures instead of high-dimensional Monte Carlo integration.

The authors apply this idea to several existing neural spectral methods, including SpIN, NeuralEF, and NeuralSVD (in the main text), and also discuss a power-method-based eigensolver (in Appendix). For non-separable terms such as the Coulomb potential, they approximate the potential using a separable Gaussian expansion so that the corresponding operator term can still be evaluated using TPQ. The paper includes approximation results for the CP representation of spectral functions, variance analyses comparing TPQ and Monte Carlo quadrature, and experiments on Schrodinger equations, integral-kernel SVD, and NTK decomposition. Overall, TPQ appears to improve convergence and numerical accuracy over the corresponding dense-MLP, Monte Carlo-based versions of the spectral objectives on the tested problems.

Critical strengths

Overall, the paper is written reasonably well to convey the high level idea. The issue of high-dimensional integrals in the neural spectral learning approach is well motivated. The core computational idea is practically relevant: if the spectral functions and the operator have suitable tensor-product structure, the high-dimensional integrals appearing in neural spectral objectives can be reduced to one-dimensional quadratures. The empirical results suggest that this can substantially improve the accuracy and stability of the optimization compared with standard mini-batch Monte Carlo integration.

I also appreciate that the authors test the idea beyond a single spectral objective. SpIN, NeuralEF, and NeuralSVD use different mechanisms for learning ordered spectral functions, and the paper shows how the tensorized computation can be incorporated into each. In particular, the SpIN case requires additional treatment of its masked gradient, so not all of these extensions are completely mechanical.

Critical weaknesses

Having said that, I found several concerns regarding the presentation, novelty, and applicability of the proposed method.

  1. (Presentation) First, I found the presentation somewhat redundant than necessary. The paper develops TPQ separately for SpIN, NeuralEF, and NeuralSVD, although these objectives seem to depend largely on the same basic quantities, namely bilinear forms. I think it would be much more informative to first identify these common mathematical primitives, explain when TPQ makes each of them efficiently computable, and then present SpIN, NeuralEF, and NeuralSVD as different instantiations. This would also make the scope and limitations of the method much clearer.

  2. (Novelty) I also have a concern around the novelty relative to the existing tensor neural network (TNN) literature. Prior work on TNN already uses variable-separable neural representations so that high-dimensional integrals can be computed through products of low-dimensional quadratures, and subsequent work applies this idea to multi-eigenpair computation without Monte Carlo integration. Appendix E discusses the distinction from this literature, but most of the distinctions seem to come from replacing the Rayleigh-Ritz-style outer eigensolver with existing objectives such as SpIN, NeuralEF, and NeuralSVD. I agree that these extensions are useful, especially for non-self-adjoint SVD and the SpIN masked-gradient construction, I am not yet convinced what part of the underlying TPQ principle itself is new relative to prior TNN methods, which already exploit variable-separable representations to reduce high-dimensional integration to products of low-dimensional quadratures. The additional developments in this paper, such as incorporating this machinery into SpIN, NeuralEF, and NeuralSVD, may well be useful and technically nontrivial, but this seems different from claiming TPQ itself as the main methodological novelty. I think this distinction should be made much more precise.

  3. (Structural assumption and approximability) My last concern is the actual scope of the computational advantage. The CP representation makes quantities such as ⟨uℓ,um⟩\langle u_\ell,u_m\rangle easy to compute using one-dimensional quadratures, but the same is true for ⟨uℓ,Tum⟩\langle u_\ell,T u_m\rangle only when TT is separable, or can itself be well approximated by a low-separation-rank representation as in the Coulomb potential case.

  • For the hydrogen problem, the authors explicitly approximate the non-separable Coulomb potential using a separable Gaussian expansion.

    • Relatedly, to provide a broader context: in genuinely high-dimensional quantum chemistry problems, high-dimensional integration is typically handled using VMC/MCMC sampling of electron configurations, which does not require the potential to admit a separable representation. This makes me somewhat unclear about the practically relevant high-dimensional regime in which TPQ is expected to provide an advantage over the standard sampling-based approach.
  • For the NTK problem, where the operator is not assumed to be separable, the operator term still requires evaluation on the tensor-product grid.

    The paper provides approximation theory for the CP representation of the spectral functions, but I could not find a corresponding analysis of when a general non-separable operator can be approximated efficiently in this separable manner, how large the required separation rank is, or how the approximation error affects the learned spectrum. This makes it difficult for me to understand how broadly the claimed high-dimensional scalability should be interpreted.


Given these concerns, I am currently leaning toward a weak reject. However, if the authors can improve the presentation, clarify the applicability and scope of the method, and more sharply distinguish the current manuscript from the existing TNN work based on Rayleigh-Ritz methods, I may increase my score.

Questions for authors

I would greatly appreciate if the authors can provide answers to the following questions, related to the weaknesses written above.

  • (On novelty): Compared with the TNN numerical-integration framework and its application to multi-eigenpair computation, what exactly is the new technical contribution of TPQ itself? My current understanding is that the underlying separable representation and tensorized quadrature principle are already present in the TNN literature, while the main new aspect here is to use this machinery inside SpIN, NeuralEF, NeuralSVD, and related objectives. Is this the intended distinction?

  • (On the scope of TPQ): Can the authors state more explicitly the conditions on the spectral-function representation, integration measure, and operator TT under which ⟨uℓ,Tum⟩\langle u_\ell,T u_m\rangle can be computed using only one-dimensional TPQ operations? What happens computationally when TT is genuinely non-separable and does not admit an accurate low-separation-rank approximation?

  • (On the operator approximation): For the Gaussian approximation of the Coulomb potential, can the authors quantify the approximation error of VV by the separable surrogate in a norm relevant to the spectral problem? How does this error propagate to the resulting eigenvalues or eigenspaces? More generally, is there a condition relating the separation rank required to approximate TT, the computational complexity, and the final spectral error?

  • (On experiments) Could the authors include a controlled comparison between the same CP-separable network trained with ordinary full-dimensional Monte Carlo quadrature and with TPQ? The current comparisons change both the function parameterization (dense MLP to CP network) and the integration scheme (MC to TPQ), making it difficult to isolate the contribution of TPQ itself.

  • (On high-dim. quantum applications): I am also curious about the intended regime of TPQ for genuinely high-dimensional many-body quantum problems. In practical quantum chemistry, the high-dimensional integration problem is typically handled using VMC-style sampling. Is TPQ intended to replace this sampling step, to complement VMC through a structured wavefunction representation or variance reduction, or mainly to target a different class of (more low-dimensional) spectral problems?

14 comments
AuthorsOpenReview ↗

Response To Reviewer aXxY

Response To Reviewer aXxY

We sincerely thank the reviewer for the thoughtful comments and questions. We have revised the presentation and clarified the novelty and applicability of our method. The revised part is highlighted in blue in the uploaded revised manuscript. Our point-by-point responses are provided below.

W1: (Presentation) First, I found the presentation somewhat redundant than necessary. The paper develops TPQ separately for SpIN, NeuralEF, and NeuralSVD, although these objectives seem to depend largely on the same basic quantities, namely bilinear forms. I think it would be much more informative to first identify these common mathematical primitives, explain when TPQ makes each of them efficiently computable, and then present SpIN, NeuralEF, and NeuralSVD as different instantiations. This would also make the scope and limitations of the method much clearer.

Response: Thank you for this helpful suggestion. We have reorganized Section 2 of the revised manuscript to improve clarity and reduce redundancy. Specifically, we added Section 2.1 (General TPQ: Reusable Bilinear-Form Primitives), which first introduces the common bilinear forms (mass and stiffness matrices) shared by SpIN, NeuralEF, and NeuralSVD. We then explain how TPQ efficiently evaluates these quantities, introduce the proposed Gaussian mixture method for non-separable potentials, and clarify the required assumptions on operator separability. Based on this unified formulation, Sections 2.3–2.5 present SpIN-TPQ, NeuralEF-TPQ, and NeuralSVD-TPQ as three specific applications, focusing on their respective objective functions and implementation details. These changes are included in Section 2 of the revised manuscript.

AuthorsOpenReview ↗

Response To Reviewer aXxY

W2: (Novelty) I also have a concern around the novelty relative to the existing tensor neural network (TNN) literature. Prior work on TNN already uses variable-separable neural representations so that high-dimensional integrals can be computed through products of low-dimensional quadratures, and subsequent work applies this idea to multi-eigenpair computation without Monte Carlo integration. Appendix E discusses the distinction from this literature, but most of the distinctions seem to come from replacing the Rayleigh-Ritz-style outer eigensolver with existing objectives such as SpIN, NeuralEF, and NeuralSVD. I agree that these extensions are useful, especially for non-self-adjoint SVD and the SpIN masked-gradient construction, I am not yet convinced what part of the underlying TPQ principle itself is new relative to prior TNN methods, which already exploit variable-separable representations to reduce high-dimensional integration to products of low-dimensional quadratures. The additional developments in this paper, such as incorporating this machinery into SpIN, NeuralEF, and NeuralSVD, may well be useful and technically nontrivial, but this seems different from claiming TPQ itself as the main methodological novelty. I think this distinction should be made much more precise.

Response: Thank you for this comment. We agree that the underlying TPQ principle is not new. We have clarified our methodological contributions at the beginning of Section 2 and expanded the comparison with TNN methods in Appendix Section F of the revised manuscript.

(1) Different roles of TPQ in spectral computation. Both TNN and our method use TPQ for efficient high-dimensional integration. However, TNN-based Rayleigh–Ritz methods use TPQ to learn a trial subspace and then recover eigenfunctions by solving a projected generalized eigenvalue problem. In contrast, our framework uses TPQ as a high-dimensional quadrature module and adapt it into different neural spectral objectives, including SpIN, NeuralEF, and NeuralSVD. This enables direct optimization of spectral functions without requiring a separate Rayleigh–Ritz post-processing step.

(2) Our technical contributions. As noticed by the reviewer, integrating TPQ into these objectives requires several nontrivial developments, as summarized in Section 2:

  • A pre-trained Gaussian-mixture surrogate for handling non-separable potentials.
  • A boundary-compatible masked gradient construction for SpIN-TPQ.
  • A factorwise normalization technique for NeuralEF-TPQ for avoiding full-grids.
  • Two-branch nested low-rank assembly for NeuralSVD-TPQ that enables operator SVD.

Moreover, we have extended the Gaussian-mixture approximation and TPQ framework to handle pairwise electron–electron Coulomb interactions in many-body quantum systems, and validated this extension through a new 6D two-electron experiment. Please see our response to your Comment W3.1 for details.

(3) Benefits over TNN-based spectral solvers. Our framework enables direct and mode-resolved spectral learning and extends TPQ-based computation to non-self-adjoint operator SVD. Moreover, NeuralEF-TPQ and NeuralSVD-TPQ avoid mass-matrix inversion and generalized eigensolves, which can improve numerical stability when learning multiple spectral modes. The comparison in Fig. 8 of the revised manuscript shows that NeuralEF-TPQ achieves more stable convergence than TNN as the number of target eigenpairs increases. Therefore, though the TPQ integration principle is not new, we have proposed a unified framework that incorporates TPQ into different neural spectral optimization objectives, together with the technical developments needed to make these integrations effective.

Finally, we have revised the wording throughout the manuscript to avoid presenting TPQ itself as a new methodological contribution. In particular, we acknowledge the original TNN literature as the source of the TPQ principle, and emphasize that our contribution lies in adapting and extending this quadrature technique to different neural spectral objectives with new technical developments. These changes are reflected in the Introduction, the beginning of Section 2, and Appendix Section F of the revised manuscript.

AuthorsOpenReview ↗

Response To Reviewer aXxY

W3 (Structural assumption and approximability) My last concern is the actual scope of the computational advantage. The CP representation makes quantities such as ⟨ul,um⟩\langle u_l,u_m\rangle easy to compute using one-dimensional quadratures, but the same is true for ⟨ul,Tum⟩\langle u_l,Tu_m\rangle only when TT is separable, or can itself be well approximated by a low-separation-rank representation as in the Coulomb potential case.

Response: We agree with the reviewer. As clarified in the revised Section 2.1, the computational advantage of TPQ depends on the operator structure.

  1. Separable operators: For additively or multiplicatively separable operators, both the mass matrix Σ\Sigma and the operator-induced matrix Π\Pi can be efficiently assembled using TPQ.
  2. Non-separable operators: This advantage extends to operators admitting separable approximations, such as Gaussian-mixture approximation of the Coulomb potential that proposed in Section 2.2.
  3. General non-separable operators: When no suitable separable approximation exists, the operator-induced integrals still require full-dimensional evaluation, although the mass matrix can be efficiently computed by TPQ.

We have stated these conditions in the revised manuscript Section 2.1.

W3.1: For the hydrogen problem, the authors explicitly approximate the non-separable Coulomb potential using a separable Gaussian expansion. Relatedly, to provide a broader context: in genuinely high-dimensional quantum chemistry problems, high-dimensional integration is typically handled using VMC/MCMC sampling of electron configurations, which does not require the potential to admit a separable representation. This makes me somewhat unclear about the practically relevant high-dimensional regime in which TPQ is expected to provide an advantage over the standard sampling-based approach.

Response: Thank you for this comment. We agree that VMC/MCMC does not require a separable potential, while TPQ relies on a suitable structured representation. To clarify the practical applicability of TPQ, we have added a six-dimensional two-electron Schrödinger experiment with Coulomb interaction, directly comparing NeuralEF-TPQ with a VMC-based baseline in this setting (Section L.1 in the revised manuscript). We consider two same-spin electrons in a three-dimensional harmonic trap, giving a six-dimensional configuration space. The Hamiltonian includes the non-separable electron-electron Coulomb interaction 1/∣r1−r2∣21/|r_1-r_2|_2, and we learn the lowest 12 antisymmetric states. For NeuralEF-TPQ, we approximate the electron-electron interaction using a Gaussian expansion with K=30K=30. The interaction is then assembled through three two-dimensional coordinate-pair quadratures, avoiding the full six-dimensional tensor grid. For comparison, NeuralEF-VMC uses an MLP with 2,048 persistent Metropolis walkers and evaluates the original Coulomb potential. Both methods use the same NeuralEF objective and are trained for 10,000 iterations. The VMC batch size is adjusted to achieve comparable training iterations per second.

Table 11 of the revised manuscript reports the physical energies and relative eigenvalue errors after 10,000 iterations, we paste it here:

ModeReference energyTPQ energyVMC energyTPQ relative errorVMC relative error
14.515076524.514690884.646400428.5412×10⁻⁵2.9086×10⁻²
24.515076524.514897824.657696363.9577×10⁻⁵3.1587×10⁻²
34.515076524.515544414.661463521.0363×10⁻⁴3.2422×10⁻²
45.515076525.514023785.642072611.9088×10⁻⁴2.3027×10⁻²
55.515076525.516417035.648788532.4306×10⁻⁴2.4245×10⁻²
65.515076525.517086035.651246423.6437×10⁻⁴2.4690×10⁻²
75.515076525.517141826.051468633.7448×10⁻⁴9.7259×10⁻²
85.515076525.517386446.097744054.1884×10⁻⁴1.0565×10⁻¹
95.515076525.517714026.209733594.7824×10⁻⁴1.2596×10⁻¹
105.515076525.517823706.479816344.9812×10⁻⁴1.7493×10⁻¹
115.515076525.518265726.738264275.7827×10⁻⁴2.2179×10⁻¹
125.515076525.519499306.777890628.0194×10⁻⁴2.2897×10⁻¹
Mean error over 12 modes———3.4807×10⁻⁴9.3301×10⁻²
Maximum error over 12 modes———8.0194×10⁻⁴2.2897×10⁻¹

NeuralEF-TPQ achieves lower relative eigenvalue errors than NeuralEF-VMC for all 12 states. Furthermore, Fig. 12 of the revised manuscript shows the convergence of all 12 predicted energies over 10,000 iterations, while Fig. 13 compares the learned eigenfunction slices at 5,000 and 10,000 iterations. These results further demonstrate the improved spectral accuracy and eigenfunction approximation of NeuralEF-TPQ in this interacting system. It provides an evidence that TPQ can outperform the tested VMC baseline in a six-dimensional interacting quantum system.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised Manuscript Uploaded

Dear Reviewers and AC,

We sincerely thank the reviewers for their valuable comments and suggestions. We have carefully revised the manuscript to address the concerns raised in the reviews and uploaded the updated version. All modifications are highlighted in blue in the revised manuscript for ease of reference. We greatly appreciate the reviewers' time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.