acceptodds
Under review as a conference paper at ICLR 2027

Apparent Compression, Real Stability: The Intrinsic Dimension of Learning a Quantum Wavefunction

Abstract

How many directions in weight space does training need? The intrinsic dimension answers this with the smallest number of random directions in which training still reaches a target accuracy, and small values have motivated parameter-efficient methods such as LoRA. We measure it for variational Monte Carlo (VMC), which trains a neural network to represent the ground state of a quantum many-body system. VMC is a demanding test, because the network generates its own training samples and every gradient is noisy, and a revealing one, because the exact answer is known and every run can be scored. We train only a small latent vector that a frozen random map turns into the network's weights, with no change to the standard natural-gradient optimizer. We find that a small dimension can be misleading, while the stability it brings is real. On a magnet with a hard sign pattern, a network that cannot represent signs reaches its best energy in 8 of 28,642 directions, but only because no such network can go lower; once signs are learnable, neither the signs nor the magnitudes are cheap. The dimension rises across a quantum phase transition, so it tracks how difficult a state is at far less compute than fitting a scaling law, yet it never falls below a floor set by the random subspace itself, even where the ground state is nearly trivial. Training in the subspace, in contrast, never diverged in our experiments, whereas full-parameter training with the same settings did, and a control with matched solvers attributes the difference to the reduced dimension.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural quantum states, variational Monte Carlo, intrinsic dimension, subspace training, mapping networks, quantum many-body physics

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4ALrOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates the intrinsic dimension of training neural quantum states (NQS) using variational Monte Carlo. Following existing approaches to measuring intrinsic dimension, the authors restrict optimization to random low-dimensional subspaces of the network parameters, using stochastic reconfiguration in the latent space.

The experiments focus on the J1–J2 Heisenberg model and the transverse-field Ising model. The main findings are that low-dimensional training can improve optimization stability, that apparent compressibility can reflect limitations of the variational ansatz rather than the complexity of the ground state, and that the measured intrinsic dimension changes across a quantum phase transition.

Critical strengths
  1. Potentially useful diagnostic. Measuring intrinsic dimension could provide an inexpensive proxy for the difficulty of learning different quantum systems, particularly if it can be related to physical properties or computational scaling.

  2. Interesting experimental controls. The separation between amplitude and sign learning, particularly the exact-sign comparison and the identification of the positive-cone bound, helps distinguish limitations of the ansatz from optimization difficulty.

  3. Optimization stability. The reported reduction in divergence events under subspace training is encouraging. The additional experiment using a matched stochastic reconfiguration solver helps support this observation.

Critical weaknesses
  1. Limited novelty. The methodology largely transfers established techniques for measuring intrinsic dimension and optimizing in random subspaces to NQS. The stochastic reconfiguration formulation follows directly from the standard pullback of the quantum geometric tensor. The main novelty is therefore in the application and empirical observations rather than in the methodology.

  2. Limited scientific insight. My principal concern is that the paper does not develop a sufficiently strong hypothesis about what determines the intrinsic dimension of learning a quantum wavefunction. The findings are largely descriptive, and several are natural consequences of previously understood properties of random subspaces, variational bounds, and optimization. More interesting relationships between intrinsic dimension, Hamiltonian properties, expressivity, and optimization difficulty could have been theoretically motivated and empirically tested. For me to change my assesment of this paper it would require the authors to do substantial work that I think would be unreasonable and probably innacurate during a short rebuttal period.

  3. Insufficient experimental diversity. The evaluation considers only two Hamiltonian families and one neural architecture family, with relatively small system sizes. This limits the generality of the conclusions, particularly those concerning sign complexity and optimization stability.

  4. Limited statistical robustness. Several experiments use only two or three seeds, despite substantial seed dependence and multimodal optimization outcomes. This is particularly problematic when intrinsic dimension is defined by whether the median run crosses a fixed accuracy threshold.

  5. Presentation. The writing would benefit from substantial revision. The introduction is not always well structured, and parts of the experimental discussion use repetitive or unnatural phrasing that obscures the main observations.

Questions for authors
  1. What are the main insights or intuitions on the critical aspects that determine the intrinsic dimension of a wavefunction beyond Hamiltonian size and frustration? Are there any concrete hypothesis that you would formulate in this regard?

  2. How sensitive are the conclusions to the choice of architecture? Would the same Hamiltonian exhibit comparable intrinsic dimensions under substantially different NQS parameterizations? Are there any chances that the intrinsic dimension depends also on the optimization method used (e.g. SR vs. Adam) or sampling procedure (auto-regressive vs. metropolis hastings)?

  3. Given the strong seed dependence, how robust are the reported intrinsic dimensions to additional seeds and changes in the accuracy threshold?

8 comments
AuthorsOpenReview ↗

Response to Reviewer 4ALr (1/6)

W2 + Q1. What determines the measured dimension, and what hypothesis can we test?

Reviewer concern. Limited scientific insight. My principal concern is that the paper does not develop a sufficiently strong hypothesis about what determines the intrinsic dimension of learning a quantum wavefunction. The findings are largely descriptive, and several are natural consequences of previously understood properties of random subspaces, variational bounds, and optimization. More interesting relationships between intrinsic dimension, Hamiltonian properties, expressivity, and optimization difficulty could have been theoretically motivated and empirically tested. For me to change my assesment of this paper it would require the authors to do substantial work that I think would be unreasonable and probably innacurate during a short rebuttal period.

What are the main insights or intuitions on the critical aspects that determine the intrinsic dimension of a wavefunction beyond Hamiltonian size and frustration? Are there any concrete hypothesis that you would formulate in this regard?

Response.

We thank the reviewer for the careful review, and for recognizing the exact-sign control, the positive-cone bound and the matched-solver stability result.

Our hypothesis is that a random slice imposes a floor on the measured dimension, that a direction matched to the dominant physical structure removes that floor, and that the dimension that remains follows the demand of the Hamiltonian. At a 2% energy target, the uniform transverse product state is sufficient at field-to-coupling ratio 4 and insufficient at 1.2. The hypothesis predicts that making this state reachable lowers the measured dimension at 4 but leaves it unchanged at 1.2. The first tested dimension meeting the target fell from 64 to 8 at 4 and stayed at 64 at 1.2. The test added after submission used six matched seeds, replaced one direction within each pair and evaluated energies over every configuration.

The floor is the submission's observation in Section 4.3. Deep in the paramagnet, random-subspace training needs far more coordinates than the state requires. At field-to-coupling ratios 4 and 10 a parameter-free product state is within 2% of the exact energy (1.5445% and 0.2495%), while random-map training first reaches 2% at 64 coordinates.

The strong-field expansion identifies the dominant structure and gives the prediction. At ratio 4 a nearest-neighbor trial state from the expansion reaches 0.00615% error, so the product state meets a 2% target and misses a 1% target that the correlated state passes. At ratio 1.2 the best product state has 4.2231% error and meets neither.

The controlled comparisons use the 16-spin transverse-field Ising chain and vary the field, the trainable directions and the optimizer. The three properties named in the review enter as follows.

Hamiltonian properties determine which physical structures are needed at a given tolerance. When the trainable directions give access to the uniform transverse product state, lowering the field-to-coupling ratio from 4 to 1.2 raises the measured dimension from 8 to 64 at a 2% tolerance.

Expressivity determines which states the trainable directions can reach. At ratio 4, replacing one random direction by the direction that reaches that product state lowers the measured dimension from 64 to 8 at 2%, and from no passing point on the grid to 16 at 1%.

Optimization difficulty determines which reachable states are found within the training budget. Replacing the optimizer changes individual runs near a threshold and leaves the measured dimension unchanged.

AuthorsOpenReview ↗

Response to Reviewer 4ALr (2/6)

W2 + Q1 (continued)

The test compares random maps with maps that replace the sole output direction by the direction toward zero output weights, which gives access to the uniform transverse product state. The network, initialization, other directions, and scales are fixed within each pair. At a given dimension and seed, both fields use the same maps. The grid, seeds and tolerances were registered before the 36 pairs at 8, 16 and 64 coordinates were run, and every state is scored by summation over all 65,536 configurations. The table gives the median error and individual 1% passes for six matched seeds per cell under one exact-energy L-BFGS-B protocol, and the first tested dimension whose median error meets each tolerance.

Field/couplingMap8163264First dimension meeting 1%First dimension meeting 2%
4Random22.106% (0/6)11.250% (0/6)4.610% (0/6)1.435% (0/6)None on grid64
4Aligned1.357% (0/6)0.924% (4/6)0.410% (6/6)0.172% (6/6)168
1.2Random13.801% (0/6)6.574% (0/6)3.634% (0/6)1.978% (0/6)None on grid64
1.2Aligned9.365% (0/6)6.595% (0/6)2.747% (0/6)1.405% (0/6)None on grid64

Rows 1 and 2 compare the two maps at ratio 4. Replacing one direction lowers the first passing dimension from 64 to 8 at 2%. At 1% the random map has no passing point on the grid and the aligned map passes at 16. The intervention changes one trainable direction and nothing else. A separate sampled-SR comparison with 6,000 updates at 32 coordinates gives the same contrast, 0/6 against 6/6 passes at 1%, and its energy decomposition shows a lower transverse-field cost with bond correlations still present.

Rows 2 and 4, and rows 1 and 3, change the Hamiltonian. With the aligned map, changing the field ratio from 4 to 1.2 raises the first passing dimension from 8 to 64 at 2% and removes every 1% pass on the grid. With random maps both fields first reach 2% at the last grid point (medians 1.435% and 1.978% at 64 coordinates) and neither reaches 1%. The aligned map separates the two Hamiltonians, 8 against 64 at 2%, as the hypothesis predicts.

Optimization changes individual attainment near a threshold and leaves the first passing dimension unchanged. At ratio 4 the random-map dimension at 2% is 64 under the submitted SR training, independently rescored, and under exact-energy L-BFGS-B on a new cohort of maps. At ratio 0.95, SR and Adam on paired maps both first pass 2% at 64 coordinates (response to W3 + Q2). On the aligned maps of the table at ratio 4, replacing L-BFGS-B by 6,000 sampled-SR updates gives median errors of 1.358% at 8 coordinates and 0.882% at 16, against 1.357% and 0.924%. Individual 1% passes at 16 coordinates change from 4/6 to 5/6, and the first passing dimension stays at 16 at 1% and 8 at 2%.

On the frustrated lattice, expressivity enters through a class bound. No wavefunction with non-negative amplitudes can go below 52.7% error, and the non-negative network approaches that bound with 8 of 28,642 directions (Section 4.1). In the sign-capable model at a fixed total of 2,048 coordinates, moving coordinates from amplitude to phase lowers the median error from 23.03% to 5.20% and lowers the error in every matched pair, and supplying the exact sign at the same 1,938 amplitude coordinates lowers the median to 3.48%.

The submission tests competing explanations at three points. It tests a class bound against the state for the 8-direction result (Sections 4.1 and 4.2), too few sign dimensions against a costly sign for the failure of proportional allocation (Section 4.2), and a physical maximum against a threshold artifact for the 16-spin interior maximum, pre-registered and decided at 24 spins (Section 4.3 and Appendix E). The better-conditioned map trains worse (condition number 10210^{2} to 10410^{4} against 101010^{10}, Section 4.1), and with 2 or 8 amplitude dimensions the learned sign gives a lower error than the exact sign (Section 4.2).

AuthorsOpenReview ↗

Response to Reviewer 4ALr (3/6)

W1. What is new beyond applying random-subspace methods to neural quantum states?

Reviewer concern. Limited novelty. The methodology largely transfers established techniques for measuring intrinsic dimension and optimizing in random subspaces to NQS. The stochastic reconfiguration formulation follows directly from the standard pullback of the quantum geometric tensor. The main novelty is therefore in the application and empirical observations rather than in the methodology.

Response.

The Introduction identifies the contribution as a measurement for variational Monte Carlo and the empirical findings it yields, proposes no new training algorithm, and states that to our knowledge no published work measures intrinsic dimension in this setting. The random-subspace construction follows Li et al. (2018), and stochastic reconfiguration applies to the latent coordinates through the standard pullback. What is new in the methodology is a set of controls that determine what a measured dimension means. They are a restricted wavefunction class whose attainable energy bound is known, separate coordinate budgets for sign and amplitude compared against a network with the exact sign, and a product-state limit that exposes the cost of the random slice (Section 4.3). The known ground-state energy distinguishes matching the optimum of a restricted model from approximating the target state. A small dimension can reflect an attainable bound of a restricted wavefunction class or the orientation and allocation of the trainable directions.

The positive-wavefunction example in Section 4.1 provides a concrete diagnostic. Measured against the full model, as the criterion of Li et al. prescribes, 8 of 28,642 directions lose almost nothing (0.5313 against 0.5271). Measured against the exact energy, both are about 53% away, because both approach the positive-cone bound. That the standard measurement cannot see the bound is the finding. The interventions described in our response to W2 + Q1 add an experimental test of the orientation and allocation interpretation. At field-to-coupling ratio 4, changing one direction lowers the measured dimension from 64 to 8 at a 2% tolerance. At a fixed total of 2,048 coordinates, moving coordinates from amplitude to phase lowers the median independently evaluated error from 23.03% to 5.20%. These are properties of the tested learning problem that the SR pullback alone does not determine.

Reviewer_GtJLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the intrinsic dimension of training neural quantum states (NQS) with variational Monte Carlo (VMC). Training is restricted to a frozen random low-dimensional parametrization (a linear random subspace, plus a nonlinear tanh variant inspired by Mapping Networks), and the observable d* is the smallest tested dimension at which the median relative energy error over seeds falls below a threshold within a fixed training budget. Experiments on the frustrated J_1–J_2 Heisenberg model (4×4 and 6×6) and on the transverse-field Ising chain (L=16,24) examine how this measurement depends on sign structure, Hamiltonian coupling, training budget, and optimization stability.

The authors report three findings. (i) A small measured dimension can reflect a restrictive variational bound rather than an accurate approximation of the ground state: a network with non-negative amplitudes reaches its optimum in 8 of 28,642 directions, but this optimum is far from the true ground-state energy. Controls that separately vary the amplitude and sign dimensions (including an exact-sign control) show that neither is cheap. (ii) On the Ising chain, d* increases about 4× across the transition and then relaxes only to a floor set by the random subspace itself. (iii) Subspace training logged no divergence events, whereas full-parameter training with the same optimizer settings diverged repeatedly on the 6×6 lattice. A control with the full-parameter solver applied to the subspace is used to attribute the difference to the reduced dimension. No new training algorithm is proposed. The contribution is a measurement protocol and its empirical findings.

Critical strengths
  1. Careful interpretation of apparent compression. The positive-cone bound provides a concrete example in which training with very few directions reaches the restricted model class’s optimum while remaining far from the true ground-state energy. This is a useful caution when interpreting small intrinsic-dimension measurements (Section 4.1).
  2. Informative amplitude and sign controls. Supplying the exact ground-state sign and separately varying the dimensions allocated to amplitude and phase helps distinguish limitations that proportional allocation alone would conflate. This makes the experimental analysis more informative than a sweep over the total trainable dimension (Section 4.2).
  3. Controls addressing alternative explanations. The authors compare two training budgets for the Ising experiments and rerun a subspace configuration with the full-parameter (minSR) solver. These controls strengthen the evidence that the reported effects cannot be explained solely by the shorter budget or the choice of solver (Sections 3.3, 4.3 and 4.4).
  4. Transparent reporting of limitations. The manuscript reports instability events, unsuccessful configurations, sensitivity to thresholds, and variation between reruns. It also limits the stability claim to the optimizer configuration studied and acknowledges that the 6×6 experiment cannot separate amplitude and sign deficits (Sections 4.2 and 4.4, Appendices A, D and E).
  5. Exact references. Every run is scored against an exact ground-state energy, including a new exact diagonalization of the 6×6 torus (Appendix C), which makes the error measurements unambiguous.
Critical weaknesses
  1. Uncertainty in the measured dimension needs clearer quantification. Several comparisons use two or three seeds, while the results exhibit substantial variation between seeds (e.g., full-parameter 4×4 seeds at 0.0382, 0.0444 and 0.1920). Appendix D also reports that rerunning the same seeds changes which attractor is reached. Since d* is determined by whether a median crosses a threshold, this variability can change the reported dimension; with two seeds the median is simply the mean of two values. In addition, the reported “best ε_rel” appears to be the best Monte Carlo estimate over training, which is biased downward, and this matters most for cells close to a threshold. Reporting uncertainty in threshold attainment (e.g., bootstrap or per-seed pass rates) would help establish which differences are reliable (Sections 3.5 and 4.1, Appendices D and E).
  2. The evidence does not establish that the entire sign block is necessary. At 1024 amplitude dimensions, increasing the sign dimension from 402 to 1633 reduces the median error from approximately 0.049 to 0.039 (Table 5). The seed ranges overlap (0.044–0.050 versus 0.038–0.046, Section 4.2), so with three seeds this is at most suggestive of a modest benefit, not a demonstrated one. Moreover, the sign sweep contains no allocation between 402 and 1633, so the conclusion that the sign “needs its whole block” (Introduction, Section 4.2, Figure 7) should be restricted to the tested allocations or supported by a finer sweep with more seeds. Relatedly, the exact-sign values at 415 and 1024 amplitude dimensions in Table 5 are interpolated (log-log) between runs rather than measured, so the statement that the learned sign matches the exact one within 6% from 415 dimensions on rests on interpolation.
  3. The central observable d* is not obtained on the frustrated testbed. In the J_1–J_2 experiments, the gated dimension is not reported at 4×4 because the full-parameter reference is bimodal across seeds (Appendix E), and at 6×6 it is right-censored (>3072) at every coupling (Section 4.2, Appendix A). The frustrated-magnet results are therefore comparisons of error at fixed d, while the only actual d* values come from the Ising chain. This is not stated clearly in the abstract or introduction, and the title and framing suggest that the intrinsic dimension of learning the frustrated wavefunction was measured.
  4. The claim that d* “tracks” the phase transition is stronger than the evidence. The rise in d* amounts to one or two steps on a coarse power-of-two grid (e.g., 64→128 at L=16), several of the relevant cells are marked as threshold-sensitive (†), d* remains at 128 up to h/J=2 rather than peaking at the transition, and only two system sizes are used without finite-size analysis. The authors themselves note that the staircase is a level cut of a smooth error surface (Appendix E, Fig. 9) and that its location drifts with the threshold, which suggests that d* reflects a smooth h-dependence of difficulty at least as much as the transition. In addition, the statements that d* costs “far less compute” than fitting scaling laws (Abstract, Section 5.2: “roughly two orders of magnitude”) are not supported by a compute accounting in the paper (Sections 4.3 and 5.2).
  5. The attribution of the stability improvement to reduced dimension remains partly unresolved. Repeating the subspace experiment with minSR is a useful control against a solver-specific explanation. However, the restricted parametrization also changes the available update directions and their scaling (each group is scaled by s_k), and the full-parameter baseline is plain SGD with a fixed learning-rate policy and a ∼1000-step budget. Better-tuned baselines (learning-rate schedule, QGT damping, adaptive optimizers, more samples) are not evaluated, as acknowledged in Appendix A. The seed-spread comparison is also unbalanced: 1.0–3.3× over two seeds for the subspace versus 10–52× over five seeds for the reference, and the range of a sample grows with its size. Finally, the divergence detector is the authors’ own protocol, so “zero divergence events” is defined relative to it. The observed stability advantage is useful even without establishing its mechanism, but the title and abstract language (“real stability”, “attributes the difference to the reduced dimension”) should reflect this scope (Sections 3.2–3.4 and 4.4).
  6. Claims of budget independence and convergence should be qualified. Agreement between the 2000-step and 6000-step protocols is established for 7 shared cells at a single threshold τ=10^(-2), and the control fails at tighter thresholds (3×10^(-3) mixed, 10^(-3) one cell moves 128→64; Appendix E). This supports robustness over the tested budgets at that threshold; it does not establish independence from optimization time in general. Likewise, a plateau over the final portion of training (Sections 4.2 and 4.3) does not distinguish a representational limitation from an optimization plateau. These observations support a finite-budget measurement, but stronger interpretations of the measured dimension require additional evidence (Section 4.3, Appendix E).
Questions for authors
  1. Energy evaluation. Are the reported best energies independently re-evaluated using fresh samples after checkpoint selection, or are they the minimum Monte Carlo estimates observed during training? Please report the corresponding statistical uncertainties and the number of samples per step, particularly for results close to the thresholds that define the intrinsic dimension.
  2. Variability and reproducibility. Appendix D reports different attractors when rerunning the same seeds. Which sources of randomness remain uncontrolled in these reruns? How sensitive are the main threshold crossings to this variability, particularly the cells marked as threshold-sensitive in Table 6? Could you report d* with an uncertainty interval (e.g., bootstrap over seeds), or increase the number of seeds for the cells that decide d*?
  3. Sign-block requirements. Could you add intermediate sign allocations (e.g., roughly 800–1000 dimensions) at 1024 amplitude dimensions, with more than three seeds, and report whether the 1.24× gap between 402 and 1633 sign dimensions exceeds the seed variation? Could the exact-sign control also be run directly at 415 and 1024 amplitude dimensions instead of interpolated?
  4. Stability comparison. Please specify the QGT damping or regularization, learning rates, and their schedules for the solver-matched comparison. Are the effective update magnitudes comparable between subspace and full-parameter training? Does the stability gap persist for a tuned full-parameter baseline (learning-rate schedule, larger damping, Adam) and with equal numbers of seeds?
  5. Role of sampling noise. For the 4×4 and L=16 systems, exact summation over the Hilbert space is feasible. Would exact-gradient runs separate the effect of sampling noise from the effect of reduced dimension on stability and on the seed-dependent attractors?
  6. Frustrated testbed. Is a gated d* obtainable for the J_1–J_2 model with a robust reference (e.g., best-seed gate or a better-converged reference)? If not, I suggest stating in the abstract that d^* is measured on the Ising chain only.
  7. Phase-transition claim. Can the location of the step in d* (h) be shown to scale with L (more sizes or a finer d grid), or be distinguished from a smooth level cut of ε(h,d)? What compute was used for the d* sweeps, to support the comparison with scaling-law fits?
15 comments
AuthorsOpenReview ↗

Response to Reviewer GtJL (1/6)

W1 + Q1 + Q2. How were energies scored, and how reliable are threshold crossings across seeds?

Reviewer concern. Uncertainty in the measured dimension needs clearer quantification. Several comparisons use two or three seeds, while the results exhibit substantial variation between seeds (e.g., full-parameter 4×4 seeds at 0.0382, 0.0444 and 0.1920). Appendix D also reports that rerunning the same seeds changes which attractor is reached. Since d* is determined by whether a median crosses a threshold, this variability can change the reported dimension; with two seeds the median is simply the mean of two values. In addition, the reported “best ε_rel” appears to be the best Monte Carlo estimate over training, which is biased downward, and this matters most for cells close to a threshold. Reporting uncertainty in threshold attainment (e.g., bootstrap or per-seed pass rates) would help establish which differences are reliable (Sections 3.5 and 4.1, Appendices D and E).

Energy evaluation. Are the reported best energies independently re-evaluated using fresh samples after checkpoint selection, or are they the minimum Monte Carlo estimates observed during training? Please report the corresponding statistical uncertainties and the number of samples per step, particularly for results close to the thresholds that define the intrinsic dimension.

Variability and reproducibility. Appendix D reports different attractors when rerunning the same seeds. Which sources of randomness remain uncontrolled in these reruns? How sensitive are the main threshold crossings to this variability, particularly the cells marked as threshold-sensitive in Table 6? Could you report d* with an uncertainty interval (e.g., bootstrap over seeds), or increase the number of seeds for the cells that decide d*?

Response.

We thank the reviewer for the detailed review. Several of its suggestions are now in the revision, including independent scoring, per-seed reporting and the best-seed gate.

The submitted tables report the lowest Monte Carlo energy estimate reached during training, at 4,096 samples per update. We have independently scored all 94 checkpoints that decide a value in Table 6, that is, every seed at the submitted dimension and at the preceding tested dimension where one exists. For 16 spins, scoring sums over all 65,536 configurations, which removes evaluation sampling error. For 24 spins, each checkpoint uses four fresh streams of 262,144 samples.

Under independent scoring the rise across the transition remains bracketed at the 1% tolerance at both sizes. In nine cells where the preceding tested dimension fails and the stated one passes, the dimension is 64 at the lower field ratios and 128 at the higher ones (response to W4). None of the nine is marked † in the submitted Table 6, and 15 of their 18 individual checkpoints pass at the stated dimension. At the submitted dimensions the independent medians of all 22 fields lie between 0.49% and 1.23%. At 1%, 10 pass (5 of 14 for 16 spins, 5 of 8 for 24 spins), 10 fail and 2 are unresolved. The ten that fail include all six †-marked values. At 2% all 22 pass. The revised Table 6 and the new Table 7 report the independently scored dimensions and every seed's error, with the four-stream intervals for the 24-spin checkpoints.

We added four new training seeds at the passing cell closest to the 1% tolerance (16 spins, field-to-coupling ratio 0.95, 6,000 updates, 4,096 samples per update). The 2% bracket reproduces. The 32-coordinate median fails (2.135%) and all four runs pass at 64 coordinates (median 1.171%). At 1% the 64-coordinate median moves from 0.969% in the two submitted seeds to 1.171% in the four new ones, so this cell lies within seed variation of the tolerance, and the nine brackets in the response to W4 do not use it.

For the same-seed reruns of Appendix D we cannot isolate which runtime or sampling difference changed the attractor. The new runs fix the initial weights, maps and sampler state and record the runtime configuration. The six-seed, exactly scored sign comparisons are in the response to W2.

AuthorsOpenReview ↗

Response to Reviewer GtJL (2/6)

W2 + Q3. How much of the sign block is needed, and can the exact-sign comparisons be measured directly?

Reviewer concern. The evidence does not establish that the entire sign block is necessary. At 1024 amplitude dimensions, increasing the sign dimension from 402 to 1633 reduces the median error from approximately 0.049 to 0.039 (Table 5). The seed ranges overlap (0.044–0.050 versus 0.038–0.046, Section 4.2), so with three seeds this is at most suggestive of a modest benefit, not a demonstrated one. Moreover, the sign sweep contains no allocation between 402 and 1633, so the conclusion that the sign “needs its whole block” (Introduction, Section 4.2, Figure 7) should be restricted to the tested allocations or supported by a finer sweep with more seeds. Relatedly, the exact-sign values at 415 and 1024 amplitude dimensions in Table 5 are interpolated (log-log) between runs rather than measured, so the statement that the learned sign matches the exact one within 6% from 415 dimensions on rests on interpolation.

Sign-block requirements. Could you add intermediate sign allocations (e.g., roughly 800–1000 dimensions) at 1024 amplitude dimensions, with more than three seeds, and report whether the 1.24× gap between 402 and 1633 sign dimensions exceeds the seed variation? Could the exact-sign control also be run directly at 415 and 1024 amplitude dimensions instead of interpolated?

Response.

We agree that “the sign needs its whole block” could be read as a statement about every allocation between 402 and 1,633, and the submitted Section 4.2 itself reports that the three-seed ranges at 402 and 1,633 overlap at 1,024 amplitude dimensions. We therefore replace it with “the sign block needs more than its proportional share” in Section 4.2 and the caption of Figure 7 and with “allocating dimensions by block size starves the sign” in the Introduction, and we remove the 1.24× comparison between 402 and 1,633 from Sections 4.2 and 5.1 and from the caption of Table 5.

The replacement rests on a comparison with six matched seeds, 1,500 updates and terminal energies summed exactly over all 12,870 configurations of the zero-magnetization sector. At a fixed total of 2,048 coordinates, the proportional allocation gives the sign block 110 coordinates and ends at a median error of 23.032%. Giving the sign block its 1,633 coordinates, with 415 left for the amplitude, ends at 5.198% and is lower in all six pairs.

We ran the exact-sign control directly at 415 and at 1,024 amplitude dimensions in the same six-seed protocol. At 415 it gives a median error of 5.777%, close to the interpolated 0.057 of Table 5, and the learned sign with its complete block gives 5.198%. As Section 4.2 notes, the exact ground-state sign need not be the best sign for a restricted amplitude. At 1,024 the exact-sign control gives 4.314%, and the learned sign with its complete block gives 5.029%. We agree that “From 415 dimensions on it matches the exact-sign control within 6%” could be read as a directly measured comparison, whereas Table 5 marks the exact-sign entries at 415 and 1,024 as interpolated. We therefore replace it with the direct measurements. With its complete block the learned sign stays within 17% of the exact-sign control at both amplitude dimensions, 10% below it at 415 and 17% above it at 1,024.

AuthorsOpenReview ↗

Response to Reviewer GtJL (3/6)

W3 + Q6. Can a threshold dimension be obtained for the frustrated lattice using a robust reference?

Reviewer concern. The central observable d* is not obtained on the frustrated testbed. In the J_1–J_2 experiments, the gated dimension is not reported at 4×4 because the full-parameter reference is bimodal across seeds (Appendix E), and at 6×6 it is right-censored (>3072) at every coupling (Section 4.2, Appendix A). The frustrated-magnet results are therefore comparisons of error at fixed d, while the only actual d* values come from the Ising chain. This is not stated clearly in the abstract or introduction, and the title and framing suggest that the intrinsic dimension of learning the frustrated wavefunction was measured.

Frustrated testbed. Is a gated d* obtainable for the J_1–J_2 model with a robust reference (e.g., best-seed gate or a better-converged reference)? If not, I suggest stating in the abstract that d^* is measured on the Ising chain only.

Response.

We use the best-seed gate that the reviewer proposes. It is the robust gate of the submission, in which the median must come within twice the best full-parameter seed (Section 4.2 and Appendix A). Under it the dimension of the sign-capable model at 6×6 is right-censored above 3,072 at every coupling, and this bound is the measurement. At 4×4 and coupling ratio 0.5 the submission gates against twice the full-parameter reference, 0.0888, and proportional allocation does not meet this gate at any tested dimension up to 512 (Section 4.2). For the non-negative network, Section 4.1 reports that 8 of 28,642 directions reach the positive-cone bound. The title calls the compression “apparent” for this reason. The small dimension is a property of the model class and says nothing about the ground state.

We agree that “We measure d* on two spin models” could be read as reporting an attained threshold for the sign-capable frustrated model. We therefore replace it with “We measure attained thresholds on the Ising chain; on the frustrated magnet we report the class bound of the non-negative model and a censored threshold for the sign-capable model, above 3,072 dimensions at 6×6”. The revised abstract states that on the larger lattice the dimension exceeds 3,072, the largest subspace we train.

Reviewer_VsL4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the intrinsic dimension of training neural quantum states (NQS) using variational Monte Carlo (VMC), with particular attention to optimization stability and the interpretation of low-dimensional solutions. Following existing random-subspace methods, the authors train a small latent vector mapped to the full network parameters through a frozen random projection. Stochastic reconfiguration is applied in the latent space.

Experiments on frustrated Heisenberg models and transverse-field Ising chains examine the effects of restricting trainable dimensions, separating sign and amplitude parameters, and varying the Hamiltonian coupling. The authors find that a small intrinsic dimension can be misleading when the model cannot represent the correct wavefunction signs. They also report more stable optimization in random subspaces and an increase in measured dimension near the TFIM phase transition, although the measurement retains a floor imposed by the random parameterization.

Critical strengths
  1. In Section 4.1, the sign-free model reaches almost the same energy with 8 trainable directions as with all 28,642 parameters, despite being unable to represent the true ground-state sign structure. Both solutions remain constrained by the positive-cone bound. The comparisons with sign-capable models and the exact-sign control in Section 4.2 help explain why the apparent compression is misleading.
  2. Separating the sign and amplitude parameter budgets in Section 4.2 gives a more detailed account of the optimization difficulty than simply reporting one intrinsic-dimension value.
  3. The experimental design includes several useful controls, particularly the matched-solver comparison for optimization stability and the examination of training-budget dependence in the TFIM experiments. I also appreciate that the authors report cases where subspace training is more stable but achieves worse energies than successful full-parameter training.
Critical weaknesses
  1. Random-subspace training and intrinsic-dimension measurements have already been studied extensively [1], and the extension to VMC mainly involves applying stochastic reconfiguration to the latent parameters through the usual reparameterization. I understand that the paper is intended as an empirical investigation rather than a new algorithm. However, I am not convinced that the resulting observations offer a sufficiently substantial advance in our understanding of ML optimization or representation learning for ICLR.
  2. Although the paper reports several interesting measurements, it remains unclear how an NQS practitioner should use them to make better training decisions. Section 5.2 suggests restricting training to a random subspace when full-parameter optimization is unstable, but no procedure is developed for choosing an appropriate dimension or balancing stability against accuracy. In the 6 × 6 experiments, subspace training avoids the divergence events seen in full-parameter training, yet performs worse at the three couplings where the full-parameter reference avoids its bad basin. The sign-amplitude experiments raise a similar issue, since they demonstrate that proportional allocation can leave the sign network with too few trainable directions without establishing how these directions should instead be allocated. One could imagine developing an adaptive allocation rule from these findings, but that is not explored. As it stands, the paper provides diagnostics and observations rather than much operational guidance for training NQS.
  3. The increase in intrinsic dimension near the TFIM phase transition is interesting, but its interpretation becomes difficult once the effects of the random parameterization are considered. At (h/J=10), a parameter-free product state already gives a relatively accurate approximation, while the 32-dimensional random-subspace model remains substantially worse. This suggests that the measured dimension can reflect the difficulty of reaching a solution through a particular parameterization rather than the complexity of the target state. The authors also acknowledge that the location of the increase depends on the accuracy threshold, while the dimension itself remains bounded below by the random-subspace floor. Given these dependencies, I am not sure what information the measurement provides about physical complexity beyond the optimization behavior of the chosen model. More evidence would be needed before treating it as a generally useful probe.
  4. With 41 divergence events across 35 full-parameter runs on the larger frustrated system and none in the tested subspace runs, the stability difference is substantial under the reported settings. The matched-solver control helps rule out one possible explanation, but the full-parameter optimizer is not independently tuned to achieve comparable stability. Could a smaller learning rate or different regularization produce similar behavior without restricting the model's trainable directions?
  5. It is difficult to judge the generality of the results from experiments using a single autoregressive NQS architecture, particularly when some conclusions depend on how that architecture represents signs and amplitudes.
  6. Much of the strongest evidence concerns the properties of specific quantum systems, including frustrated sign structures and the TFIM phase transition, while the implications for broader ML problems remain mostly speculative. For example, Section 5.1 relates the observed imbalance between sign and amplitude parameter budgets to parameter-efficient learning methods such as LoRA [2]. However, the paper does not investigate low-rank adaptation or show that the same issue arises in more conventional neural network architectures. I think the physical analysis has value, but the connection to general representation learning and optimization needs to be established more directly. Without that connection, the work feels closer to a characterization of NQS behavior in particular physical settings than a broadly applicable ML contribution.

[1] https://openreview.net/forum?id=ryup8-WCW

[2] https://openreview.net/forum?id=nZeVKeeFYf9

Questions for authors
  1. Since the paper identifies both improved stability and an accuracy cost from subspace restriction, how should practitioners decide when to use it and what dimension to choose? Is there evidence that measuring intrinsic dimension can lead to better training decisions, rather than serving only as a diagnostic after training?
  2. Have the authors investigated whether the stability advantage persists when the full-parameter optimizer is tuned independently, for example by adjusting its learning rate or regularization?
  3. Given that the measured intrinsic dimension includes a floor associated with the random parameterization and depends on the accuracy threshold, how can its variation be distinguished from changes in optimization difficulty? This distinction seems particularly important for interpreting the TFIM phase-transition results.
6 comments
AuthorsOpenReview ↗

Response to Reviewer VsL4 (1/5)

W1 + W6. What is new, and what can machine-learning researchers take from the results?

Reviewer concern. Random-subspace training and intrinsic-dimension measurements have already been studied extensively [1], and the extension to VMC mainly involves applying stochastic reconfiguration to the latent parameters through the usual reparameterization. I understand that the paper is intended as an empirical investigation rather than a new algorithm. However, I am not convinced that the resulting observations offer a sufficiently substantial advance in our understanding of ML optimization or representation learning for ICLR.

Much of the strongest evidence concerns the properties of specific quantum systems, including frustrated sign structures and the TFIM phase transition, while the implications for broader ML problems remain mostly speculative. For example, Section 5.1 relates the observed imbalance between sign and amplitude parameter budgets to parameter-efficient learning methods such as LoRA [2]. However, the paper does not investigate low-rank adaptation or show that the same issue arises in more conventional neural network architectures. I think the physical analysis has value, but the connection to general representation learning and optimization needs to be established more directly. Without that connection, the work feels closer to a characterization of NQS behavior in particular physical settings than a broadly applicable ML contribution.

[1] https://openreview.net/forum?id=ryup8-WCW

[2] https://openreview.net/forum?id=nZeVKeeFYf9

Response.

We thank the reviewer for the thoughtful review, and for recognizing the positive-cone example, the separate sign and amplitude budgets and the matched-solver control.

A neural network trained by natural gradient to find a quantum ground state needs both a suitable representation and reliable optimization. In our experiments, restricting the trainable directions improves stability, yet a small intrinsic dimension can coexist with a large energy error set by the representation. The distinction matters when deciding whether few trainable directions suffice for the learning task, or only for matching the full model.

The intrinsic dimension of Li et al. (2018) compares training in a random subspace with the performance of the full model. It is the smallest subspace that reaches 90% of that performance. The submission scores every run against an exact ground-state energy instead (Introduction and Section 3.1), which the systems studied here provide.

Section 4.1 shows what the full-model criterion cannot see. A network with non-negative amplitudes reaches a relative energy error of 0.5313 with 8 of its 28,642 directions and 0.5271 with all of them. Relative to the full model, the subspace loses almost nothing with about 3,500 times fewer directions. Against the exact ground-state energy, both errors are about 53%, because both approach the lowest energy that any non-negative wavefunction can reach. A small dimension under the full-model criterion establishes agreement with the full model. It does not establish proximity to the ground state.

The submission has no experiment on LoRA or on conventional architectures. Section 5.1 mentions LoRA because LoRA also divides a budget of trainable dimensions among parameter blocks, and the section presents the dissection as “a measured instance of how this can fail”. We agree that “Low-rank adaptation with budgets set by shape gives too little to any small block whose demand exceeds its share” could be read as a tested statement about LoRA. We therefore replace it with the measured case. A budget set by block size gives the sign block 110 of 2,048 coordinates and a median error of 23.032%, against 3.484% with the exact sign at the same 1,938 amplitude coordinates, and 5.198% when the 2,048 coordinates are reallocated to give the sign block all 1,633 of its coordinates.

Two interventions added after submission show that the coordinate count alone does not determine the outcome. In the frustrated system, at the same total of 2,048 trainable coordinates, moving coordinates from the amplitude block to the sign block lowers the median independently evaluated terminal error from 23.032% to 5.198% and improves all six matched seeds. On the 16-spin Ising chain at field-to-coupling ratio 4, two maps that are paired at each tested coordinate count and differ in one output direction first meet a 2% tolerance at 64 and at 8 coordinates, and the effect of the direction on the energy is reproduced in a fully connected tanh network (response to W5). At a fixed count, the outcome depends on how the coordinates are allocated among blocks and on which directions they span.

AuthorsOpenReview ↗

Response to Reviewer VsL4 (2/5)

W2 + Q1. When should practitioners use subspace training, and how should they choose the dimension and allocation?

Reviewer concern. Although the paper reports several interesting measurements, it remains unclear how an NQS practitioner should use them to make better training decisions. Section 5.2 suggests restricting training to a random subspace when full-parameter optimization is unstable, but no procedure is developed for choosing an appropriate dimension or balancing stability against accuracy. In the 6 × 6 experiments, subspace training avoids the divergence events seen in full-parameter training, yet performs worse at the three couplings where the full-parameter reference avoids its bad basin. The sign-amplitude experiments raise a similar issue, since they demonstrate that proportional allocation can leave the sign network with too few trainable directions without establishing how these directions should instead be allocated. One could imagine developing an adaptive allocation rule from these findings, but that is not explored. As it stands, the paper provides diagnostics and observations rather than much operational guidance for training NQS.

Since the paper identifies both improved stability and an accuracy cost from subspace restriction, how should practitioners decide when to use it and what dimension to choose? Is there evidence that measuring intrinsic dimension can lead to better training decisions, rather than serving only as a diagnostic after training?

Response.

The submission's budget protocol showed that proportional allocation leaves the sign block too few dimensions (Section 4.2). We tested the resulting choice in a comparison registered before it was run, on six new matched seeds of the 4 × 4 frustrated model at coupling ratio 0.5, with the same total of 2,048 coordinates and the same 1,500-update budget in both arms. The proportional allocation (110 sign and 1,938 amplitude coordinates) ends at a median independently evaluated error of 23.032%. The sign-focused allocation (1,633 sign and 415 amplitude coordinates) ends at 5.198% and is lower in all six pairs. Acting on the measurement lowers the error 4.4 times at a fixed budget.

On when to use subspace training, Section 4.4 reports both sides of the 6 × 6 comparison at equal steps. The 3,072-coordinate subspace beats the full-parameter median by 1.6–4.6× at the four couplings where that median ends in the bad basin, and loses at the three the reviewer cites. It is therefore worth using where full-parameter training is fragile (Section 5.2).

On the dimension, the quantity the submission measures is the selection rule, the smallest tested dimension whose median error meets the requested tolerance at a fixed budget (Section 3.5). A direction matched to known physical structure lowers it. On the 16-spin Ising chain at field-to-coupling ratio 4, in a paired exact-energy experiment, replacing one random output direction by the direction that gives access to the transverse product state lowers the first dimension meeting a 2% tolerance from 64 to 8.

AuthorsOpenReview ↗

Response to Reviewer VsL4 (3/5)

W3 + Q3. How much of the measured dimension reflects physical complexity, parameterization, or optimization difficulty?

Reviewer concern. The increase in intrinsic dimension near the TFIM phase transition is interesting, but its interpretation becomes difficult once the effects of the random parameterization are considered. At (h/J=10), a parameter-free product state already gives a relatively accurate approximation, while the 32-dimensional random-subspace model remains substantially worse. This suggests that the measured dimension can reflect the difficulty of reaching a solution through a particular parameterization rather than the complexity of the target state. The authors also acknowledge that the location of the increase depends on the accuracy threshold, while the dimension itself remains bounded below by the random-subspace floor. Given these dependencies, I am not sure what information the measurement provides about physical complexity beyond the optimization behavior of the chosen model. More evidence would be needed before treating it as a generally useful probe.

Given that the measured intrinsic dimension includes a floor associated with the random parameterization and depends on the accuracy threshold, how can its variation be distinguished from changes in optimization difficulty? This distinction seems particularly important for interpreting the TFIM phase-transition results.

Response.

The reviewer's reading is the one the submission draws in Section 4.3, “the floor belongs to the random slice rather than to the state”. The controls added after submission vary the target state, the parameterization and the optimizer in controlled comparisons. In the 16-spin chain, the product state has 0.2495% energy error at field-to-coupling ratio 10, so it already satisfies a 1% target. At ratio 4 its error is 1.5445%, so a 1% target requires correlations beyond a product state. The same accuracy requirement therefore resolves different physical structure in these two regimes.

Within the neural model at ratio 4, a paired exact-energy experiment fixes the Hamiltonian, target, architecture, initialization, other directions, scales and optimizer and changes one output direction. The first passing dimension falls from 64 to 8 at a 2% tolerance, and the aligned map first passes 1% at 16 coordinates. This identifies a contribution from how the parameterization gives access to useful states. At ratio 1.2, near the transition, the same replacement leaves the first passing dimension unchanged. It is 64 with either map at 2%, and there is no passing point on the tested 8–64 grid at 1%. With the aligned direction supplied, the first passing dimension at 2% is therefore 64 near the transition against 8 at ratio 4. The direction that lowers the strong-field floor does not account for the dimension near the transition.

The submission does not present the dimension as a property of the state alone. Section 5.2 calls it “a cheap probe with a floor” whose “value includes what the random slice itself costs”. Its variation is not specific to an optimizer. On the aligned maps at ratio 4, exact-energy L-BFGS-B and 6,000 sampled-SR updates give the same first passing dimension, 16 at 1% and 8 at 2%, and at ratio 0.95 SR and Adam both first pass 2% at 64 coordinates.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.