- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper represents score functions as superpositions of electrostatic fields generated by spherical and slab components. This construction provides conservative fields with analytic divergences. For fixed geometries and a fixed base field, coefficient estimation reduces to a regularized quadratic problem. Experiments cover synthetic score estimation and latent score estimation in Wasserstein autoencoders.
Critical strengths
A tractable construction of conservative score fields. The proposed components have analytic fields and divergences, and coefficient estimation reduces to a regularized quadratic problem when the geometries and base field are fixed. This makes the score-matching objective directly computable while preserving the gradient-field structure, providing a useful alternative to unconstrained vector-field models.
Promising results with compact mixed dictionaries. On the synthetic GMM benchmark, the mixed model achieves a score MSE of with approximately 21 selected components, outperforming the reported spherical-only and slab-only variants. This provides concrete evidence of the approach’s practical promise for structured score-estimation problems.
Critical weaknesses
The source of the empirical improvements is insufficiently isolated. Although the appendix includes useful ablations, some comparisons differ in base-field fitting, regularization, dictionary size, or validation criteria. Given the close connections to kernel exponential families and tied-weight ReLU models, stronger controlled comparisons are needed to establish the benefit of the proposed mixed construction beyond these implementation choices. This uncertainty limits the evidence for its distinct empirical contribution.
The efficiency evaluation does not capture the complete training pipeline. The reported timings exclude dictionary construction and selection, which can involve clustering, ICA, adaptive search, and repeated fitting. A fast coefficient solve for a fixed dictionary therefore does not establish an end-to-end computational advantage. Since computational tractability is a central motivation of the paper, complete timing measurements under comparable tuning budgets are important to substantiate its practical benefits.
Questions for authors
Comparability of the experimental settings. Are base-field fitting and validation criteria matched between the proposed method and the closest KEF and tied-weight ReLU baselines? Which existing controlled comparisons best isolate the benefit of the mixed dictionary from these other choices?
End-to-end computational cost. For one representative experiment, could you provide a runtime breakdown covering dictionary construction, adaptive selection, and coefficient/base-field fitting? Does the reported speed advantage persist when these stages are included and the baselines are timed under the same accounting rules?
Scope of the density interpretation. Are any constraints imposed to ensure that the fitted potential defines a normalizable density? If not, is the method intended primarily as a score estimator, and do any of the reported experiments require the fitted field to correspond to a normalizable density?
Thank you for identifying the comparisons needed to clarify the contribution. We have revised the manuscript with matched geometry and base-field comparisons (Appendix G.1), full search-cost measurements (Appendices G.4 and G.7), and a clarification of normalizability (Appendix A.4). These additions are highlighted in green in the revised manuscript.
1. Comparability of the experimental settings.
“Are base-field fitting and validation criteria matched between the proposed method and the closest KEF and tied-weight ReLU baselines?”
Yes. We added frozen-CIFAR comparisons using the same representation, splits and validation criterion, with 1,024 and 2,048 total components: K spheres for Gaussian KEF, K slabs for ReLU models, and K/2 of each for mixed. Each is tested without a base, with a fixed Gaussian base, and with joint symmetric base refitting. Fixed geometries share the applicable regularization grids; learned tied/untied ReLUs share their training-setting search.
Mixed has the lowest mean test J in every base setting at both budgets. With 2,048 components and joint refitting, it obtains −1628.83 ± 6.26, versus −1495.32 ± 0.65 for Gaussian KEF, −1538.09 ± 6.20 for fixed slabs, and −1544.25 ± 6.03 for learned untied ReLU; lower is better.
Thus, matching component budgets and base treatment preserves the advantage (Appendix G.1). Under the same SM objective, neural SM obtains −1531.85 ± 3.05, also outperformed by the mixed geometry model (Appendix G.2).
2. End-to-end computational cost.
“For one representative experiment, could you provide a runtime breakdown covering dictionary construction, adaptive selection, and coefficient/base-field fitting? Does the reported speed advantage persist when these stages are included and the baselines are timed under the same accounting rules?”
Yes. Full red/white wine searches take 3.28/3.57 seconds, including geometry construction, cross-validation, and final refitting, versus 199.59/269.75 for SSM. Appendix G.7 separates geometry proposals, moments, coefficient/base solves, validation and overhead. Both exclude preprocessing and checkpointing required from GPU use. SSM has lower mean wine loss; mixed provides a substantially faster fit with competitive performance results.
On the d=30 Gaussian mixture, mixed is both faster and more accurate: MSE 4.85 in 4.8 seconds, versus 12.67 in 34.5 seconds for neural SSM, including geometry construction and search for mixed (Appendix G.4). Wine selection compares fixed dictionaries; Appendix G.8 separately measures greedy additions and refitting beyond validation stopping, conditional on selected settings.
3. Scope of the density interpretation.
“Are any constraints imposed to ensure that the fitted potential defines a normalizable density? If not, is the method intended primarily as a score estimator, and do any of the reported experiments require the fitted field to correspond to a normalizable density?”
Our reported fits are score estimators; we do not impose a global normalizability constraint, and our experiments do not require a normalizing constant. Conservativeness means that the score can be written as −∇V. To additionally define a probability density over all space, we need ∫e−V(x)dx<∞. A conservative field alone does not guarantee this.
Appendix A.4 explains a sufficient condition. Intuitively, the linear base must pull inward strongly enough that positive charge extending to infinity does not overcome it. Gaussian spherical potentials are bounded, and finite-thickness slab potentials grow at most linearly, so a confining quadratic base dominates these contributions far from the data. Our fitted one-sided slab boundaries can instead contribute quadratic growth. A sufficient condition is
\qquad \varepsilon > 0,$$ where $\Omega$ is the symmetric linear base, $\theta_j$ is a slab coefficient, and $u_j$ its unit direction. This ensures $V(x) \geq \varepsilon \lVert x \rVert^2/2 - C\lVert x \rVert - C$ for some finite $C$, making the integral finite. Negative slab coefficients help confinement. Alternatively, a continuous potential restricted to a bounded box defines a normalizable density on that box. Neither restriction is imposed in the reported experiments. **Score MSE and J evaluate the score field directly and do not require a normalizing constant.** WAE generation and AIS use the model defined by the Gaussian latent prior and decoder, rather than normalizing the fitted score potential. The finite-step diffusion experiment likewise demonstrates sampling without claiming that its estimated fields define a normalizable static density.