- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies evaluating models via the covariance discrepancy , motivated by the observation that likelihood, marginal-fit, and perceptual criteria can miss covariance changes that directly affect regression and PCA. Three theoretical results characterize what can and cannot establish — marginals do not determine covariance, matched marginal variances make exactly determine the difference in bivariate population regression slopes, and covariance perturbation bounds PCA eigenvalue and principal-subspace errors, yielding a subspace certificate informative when . Synthetic data show that matched marginals can conceal slope reversals and tail-dependence differences, and is positioned as a task-linked complement to FID, MMD, copula diagnostics, and direct downstream evaluation.
Critical strengths
- The paper is relatively well structured and written, especially in the first half.
- The authors make due effort to scope the work.
Critical weaknesses
-
There is a large gap between the linear theory and practical generative models.
- Rezaei et al. (2026), the paper's main reference, concerns linear models, for which covariance-based analysis is natural and effective. A neural classifier often has a linear head, but that analysis's relevance to neural classifiers is a theoretically motivated heuristic supported by experiments. The theoretical gap remains substantial even with a linear final layer.
- All of the above applies to the current submission, and even more so. Although the Limitations section acknowledges that "Covariance does not characterize nonlinear, conditional, higher-order, or tail dependence" and that the work is "for linear and spectral tasks," it would not be unreasonable to state bluntly that the current submission concerns only linear models, in both theory and experiments.
-
The experiments do not close the gap described above.
- First, the experiments all appear to be purely synthetic (as suggested by the section title "SYNTHETIC EXAMPLES"); or are they?
- Are neural networks used in any of them? Only the VAE-on-MNIST experiment seems to use neural networks — tho it may be drastically simplified.
- The datasets appearing under EMPIRICAL SUMMARY are never explained, let alone the results. That subsection contains only a table, which is not appropriate for a main-text subsection.
-
The theory itself consists of elementary examples or straightforward corollaries of classical (textbook) results.
-
Theorem 1 is an elementary consequence of a stronger classical result. The underlying classical fact is that univariate marginals do not determine the joint distribution. Sklar’s theorem expresses the remaining freedom through the copula; with continuous marginals, that copula is unique. See Fermanian and Scaillet, §2, recalling Sklar’s theorem.
Embrechts, P., McNeil, A. J., & Straumann, D. (2002). Correlation and dependence in risk management: Properties and pitfalls. Cambridge University Press. Theorem 4 in the author version addresses the possible correlations with fixed marginal distributions.
Nelsen, R. B. (2006). An introduction to copulas (2nd ed.). Springer. https://doi.org/10.1007/0-387-28678-0. Relevant result: Sklar’s theorem, Theorem 2.3.3, §2.3.
-
Theorem 2 is an immediate algebraic identity. The standard population linear-projection formula is It requires finite second moments and positive predictor variance; it does not require Gaussianity, a linear conditional mean, or homoscedasticity. See Yen-Chi Chen’s regression notes, “Linear prediction”. And:
Hansen, B. E. (2022). Econometrics. Princeton University Press. (This is an undergrad textbook.) §2.21, Theorem 2.10, p. 42, Eq. (2.40), gives the population regression coefficient with an intercept:
-
Theorem 3 is also a standard consequence of known results: Eq. (18) is Weyl’s eigenvalue perturbation bound, and Eq. (20) follows from the Davis–Kahan theorem.
Yu, Y., Wang, T., & Samworth, R. J. (2015). A useful variant of the Davis–Kahan theorem for statisticians. Biometrika, 102(2), 315–323. https://doi.org/10.1093/biomet/asv008. Relevant material: §1, Theorem 1 and the following operator-norm remark; §2, Theorem 2, develops bounds using only a population eigengap. This is particularly close to your formulation.
Fan, J., Wang, K., Zhong, Y., & Zhu, Z. (2021). Robust high-dimensional factor models with applications to statistical machine learning. Statistical Science, 36(2), 303–327. https://doi.org/10.1214/20-STS785. §2.4 of the author manuscript states Weyl’s eigenvalue inequality and then Corollary 2.1, which gives the operator-norm sine-of-principal-angles bound with factor 2 and an eigengap from the unperturbed matrix only.
-
Questions for authors
See the questions about the experiments raised in the Critical Weaknesses section above.