- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper considers an analysis of approximate group equivariance by reformulating it as linear subspace constrained least-squares problem. Under a simple Gaussian model, they exhibit a natural bias-variance trade-off. The model is first isotropic and then made more general, and some synthetic experiments are proposed for illustration.
Critical strengths
I like the idea of reformulating the approximate equivariance problem as a constrained problem (but this idea is not really new, see https://proceedings.mlr.press/v139/finzi21a/finzi21a.pdf).
Critical weaknesses
- Once the paper reformulates the least-squares problem into a constrained one, this is simply reusing simple traditional least-squares analysis for random design and Gaussian data. This is very classical and should be acknowledged by the authors with references, e.g., the book An Introduction to Multivariate Statistical Analysis by T. W. Anderson (1958). The particularities of equivariances are then ignored. Given that the reformulation is not new, this leads to a strong lack of novelty.
- In terms of significance, the OLS type analysis has been superseded by more complex ones, in particular ridge regression, without the need for Gaussian data. Moreover, linear regression is too simplistic here.
- In (7), the selected set of estimators is natural, but it does not seem to be the OLS estimator for the constrained class (unless X'X =I). This makes the math easier, but deviates from what people would do in practice.
Questions for authors
- Theorem 2 seems to imply that this is a minimax result for all measurable estimator, as defined by M_n(rho), but it is unclear if this is in fact for the one-parameter family. Can you clarify?
- Section 4 goes beyond isotropic, which is good, but isn't knowing Sigma too much to ask. What would you do in practice?
- Since you are using least-squares, it seems an approach based on data augmentation would be do able and easy to implement, in particular since only moments are needed. Have you considered it?