- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors address the problem of automatically reporting the recovery of individual latent spikes in a multi-spiked tensor PCA model, as they are achieved along the run of an SGD algorithm. This is done by exploiting statistics computed from a sequence of tensor observations containing the same signal but with distinct and independent noise realizations. The first statistic is a contraction of the observations with the current iterates, while the second one relies on the spectrum of the so-called traverse Riemannian Hessian.
While the paper seems novel and the subject is definitely interesting for researchers working in high-dimensional inference, in particular involving tensor models, it needs in my view a substantial revision combined with a great pedagogical effort in order to improve its readability.
Critical strengths
-
The subject is timely.
-
The results seem to have important implications for practitioners who rely on tensor spectral methods.
Critical weaknesses
- In its current form, this Reviewer finds that the paper is unfortunately extremely hard to read, which obscures the main messages and makes it difficult to assess its soundness and its contributions. This difficulty is due to several reasons:
- Many expressions and sentences have an unclear (or yet undefined) meaning in their surrounding context. Examples are: "overlap-defined column signal", "windowed threshold" and "latent greedy recovery order" (Abstract); "initialization with separated greedy weights", "transverse column-j Hessian", "transverse Hessian" and "stage certification" (Introduction); "Haar-probability claim", "Hessian compression", "quadratic-form compression", "lower-bulk theorem", "null-replicate eigensolver errors", "full-law noise simulator" (Section 2). And the list goes on.
- Many math symbols are used before they are defined, such as in line 83, in line 109, in line 134, and in line 161, in line 253, in line 269. Furthermore, is reused ( and seem to mean quite different things), as well as ( and ).
- The presentation is highly technical and far from pedagogical, which is in my view an issue for a conference aimed at a quite broad audience. A plethora of different symbols and definitions is given throughout the paper, but often without an accessible description of the overall/high-level reasoning being followed. It is thus quite difficult to keep track of its contents and main messages.
- The meaning of certain concepts is not sufficiently explained. For instance: What exactly are the certified regions of lines 160-161? What exactly are the quantities plotted in Figure 1 (are they those defined in (35))?
-
Model (1) assumes that a sequence of tensor-valued observations is given as an input to the algorithm. However, it is not clear that this model is relevant for applications, in particular the ones given in the Introduction as motivation. In the case of moment tensors, one could in principle partition the observations in order to compute a sequence of independent estimators of the target tensor, but this would increase the noise variance. Furthermore, the condition in (12) seems to imply that a large number of observed tensors would be required. This issue is not discussed in the paper.
-
The discussion on related work and on relevant properties of the tensor PCA model is insufficient. In particular, the paper does not mention the fact that, in the regime , there is an algorithmic threshold that each spike weight must attain for its associated spike to be recoverable by the considered algorithm. Hence, the assertion that individual overlaps will grow, as discussed in the Introduction, hinges upon all spikes being in this (weakly) recoverable regime, but this assumption is not discussed.
Questions for authors
- Shouldn't the given statements include a condition that spike strengths (the of your model) are beyond the algorithm threshold (which scales as for randomly initialized local search algorithms, or as if, say, a spectral initialization is used)?