- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper focuses on modern learning systems across noise and local persistence scenarios. When a learning system repeatedly observes samples that violate established patterns, how should one determine which specific type of situation these deviations represent? This work observed that the distinction among noise, local persistence, and reusable structure becomes sharper. To that end, the authors proposed a statistical model based on binary residuals and latent variables, constructed it with symmetric noise, and also observed the same distinction in three views (simulation, CLINC150, and Omniglot).
Critical strengths
-
The focused question is valuable, and a clear statistical framework has been established.
-
The two theories yield clear and interpretable conclusions, and the appendix provides a relatively complete mathematical proof.
-
The experimental results are not only in one dataset, but separate two goals.
Critical weaknesses
-
The theoretical novelty of the proposed allocation laws relative to classical hypothesis testing and optimal experimental design. For example, the sparse-replication strategy in Theorem 1 is insightful, but the authors do not sufficiently articulate the fundamental differences between this conclusion and existing theories regarding grouped observations, optimal experimental design, or sequential hypothesis testing. (Can the authors also discuss Theorem 2?)
-
In my view, the main contribution of this paper is that repetition establishes persistence, while supporting-type diversity enables generalization. I think the novelty is not sufficient.
-
The experiment should be more reliable. Residual labels are manually constructed; N, E, and R may be found in one scenario in the real world; the authors need more baselines to compare.
-
Several equations are not numbered, which makes them difficult to reference. The authors are encouraged to number these equations for better readability.
Questions for authors
See the weaknesses.