- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper focuses on online fitting of sparse additive models. The paper proposes to integrates the smooth backfitting objective function with group LASSO and SCAD regularizations and develops online estimates and simultaneous confidence bands together with theoretical guarantees. The practical performance of the proposed method is compared with existing methods by simulation and two real data applications.
Critical strengths
The paper studies online sparse additive models, which is an extension of offline sparse additive models to the online case as well as an extension of online additive models to handle high-dimensional covariates. It potentially enriches the theoretically guaranteed methodological toolbox in nonparametric statistics and machine learning.
Critical weaknesses
-
The proposed method is not well justified. As illustrated in Figure 1, the proposed method involves multiple steps, but it is not clearly explained what purpose each step is needed for (e.g., smooth backfitting, group LASSO, SCAD) or whether and why each step is the best choice compared to other options. For example, it is not justified why the discretized smooth backfitting criterion is a better choice than approximating each component function by global basis functions, e.g., B-spline basis functions. Although the smooth backfitting criterion is adopted, the estimation procedure does not involve iterative updates like the original smooth backfitting algorithm and the oracle asymptotic distribution of each fitted component function is not developed, which is the key theoretical feature/advantage of smooth backfitting. Definitions in Lines 174-176 are also unclear.
-
The paper is somewhat weak in both theoretical innovations and practical utility. First, as a “learning theory” paper, it fails to demonstrate substantial theoretical innovations, e.g., what key theoretical difficulties this paper can address while existing theoretical tools cannot. Comments are missing about the connections between the developed theoretical results and literature. Second, the practical utility of the proposed method is not convincing. Although this paper is categorized into the “learning theory” Primary Area by the author(s), it is more suitable to treat it as a methodology paper, but its practical utility, which is an important assessment metric for such papers, is very weak. For the two real data applications, necessary details are not provided in the main text, existing methods are not compared, e.g., Haris et al. (2022) not cited by this paper and those cited in the simulation study, and new practical insights revealed by the proposed method are not given. The Bikeshare data application is also problematic since null covariates are artificially added.
Reference:
Haris, A., Simon, N., & Shojaie, A. (2022). Generalized sparse additive models. Journal of Machine Learning Research, 23(70), 1-56.
Questions for authors
- Why is the discretized smooth backfitting criterion used instead of its original version? Why not approximating each component function by global basis functions, e.g., B-spline basis functions?
- What are Gaussian-copula additive models in Theorem 2?
- Can Gaussian autoregressive errors be relaxed to sub-Gaussian or sub-exponential?
- Line 178: Taking the derivative of the SCAD penalty as a weight seems an unusual way of using SCAD. What is the justification of it?
- What is the connection between defined in Line 134 and the estimated component function obtained by Eq. (1)?
- The main text should be self-contained. For example, Section 4 discusses many results not given in the main text, e.g., F-SGD and Sieve-SGD are not included in Table 1 in the main text.
- Centering component functions is a standard practice to address identifiability in additive modeling. It is overstated to emphasize it as an obstacle.
- The logical flow in this paper is unclear. Many consecutive sentences lack logical connections between them.