- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies feature bagging in adaptive search in two parts. In the first part, the authors study when feature bagging helps. They first establish a posterior signal-to-noise ratio (SNR) boundary for grouped nonlinear search under an isotropic Gaussian signal model, complemented by a fixed-signal bound under general feature covariance and a budget ordering under orthogonal weak signals. They then extend the SNR identity to recursive adaptive search under a common-projection condition. In the second part, the authors propose an algorithm for tuning mtry that couples full-depth trees across budgets to share split searches and selectively refines promising candidates. The numerical study covers 12 wide-feature regression tasks, each evaluated on five outer folds.
Critical strengths
The paper contains publishable results, including an exact posterior SNR identity for grouped adaptive search with arbitrary candidate correlations and its extension to recursive adaptive search. The proposed algorithm combines shared split search with successive halving and a corrected OOB target, resulting in a new selector that achieves accuracy comparable to successive halving with reduced measured wall time.
Critical weaknesses
(1) Limited novelty and practical scope of the theory. The theory in Section 2 essentially establishes that feature bagging helps at low SNR, which has been argued in prior work (Mentch & Zhou, 2020, as the authors cite). The authors derive the same conclusion under different model settings. The statement that may address strong signals, the boundary in Eq. 7, depends on D and V, which cannot be calculated in advance, so it provides little guidance for the moderate- and high-SNR settings where choosing mtry matters most. In Theorem 3.1, the authors extend the theory to recursive adaptive search, but part (i) conditions on shared non-feature randomness and part (ii) further requires a common projection geometry, which fails across bootstrap samples; the theorem therefore does not cover standard random forests with per-tree bootstrap. In addition, as the authors note, its low-SNR conclusion requires D_T ≥ 0, which can fail; this condition also cannot be verified in advance. More importantly, the theory developed in Sections 2 and 3 has only a weak connection with the mtry selector developed in Section 4 and evaluated in Section 5.
(2) Limited methodological novelty and a confounded speedup. The corrected OOB score is a per-row variant of the ECV finite-ensemble correction (Du et al., 2024). The corrected halving baseline, which combines the corrected score with successive halving, performs worse than plain SH (1.56% vs. 1.39%). Moreover, the proposed geometric refinement does not significantly outperform SH (the 95% CI of the paired difference contains 0). Although the proposed algorithm has the lowest wall time among the tuning methods (about half that of SH, the only directly measured comparison), it is unclear how much of the speedup comes from the lighter tree schedule (4–8–16 for the proposed method vs. 8–16–32 for SH). The reported speedup may therefore be substantially attributable to the schedule rather than to shared search.
Questions for authors
(1) Beyond providing exact formulas, what additional conceptual contribution does Section 2 make relative to prior low-SNR explanations of feature randomization (e.g., Mentch & Zhou, 2020)? Does it also provide practical guidance for moderate- and high-SNR settings, where the choice of mtry is more important?
(2) Beyond analytical convenience, what computational capabilities does Eq. 3 provide? At a single node, the winning subset can be identified by a lookup among the p featurewise winners, and the subset probabilities could likewise be estimated by sampling.
(3) How does Theorem 3.1 apply to standard random forests based on per-tree bootstrap samples, given that part (i) conditions on shared non-feature randomness and part (ii) requires a common projection geometry?
(4) Is there empirical evidence that the posterior SNR boundary in Eq. 15 predicts out-of-sample test performance? How should practitioners address the condition D_T ≥ 0, which may fail and cannot be verified in advance?
(5) The grouped-search structure is the standard CART split search, and the theory compares a fixed budget with a full search, whereas the selector compares budgets with one another. What does the theory contribute specifically to the selection problem?
(6) How much of the speedup over SH is attributable to shared search, and how much to the lighter 4–8–16 schedule? Can you report an ablation under a common schedule, with and without sharing?
(7) How sensitive are the results to the individual design choices: the schedule, retaining half, the half-corrected score, the two refinement triggers, and the 50-tree comparison? In the manuscript, only the midpoint has been ablated.
(8) Could Table 1 include confidence intervals so that readers can see directly that most accuracy differences are not statistically significant?