When Feature Bagging Helps Adaptive Search: SNR Theory and Prospective mtry Selection

Feature bagging averages predictors fitted after restricting an adaptive search to random feature subsets. We study when this improves prediction and use the same search structure to tune mtry, the number of features searched at each forest split. Under an isotropic Gaussian signal model, an exact restriction–diversity identity gives a posterior signal-to-noise ratio (SNR) boundary for grouped nonlinear search with arbitrary candidate correlations. A complementary fixed-signal bound permits general feature covariance; under orthogonal weak signals, smaller feature fractions have lower asymptotic risk. A common-projection result extends the SNR identity to completed recursive fits. For tuning, we couple full-depth trees across budgets to share split searches and selectively refine promising candidates. On a development benchmark of 12 wide-feature regression tasks and five outer folds, with 39–85 candidate budgets, the selector attains 1.336% mean regret relative to the best 256-tree test-grid candidate, compared with 1.385% for successive halving. Its measured tuning and total wall-time ratios to halving are 0.3710 and 0.5123, respectively; total time includes the selected 256-tree forest.