- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes FetNAS, a weight-sharing NAS method that jointly searches over the number of input features, network depth, and layer width. It adds an elastic-features stage to progressive shrinking and uses a gradient-based feature ranking to construct nested feature subsets. After supernet training, multi-objective evolutionary search finds subnetworks that balance predictive performance, parameter count, and feature count using inherited weights without retraining. Experiments on TabArena show improved performance–complexity trade-offs for classification, while gains for regression are smaller and depend on the feature budget.
Critical strengths
The paper studies a relevant problem by considering feature selection and neural architecture search together for resource-constrained applications. The evaluation covers 51 TabArena datasets and shows some benefits for classification under limited feature and parameter budgets.
Critical weaknesses
- FetNAS adds an elastic-features stage to progressive shrinking and combines feature importance ranking with multi-objective evolutionary search. However, the novelty beyond this combination seems somewhat limited.
- FetNAS uses the same fixed feature ranking for all subnetworks and searches only over the number of features. This may limit its ability to select suitable feature subsets for different architectures.
- The work does not report comparisons of actual training time, search time, or computational resource usage, making it unclear how much computational cost FetNAS saves in practice.
- The baselines mainly combine UF or mRMR with Huang’s rule of thumb or evolutionary NAS. Comparisons with embedded feature selection methods and existing methods for joint feature selection and architecture search are missing.
Questions for authors
My questions are reflected in the concerns raised in the Weaknesses part.
3 comments
Thank you for the detailed response. I still have concerns about the methodological novelty. Therefore, I keep my original rating.
We thank the reviewer for taking their time to go to our response.
Thank you for your response. However, we still believe that our proposed approach to jointly optimize feature selection and neural architecture should be considered a contribution to the literature. To the best of our knowledge, no existing NAS method treats feature selection as a searchable dimension in a weight-sharing supernet within the NAS loop. Consequently, FetNAS can sample architectures for arbitrary feature and parameter budgets from a single supernet without requiring retraining or a separate supernet for each feature budget.
In addition, we propose an elastic-features stage that enables subnetworks with different input feature sets to share weights. Unlike depth or width, removing a feature removes information, so sharing weights across subnetworks with different inputs is not a straightforward extension. We also define and study several feature-ranking methods and compare them with each other and with random feature sampling. Finally, through our evaluation on the TabArena benchmark, we show that our method is valuable and competitive with sequential feature-selection-and-NAS pipelines, embedded feature-selection methods and handcrafted architectures.
We will make sure that this is clarified in the revised version of the paper.
We thank the reviewer for the valuable comments. Please find our response to your questions below.
77Bx-Q1
Please see "Contribution and novelty" in the general response.
77Bx-Q2
Please see "Fixed ranking vs. arbitrary feature subsets" in the general response. We are including a non-ranking variant which samples over arbitrary feature subset per subnetwork. Random subsets are competitive, though ranking methods still yield better results and serve as a good heuristic (difference of +0.011 [+0.004, +0.017] F1/R2 in favour of the ranking at matched budgets).
77Bx-Q3
Please see "Computational cost" in the general response: FetNAS covers the five feature budgets of the paper with 28% of the GPU time of the sequential pipelines.
77Bx-Q4
For embedded feature selection we added LassoNet, see "Embedded feature-selection baseline" in the general response.
For the joint methods, we agree that prior work exists, e.g. [1], which co-evolves neural architectures and input features. However, we want to point out that this approach trains every candidate model from scratch during the search. In FetNAS, the supernet is trained only once and every candidate inherits its weights, so it is evaluated without retraining. A direct comparison under our protocol is therefore not possible.
[1] Hafiz, Faizal, Jan Broekaert, Davide La Torre, and Akshya Swain. “Co-Evolution of Neural Architectures and Features for Stock Market Forecasting: A Multi-Objective Decision Perspective.” Decision Support Systems 174 (November 2023): 114015.