- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
INFER casts AutoFE as a hierarchical RL problem that is trained once across datasets and applied with frozen weights to new ones. An outer PPO agent picks k = 4 columns from 129-dim dataset meta-features ϕ(D) (Sec. 3.3). An inner DRQN agent builds a feature with a stack machine over unary/binary/GroupBy operators (Sec. 3.2), using shaped rewards (+5/+10/+1/−1) and an RRE-in-AUC outer reward (Eq. 1). At deployment (Alg. 1), N = 50 runs × m = 5 features produce a pool of 250 candidate programs. Selection then happens with OOF-based Top-K ensembling, LR-stacking, NNLS, or a "Combined" meta-selection over the three. All of it is strictly nested within outer training folds, which leakage-invariance tests confirm (Sec. 4.2). On 75 OpenML binary tasks with a dataset-level 4-fold transfer split, INFER (Combined) improves over original features on 81% (XGB) / 83% (RF) of datasets. The mean RRE is +5.9%/+6.0%, and INFER beats DIFER and OpenFE under XGB while tying with OpenFE under RF (Table 1, Fig. 2).
Critical strengths
- [Importance of the research problem] The paper targets a real cost: nearly every AutoFE method reruns a full search on each new dataset, so nothing is amortized and users cannot bound compute (Sec. 1).
- [Importance of the research problem] Treating the number of generation runs N as an explicit budget knob answers a stated usability need.
- [Novelty of the approach] The inductive setting is new for deep-RL AutoFE. Frozen agents trained on source datasets are applied to unseen datasets under a dataset-level 4-fold split (Sec. 4.2).
- [Novelty of the approach] Splitting the task into column selection (outer PPO agent over 129-dim meta-features) and operator composition (inner DRQN stack machine) is clean (Secs. 3.2–3.3).
- [Soundness and feasibility] The pipeline is leakage-free by construction. Operators are fitted on training rows only, candidates are stored as replayable programs, and the leakage-invariance tests pass (Secs. 3.5 and 4.2).
- [Soundness and feasibility] Selection is fully training-side, including the Combined meta-selection.
- [Experiments and practical impact] The protocol is rigorous: 75 datasets, 5×5 nested CV, Holm-corrected Wilcoxon tests, bootstrap CIs and medians.
- [Experiments and practical impact] INFER improves over the original features on 61/75 (XGBoost) and 62/75 (RandomForest) datasets, with mean RRE +5.9% and +6.0% (Table 1).
- [Experiments and practical impact] Reporting is honest: the outlier dataset is disclosed (+4.8% and +5.1% without it), and parity with OpenFE under RandomForest (35/1/39, p = 0.73) is acknowledged.
- [Theoretical and application-level implications] The results suggest that feature-construction skill can partly transfer across tabular datasets through meta-features alone, without textual semantics.
- [Theoretical and application-level implications] The leakage-tested nested protocol is a reusable methodological contribution for AutoFE evaluation.
- [Theoretical and application-level implications] The design suits settings with many similar tabular problems: the one-time cost (about 1.3 GPU-hours plus about 614 core-hours) breaks even after about 27 datasets against DIFER.
Critical weaknesses
- [Importance of the research problem] The scope is restricted to binary classification on datasets with 4–20 columns, which limits how far the problem is actually addressed.
- [Novelty of the approach] Hierarchical and multi-agent RL feature generation and meta-learned feature generation already exist (MARLFS; Gao et al., 2025; ExploreKit; Reyes-Amezcua et al., 2022). The individual components are therefore incremental.
- [Soundness and feasibility] Top-K averages up to K = 10 classifiers, and LR-Stacking and NNLS combine many classifiers, while the baselines are single models. Gains may come from ensembling rather than better features.
- [Soundness and feasibility] There is no random-policy ablation (random subsets and random valid programs through the same pipeline), so the contribution of the RL agents is not isolated.
- [Soundness and feasibility] The inner reward is hand-shaped (+5/+10/+1/−1, Sec. 3.1), and its sensitivity is not analyzed.
- [Experiments and practical impact] Absolute AUC gains are small (+0.011 and +0.014). Gains are weak on hard datasets (53% win rate and −0.6% RRE under XGBoost, Fig. 4).
- [Experiments and practical impact] Per-dataset cost is about 4.0 core-hours at N = 50, versus 0.22 hours for OpenFE (Table 2). There is no accuracy-vs-N curve.
- [Theoretical and application-level implications] There is no theory of when transfer should work, e.g., as a function of the shift between source and target meta-features.
- [Theoretical and application-level implications] The ensemble outputs add inference latency and reduce interpretability. The method needs labeled data, and multiclass and regression tasks are unsupported.
Questions for authors
- Can you add ensemble controls for the ensembling concern above, especially Top-K/LR-stacking/NNLS over 50 seeded or bootstrapped models on the original features, and the same selection over OpenFE candidates?
- Can you report a random-policy baseline (random subsets + random valid programs) through the identical generation-then-selection pipeline?
- What is AUC as a function of N ∈ {1, 5, 10, 25, 50}, and where does INFER cross OpenFE in accuracy per core-hour?
- How many generated features end up in the final model on average, and are they interpretable? A few examples would help.
- Does a single pair of agents trained on all 75 datasets transfer to an external benchmark, e.g., datasets from the OpenFE or CAAFE papers?