acceptodds
Under review as a conference paper at ICLR 2027

INFER: Feature Engineering as Inference with Transferable Reinforcement Learning Agents

Abstract

Feature engineering, the transformation of raw columns into predictive features, is one of the most time-consuming steps of tabular machine learning. Automated feature engineering (AutoFE) faces an exponentially growing candidate space and expensive per-candidate evaluation, and nearly every existing method pays for a fresh search on every new dataset. We propose INFER (INductive Feature Engiמeering via Reinforcement), which learns feature engineering once, offline, and applies it to any new dataset by inference. INFER is a hierarchical deep reinforcement learning framework with two cooperating agents: an outer agent reads dataset meta-features and selects a small subset of promising columns, and an inner agent composes operators over that subset to build a new feature. Both agents are trained on a collection of datasets and then applied, with frozen weights and no retraining, to previously unseen datasets. Feature generation on a new dataset therefore costs only forward passes and a lightweight selection step, with the number of generation runs as an explicit compute budget. On 75 binary-classification datasets, under a strict nested protocol in which every candidate is generated and fitted on training rows only, INFER improves over the original features on 81–83% of datasets with both XGBoost and RandomForest. It outperforms the RL-based method DIFER under both classifiers and the state-of-the-art OpenFE under XGBoost, and matches OpenFE under RandomForest. Code: https://github.com/infer2026/INFER

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Automated feature engineering, Tabular data, Deep reinforcement learning, Cross-dataset transfer, AutoML

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_u3AFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

INFER casts AutoFE as a hierarchical RL problem that is trained once across datasets and applied with frozen weights to new ones. An outer PPO agent picks k = 4 columns from 129-dim dataset meta-features ϕ(D) (Sec. 3.3). An inner DRQN agent builds a feature with a stack machine over unary/binary/GroupBy operators (Sec. 3.2), using shaped rewards (+5/+10/+1/−1) and an RRE-in-AUC outer reward (Eq. 1). At deployment (Alg. 1), N = 50 runs × m = 5 features produce a pool of 250 candidate programs. Selection then happens with OOF-based Top-K ensembling, LR-stacking, NNLS, or a "Combined" meta-selection over the three. All of it is strictly nested within outer training folds, which leakage-invariance tests confirm (Sec. 4.2). On 75 OpenML binary tasks with a dataset-level 4-fold transfer split, INFER (Combined) improves over original features on 81% (XGB) / 83% (RF) of datasets. The mean RRE is +5.9%/+6.0%, and INFER beats DIFER and OpenFE under XGB while tying with OpenFE under RF (Table 1, Fig. 2).

Critical strengths
  • [Importance of the research problem] The paper targets a real cost: nearly every AutoFE method reruns a full search on each new dataset, so nothing is amortized and users cannot bound compute (Sec. 1).
  • [Importance of the research problem] Treating the number of generation runs N as an explicit budget knob answers a stated usability need.
  • [Novelty of the approach] The inductive setting is new for deep-RL AutoFE. Frozen agents trained on source datasets are applied to unseen datasets under a dataset-level 4-fold split (Sec. 4.2).
  • [Novelty of the approach] Splitting the task into column selection (outer PPO agent over 129-dim meta-features) and operator composition (inner DRQN stack machine) is clean (Secs. 3.2–3.3).
  • [Soundness and feasibility] The pipeline is leakage-free by construction. Operators are fitted on training rows only, candidates are stored as replayable programs, and the leakage-invariance tests pass (Secs. 3.5 and 4.2).
  • [Soundness and feasibility] Selection is fully training-side, including the Combined meta-selection.
  • [Experiments and practical impact] The protocol is rigorous: 75 datasets, 5×5 nested CV, Holm-corrected Wilcoxon tests, bootstrap CIs and medians.
  • [Experiments and practical impact] INFER improves over the original features on 61/75 (XGBoost) and 62/75 (RandomForest) datasets, with mean RRE +5.9% and +6.0% (Table 1).
  • [Experiments and practical impact] Reporting is honest: the outlier dataset is disclosed (+4.8% and +5.1% without it), and parity with OpenFE under RandomForest (35/1/39, p = 0.73) is acknowledged.
  • [Theoretical and application-level implications] The results suggest that feature-construction skill can partly transfer across tabular datasets through meta-features alone, without textual semantics.
  • [Theoretical and application-level implications] The leakage-tested nested protocol is a reusable methodological contribution for AutoFE evaluation.
  • [Theoretical and application-level implications] The design suits settings with many similar tabular problems: the one-time cost (about 1.3 GPU-hours plus about 614 core-hours) breaks even after about 27 datasets against DIFER.
Critical weaknesses
  • [Importance of the research problem] The scope is restricted to binary classification on datasets with 4–20 columns, which limits how far the problem is actually addressed.
  • [Novelty of the approach] Hierarchical and multi-agent RL feature generation and meta-learned feature generation already exist (MARLFS; Gao et al., 2025; ExploreKit; Reyes-Amezcua et al., 2022). The individual components are therefore incremental.
  • [Soundness and feasibility] Top-K averages up to K = 10 classifiers, and LR-Stacking and NNLS combine many classifiers, while the baselines are single models. Gains may come from ensembling rather than better features.
  • [Soundness and feasibility] There is no random-policy ablation (random subsets and random valid programs through the same pipeline), so the contribution of the RL agents is not isolated.
  • [Soundness and feasibility] The inner reward is hand-shaped (+5/+10/+1/−1, Sec. 3.1), and its sensitivity is not analyzed.
  • [Experiments and practical impact] Absolute AUC gains are small (+0.011 and +0.014). Gains are weak on hard datasets (53% win rate and −0.6% RRE under XGBoost, Fig. 4).
  • [Experiments and practical impact] Per-dataset cost is about 4.0 core-hours at N = 50, versus 0.22 hours for OpenFE (Table 2). There is no accuracy-vs-N curve.
  • [Theoretical and application-level implications] There is no theory of when transfer should work, e.g., as a function of the shift between source and target meta-features.
  • [Theoretical and application-level implications] The ensemble outputs add inference latency and reduce interpretability. The method needs labeled data, and multiclass and regression tasks are unsupported.
Questions for authors
  1. Can you add ensemble controls for the ensembling concern above, especially Top-K/LR-stacking/NNLS over 50 seeded or bootstrapped models on the original features, and the same selection over OpenFE candidates?
  2. Can you report a random-policy baseline (random subsets + random valid programs) through the identical generation-then-selection pipeline?
  3. What is AUC as a function of N ∈ {1, 5, 10, 25, 50}, and where does INFER cross OpenFE in accuracy per core-hour?
  4. How many generated features end up in the final model on average, and are they interpretable? A few examples would help.
  5. Does a single pair of agents trained on all 75 datasets transfer to an external benchmark, e.g., datasets from the OpenFE or CAAFE papers?
Reviewer_2Q95OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces INFER, a hierarchical reinforcement learning approach to automated feature engineering within a generate-then-select framework. An outer agent selects four columns, and an inner agent sequentially composes operators from a predefined library to construct a new feature. The agents are trained on source datasets and applied with frozen weights to unseen datasets. Multiple candidate feature sets are generated and evaluated through downstream classifiers, whose predictions are selected or combined using Top-K, LR-Stacking, NNLS, or a combination of these strategies. Experiments on 75 binary-classification datasets with XGBoost and RandomForest report statistically significant aggregate improvements over using the original features alone.

Critical strengths
  1. A well-motivated and accessible approach to transferable feature generation. Learning a reusable feature-generation policy addresses the cost of repeating feature search for every new dataset. The decomposition into column selection and sequential operator composition is straightforward and clearly presented. The positive downstream results obtained with compact MLP and recurrent networks make the approach informative: they demonstrate the feasibility of reusing learned generation policies across datasets without updating those policies on each new dataset.

  2. Broad empirical evaluation and detailed documentation. The 75-dataset evaluation covers two downstream classifier families, and the comparisons among Top-K, LR-Stacking, NNLS, and Combined show that the same learned candidate generator can support multiple selection and ensemble strategies. This provides evidence that its usefulness is not confined to a single postprocessing choice. The implementation details, hyperparameters, and evaluation protocol facilitate reproduction, while the explicit reporting of offline and per-dataset computation helps readers assess practical costs.

Critical weaknesses
  1. Generality beyond the evaluated downstream classifiers remains uncertain. The experiments use XGBoost and RandomForest, with a separate pair of policies trained for each classifier. They therefore establish transfer across datasets within these configurations, while leaving applicability to other tree ensembles, such as LightGBM, and neural tabular models untested. The benefit of explicit feature construction may depend on the downstream model's capacity to learn transformations and interactions itself. This limits the demonstrated scope of the approach.

  2. The relatively high generation cost needs a more detailed explanation. Under the reported five-fold protocol, INFER requires approximately 4.0 core-hours per dataset for generation at N = 50, plus about 0.2 core-hours for selection, compared with 0.22 core-hours for OpenFE's full search. This comparison excludes INFER's offline policy-training cost. Given the compact agent networks, it is difficult to assess the runtime bottleneck from the aggregate numbers. The paper attributes most generation time to candidate materialization, but does not quantify the contributions of policy inference, meta-feature computation, and operator execution or fitting. This limits the interpretation of the efficiency claims and the practical trade-off relative to OpenFE.

Questions for authors

Clarification requests:

  1. Could you provide a breakdown of the reported generation runtime into agent inference, meta-feature computation, and candidate materialization, including operator execution and fitting? In particular, how are the classifier-based dataset meta-features computed and reused, and how much do they contribute to generation time rather than the separately reported selection time?
  2. Is there existing evidence or analysis that clarifies how the benefit of generated features depends on the downstream classifier's capacity or model family? Please distinguish the demonstrated transfer across datasets from any expected transfer across downstream classifiers.

Suggestions for future work: Evaluating additional tree ensembles and neural tabular models would help establish broader applicability. Exploring higher-capacity agent backbones, potentially including frontier models, is also an interesting direction, although any benefit over the current compact networks remains to be demonstrated.

Reviewer_aKUcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work propose INFER. It trains two RL agents on a collection of tabular datasets: An outer agent read dataset statistics and picks 4 columns, a inner DRQN combines them with several operators (e.g. +, -, x, , binning, normalization, etc) into a new feature.

Finally, on a new dataset, INFER generate 50 runs x 5 condidate features, followed by feature selection. On 75 OpenML datasets, INFER marginally improves ~80% datasets (mean +0.011 with XGBoost, +0.014 with RandomForest).

Critical strengths
  • This work performs leakage-invariance tests.
  • This work evaluate methods on a large collection of tasks.
  • Outlier dataset is flagged.
Critical weaknesses
  • Limited Novelty, Meta-learned (RL) feature engineering that transfers across datasets has already been studied long ago [1,2,3].
  • No evidence that the RL agents matter. There is no ablation replacing the agents with random column selection and random operator programs under the same generation and selection pipeline, and no comparison with untrained agents. The selection step could be doing all the work.
  • The improvement is marginal (mean +0.011 with XGBoost, +0.014 with RandomForest).

[1] Katz, Gilad, Eui Chul Richard Shin, and Dawn Song. "Explorekit: Automatic feature generation and selection." 2016 IEEE 16th international conference on data mining (ICDM). IEEE, 2016.

[2] Fatemeh Nargesian, Horst Samulowitz, Udayan Khurana, Elias B. Khalil, and Deepak Turaga. 2017. Learning Feature Engineering for Classification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17. IJCAI, 2529–2535.

[3] Zhang, Jianyu, et al. "Automatic feature engineering by deep reinforcement learning." Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems. 2019.

Questions for authors
  • The final predictor ensembles up to 10 classifiers (Top-K, stacking, NNLS), while the original-features baseline is a single model. How much of the +0.011 AUC (XGBoost) remains against an equally sized ensemble on the original features?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.