Co-Evolving LLM Agents and Bayesian Optimizers for Automated Feature Engineering

Automated feature engineering (AutoFE) involves searching a combinatorial space of compositional transformations under expensive downstream evaluations, making efficient exploration and effective reuse of search experience essential. We introduce CoEvolve, a collaborative evolution framework that couples an LLM agent for local exploration with a global optimizer for selecting promising feature set anchors. Starting from an anchor, the agent invokes FE-MDP, a surrogate network that predicts one-step performance gains conditioned on the current feature set and candidate action, to guide exploratory beam expansion and collect local search experience. A dual-memory mechanism distills this experience into Conditional Agent Memory (CAM), which captures context-specific evidence about effective and ineffective transformations, and Unconditional Agent Memory (UAM), which accumulates reusable knowledge across feature sets. Together, these memories guide the agent in proposing candidate transformations for downstream evaluation. To coordinate global exploration, we introduce the Deep Set Global Optimizer (DSGO), which combines a permutation-invariant Set Transformer encoder with a Bayesian linear regression posterior to estimate downstream performance and predictive uncertainty. DSGO selects promising anchors through expected improvement, directing local exploration toward promising regions of the search space. As newly evaluated candidates expand the search pool, the surrogate models and agent memories are iteratively updated, allowing global optimization and local exploration to inform one another. Experiments on public tabular benchmarks demonstrate that CoEvolve outperforms existing feature selection and generation methods, achieving average relative performance improvements of 21.68% over the raw-feature baseline and 3.59% over the strongest competing baseline on each dataset.