- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CoEvolve, a framework for automated feature engineering. It combines DSGO for selecting initial feature sets via Bayesian optimization, FE-MDP for estimating transformation gains during local search, and an LLM guided by conditional memory (CAM) and accumulated memory (UAM) to propose further transformations. Evaluated candidates are used to update the models and memory components.
Critical strengths
- Conditioning transformations on the current feature set is well motivated. Equation (4) accounts for both generation and selection, and Appendix B demonstrates that the same transformation can benefit one feature set while degrading another. This supports the paper's focus on context-aware feature search.
- The experiments cover both classification and regression datasets and compare CoEvolve with a broad set of feature selection and generation methods, providing a reasonably broad empirical basis for evaluating the method.
Critical weaknesses
- The abstract describes general challenges in AutoFE but does not identify a specific limitation of existing methods before introducing CoEvolve. The authors should clarify which unresolved problem motivates the proposed method.
- Section 2 and Table 1 omit several recent and closely related methods [1-4]. Why are these works neither reviewed nor compared with CoEvolve? Please explain the distinctions from these methods and provide appropriate comparisons under matched evaluation budgets, or justify why such comparisons would be unsuitable. [1] Continuous Optimization for Feature Selection with Permutation-Invariant Embedding and Policy-Guided Search. KDD 2025, 1857-1866. DOI: 10.1145/3711896.3736891. [2] Adaptive Context-Infused Performance Evaluator for Iterative Feature Space Optimization. CIKM 2025, 3448-3457. DOI: 10.1145/3746252.3761123. [3] GPT-FT: An Efficient Automated Feature Transformation Using GPT for Sequence Reconstruction and Performance Enhancement. Web and Big Data (APWeb-WAIM 2025), LNCS 16113, 3-19, 2026. DOI: 10.1007/978-981-95-5640-3_1. [4] Heterogeneous Multi-Agent Reinforcement Learning with Attention for Cooperative and Scalable Feature Transformation. KDD 2026, 2010-2019. DOI: 10.1145/3770854.3780231.
- Section 3.2 relies on an LLM for transformations and memory updates, but the manuscript does not identify the model or provide the prompts used. These omissions make it difficult to reproduce the agent's behavior.
- Sections 4 and 4.1 omit key experimental details, including the downstream learners, classification metric, train/test split ratio, and baseline tuning procedures. As a result, readers cannot reconstruct the evaluation or assess whether all methods were tested under comparable settings.
- Section 4.3 mentions retaining 20 candidates, but this does not fully define the evaluation budget, and neither the total number of evaluations nor the baseline budgets are reported. The reported gains could therefore reflect more evaluations rather than a more effective search strategy.
- In Figure 3, the variant without DSGO slightly outperforms CoEvolve on OpenML-586 (0.7282 versus 0.7279), while the full method improves by only 0.0032 and 0.0079 on OpenML-589 and UCI Credit, respectively. Please explain why the simpler variant performs better or similarly on these datasets and whether these small differences persist across independent runs. Without uncertainty estimates or a comparison of computational cost, it remains unclear whether DSGO's additional complexity is justified on these datasets.
Questions for authors
- What specific limitation of existing AutoFE methods motivates CoEvolve, and could this gap be stated explicitly in the abstract?
- Why were recent related works [1-4] neither reviewed nor compared with CoEvolve? Please clarify how CoEvolve differs from these methods and provide comparisons under matched evaluation budgets, or justify why such comparisons would be unsuitable.
- Which LLM, prompts, and generation settings were used to propose feature transformations and update CAM/UAM?
- Which downstream learners, metrics, train/test splits, and baseline tuning procedures were used to produce Table 1?
- How many cross-validation evaluations and LLM calls did each method use, and does CoEvolve still outperform the baselines under the same evaluation budget?
- Why does the variant without DSGO slightly outperform CoEvolve on OpenML-586, while the full method provides only small gains on OpenML-589 and UCI Credit? Do these differences persist across independent runs, and do the gains justify DSGO's additional computational cost?
6 comments
Response to Weakness 1 and Weakness 2
: The abstract describes general challenges in AutoFE but does not identify a specific limitation of existing methods before introducing CoEvolve. The authors should clarify which unresolved problem motivates the proposed method.
: We agree that the abstract should identify the specific problem behind the large search space and high evaluation cost. Each candidate requires two choices: a feature set to start from and a transformation to apply. Since only a small fraction of these combinations can be evaluated, the challenge is to decide where to spend the next evaluations and what to try there.
CoEvolve addresses this challenge through a divide-and-conquer approach built around two alternating decisions that share evaluation feedback. DSGO first selects a feature set, or anchor, for further exploration. Given that anchor, FE-MDP and the LLM agent choose local transformations to evaluate. When a transformation produces a new feature set, its score informs which set to explore next, while its gain or loss relative to its parent informs which transformations to propose in subsequent rounds. The next global decision can therefore select a set discovered by local search, and the next local decision can draw on evidence gathered at earlier anchors. Our depth-two experiment supports the rationale for choosing the anchor separately. Under the same local evaluation budget, higher-scoring anchors tended to yield higher-scoring successors on all eight datasets. The Spearman correlation between the anchor’s training CV score and its best depth-two successor’s training CV score ranged from 0.3776 to 0.8473, with a mean of 0.5987. Thus, anchor quality provides useful guidance about where to search, even before the local agent determines which transformation to try.
| Dataset | OpenML-586 | OpenML-589| OpenML-607 | OpenML-620 | UCI Credit | SpamBase | svmguide3 |German Credit |
| Spearman | 0.3776 | 0.5172| 0.6724 | 0.5829 | 0.8473| 0.6524 | 0.7128 | 0.4271|
: The “w/o LLM” variant also replaces the local search method with embedding-based neighborhood search. Thus, it does not isolate the benefit of the LLM and its memories. Keeping DSGO and FE-MDP beam search while removing only LLM proposals, under the same evaluation budget, is needed to establish this central contribution.
: Thank you for identifying these omissions. We perform supplementary experiments to compare CoEvolve with CAPS and HAFT in Table 2.The public implementation of EASE and GPT-FT could not be found, thus they are not included in the supplementary experiment.
In the new comparison, CoEvolve scored higher than both CAPS and HAFT on seven of eight datasets. Across the four regression tasks, the mean gains in were 0.1008 over CAPS and 0.0674 over HAFT. On UCI Credit, CoEvolve scored 0.8061, compared with 0.8076 for CAPS and 0.8063 for HAFT. These runs used the same downstream learner, splits, and metrics. The total numbers of downstream search evaluations have not been matched, so the results establish performance under the reported settings. Thus, CoEvolve outperforms CAPS and HAFT.
Response to Weakness 3 part 1
: Section 3.2 relies on an LLM for transformations and memory updates, but the manuscript does not identify the model or provide the prompts used. These omissions make it difficult to reproduce the agent's behavior.
: We agree that Section 3.2 should identify the model and make the prompts easy to locate. Here we provides these details and include them in the appendix of revision. 1.Our benchmark configuration specifies Qwen3-30B-Instruct. 2. System prompts and fixed in-context example We use four system prompts and one fixed in-context example. The example accompanies the Conditional Agent Memory Writer. (1) Conditional Agent Memory Writer — CAM_SYSTEM_PROMPT You are the Conditional Feature Direction Inference Agent. You are given a fixed current feature set S and a collection of observed transitions (S, S_prime, score). Your task is to use these observations to identify promising directions for local search around S. The score of a transition is defined as score = metric(S_prime) - metric(S). A positive score indicates that the change improved performance relative to S. A negative score indicates that the change reduced performance relative to S. Assign each item to one of four priority levels. Priority 1 — Most promising directions to explore A direction appears repeatedly across multiple positive-scoring transitions, or a positive-scoring transition directly supports a single operation. These directions are the most promising candidates for further exploration around S. Priority 2 — Individual contribution unknown; prioritize for ablation A specific feature addition, removal, or replacement is part of the best positive-scoring transition. That transition changes multiple features, so the individual contribution of each component remains unknown. Priority 3 — Lower priority with insufficient evidence of harm The observed gain is small, the evidence is mixed, or the direction has only weak positive support. The direction is a lower priority for the current search, but the evidence gives no clear indication that it is harmful. Priority 4 — Negative directions with low confidence The direction appears only in negative-scoring transitions. The evidence comes from few observations or lacks repeated validation. Lower its search priority. The available evidence does not establish that the direction is harmful across all contexts. Rules 1.Attribute a score to an individual operation only when S and S_prime differ by exactly that operation. 2.If a transition adds or removes multiple features, attribute its observed score only to the combination of changes. Do not assign the score to any individual item. 3.Assign the individual components of the best positive-scoring multi-operation transition to Priority 2. 4.Assign each item to at most one priority level. 5.Return at most K items per priority level. 6.Return fewer than K items when the evidence is insufficient. Do not invent items to fill the list. 7.Limit CAM findings to the current S or similar feature sets. Do not generalize them directly to all feature sets. 8.Avoid claims such as “statistically significant” or “conclusively effective” unless the input includes repeated experiments and variance estimates. 9.Describe each item as a concrete operation or search direction. Examples include adding f1*f2, removing the standalone feature log(f1), or replacing {f1, f2} with {f1/f2}. 10.Return only JSON in the specified format. Output format { "best_anchor": { "transition_id": "", "score": 0.0 }, "analysis": [ { "finding": "", "evidence": [], "confidence": "high|medium|low" } ], "priorities": { "priority1": [ { "rank": 1, "item": "", "reason": "", "next_action": "", "confidence": "high|medium|low" } ], "priority2": [], "priority3": [], "priority4": [] }, "do_not_conclude": [] } (2) Fixed in-context example — CAM_ICL_EXAMPLE K=2 CAM={ "cam_id":"C1", "S":["f1","f2","f3","log(f1)","f1+f2"], "S_prime":[ [["f1","(f1+f2)/log(f1)","f3"],0.20], [["f1","f2","f3","f1+f2"],0.06], [["f1","f2","f3","log(f1)","f1+f2","f2+f3"],0.01], [["f1","f2","f3","log(f1)","f1+f2","sin(f1)"],-0.08] ] }
Full Ablation Study and Additional Baselines
We appreciate the reviewer’s thoughtful comments. In order to to isolate the contribution of each component in CoEvolve. We have expanded the ablation study to compare CoEvolve with six variants: (1) DSGO + FE-MDP, (2) DSGO + embedding-neighborhood search (ENS), (3) DSGO + LLM + CAM, (4) DSGO + LLM + UAM, and (5) FE-MDP (w/o DSGO) (6) DSGO (w/o FE-MDP). All variants use the same initialization and downstream CV budget. We use the same DSGO configuration wherever DSGO is present. For comparisons that isolate the proposal mechanism, we also hold the anchor-selection procedure, the initial beam search, and the legal action set fixed. By proposal mechanism, we mean how a method chooses which additional feature transformations to evaluate from the selected anchor after the initial beam search. In the DSGO + FE-MDP control, FE-MDP ranks untried legal actions by their predicted one-step gain and selects the highest-ranked actions. In the full CoEvolve model, the LLM selects actions from the same set using CAM, which summarizes which transformations helped or hurt at the current anchor, and UAM, which captures experience across anchors and search rounds. Both methods apply their selected actions to the original anchor and evaluate the resulting feature sets by CV under the same remaining budget.
Table 1
We also perform additional experiments to compare CoEvolve with more recent AutoFE methods, including CAPS (Liu et al., KDD 2025) and HAFT (Zhe et al., KDD 2026). CAPS uses a permutation-invariant encoder–decoder and a PPO policy to select a subset of existing columns. HAFT (Zhe et al., KDD 2026) uses two feature agents and an operation agent coordinated by a shared critic. Table 2 shows the eperimental results.
Table 2