- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates methodological failures in agentic data science systems that conventional evaluations, focused on predictive performance, may overlook. The authors conduct three complementary studies: (1) a large-scale analysis of 15 failure modes across 220 data science tasks and nine agent–model configurations, (2) a controlled test suite targeting six methodological pitfalls, and (3) a comparison of agent-generated pipelines with expert solutions from Kaggle competitions.
Critical strengths
-
The paper highlights a critical limitation of existing data science agent benchmarks: high predictive performance does not necessarily imply methodological correctness—an increasingly important issue as capable agents automate complex ML workflows.
-
The evaluation spans 220 tasks, multiple agent frameworks and backbone models, controlled failure-mode tests, and comparisons with expert Kaggle solutions, offering complementary insights into the limitations of current data science agents.
-
Particularly noteworthy is the finding that data leakage is associated with better predictive rankings. The distinction between failures caused by missing pipeline stages and those arising within more sophisticated pipelines also provides valuable guidance for future agent development.
Critical weaknesses
- The large-scale analysis relies heavily on a two-stage LLM judge to identify methodological failures, yet the paper provides no systematic human-annotated validation of the precision and recall of the 15 failure categories. Since the main conclusions depend on these judgments, stronger validation of the evaluation framework is necessary.
- Several failure modes, such as using only one evaluation metric, naive ensembling, and the absence of feature engineering , do not necessarily indicate methodological errors in all scenarios. Their validity depends on task requirements and model characteristics. The paper should better distinguish objectively invalid practices from potentially suboptimal design choices, as this distinction directly affects the reported failure rates.
- The paper reports that leakage-related failures are associated with improved predictive rankings, but the regression analysis controls only for agent and model identities, without fully accounting for task characteristics or pipeline complexity. A controlled comparison of identical pipelines before and after leakage correction would provide stronger evidence.
Questions for authors
-
How were the 15 failure modes validated against expert data scientists' judgments? Can the authors report category-level precision, recall, or inter-annotator agreement?
-
How does the failure prevalence change when subjective categories such as FM07, FM12, and FM15 are excluded?
-
Can the authors conduct controlled leakage-removal experiments to determine whether correcting leakage consistently reduces the reported predictive performance?
-
How representative are the controlled test results of real-world data science tasks, particularly for failures requiring domain-specific knowledge?
4 comments
Part 2/2
W3 & Q3: "The paper reports that leakage-related failures are associated with improved predictive rankings, but the regression analysis controls only for agent and model identities... Can the authors conduct controlled leakage-removal experiments to determine whether correcting leakage consistently reduces the reported predictive performance?"
R3: We ran the controlled experiment as suggested: for pipelines where the judge flagged leakage, we used an LLM to fix the leakage, and another LLM judge to verify if there is any remaining leakage. We repeat this fix-verify loop until the LLM check confirms there is no more leakage, and we re-execute the pipeline on the same test set to compare scores.
We sampled 152 pipelines across 109 tasks from all five data sources, stratified by agent (ReAct, AIDE, MLEvolve), model (Gemini 3.7, Qwen 27B, Qwen 4B), leakage mode, maximum severity, number of leakage findings (1-3), and evaluation metric. 145 pipelines reached leakage-free status (5 timed out; 2 retained leakage upon manual review and were excluded).
Across the 145 repaired pipelines, 37% scored worse once leakage was removed, 10% scored better, and 52% changed by less than 0.5% as noise. Among inflated pipelines, the median inflation was 4.4%, 18 pipelines lost more than 10% of their score, and the most extreme case (where an engineered feature was equivalent to the target) essentially lost all predictive performance.
Seeing these scores change confirms the paper’s finding that leakage positively correlates with downstream performance. This complementary experiment shows the inflation is directly caused by the leakage rather than other confounding factors. We integrate this in our main text as complementary measures.
Q4: "How representative are the controlled test results of real-world data science tasks, particularly for failures requiring domain-specific knowledge?"
R4: While the empirical representativeness across domains is established by our 220-task real-world study from multiple sources (domain coverage in Table 8, Appendix A.3), the controlled suite is designed as a diagnostic unit-test tool. Each of the tasks is built on real-world datasets with a manually introduced canonical failure and an unambiguous ground-truth practice check, which is a limitation in the large-scale analysis where ground-truth is not available. The suite tests reasoning patterns where domain knowledge is required for valid modeling, such as understanding physical interaction laws (torque = weight x distance in FM04) or domain-specific data encodings (FM02). These tests measure whether agents inspect and reason based on data semantics or blindly apply common data science templates. The diagnostic findings reflect real-world behavior observed from our expert solution comparisons, where human experts spend substantial effort validating domains while agents often ignore them and look for shortcuts.
We would be grateful if you would consider updating your score after these clarifications and additions. Please let us know if you have any remaining questions.
I thank the authors for their detailed rebuttal and additional experiments, which have meaningfully strengthened the empirical analysis and addressed several of my concerns.
However, I remain concerned about the reliability of the LLM-based failure identification, the limited causal evidence linking data leakage to performance inflation, and the sensitivity of the reported failure rates to subjective failure definitions. These issues continue to affect the confidence and generalizability of the paper's central findings.
Therefore, while I appreciate the authors' substantial efforts, I will maintain my original rating of 2.
Thank you for the response.
We would like to emphasize the setup we have for each of the evaluation components ensuring reliability. Experts spent 30+ person-hours annotating 19 end-to-end data science pipelines across all 15 failure modes (two experts, independently) and verifying 42 further judge findings with their severity. The experts' own labels, without any LLM, reproduce the paper's prevalence: 95% of pipelines contain at least one failure (paper: 94.6%). And, our controlled suite does not rely on open-ended judging. Each test has a carefully designed, iterated pitfall with ground-truth checks.
For leakage presence, the observational and interventional estimates agree. Our rank regression shows leaky pipelines rank 0.41 positions higher on their task leaderboard, adjusted for agent and model. Subsequently, removing the leakage from 143 of these pipelines moves them down by 0.52 positions.
To our knowledge, this is the first study of silent failures in data science agents at scale (220 tasks covering different domains, 3 agents with different framework complexity and 3 models with different sizes). We provide evidence of how detrimental generated code can be when leakage is present. Our insights and findings consistently show that a leaderboard score alone cannot tell if the method is sound, and we need to audit the pipelines. Besides showing score is not enough, the analysis surface persistent critical failures such as leakages with different forms and flavors , that informs future research and better deployment practices.
Part 1/2
Thank you for the constructive review, and helpful suggestions. We address the raised questions and remarks below.
R1: Our two-step judging procedure, prompt and model selection were initially developed by aligning against human assessments on pilot pipelines. We now add a formal human evaluation (Sec. 3.1 & 4.1).
Two data science experts independently annotated 19 randomly sampled pipelines across all failure modes. The two experts agreed on 0.947 of labels (Cohen's κ=0.80). Our LLM judge reached 0.895 and 0.926 agreement with the two annotators (κ=0.62 and 0.71, substantial agreement).
Sampled pipelines do not cover all FMs, so we performed targeted annotations on FM01 wrong task formulation, FM02 peeking at test data, FM09 feature selection before splitting, FM10 wrong model type and FM13 reusing the test set. The resulting precision is 83% overall and 100% on the two leakage modes (FM02 and FM09).
Per-category P, R, and F1 (to be included in the appendix) show no significant inter-annotator disagreement; rare categories are covered by the targeted review above.
R2: Each of these failure modes can be harmful within specific contexts. Therefore, we have used a two-stage judging tool that integrates the applicability check on a stage-level in a structured way. We agree that there could be another level of context needed, such as the severity level on the instance-level. Instances with lower severity often reflect suboptimal practices rather than strictly invalid methodology.
The judge assigns a severity score (1-3) to every finding. Across all judged pipelines, occurrences in FM07, FM12, and FM15 are indeed concentrated at severity 1-2, whereas leakage and validation failures are mainly at severity 2-3. FM14 also occurs most frequently at severity level 1-2.
We capture both confidence and severity; human-judge severity agreement reaches 82% (all but one within one level; full results in the extra page).
Removing those less critical failure modes drops the percentage of failure findings and the mean number of findings, with 67.1% (and 63%) of pipelines still containing at least one finding of severity $\ge$2. In the revision, we revise the distinction between failures and bad practices and report prevalence on both groups based on the severity level.