- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses an important yet underexplored limitation of current coding agents in data lake settings: each task is tackled from scratch, so knowledge and artifacts from previous work are discarded rather than reused. To address this, the authors propose TEAL, a framework for tool evolution in data lake analytics that progressively builds and refines a library of reusable tools together with useful skills distilled from past tasks. TEAL is a multi-agent system in which agents take on distinct, specialized roles. The authors further introduce TEAL-Q, a component that generates synthetic questions to drive and support tool evolution. Experiments on KramaBench and LakeQA show that the proposed approach is effective.
Critical strengths
- The paper is well organized and easy to follow. The illustrations are vivid and easy to understand.
- The proposed method is simple, intuitive, and easy to understand. The experimental results demonstrate a certain degree of effectiveness.
Critical weaknesses
- The method is relative simple and its multi-agent design (TEAL) lacks a clearly novel technical component, limiting the incremental contribution.
- Library-evolution variants degrade notably between 20 and 30 steps, in some cases severely; this contradicts the paper's main claim and is left unexplained — a theoretical account and finer-grained experiments are needed.
- Ablations isolating each agent's role and contribution are missing, leaving the empirical support incomplete.
Questions for authors
N/A
3 comments
Dear Reviewer, The authors have posted their rebuttal responses. I would kindly encourage you to review the authors' responses and continue the discussion by providing folow-up comments where appropriate. In particular please indicate whether the authors responses have adequately addressed your concerns and update your assessment if necessary Thank you for your time and continued efforts! Your AC
Looking forward to your feedback
Dear Reviewer oo4E,
Thank you again for your time and thoughtful review. As the discussion period ends in a few hours, we would like to check whether our responses address your concerns. In particular, we clarified our main claim and added an analysis of why the severe drops occur only for Gemini-3.8-Flash (Q2). For per-agent ablations, we added an Explorer ablation on LakeQA and a gold-answer audit of the Analyst checks (Q3). We would be glad to clarify anything further before the deadline.
Best regards,
Authors of Submission46103
Response to Reviewer oo4E [1/1]
Q1: Novelty
Thank you for this comment. We believe TEAL provides a new tool evolution system for data lake analytics, with three contributions.
search_dataset_catalogin all 80 LakeQA test tasks. Without the Explorer, Gemini-3.1-Pro's accuracy on LakeQA at k=15 drops from 50.83 to 45.83.Q2: Drops below the baseline at 20 to 30 steps
Our main claim is about evolving under a tight budget and deploying at any looser budget (Sec. 4.2). Every library that falls below the baseline in Fig. 3 was evolved at 20 steps or more, and libraries evolved at a tight budget stay above the baseline at every looser budget, for every Worker on both benchmarks.
The severe drops (5 points or more) occur only for Gemini-3.8-Flash. Our further analysis shows that this Worker differs from the other three as follows.
search_catalog, thenquery_files). This coincides with more empty answers (28% vs. 13% for the k=30 baseline).Q3: Per-agent ablations
Thank you for the suggestion. Sec. 4.5 ablates the Explorer, the Skill Writer, and the label-free verification, with Sonnet-5 as the Worker at k=15 on KramaBench. We add an Explorer ablation on LakeQA and an audit of the Analyst.
Thank you for your time and effort in reviewing our paper. We have tried to address each of your concerns with clarifications and new results, and we would be grateful if you could take them into account.