- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies two failure phenomena in time series agents—Human-Agent Tool Misalignment (expert-curated tool libraries help on some tasks but hurt on others) and Silent Harm (self-revision fixes some answers but breaks other previously correct ones)—and proposes TimeEvo to help the agent evolve its own tool library from diagnosed failures. TimeEvo clusters errors into failure buckets, plans a measurement contract per bucket, synthesizes evidence-only tools, and admits them only after two-level validation: a per-tool prefilter requiring an attributable repair, and a whole-library paired gate that explicitly weighs helped against harmed questions and enforces a positive statistical lower bound (with a one-shot prune-and-retest fallback). Experiments demonstrate the effectiveness of TimeEvo.
Critical strengths
-
The two failure phenomena are practical problems, which should be addressed in the real-life applications
-
The two-level validation is well-targeted and reasonable.
-
The results are strong and complete within the tested setting.
Critical weaknesses
-
The paper presents Human-Agent Tool Misalignment and Silent Harm as two co-equal motivations, but they concern two distinct problems—"mismatch in externally supplied tools" and "hidden harm from self-revision"—and the paper never establishes a causal or structural relationship between them. This gives the motivation the appearance of being assembled from two unrelated observations.
-
The evolved library mixes executable tools with prompt patches, yet the paper never reports the separate contributions of these two artifact types to helped/harmed counts, leaving the attribution of Silent Harm unclear.
-
The paper only tests cross-model transfer within the same task; it does not validate cross-task or cross-domain generalization. Moreover, the method only patches capabilities for the current failure buckets and never defines a global capability space, so it is impossible to judge whether the library is complete or whether it contains redundancy.
-
The paper claims to cluster by "missing capability," but it never defines how capabilities are represented, how similarity between erroneous questions is measured, or how stable the clustering is and how often it fails; nor does it provide any threshold sensitivity analysis. As a result, this core mechanism is neither reproducible nor verifiable.
-
The paper does not compare against memory-based methods or tool-selection methods, so it cannot rule out that simply "recording execution experience" or "optimizing selection over a fixed library" would achieve similar results.
-
The paper calls itself as self-evolving, yet from the experimental study, the evolving is from empty root library with one evolution round
-
All ten tasks are closed-set multiple-choice questions with automatically scorable answers. There is a lack of open-ended, multi-hop, and real-world domain tasks, raising doubts about the method's external validity.
Questions for authors
Please check the weak points above.
Thank you for the authors' response. I have reviewed the rebuttal carefully. The authors have addressed the points I raised, and the clarifications improve my understanding of the submission. I'll consider raise my score.