- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TimesEval addresses tool overuse in LLM agents for time series reasoning. Agents invoke analytical tools and usually execute every proposed tool call. Some calls are redundant given the available evidence, while others produce misleading outputs, leading to wasted computation and reduced accuracy. Instead of post-training the backbone LLMs, TimesEval proposes a lightweight learned module for deciding whether to execute a proposed tool call, reject the tool call, or drop the tool call and answer immediately. The module is trained via paired counterfactual rollouts, learning which decision leads to a better final outcome given the evidence so far. On TSRBench, the final model outperforms baseline tool-use strategies while making fewer tool calls
Critical strengths
- The method is practical. A lightweight MLP module learns to gate efficient tool usage with the backbone agents frozen, addressing a problem that otherwise may require expensive RL post-training.
- The counterfactual supervision is well-grounded. Branching decisions from the same state isolates each decision’s effect on the final outcome. The paper shows thorough analysis on paired versus unpaired training: unpaired training loses 8.6% (table 2), and paired evaluation turns more wrong cases into right ones than the reverse (fig. 5).
Critical weaknesses
- Evaluation is only on the in-distribution TSRBench. The policy is trained and selected on 1,200 TSRBench problems drawn from the same 15 tasks it is tested on, whereas baselines are training-free or zero-shot (including the RL models in sec 5.3.1). The method can prove to be much stronger if it is demonstrated to generalize to OOD tasks and samples.
- The ablation in section 5.3 is limited to Qwen3-8B. On other backbone in stable 1, no ablation is reported alongside, so the gains over ReAct, LATS, and ToolTree cannot be attributed solely to the learned gate. The method can be more convincing if similar ablation results show the same conclusion as on Qwen3-8B.
- The efficiency claim is not well supported. The main text reports cost only for Qwen3-8B against Always-Commit (table 2), where it drops from 3.90 to 3.33. The cost also excludes the evaluator’s own forward pass through the backbone LLM at every evaluation round. The claim would be more convincing if end-to-end cost, including the evaluator, were reported for all backbones and baselines.
Questions for authors
- Can the policy trained on a subset of TSRBench better guide tool usage on some held-out TSRBench samples or in other benchmarks.
- The policy is trained on TSRBench training problems while the baselines are not. How do the baselines perform when given the same data, for example as in-context examples or for fine-tuning?
- What are the accuracy and cost of TimesEval on other backbone models: TimeOmni-1-7B, Time-MQA-7B, and TS-Reasoner-7B?
- Does the reported cost include the evaluator’s forward pass through the backbone LLM, and what are the end-to-end cost and latency of TimesEval relative to Always-Commit?
Thanks for the response. The additional experiements have resolved my main concerns. I'll increase the rating to 3.