- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TUNE-R2, an LLM-guided hyperparameter optimization (HPO) framework that amortizes LLM reasoning across multiple objective evaluations. The proposed method compresses optimization history into a bounded search state, induces structured search policies, generates candidate configuration queues, and adaptively determines when to refresh the policy. Experiments across OpenML, FCNet, and NASBench-201 demonstrate competitive optimization performance while reducing Tokens@Best by 68–93% compared with existing LLM-guided HPO methods.
Critical strengths
- Practically relevant cost-performance trade-off: This study focuses on the critical trade-off between LLM inference cost and optimization performance, providing valuable insights for LLM-guided optimization and agent systems requiring repetitive decision-making.
- Decoupled reasoning and policy reuse: The framework decouples policy reasoning from objective function evaluation, reduces redundant inference overhead through policy reuse, and requires no fine-tuning or additional training of the underlying LLMs.
Critical weaknesses
- Incomplete validation of algorithmic novelty: The novelty of the approach has not been fully validated. Without a controlled comparison against a fairly configured vanilla batch search baseline, it is difficult to determine how much of the performance gain genuinely stems from the structured "policy" mechanism rather than simple batching.
- Unexplained discrepancy in normalized regret: There is a noticeable discrepancy in normalized regret between Table 1 and Table 9 that requires clarification and reconciliation.
Questions for authors
- How does TUNE-R2 compare against a vanilla batch search baseline under a matched evaluation budget and comparable candidate generation/filtering settings? Please also report the corresponding token consumption.
- Please reconcile the normalized regret values between Table 1 and Table 9. Were identical normalization procedures and experimental settings used across both tables?
- Could you report the total token consumption and end-to-end runtime at a fixed evaluation budget for the main benchmark suite, alongside the reported Tokens@Best metric?