acceptodds
Under review as a conference paper at ICLR 2027

TUNE-R2: Amortized Search Policies for LLM-Guided Hyperparameter Optimization

Abstract

Hyperparameter optimization (HPO) automates the selection of effective configurations in machine learning. Recent work uses large language models (LLMs) to guide this process through semantic priors and feedback from prior trials. However, most existing methods tightly couple reasoning with acting, repeatedly invoking the LLM over a growing trial history after each evaluation. This increases reasoning cost and can dilute decision-relevant signals with redundant context. In this paper, we introduce TUNE-R2, an amortized search policy framework that decouples LLM reasoning from individual objective evaluations. Instead of reasoning anew for every trial, TUNE-R2induces a reusable search policy from a bounded search state and refines it into a queue of candidate configurations for multiple subsequent evaluations. A state-adaptive policy horizon determines how many evaluations reuse each policy before it is refreshed with new feedback, balancing reasoning amortization against feedback freshness. Across OpenML, FCNet, and NASBench-201, TUNE-R2 achieves the best overall regret, AOC, mean rank, and win rate among evaluated LLM-guided HPO methods while requiring 68%–93% fewer Tokens@Best. Additional experiments on LLM fine-tuning and kernel-SVM HPO further support the generality of the approach. These results demonstrate the effectiveness of amortizing LLM reasoning through reusable search policies.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Hyperparameter Optimization, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_H2U3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TUNE-R2, an LLM-guided hyperparameter optimization (HPO) framework that amortizes LLM reasoning across multiple objective evaluations. The proposed method compresses optimization history into a bounded search state, induces structured search policies, generates candidate configuration queues, and adaptively determines when to refresh the policy. Experiments across OpenML, FCNet, and NASBench-201 demonstrate competitive optimization performance while reducing Tokens@Best by 68–93% compared with existing LLM-guided HPO methods.

Critical strengths
  1. Practically relevant cost-performance trade-off: This study focuses on the critical trade-off between LLM inference cost and optimization performance, providing valuable insights for LLM-guided optimization and agent systems requiring repetitive decision-making.
  2. Decoupled reasoning and policy reuse: The framework decouples policy reasoning from objective function evaluation, reduces redundant inference overhead through policy reuse, and requires no fine-tuning or additional training of the underlying LLMs.
Critical weaknesses
  1. Incomplete validation of algorithmic novelty: The novelty of the approach has not been fully validated. Without a controlled comparison against a fairly configured vanilla batch search baseline, it is difficult to determine how much of the performance gain genuinely stems from the structured "policy" mechanism rather than simple batching.
  2. Unexplained discrepancy in normalized regret: There is a noticeable discrepancy in normalized regret between Table 1 and Table 9 that requires clarification and reconciliation.
Questions for authors
  1. How does TUNE-R2 compare against a vanilla batch search baseline under a matched evaluation budget and comparable candidate generation/filtering settings? Please also report the corresponding token consumption.
  2. Please reconcile the normalized regret values between Table 1 and Table 9. Were identical normalization procedures and experimental settings used across both tables?
  3. Could you report the total token consumption and end-to-end runtime at a fixed evaluation budget for the main benchmark suite, alongside the reported Tokens@Best metric?
Reviewer_GJwBOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TUNE-R2 reduces the frequency and context size of LLM reasoning in hyperparameter optimization. Each refresh uses a bounded trajectory summary to induce structured search guidance, then makes a second LLM call to generate an over-complete proposal pool. Local validation, deduplication, and ranking produce a queue whose length is selected adaptively between one and five evaluations. Experiments cover a 17-task HPO/NAS suite and two additional tuning tasks, with state, induction, horizon, refinement, prompt-interface, and backbone analyses. The reported headline result combines improved aggregate regret with fewer Tokens@Best.

Critical strengths
  1. A simple and useful separation between reasoning and objective evaluation. The bounded-state and queue-based design directly addresses repeated LLM inference over growing histories. The policy horizon makes the cost–feedback trade-off explicit. Appendix E supplies prompts and schemas, and Appendix G provides a concrete state, reflection, queue, and run-level example, making the overall procedure substantially easier to understand.

  2. The controlled horizon study supports more than batching alone. Table 4 compares adaptive reuse with fixed horizons under the same evaluation budget. Adaptive reuse improves regret and AOC over h=1,3,5 while using fewer tokens than h=1 or h=3; h=5 is cheaper but less accurate. The matched-state raw-history test and fixed-proposal refinement test are also useful attempts to separate the individual mechanisms. The additional LoRA and kernel-SVM experiments extend the evaluation beyond lookup benchmarks.

Critical weaknesses
  1. The reported aggregate metrics need a numerical and protocol audit. Table 1 reports TUNE-R2's 17-task mean normalized Regret@30 as 0.0250, while Table 9 reports 0.6553 for Qwen3-32B on four tasks drawn from that suite. If these are the same nonnegative task-normalized values, runs, and weighting convention, the four-task contribution alone implies a 17-task mean of at least 4×0.6553/17 = 0.1542, which is incompatible with Table 1. Different normalization bounds or evaluation protocols could explain the discrepancy, but they are not identified. Table 1's conventional-method win rates such as 7.7%, 13.3%, and 15.4% also require explicit denominators and tie rules under the stated 17-task protocol. These issues affect the main claim of aggregate superiority, not just presentation.
  2. The efficiency and comparative-strength claims are broader than the supporting controls. Tokens@Best compares the cost of reaching each method's own best incumbent, rather than a shared target quality, and can omit subsequent reasoning expenditure. The trajectory plots and wall-clock results on two additional tasks are helpful, but do not establish the full-suite cost advantage at matched optimization quality. Further, the seven conventional baselines consist of random search and six variants of a shared GP implementation; this does not cover several standard mixed/discrete-space optimizers. The aggregate tables provide task dispersion rather than uncertainty in paired method differences, so the evidence for a reliable advantage over strong conventional HPO remains incomplete.
Questions for authors
  1. Reconcile the metrics and provide an auditable aggregation. Please release or tabulate the per-task, per-seed final objectives, normalized regrets, normalization reference values, ranks, and wins underlying Tables 1 and 9. Are the Qwen3-32B runs and normalization bounds the same in both tables? What denominators, tie handling, and missing-task rules produce each Table 1 win rate? Please also define Queue Regret and key-state reference density and state the task/seed aggregation used in Tables 2, 4, and 5.
  2. Compare costs at a common quality level and full budget. Could the authors report cumulative tokens and elapsed time at a shared task-specific target, together with total cost through all 30 evaluations, counting both refresh calls and any repair/retry/fallback operations? Paired confidence intervals or a task-level statistical comparison would clarify whether the optimization improvements are reliable. Comparisons with a well-tuned TPE or mixed-space optimizer would strengthen the conventional-HPO claim.
  3. Specify the local controller and failure behavior. What are the exact capacities and update rules for the bounded state, the mapping from search regime/plateau to the fallback horizon, the proposal-pool size rule, and the novelty metric/weight? What happens when fewer than h valid distinct candidates remain, or the entire queue is empty? These local choices determine both the realized horizon and the claimed two-call cost, but are not fully specified by the prompt templates.
Reviewer_xr4jOpenReview ↗
Rating
3
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TUNE-R2 reduces the cost of LLM-guided hyperparameter optimization by reusing search guidance across several objective evaluations. At each refresh, it constructs a bounded summary of the optimization state, asks the LLM for structured guidance and candidate configurations, and builds a locally filtered queue. The queue is evaluated for an adaptive number of trials before new feedback is incorporated. The evaluation covers tabular HPO, FCNet, and NASBench-201, with additional tasks and studies of reasoning cost, history representation, policy lifetime, and candidate refinement.

Critical strengths

The paper addresses a practical limitation of LLM-based optimization: repeatedly querying the model can be expensive relative to the useful guidance it provides. The results suggest that a bounded state and short candidate queue can retain competitive optimization performance while reducing this overhead. Figure 3 is particularly helpful because it shows optimization quality against cumulative tokens throughout the search, complementing the aggregate results in Table 1.

The component experiments provide meaningful support for the design. Under the same 30-evaluation budget, the adaptive horizon achieves lower regret than fixed horizons of 1, 3, and 5 (Table 4). The longest fixed horizon is cheaper but performs worse, so the result supports choosing when to refresh rather than simply maximizing reuse. The bounded-history comparison at matched checkpoints and the refinement experiment using a fixed candidate pool are also useful controls. Together, these studies give the method a stronger empirical basis than the overall benchmark comparison alone.

Critical weaknesses

The aggregation in the main results needs clearer documentation. Table 1 does not explicitly identify the central statistic used for regret, and it is difficult to determine whether every method is evaluated over the same task set. The win rates are not straightforward to reconcile with a common 17-task denominator; this may reflect ties, seed aggregation, or different coverage, but the convention is not stated. Table 9 explicitly reports a mean over four tasks, and its relationship to the main table also needs explanation. These ambiguities make the overall comparison harder to assess than it should be.

Several local computations remain underspecified. The prompt schemas are helpful, but the method also depends on how trend and phase are computed, how the fallback horizon is chosen, and how novelty modifies candidate ranking. These choices affect both the state presented to the LLM and the configurations ultimately evaluated. They should be specified precisely enough to reproduce the method without making additional design decisions.

Questions for authors
  1. Please provide the per-task results and clarify the aggregation statistic, seed aggregation, normalization references, task coverage, and tie handling used in Table 1. How do these conventions relate to Table 9?
  2. Can you specify the computations for trend, phase, and the fallback horizon, together with the novelty adjustment used to rank the candidate pool?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.