acceptodds
Under review as a conference paper at ICLR 2027

From Knobs to Actions: Agentic Bayesian Optimization for Self-Improving System Tuning

Abstract

Modern systems such as Large Language Model (LLM) inference engines and databases expose large configuration spaces with strongly interacting parameters. Their workloads can change, making automated tuning essential. Bayesian optimization (BO) is sample-efficient for expensive black-box functions, but BO-based autotuners often underuse runtime metrics and parameter relationships, require many evaluations in high-dimensional spaces, and adapt slowly to workload shifts. We observe that transitions from suboptimal to better configurations follow recurring directions in parameter space. These directions remain partly stable across similar workloads, and LLMs can infer candidate directions from system knowledge. Building on this observation, we propose EvoTune, an agentic BO framework that searches over semantic Actions and refines them using trial evidence. EvoTune maintains an Action–Metric–Objective Graph to combine LLM-based diagnosis with model-based Action selection, while BO optimizes execution magnitudes and evidence and feasibility checks govern Action and graph updates. Across eight static workloads, EvoTune achieves the best observed endpoint, with average and maximum gains over a common initial reference of 33.47% and 55.19%, respectively. Under workload shifts, its post-switch throughput exceeds the strongest compared baseline by 11.42% on databases and 25.79% on Spark.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Bayesian optimization, LLM agents, system configuration tuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WVVROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents EvoTune, an agentic Bayesian optimization framework for system configuration tuning. It represents coordinated parameter changes as semantic Actions, uses an Action–Metric–Objective Graph to guide Action selection, and applies BO to optimize their execution strengths. Trial evidence and feasibility checks govern updates to Action directions and graph relationships. Evaluation covers eight static workloads and three workload-switch experiments across vLLM, PostgreSQL, and Spark, together with component ablations on Spark.

Critical strengths
  1. A coherent combination of semantic reasoning and numerical optimization. Separating coordinated adjustment directions from their execution strengths provides a useful way to incorporate system knowledge into BO. Graph-guided selection and evidence-gated representation updates give the framework a concrete mechanism for responding to changing bottlenecks.

  2. Broad and encouraging empirical validation. Experiments span three system types, both static and changing workloads, and several relevant baselines. The Spark ablations provide additional evidence for the roles of Action updates, graph updates, and selection. Explicitly including validation and failed executions in the trial budget also strengthens the evaluation protocol.

Critical weaknesses
  1. The statistical robustness of the reported improvements is insufficiently characterized. Section 5.2 mentions experiments with different initial configurations, but does not report their counts or aggregate variability. Appendix F.3 clarifies that the main static results are best-observed endpoints rather than independently confirmed performance. Reporting uncertainty and confirmation results would help establish whether the improvements over the strongest baselines are consistent rather than sensitive to measurement noise or initialization.

  2. The relationship between the specified selection rule and the implementation needs clarification. Equation 7 uses average absolute GP posterior gradients to measure Action sensitivity. However, Appendix K.3 states that the inspected v2 diagnosis code instead estimates positive posterior objective gain and combines it with confidence and repeated response support. These are distinct scores, and it is unclear which produced the reported results. This matters for reproducing the method and interpreting the Action-selection ablation.

Questions for authors
  1. Which Action-scoring rule was used for each system in Figures 5–8: the gradient-based rule in Equation 7 or the posterior-gain-based implementation described in Appendix K.3? Does this distinction materially affect the results or ablation conclusions?

  2. How many independent optimization runs and initialization settings support the reported comparisons? Please provide variability estimates and the independent confirmation results described in Appendix F.3, particularly for comparisons against the strongest baselines.

3 comments
AuthorsOpenReview ↗

Response to Reviewer WVVR

Thank you for your review. We address all your concerns below.

Which scoring rule was used for each system in Figs. 5–8? Would the alternative rule change the ablation conclusions?

vLLM, PostgreSQL, and Spark all used the absolute posterior-gradient rule in Eq. (7) whenever Action screening was enabled. The positive posterior-gain score in the previous Appendix K.3 came from an older implementation. The revised K.3 now specifies Eq. (7), the rule used in these experiments.

How many independent runs support the comparisons, and does the advantage survive endpoint confirmation?

Each method completed five independent searches per static workload, with a common first-trial configuration across methods for each seed. The advantage persisted after independent remeasurement on all eight static workloads. Figures 5–6 now show mean best-so-far curves across the five seeds, with pointwise 95% confidence bands for EvoTune and the strongest baseline. After the budget was exhausted, we locked the configuration with the best valid observed objective from each run and measured it four more times. The four repeats contribute only to the confirmation statistics. Relative to the strongest remeasured baseline, EvoTune reduced TPC-H latency by 1.17 s (paired 95% CI 0.67–1.66 s) and increased PageRank throughput by 3.98 thousand ranks/s (2.14–5.81). Appendix I.4 gives the seed mapping and endpoint sample variances. Appendix I.5 reports the paired confirmation interval for each of the eight workloads. These replication counts and uncertainty estimates apply to the static experiments in Figs. 5–6.

Dear Reviewer WVVR,

With just over one day remaining before the author–reviewer discussion deadline on November 18 (AoE), we would appreciate your feedback on our response. We clarified that all three systems used the absolute posterior-gradient rule in Eq. (7) whenever Action screening was enabled, and corrected Appendix K.3 accordingly. To address your concerns about repeatability and endpoint confirmation, we added five independent searches per static workload and four additional measurements of each selected configuration, with paired 95% confidence intervals reported in Appendices I.4–I.5. The confirmation results favor EvoTune on all eight static workloads.

Could you please share your assessment of these clarifications and additional results? We would be happy to address any remaining questions before the deadline. Thank you for your time.

Dear Reviewer WVVR, we have devoted considerable effort to strengthening the statistical evaluation through repeated searches, independent endpoint confirmation, and confidence intervals, while also clarifying the Action-scoring rule used in the experiments. As the discussion period draws to a close, we would greatly appreciate it if you could review our response and revised manuscript and share your assessment. Thank you for your thoughtful feedback and time.

Reviewer_HYcmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses configuration tuning for systems with interacting parameters and changing workloads. It identifies limitations in using runtime evidence and reusing coordinated adjustments. EvoTune encodes semantic Actions as parameter-change directions, uses an Action–Metric–Objective Graph for diagnosis and selection, and applies BO to Action strengths. Trial evidence governs representation revisions. Experiments cover eight static workloads and three workload switches across vLLM, PostgreSQL, and Spark; the authors report best observed static endpoints on all workloads, averaging 33.47% improvement over their common initial reference.

Critical strengths
  1. Novelty: The technical change is to optimize strengths of coordinated semantic directions while revising those directions and their diagnostic graph from evidence (Eq. 5; Sec. 4.5). This is a novel combination, not a new BO acquisition rule. Relative to the paper's closest references, OtterTune reuses metric/workload experience, LlamaTune uses random embeddings, GPTuner guides knob/range selection from manuals, and LLAMBO inserts LLMs into warmstarting, prediction, and candidate generation; none is described as this revisable Action representation (Sec. 2). In my judgment, the delta could suffice for ICLR if its mechanism is validated.
  2. Directional structure is tested out of sample: Prototypes are fitted on 280 development directions and evaluated on 120 held-out directions, with an isotropic reference; eight prototypes cover 95.8% at cosine threshold 0.80 (Sec. 3.1; Fig. 1). This provides concrete motivation for a compact directional library, rather than relying solely on LLM plausibility.
  3. Auditable execution safeguards: Version-compatible history prevents unchanged coefficients from being mistaken for unchanged configurations; held-out relation checks, paired direction tests, duplicate filtering, and protected confirmation share the execution budget (Appendices D–F; Eq. 29; Appendix K.4). These are valuable safeguards for operators tuning inference, database, and processing systems (Sec. 1).
  4. Reimplementation detail, with bounded completeness in terms of reproducibility.
Critical weaknesses
  1. Method–implementation mismatch — Eq. 7 ranks Actions using absolute posterior-gradient sensitivity, whereas Appendix K.3 explicitly states that inspected v2 code estimates positive posterior objective gain and combines it with confidence and repeated response support. These are acknowledged to be distinct scores. Which rule generated each performance figure is not reported in the main text or appendix, preventing attribution to the stated selection mechanism. Identify the exact implementation for Figs. 5–8 and compare both scoring rules under otherwise identical settings.
  2. Endpoint superiority lacks uncertainty estimates — Main gains concern best observed measurements, not independently confirmed outcomes (Appendix F.3). The different-initialization claim is qualitative (Sec. 5.2); independent-run counts, seed lists, and endpoint mean/variance are not reported in the main text or appendix. Moreover, 33.47% is improvement over a common initial reference, not over the strongest baseline (Appendix I.3). Report repeated, paired-seed comparisons with independently remeasured endpoints and confidence intervals; this determines whether the observed ranking survives measurement variability.
  3. Baseline fairness cannot be established — Equal execution counts and shared initial seeds are specified, but shared seeds need not produce identical initial samples (Appendix F.2). Baseline implementation provenance, LLM identities, hyperparameter-tuning budgets, and post-switch reset/history policies are not reported in the main text or appendix (Sec. 5.1; Appendices I, J, K). Consequently, stronger language-model guidance or different adaptation policies remain unexcluded explanations. Provide per-baseline configurations and a common initialization, backbone, and switch-policy comparison.
  4. Semantic contribution is not isolated — Existing ablations retain the semantic library or change graph/direction updates; the no-LLM replacement procedure is not reported in the main text or appendix (Sec. 5.4; Appendix I.5). Thus Fig. 8 does not distinguish semantic knowledge from generic dimension restriction. Compare with feasible random directions matched in count, scales, and update budget, and specify the no-LLM controller. This directly tests the claimed representational insight rather than another complete-system comparison.
Questions for authors

Q1. Which scoring rule generated Figs. 5–8? If a matched rerun of Eq. 7 preserves the reported advantage, I would raise my score; if only the distinct v2 heuristic succeeds, the stated mechanism needs revision.

Q2. Can you provide matched-backbone, paired-seed confirmed endpoints and baseline switch policies? If superiority persists under these controls, I would raise my score; if it disappears, I would lower it.

Q3. Does EvoTune outperform count/scale/budget-matched random-direction BO, and what exactly replaces LLM diagnosis and revision in Fig. 8?

4 comments
AuthorsOpenReview ↗

Response to Reviewer HYcm

Thank you for your review. We address all your concerns below.

Which scoring rule generated Figs. 5–8, and how does it compare with the alternative?

Figures 5–8 used the absolute posterior-gradient rule in Eq. (7) wherever Action screening was enabled.

Sorry for the confusion. The previous Appendix K.3 described the positive posterior-gain score from an older implementation. We have updated Appendix K.3 to specify Eq. (7) throughout. We use this rule to stabilize Action screening. Averaging the absolute gradient over several feasible probe points reduces the influence of local sensitivity at any one configuration. We have double checked the main paper and Appendix to ensure that there is no such mistakes.

Does the endpoint advantage persist across paired seeds and independent remeasurements?

Yes, on all eight static workloads. Each method completed five independent searches per workload using paired seeds. We locked the best observed configuration from each search and measured it four more times, excluding its original search score. These four post-search measurements are excluded from the 35- or 45-trial budget and from Figs. 5–6. Across 57 workload–method combinations and five seeds, this yielded 1,140 new measurements for 285 selections. We averaged the four repeats within each seed and computed a paired 95% interval from the five seed-level differences. All eight intervals favored EvoTune over the strongest remeasured baseline for the corresponding workload. The table gives the mean advantage and its paired 95% interval in each workload’s own units. A positive value denotes lower latency on TPC-H and higher throughput elsewhere.

Workload (unit)Strongest remeasured baselineEvoTune meanBaseline meanPaired advantage [95% CI]
KV (tokens/s)Pure LLM887.40772.07115.33 [69.66, 160.99]
Batch (tokens/s)Pure LLM2,226.612,007.86218.76 [152.93, 284.58]
Prefill (tokens/s)Pure LLM233.60208.2325.37 [18.50, 32.24]
Prefix (tokens/s)OtterTune880.90783.6497.25 [53.65, 140.86]
TPC-H (s)SCOOT9.4310.591.17 [0.67, 1.66]
pgbench (TPS)Pure LLM2,312.942,103.07209.87 [144.84, 274.89]
Sort (thousand rows/s)SCOOT984.12886.1497.98 [42.48, 153.48]
PageRank (thousand ranks/s)Pure LLM42.5538.573.98 [2.14, 5.81]

Appendix I.4 gives the seeds and search-endpoint variances. Appendix I.5 reports the confirmation intervals separately by workload.

Were initialization, LLM settings, evaluation budgets, and workload-switch policies matched?

Yes for these four controls. The platform passes the same concrete first-trial configuration to every method within a workload and replicate. All methods that call an LLM online use deepseek-flash with temperature 0.1, a 4,096-token output limit, and a 120-second timeout. The static search budget is 45 target executions for vLLM and Spark and 35 for PostgreSQL, including the common first trial and attempts that fail after execution starts. An executed in-budget confirmation counts once, and its valid measurement enters the running best at that trial. At trial 21 of a switch experiment, the platform changes the workload and continues the same run, retaining the configurations, measurements, execution statuses, and workload labels from trials 1–20. Section 5.1 identifies each baseline and its search mechanism. The shared controls are documented in Appendices F.2, I.1, I.6, and J.

What replaces the LLM in Fig. 8, and does the comparison isolate semantic knowledge?

The no-LLM controller uses BO to optimize all coefficients of a fixed random-direction library. It skips metric diagnosis, Action screening, and revisions to either the directions or the graph. The number of directions matches the semantic library, and both controllers receive 40 target executions under the same Spark Sort-to-PageRank switch. EvoTune reaches higher final PageRank throughput. This result supports the contribution of semantic knowledge in EvoTune: using it to construct, select, and revise Actions yielded higher final throughput than optimizing equally many fixed random directions (§5.4).

Dear Reviewer HYcm,

With just over one day remaining before the author–reviewer discussion deadline on November 18 (AoE), we are following up on our response. We clarified the scoring rule used in Figures 5–8 and corrected Appendix K.3, added five-seed results with independent remeasurements and paired 95% confidence intervals (Appendices I.4–I.5), and documented the shared initialization, LLM settings, evaluation budgets, and workload-switch protocol. We also clarified precisely what replaces the LLM in the Figure 8 comparison (§5.4).

We would be grateful if you could review these updates and share your thoughts, particularly whether any concerns remain. We would welcome the opportunity to respond before the discussion closes. Thank you for your time.

Dear Reviewer HYcm,

As the discussion deadline approaches, we would appreciate your feedback on the additional experiments we conducted to address your concerns about statistical reliability and endpoint confirmation.

For every method–workload pair across the eight static workloads, we conducted five independent searches with paired seeds. We then fixed the best configuration from each search and measured it four additional times, excluding its original search score. This produced 1,140 new measurements for 285 selected configurations. The paired 95% confidence intervals favor EvoTune over the strongest remeasured baseline on all eight workloads (Appendices I.4–I.5).

Alongside these experiments, we clarified the scoring rule, shared initialization, LLM settings, evaluation budgets, workload-switch protocol, and the exact behavior of the no-LLM controller.

We hope this additional evidence helps you assess the reliability of our results. We would be grateful for your response, particularly regarding any concerns that remain, and would welcome the opportunity to address them before the deadline. Thank you for your time.

Dear Reviewer HYcm, we have undertaken substantial additional experiments and revisions in response to your feedback, including five-seed comparisons, independent endpoint remeasurements, and clarification of the scoring rule, evaluation controls, and no-LLM comparison. As the discussion deadline approaches, we would be grateful if you could review these updates and let us know whether the additional evidence addresses your concerns. We welcome further questions and would be happy to provide clarification.

Reviewer_7qt9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces EvoTune which is an LLM based bayesian optimisation framework to tune the parameters/configurations of different frameworks (vLLM, Spark and PostgreSQL). It uses Deepseek-flash LLM to read the documentation of these frameworks and define a small set of high level candidate changes called Actions. Each action can modify multiple parameters in a fixed direction e.g., in case of vLLM which is a inference engine for LLMs, it can suggest increasing batch size, kv cache or turning on prefix caching. A subsequent bayesian optimisation is used to optimise these proposed action strengths. After each trial llm then reads the metrics again and selects the ones it believe are candidate bottlenecks. The bayesian optimisier then tunes only those actions in the next trial which reduces the space.

Critical strengths
  • The proposed use of the LLM only for choosing the direction of a parameter update and using bayesian optimisation to update the step size is a reasonable design and to my knowledge this split has not been used in prior LLM based tuners. A good thing about the idea the use of LLMs in combination with the bayesian optimisation as it keeps the role of llms small enough so that its decisions can be inspected
  • The proposed validation step where revisions suggested by the LLM are tested with additional trials before they are adopted helps avoiding bad suggestions.
  • The preliminary study provided in Section 3 is a useful result on its own as it shows that the update directions of standard bayesian optimisation on vLLM (i) cluster into around eight prototypes, (ii) cover approx. 96% of the held out directions under a train/test split and (iii) are clearly distinct from an isotropic baseline.
Critical weaknesses
  • The method only searches over the Actions defined by the LLM at the start and the strengths are restricted in a range so the optimiser can only move in the directions the LLM proposed and cannot generalise to configurations outside of them. A major concern I have is e.g., if new version of vLLM adds a new scheduler parameter, the Action library has to be regenerated and all previous trials become incomparable which is not an ideal case.
  • The authors do not show whether this restriction matters, e.g., there is no comparison with the same bayesian optimisation run on (i) the raw parameters or (ii) a random subspace of the same size, which is the obvious baseline for this kind of representation.
  • The reported performance gains are measured against the lowest trial-1 configuration across all methods and not against the best baseline. Also, the margin over the best baseline is never stated anywhere in the paper.
Questions for authors
  • How does the framework handle a parameter that is not covered by the initial Action library, e.g., imagine a new scheduler flag is added in a newer vLLM release. As far as I can tell the revision step can only change the direction of existing Actions and library will have to be regenerated.
  • Can you also please elaborate on graph based selection's impact on the number of free coefficients in different runs.
  • Can you run the same bayesian optimisation on (i) the raw parameters and (ii) a random subspace of the same size as the Action library, at least for the vLLM profiles, so we can see if the Actions themselves are helping or if it is just the bayesian setup?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 7qt9

Thank you for your review. We address all your concerns below.

Can EvoTune handle a new parameter without regenerating the Action library?

Yes. We added a generalization experiment in §5.5 to test whether EvoTune can reuse its Action library after a simulated vLLM version change. We kept the Prefill workload fixed. After trial 20, we added --watermark and --max-num-scheduled-tokens to the parameter catalog and removed --swap-space. We gave each of three strategies 25 further trials. A (frozen) kept the original directions on the surviving parameters and optimized only their strengths. B (incrementally revised) retained the trial history and existing Actions and incorporated the new parameters through validated direction edits. C (regenerated) built a new Action library from the revised catalog. All three strategies started from a trial-20 best of 206.06 tokens/s. Their trial-45 best throughputs were 226.01, 236.41, and 215.72 tokens/s, respectively. Incremental revision improved on the common starting value by 14.73% and finished 4.60% above the frozen library. It achieved 9.59% higher best throughput than regeneration (Fig. 9). In this experiment, EvoTune adapted to the changed parameter catalog by revising its existing Action vectors without rebuilding the library.

How many coefficients remain free after graph screening?

The count varies by trial. The limit of three applies to the selected metrics. The number of free coefficients depends on which metrics are selected and how they connect to Actions in the graph. We take the union of the Actions adjacent to the selected metrics. Each distinct Action in that union contributes one free coefficient, and all other coefficients are fixed at zero. In the example in Appendix K.2, three selected metrics together activate all seven vLLM Actions, so all seven coefficients are free in that step.

Does the evaluation include BO over raw parameters and over a random subspace of the same size?

Yes. The original manuscript did not clearly explain how the baselines operate. We expanded the baseline description in §5.1 and the ablation description in §5.4 to identify both comparisons. SCOOT performs BO directly over the original system parameters and is included on all four vLLM profiles. The no-LLM variant in the Spark ablation performs BO in a random subspace. It uses as many random directions as there are semantic Actions, holds those directions fixed, and jointly optimizes their strengths (Fig. 8).

What is the margin over the strongest baseline?

You are correct that the percentage in the original submission measured improvement over the common initial reference. In the revised five-seed analysis, the gain over the common initial reference averages 43.52% across the eight equally weighted workloads and reaches 69.99% on Prefix reuse. Section 5.2 reports the strongest-baseline comparison separately. At the five-seed mean search endpoint, EvoTune improves on the strongest baseline by 11.00% on average across workloads. Gains denote lower latency on TPC-H and higher throughput elsewhere.

WorkloadStrongest baselineGain over strongest baselineGain over initial reference
vLLM KVPure LLM14.38%62.38%
vLLM BatchPure LLM9.99%60.08%
vLLM PrefillPure LLM9.94%21.65%
vLLM PrefixOtterTune12.82%69.99%
PostgreSQL TPC-HSCOOT11.01%40.76%
PostgreSQL pgbenchPure LLM9.23%20.27%
Spark SortSCOOT10.28%52.22%
Spark PageRankPure LLM10.34%20.81%
Equal-weight mean11.00%43.52%

Appendix I.5 separately reports independent remeasurements of the selected configurations.

AuthorsOpenReview ↗

Response to Reviewer 7qt9

Dear Reviewer 7qt9,

With just over one day remaining before the author–reviewer discussion deadline on November 18 (AoE), we would appreciate your feedback on our response and revised manuscript. In response to your questions, we added an experiment on adapting to a changed parameter catalog without rebuilding the Action library (§5.5), clarified the number of free coefficients after graph screening and the raw-parameter and random-subspace BO comparisons, and separately reported the improvement over the strongest baseline—11.00% on average across the eight workloads (§5.2).

Could you please let us know whether these additions and clarifications address your concerns? We would be happy to discuss any remaining questions before the deadline. Thank you for your time.

Dear Reviewer 7qt9, we have devoted considerable effort to addressing your concerns, including a new experiment on adaptation to parameter changes, clarification of the BO comparisons and Action screening, and explicit reporting of gains over the strongest baselines. As the discussion period draws to a close, we would greatly appreciate it if you could review our response and revised manuscript and share any remaining concerns. Thank you for your time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Reviewers,

Thank you for your careful reading of our manuscript and for the specific comparisons and checks you suggested. We have revised the paper to address your comments. The main additions and clarifications are summarized below.

  • Generalization to new parameters. We added a vLLM Prefill experiment that simulates a version change by adding two scheduler parameters and removing one after trial 20. Starting from the same best throughput of 206.06 tokens/s, we compared freezing, incrementally revising, and regenerating the Action library over 25 further trials. Incremental revision reached 236.41 tokens/s, exceeding the frozen library by 4.60% and the regenerated library by 9.59% (Section 5.5, Figure 9). (Reviewer 7qt9\color{red}{\mathrm{7qt9}})

  • Improvement over the strongest baseline. We now report both comparison points explicitly. Across eight static workloads, EvoTune improves by 43.52% over the common initial reference and by 11.00% over the strongest baseline for each workload, averaged equally across workloads. At the five-seed mean search endpoint, the strongest-baseline margins are 14.38%, 9.99%, 9.94%, and 12.82% on vLLM KV, Batch, Prefill, and Prefix, respectively. EvoTune also reduces TPC-H latency by 11.01% and increases throughput by 9.23% on pgbench, 10.28% on Sort, and 10.34% on PageRank (Abstract and Section 5.2). (Reviewers 7qt9\color{red}{\mathrm{7qt9}}, HYcm\color{blue}{\mathrm{HYcm}}, WVVR\color{orange}{\mathrm{WVVR}})

  • Repeated searches and uncertainty estimates. We ran five paired-seed searches per method and static workload. Figures 5–6 now show mean best-so-far performance across seeds, with pointwise 95% confidence intervals for EvoTune and the strongest baseline. Appendix I.4 reports the seed assignments, endpoint means, sample variances, and paired differences (Section 5.2, Figures 5–6, Appendix I.4). (Reviewers HYcm\color{blue}{\mathrm{HYcm}}, WVVR\color{orange}{\mathrm{WVVR}})

  • Independent confirmation of selected configurations. We locked the best observed configuration from each search and measured it four additional times, yielding 1,140 new measurements for 285 selections. These measurements are separate from the search budget and the curves in Figures 5–6. On all eight workloads, the paired 95% interval favored EvoTune over the strongest remeasured baseline. For example, on KV the confirmed advantage over Pure LLM was 115.33 tokens/s (95% CI 69.66–160.99) (Section 5.2, Appendix I.5). (Reviewers HYcm\color{blue}{\mathrm{HYcm}}, WVVR\color{orange}{\mathrm{WVVR}})

  • BO over raw parameters and random directions. We clarified that SCOOT performs BO directly over the original system parameters and is included on all four vLLM profiles. We also described the no-LLM control in the Spark ablation: BO jointly optimizes the strengths of fixed random directions matched in number to EvoTune’s semantic Actions. This control skips diagnosis, Action screening, and revision. EvoTune achieved higher final PageRank throughput under the same 40-trial workload switch (Sections 5.1 and 5.4, Figure 8). (Reviewers 7qt9\color{red}{\mathrm{7qt9}}, HYcm\color{blue}{\mathrm{HYcm}})

  • Scoring rule used in the reported experiments. We corrected Appendix K.3, which previously described the positive posterior-gain score from an older implementation. All experiments in Figures 5–8 that used Action screening used the absolute posterior-gradient rule in Equation (7). The revised appendix specifies how sensitivity is computed over feasible probe points (Section 4.3, Appendix K.3). (Reviewers HYcm\color{blue}{\mathrm{HYcm}}, WVVR\color{orange}{\mathrm{WVVR}})

  • Number of free Action coefficients. We clarified that the cap of three applies to selected metrics. Each distinct eligible parent Action of those metrics contributes one free coefficient. The worked example shows how three metrics can activate all seven vLLM Actions, making explicit when screening leaves the full Action space available (Section 4.3, Appendix K.2). (Reviewer 7qt9\color{red}{\mathrm{7qt9}})

  • Fairness of the comparison protocol. We documented the identical first-trial configuration and seed assigned to all methods within each replicate, the common DeepSeek Flash settings for online LLM calls, and the fixed budgets of 45 target executions for vLLM and Spark and 35 for PostgreSQL. We also clarified that each workload switch takes place at trial 21 within a continuous run, with the preceding configurations and measurements retained under their workload labels (Section 5.1, Appendices F.2, I.1, I.6, and J). (Reviewer HYcm\color{blue}{\mathrm{HYcm}})

Thank you again for helping us improve the paper.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.