- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TS-DCPPA addresses combinatorial optimization heuristic design for heterogeneous instances through a two-stage pipeline. It first discovers complementary algorithms and fuses them into a parameterized algorithm, then fixes that algorithm and learns a state-dependent configuration program. Deployment requires no LLM calls. Experiments cover TSP, OVMP, and FJSP.
Critical strengths
The two-stage design—fusing complementary algorithms before learning a state-dependent parameter policy—is sensible and requires no LLM calls at deployment. TS-DCPPA performs best across the reported ID and OOD settings, with especially strong TSP OOD results. The study includes ablations, repeated runs, paired tests, and a partial replication with another LLM backend.
Critical weaknesses
-
The “no fusion” ablation changes both the algorithm representation and the policy space, so it does not isolate dynamic parameter generation. A cleaner comparison would fix the fused algorithm and compare optimized fixed parameters, dynamic source-preset selection, and free parameter generation. The uneven warm-start benefit and the shared TSP scoring terms also motivate a simple coefficient-based parameterization. Without these controls, the gains cannot be attributed specifically to continuous adaptation.
-
The experiments omit relevant configuration and adaptive baselines. TIDE would test whether fixed-structure tuning recovers similar gains, while DASH and DyCA are closer adaptive alternatives. Comparisons under compatible interfaces and budgets, or a concrete explanation of infeasibility, are needed. The OVMP policy relies heavily on discrete threshold branches before continuous modulation, so it does not isolate whether smooth adaptation drives the OOD gain.
-
The compute comparison is not fully controlled: candidate timeouts differ, and the reported evaluation budget excludes fusion, review, and revision calls. The paper should report total LLM calls, wall-clock search cost, final-algorithm runtime, and run-to-run variation. Reproducibility is also limited because the OVMP OOD evaluation relies entirely on the non-public BD dataset. The authors should report how often fusion reached the soft limit and was accepted without structural approval.
Questions for authors
-
With the fused algorithm and compute fixed, how do optimized fixed parameters, dynamic preset selection, and dynamic parameter generation compare?
-
Under comparable interfaces and budgets, how does TS-DCPPA compare with DASH and DyCA? On TSP, does automatic fusion outperform a simple coefficient parameterization of the same source algorithms?
-
What are the total search cost, final-algorithm runtime, and run-to-run variation? How often was fusion accepted without structural approval, and can the OVMP OOD result be verified on public data?
6 comments
[2/3]
Baselines and parameterization (W2/Q2)
We evaluate TIDE, DASH, and DyCA under the same budget of 1,000 candidate evaluations. Like EoH-S, DyCA discovers a set of complementary algorithms in training without a test-time selector, so it naturally serves as an alternative Stage-1 searcher in TS-DCPPA (500 evaluations in Stage 1 + 500 in Stage 2). The table reports five-run mean test Gap (%; lower is better):
| Method | TSP ID | TSP OOD | OVMP ID | OVMP OOD | FJSP ID | FJSP OOD |
|---|---|---|---|---|---|---|
| TIDE | 23.01 | 30.46 | 1.79 | 2.41 | 11.77 | 11.55 |
| DASH | 24.59 | 26.39 | 3.15 | 4.63 | 13.45 | 11.04 |
| TS-DCPPA (DyCA in Stage 1) | 19.89 | 20.71 | 1.59 | 1.52 | 11.60 | 10.84 |
| TS-DCPPA (EoH-S in Stage 1) | 18.49 | 20.99 | 1.03 | 1.65 | 11.07 | 8.89 |
Both TS-DCPPA variants beat TIDE and DASH in all six columns. Moreover, plugging DyCA into Stage 1 remains superior to all baselines and even achieves the lowest OOD gaps on TSP (20.71) and OVMP (1.52), confirming that TS-DCPPA's gains stem from the two-stage framework itself rather than a specific Stage-1 method.
Automatic fusion vs. simple coefficient parameterization. We use automatic LLM fusion to construct rather than prescribing a fixed coefficient template for three reasons:
- No manual term alignment. Instantiating a coefficient template for the TSP source programs requires manual identification and alignment of their scoring terms. LLM fusion automates this process directly from the source code.
- Beyond simple parameterization. LLM fusion identifies shared computations across the source programs and exposes differences among them as parameters. For example,
regret_sharpnessin the fused TSP algorithm controls the distance threshold that determines which candidate nodes receive a regret adjustment, rather than simply adjusting the overall weight of the regret term. - Applicability across tasks. Extending a coefficient template to source programs with different control flows or composite decisions, such as
(operation, machine)assignments in FJSP, requires additional task-specific design. In contrast, LLM fusion constructs a task-specific parameterization directly from the source programs without requiring a manually designed template.
Threshold branches on OVMP. DCPPA favors continuous parameters for but also allows discrete or mixed parameters (such as integer window sizes and feature toggles on OVMP), and it does not restrict how the programmatic policy maps states to parameters. Using threshold branches on state variables to identify workload regimes before continuously modulating parameters within each regime is a natural way for to combine regime switching with fine-grained adjustment. Moreover, the comparison on the same (W1/Q1) shows that generating non-source parameters freely beats both fixed parameters and discrete preset selection.
[3/3]
Cost, variation, and reproducibility (W3/Q3)
Thank you for these concrete reporting suggestions.
(1) Search cost. TS-DCPPA matches the 1,000 candidate evaluations of all Table-1 baselines, and each candidate evaluation uses one LLM call. The wall-clock time is shown as follows.
| Method / stage | Candidate evaluations | TSP (min) | OVMP (min) | FJSP (min) |
|---|---|---|---|---|
| EoH | 1,000 | 32.4 | 81.6 | 77.1 |
| ReEvo | 1,000 | 95.0 | 99.5 | 156.0 |
| PartEvo | 1,000 | 65.6 | 69.7 | 35.6 |
| HiFo | 1,000 | 34.2 | 59.5 | 25.6 |
| RefineEvo | 1,000 | 157.6 | 176.3 | 140.4 |
| InstSpecHH | 1,000 | 19.9 | 22.5 | 17.4 |
| TS-DCPPA | 1,000 | 53.8 | 102.3 | 53.1 |
| TS-DCPPA: Stage 1 | 500 | 19.0 | 55.3 | 24.9 |
| TS-DCPPA: Fusion | - | 2.0 | 10.9 | 4.6 |
| TS-DCPPA: Stage 2 | 500 | 32.7 | 36.0 | 23.6 |
(2) Fusion acceptance. The following table summarizes five fusion results per task. Note that acceptance without structural approval is allowed after the 20-round soft limit only if the behavioral checks pass.
| Task | Mean LLM calls | Mean rounds | Behavioral checks passed | Accepted without structural approval |
|---|---|---|---|---|
| TSP | 11.4 | 6.2 | ||
| OVMP | 17.8 | 9.0 | ||
| FJSP | 27.6 | 12.0 |
(3) Run-to-run variation. Below is Table 1 with the SD over five runs.
| Method | TSP ID | TSP OOD | OVMP ID | OVMP OOD | FJSP ID | FJSP OOD |
|---|---|---|---|---|---|---|
| Seed heuristic | 26.33 | 28.13 | 2.30 | 3.84 | 25.41 | 27.49 |
| EoH | 19.55 ± 0.42 | 21.44 ± 0.38 | 1.67 ± 0.40 | 2.75 ± 0.55 | 12.77 ± 1.11 | 12.88 ± 4.14 |
| ReEvo | 22.42 ± 4.26 | 24.56 ± 4.50 | 2.55 ± 1.37 | 3.57 ± 0.47 | 13.02 ± 2.11 | 11.69 ± 2.26 |
| PartEvo | 20.56 ± 1.02 | 21.50 ± 1.63 | 1.94 ± 0.55 | 2.67 ± 0.87 | 14.10 ± 2.00 | 14.63 ± 2.59 |
| HiFo | 20.12 ± 0.43 | 21.27 ± 0.52 | 1.48 ± 0.71 | 2.73 ± 1.20 | 11.71 ± 1.92 | 11.27 ± 2.00 |
| RefineEvo | 20.82 ± 2.87 | 70.10 ± 108.79 | 2.08 ± 0.82 | 2.31 ± 0.61 | 11.71 ± 1.19 | 11.54 ± 4.17 |
| InstSpecHH | 26.44 ± 2.81 | 30.20 ± 3.95 | 3.80 ± 1.60 | 4.78 ± 1.95 | 16.77 ± 2.59 | 13.83 ± 3.40 |
| TS-DCPPA (Ours) | 18.49 ± 0.48 | 20.99 ± 0.66 | 1.03 ± 0.25 | 1.65 ± 0.16 | 11.07 ± 0.41 | 8.89 ± 0.32 |
(4) Candidate time limits and final-algorithm runtime. Each TS-DCPPA candidate is evaluated on all 20 training instances under a time limit. Because the time each stage needs is hard to know before the search, we leave a margin in each stage's limit. These limits are safety caps against non-terminating or very slow candidates, not actual runtimes.
To verify that the final TS-DCPPA programs finish within the baseline time limits, we evaluate the final programs on all 20 training instances. The table below reports the maximum observed runtime (seconds; InstSpecHH excludes LLM latency for algorithm selection), showing that TS-DCPPA finishes well within the baseline limits:
| Method | TSP (s) | OVMP (s) | FJSP (s) |
|---|---|---|---|
| Baseline time limit | 30 | 60 | 30 |
| EoH | 4.481 | 18.833 | 1.415 |
| ReEvo | 28.222 | 15.776 | 0.456 |
| PartEvo | 5.778 | 40.507 | 1.435 |
| HiFo | 5.684 | 19.206 | 0.658 |
| RefineEvo | 12.649 | 34.411 | 0.393 |
| InstSpecHH | 21.368 | 25.964 | 19.578 |
| TS-DCPPA | 8.818 | 19.393 | 0.244 |
(5) Public-data validation. We additionally evaluate TS-DCPPA and all six Table-1 baselines on public CERIT-SC traces, an OVMP source unseen during training. Using the five runs trained on Huawei and Azure without retraining, we test 60 instances (30 each at 50 and 80 PMs, matching our Table-1 OOD split) and normalize by first fit to avoid costly exact offline optimization over thousands of placement decisions (mean Gap %; lower is better, negative means more accepted requests than first fit):
| CERIT-SC (OOD) | EoH | ReEvo | PartEvo | HiFo | RefineEvo | InstSpecHH | TS-DCPPA |
|---|---|---|---|---|---|---|---|
| Gap (%) | -2.18 | -0.51 | -1.64 | -2.29 | -2.49 | 0.18 | -3.01 |
TS-DCPPA achieves the lowest mean Gap, confirming its OOD advantage on public data. Last but not least, we will release the code upon acceptance.
[1/3]
Thank you for your detailed and constructive review. We appreciate your positive assessment of our sensible two-stage design, TS-DCPPA's strong ID and OOD performance, and our thorough evaluation across ablations, repeated runs, paired statistical tests, and a second LLM backend. Below, we respond to your comments and questions point by point. We have revised the manuscript accordingly and highlighted the changes in blue.
Warm start and Stage-2 policy controls (W1/Q1)
Thank you for these constructive comments. We agree that discussing how Stage-2 policy search behaves under different warm-start schedules and parameter spaces is valuable.
Discussion on warm start. Warm start spends an initial share of the Stage-2 budget learning to switch among the source presets before free parameter generation. Varying this share on the same fused algorithm ϕ shows that warm start is beneficial across all three tasks, while the best fixed ratio is task-dependent (test Gap %, five-run mean, lower is better; ID = training distributions, OOD = unseen distributions):
Relative to w/o warm start, the best fixed ratio lowers the gap in every column (40% on TSP, 60% on OVMP, and 20% on FJSP), and all four nonzero ratios lower OVMP's ID gap by 28–33%. However, the sensitivity to the fixed ratio varies across tasks. This observation motivates an adaptive extension, TS-DCPPA (adaptive), which replaces the fixed ratio with Thompson sampling to choose between preset-guided and free parameter generation based on their recent search gains.
Discussion on fixed parameters vs. source presets vs. non-source parameters. Next, we examine what happens on the same ϕ and Stage-2 budget if the policy is restricted to a single learned parameter vector (fixed parameters) or to the discrete source presets (per-instance preset or per-step preset), versus generating non-source parameters freely under TS-DCPPA (original) and TS-DCPPA (adaptive):
Generating non-source parameters at each step achieves the lowest gap in all six columns, and TS-DCPPA (adaptive) alone beats all three restricted controls in every column. This confirms that dynamic non-source parameter generation brings gains beyond both a static ϕ (vs. fixed parameters) and discrete preset switching (vs. per-instance and per-step presets), and shows that better Stage-2 search allocation can further unlock the benefit of non-source parameters, which is a promising direction for future study. We have added these discussions on Stage-2 policy search to the appendix.