- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces a framework for learning fitness landscape representations from limited function evaluations. During pretraining, it combines multi-view alignment with guidance from higher-budget observations, function structure, and optimizer behavior. Only the frozen encoder is needed at deployment. The method is evaluated on the BBOB benchmark.
Critical strengths
The training setup is well motivated for black-box optimization, where function evaluations can be expensive. PGFL uses richer information offline while keeping the evaluation budget low on new problems. The algorithm-selection results also include the sampling cost, which makes the reported gains more meaningful.
Critical weaknesses
-
The ablations do not sufficiently separate the paper’s main methodological contribution from the other design choices. The overall benefit of privileged guidance is not directly isolated. Table 4 removes one component at a time, but every variant still uses some privileged supervision. A comparison using the same features, encoder, and training data with multi-view learning alone is missing. This matters because removing multi-view alignment causes the largest performance drop.
-
Both downstream tasks train and test on different instances of the same 24 BBOB functions. Recognizing familiar function types may already help predict their landscape labels and solver performance. It remains unclear whether these results carry over to unseen function types.
Questions for authors
Q1. How much does privileged guidance add over the same model trained with multi-view alignment alone?
Q2. How does PGFL perform when entire function types are held out?
Q3. Which Deep-ELA model variants and pretrained checkpoints were used for the 25d and 50d comparisons, and how were they selected?
4 comments
Weakness 2 · Limited evaluation of generalization
We agree that the original instance-based split provides limited evidence for generalization to unseen function types. We chose BBOB as a standard ELA benchmark with established landscape labels, standardized protocols, and extensive public optimizer records, enabling comparisons with existing methods.
To broaden the evaluation, we added an algorithm-selection experiment on UAV path planning in 6, 12, and 30 dimensions, using 400 problem instances per dimension and eight candidate optimizers. The PGFL and Deep-ELA encoders remained frozen, and the selectors, including PCT, were fitted using only the training folds. We report normalized fixed-budget objective scores (×100; lower is better), summarized as mean ± SD across ten sampling repeats.
| D | SBS | ELA 25d RF | ELA 25d kNN | ELA 50d RF | ELA 50d kNN | Deep-ELA Large 25d RF | Deep-ELA Large 25d kNN | Deep-ELA Large 50d RF | Deep-ELA Large 50d kNN | PCT (100) | PGFL RF | PGFL kNN |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 6 | 7.42 ± 0.00 | 7.93 ± 0.35 | 9.58 ± 0.67 | 7.70 ± 0.47 | 9.65 ± 0.63 | 7.09 ± 0.32 | 9.98 ± 0.85 | 7.12 ± 0.31 | 9.99 ± 0.56 | 6.87 ± 0.48 | 6.69 ± 0.75 | 9.75 ± 0.32 |
| 12 | 9.04 ± 0.00 | 9.96 ± 0.38 | 13.36 ± 0.81 | 9.71 ± 0.37 | 12.67 ± 1.46 | - | - | - | - | 10.13 ± 0.35 | 9.42 ± 0.36 | 12.61 ± 0.36 |
| 30 | 6.20 ± 0.00 | 6.20 ± 0.00 | 7.26 ± 0.23 | 6.20 ± 0.00 | 7.15 ± 0.30 | - | - | - | - | 6.50 ± 0.18 | 5.67 ± 0.15 | 7.13 ± 0.21 |
PGFL uses observations, averaging results from three pretrained encoder seeds within each sampling repeat. “-” denotes unsupported Deep-ELA dimensions.
PGFL remains competitive with the evaluated methods on this application benchmark, providing additional evidence for transferring representations pretrained on generated functions to applications beyond BBOB.
Appendix E.1 and E.4 already discuss dependence on the pretraining distribution and the limited empirical scope. We will clarify these boundaries in the main text. Future work will broaden pretraining through more diverse function generators and application-derived optimization traces, and further evaluate higher-dimensional, noisy, constrained, and mixed-variable problems.
I thank the authors for their response, which has addressed my concerns. I am happy to maintain my positive score.
Thank you very much for the reply. We also appreciate your recognition of our work and your constructive comments, which further improve our paper. Thank you for your contribution to the research community.
Thank you for your comments and suggestions on our paper. We address each question in turn below.
Q1: The multi-view-only baseline uses the same input descriptors and encoder architecture as PGFL. It retains both alignment objectives and disables all three privileged objectives.
The comparison supports the added benefit of privileged guidance for both HLP and AAS.
Q2: We evaluated HLP and AAS using five folds grouped by BBOB function ID. In each fold, we used the frozen PGFL encoder to extract features and fitted the downstream models using only the training function types. For HLP, we used instances 0–99 of the training types and instances 100–119 of the held-out types. For AAS, we used the available instances of the respective types. PGFL and ELA used the same folds and a sampling budget of 25d. The original PGFL results from Table 4 are included for reference.
Under function-type holdout, PGFL outperforms ELA on both HLP and AAS.
HLP is harder under this split because the classifiers must predict landscape properties for unseen function types. In particular, some Global labels occur in only one BBOB function type, so the corresponding training fold has no examples of those labels when that type is held out.
Q3: We selected Deep-ELA Large to use the same model variant across all four BBOB dimensions in our main evaluation (d∈{2,3,5,10}). Deep-ELA's dimensionality limit counts both decision variables and objectives (Seiler et al., Sections 3.2.1 and 5.1). For a 10-dimensional single-objective problem, d+m=10+1=11, which exceeds Medium's limit of 6 and fits within Large's limit of 12. We used the released large_25d_v1 and large_50d_v1 checkpoints as frozen feature extractors for the 25d and 50d comparisons, respectively.