- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes EDGE, a Bayesian optimization framework for high-dimensional mixed-variable search spaces. EDGE uses a frozen, differentiable tabular foundation model, with TabPFN as the default surrogate, and constructs search directions from an empirical second-moment matrix of acquisition-function gradients. A Marchenko-Pastur-referenced spectral rule selects the subspace rank and assigns overlap-inspired direction weights, with a random-sketch fallback when no directions are retained. Candidates are generated within a trust region and refined using optimal-transport probabilistic reparameterization, combining ground-cost smoothing with graph-transport natural-gradient updates for discrete variables. The paper provides conditional theoretical analyses and evaluates EDGE on 32 benchmarks spanning 3-450 dimensions. EDGE achieves the best overall average rank among the compared methods, leading on engineering and hyperparameter optimization tasks, while Bounce performs best on synthetic benchmarks. Additional ablations and combinatorial studies examine component contributions and encoding sensitivity.
Critical strengths
- Using acquisition-function gradients rather than predictive-mean gradients is well motivated, allowing search directions to account for both predicted objective values and uncertainty.
- OTPR incorporates discrete-space geometry into probability refinement, with ablations supporting its advantage over standard PR on the five evaluated tasks.
- The empirical evaluation is extensive, covering 32 benchmarks with 10 random seeds per problem and demonstrating competitive performance, including the best overall average rank.
Critical weaknesses
I believe this paper only have one core weakness, but it is for sure very critical: the whole workflow and ideology is very similar to GIT-BO (ICLR26), which is also mentioned in this paper. Both methods use TabPFN as a surrogate for high-dimensional BO and estimate search directions from gradient outer-product matrices (predictive-mean gradients in GIT-BO and acquisition-function gradients in EDGE). EDGE introduces several tricks to subspace construction, rank selection, candidate generation, and discrete decision refinement, with OTPR being one of the most substantive extensions. These incremental contributions could be compelling if they delivered substantial improvements over GIT-BO. However, although the paper cites GIT-BO, it does not include it as an experimental baseline. Consequently, it is difficult to determine whether the reported performance gains primarily arise from the existing GIT-BO approach (combining a TabPFN surrogate with gradient-based subspace estimation) or from EDGE’s proposed modifications.
Questions for authors
Why is GIT-BO not a compared baseline method?
6 comments
2. Additional Ablations:
We conducted two additional ablations for EDGE per request from the other reviewers. Here we presented the additional results:
a. Ablation on rank-selection of EDGE:
A separate rank-selection ablation also favors EDGE’s MP rule over the 95% explained-variance criterion (Appendix B.2, Figure 7).
Ablation on rank-selection of EDGE:
| Method variants | Avg. normalized regret (mean ± SEM over problems) | Avg. rank |
|---|---|---|
| EDGE | 0.0433 ± 0.0134 | 1.688 |
| EDGE (without MP) fixed r=10 | 0.0714 ± 0.0170 | 2.094 |
| EDGE (without MP) 95% variance | 0.0642 ± 0.0166 | 2.219 |
b. Ablation on ten additional problems than the original 32 problems:
Moreover, on ten additional problems, EDGE outperforms GIT-BO on all ten and GIT-BO+OTPR on eight (Appendix B.8, Figure 14–15). Combined, over 42 problems, EDGE ranks first among all baselines, including GIT-BO and GIT-BO+OTPR. These results support contributions from transport-aware refinement and EDGE’s remaining design choices beyond the shared GIT-BO foundation.
Average ranks of the ablation on ten additional problems:
| Method | Held-out ten problems |
|---|---|
| EDGE | 1.20 |
| GIT-BO+OTPR | 3.80 |
| Bounce | 4.00 |
| SMAC | 4.50 |
| BoDi | 4.60 |
| Casmopolitan | 5.70 |
| PR | 5.70 |
| GIT-BO | 6.70 |
| Random | 8.80 |
Average ranks with GIT-BO and GIT-BO+OTPR on all 42 problems (32-suite + 10 held-out):
| Method | All | Applied Sci. & Eng. | HPO | Synthetic |
|---|---|---|---|---|
| EDGE | 2.27 | 1.89 | 1.75 | 3.60 |
| Bounce | 3.62 | 4.33 | 3.75 | 2.20 |
| GIT-BO+OTPR | 3.79 | 3.58 | 4.00 | 3.90 |
| SMAC | 4.53 | 4.89 | 2.33 | 6.50 |
| BoDi | 4.81 | 5.36 | 5.25 | 3.30 |
| Casmopolitan | 5.45 | 4.50 | 7.58 | 4.60 |
| PR | 5.66 | 6.03 | 6.00 | 4.60 |
| GIT-BO | 6.31 | 5.69 | 6.17 | 7.60 |
| Random | 8.55 | 8.72 | 8.17 | 8.70 |
- Note: GIT-BO uses no probabilistic refinement. GIT-BO+PR and GIT-BO+OTPR add standard PR and transport-aware refinement, respectively. The GIT-BO entry in this table is not GIT-BO+PR.
We hope our responses and new experimental results clarify EDGE’s contributions to mixed-variable BO. If they address your concerns, we kindly ask you to consider raising your score. Thank you.
Dear Reviewer EU79,
As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.
Thanks!
Reviewer:
Authors:
Thank you for recognizing OTPR’s geometry-aware refinement and its empirical benefits. The controlled ablation (Appendix B.5) supports the benefit of incorporating discrete-space geometry into refinement.
Our new experiment results below also show that OTPR improves the average rank of both EDGE and adapted GIT-BO (Appendix B.7), supporting the effectiveness of the combined refinement design. Below we also present our new ablation showing that comparing standard PR and OTPR within both frameworks shows that OTPR improves both, with EDGE+OTPR ranking best (Appendix B.7, Figure 13).
Average ranks of the ablation on probabilistic refinement within EDGE and the adapted GIT-BO on all 32 problems (EDGE (OTPR) ≡ EDGE):
Reviewer:
Authors:
We appreciate the reviewer’s acknowledgment of our extensive benchmarking across 32 problems with ten seeds per problem. EDGE achieves the best average rank overall and on the engineering and HPO subsets. To answer
Reviewer reoz’s concern of generalizability outside of the set of 32 problems, we furthermore added 10 more problems from existing literature to evaluate EDGE in Appendix B.8. This ablation shows these top-ranked performances of EDGE persist even after adding adapted GIT-BO and GIT-BO+OTPR as the baseline.Reviewer:
Authors:
Thank you for raising this important comparison. GIT-BO was developed for continuous optimization and does not directly handle mixed-variable domains, which motivated its omission from the original baseline set. We agree, however, that an adapted comparison is valuable for assessing EDGE’s contributions beyond the shared TabPFN surrogate and gradient-informed search.
1. Benchmarked against GIT-BO
To address this, we adapted GIT-BO using its 95% explained-variance rank-selection rule and evaluated both GIT-BO and GIT-BO+OTPR across all 32 problems. We show the results in the following table and in Appendix B.7, Figures 11–12 of the revised submission. EDGE retains the best average rank overall and on the engineering and HPO subsets.
Average ranks with GIT-BO on all 32 problems:
Average ranks with GIT-BO+OTPR on all 32 problems: