acceptodds
Under review as a conference paper at ICLR 2027

EDGE: High-Dimensional Mixed-Variable Bayesian Optimization with Eigen-Detected Gradient Embeddings

Abstract

Hyperparameter optimization and physical-system design often require searching high-dimensional, mixed-variable spaces, yet expensive evaluations leave little room for wasted exploration. Bayesian optimization (BO) handles the expense, but existing subspace methods often rely on random embeddings or fix the number of gradient-informed directions without testing whether they carry signal. We propose EDGE, a high-dimensional and mixed-variable BO algorithm that uses a differentiable tabular foundation model (TFM) as the surrogate and thus allows us to ask the model itself which directions of the search space matter. EDGE combines parameter-free detection of informative search directions with optimal-transport probabilistic reparameterization for geometry-aware optimization of discrete variables. We evaluate EDGE on 52 mixed-variable problems spanning 3 450 dimensions under a budget of 300 evaluations after initialization. Across these diverse benchmarks, EDGE achieves the best average rank among eight methods. EDGE’s advantage is strongest on engineering problems, but does not extend to the separately evaluated purely combinatorial benchmarks.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Bayesian Optimization, Tabular Foundation Model, Mixed-Variable, Optimal Transport, Random Matrix Theory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EU79OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EDGE, a Bayesian optimization framework for high-dimensional mixed-variable search spaces. EDGE uses a frozen, differentiable tabular foundation model, with TabPFN as the default surrogate, and constructs search directions from an empirical second-moment matrix of acquisition-function gradients. A Marchenko-Pastur-referenced spectral rule selects the subspace rank and assigns overlap-inspired direction weights, with a random-sketch fallback when no directions are retained. Candidates are generated within a trust region and refined using optimal-transport probabilistic reparameterization, combining ground-cost smoothing with graph-transport natural-gradient updates for discrete variables. The paper provides conditional theoretical analyses and evaluates EDGE on 32 benchmarks spanning 3-450 dimensions. EDGE achieves the best overall average rank among the compared methods, leading on engineering and hyperparameter optimization tasks, while Bounce performs best on synthetic benchmarks. Additional ablations and combinatorial studies examine component contributions and encoding sensitivity.

Critical strengths
  1. Using acquisition-function gradients rather than predictive-mean gradients is well motivated, allowing search directions to account for both predicted objective values and uncertainty.
  2. OTPR incorporates discrete-space geometry into probability refinement, with ablations supporting its advantage over standard PR on the five evaluated tasks.
  3. The empirical evaluation is extensive, covering 32 benchmarks with 10 random seeds per problem and demonstrating competitive performance, including the best overall average rank.
Critical weaknesses

I believe this paper only have one core weakness, but it is for sure very critical: the whole workflow and ideology is very similar to GIT-BO (ICLR26), which is also mentioned in this paper. Both methods use TabPFN as a surrogate for high-dimensional BO and estimate search directions from gradient outer-product matrices (predictive-mean gradients in GIT-BO and acquisition-function gradients in EDGE). EDGE introduces several tricks to subspace construction, rank selection, candidate generation, and discrete decision refinement, with OTPR being one of the most substantive extensions. These incremental contributions could be compelling if they delivered substantial improvements over GIT-BO. However, although the paper cites GIT-BO, it does not include it as an experimental baseline. Consequently, it is difficult to determine whether the reported performance gains primarily arise from the existing GIT-BO approach (combining a TabPFN surrogate with gradient-based subspace estimation) or from EDGE’s proposed modifications.

Questions for authors

Why is GIT-BO not a compared baseline method?

6 comments

Reviewer:

OTPR incorporates discrete-space geometry into probability refinement, with ablations supporting its advantage over standard PR on the five evaluated tasks.

Authors:

Thank you for recognizing OTPR’s geometry-aware refinement and its empirical benefits. The controlled ablation (Appendix B.5) supports the benefit of incorporating discrete-space geometry into refinement.

Our new experiment results below also show that OTPR improves the average rank of both EDGE and adapted GIT-BO (Appendix B.7), supporting the effectiveness of the combined refinement design. Below we also present our new ablation showing that comparing standard PR and OTPR within both frameworks shows that OTPR improves both, with EDGE+OTPR ranking best (Appendix B.7, Figure 13).

Average ranks of the ablation on probabilistic refinement within EDGE and the adapted GIT-BO on all 32 problems (EDGE (OTPR) ≡ EDGE):

MethodAverage ranks (All)
EDGE (OTPR)2.18
GIT-BO+OTPR2.53
GIT-BO+PR2.61
EDGE (PR)2.68

Reviewer:

The empirical evaluation is extensive, covering 32 benchmarks with 10 random seeds per problem and demonstrating competitive performance, including the best overall average rank.

Authors:

We appreciate the reviewer’s acknowledgment of our extensive benchmarking across 32 problems with ten seeds per problem. EDGE achieves the best average rank overall and on the engineering and HPO subsets. To answer Reviewer reoz’s concern of generalizability outside of the set of 32 problems, we furthermore added 10 more problems from existing literature to evaluate EDGE in Appendix B.8. This ablation shows these top-ranked performances of EDGE persist even after adding adapted GIT-BO and GIT-BO+OTPR as the baseline.

Reviewer:

Why is GIT-BO not a compared baseline method?

Although the paper cites GIT-BO, it does not include it as an experimental baseline. Consequently, it is difficult to determine whether the reported performance gains primarily arise from the existing GIT-BO approach (combining a TabPFN surrogate with gradient-based subspace estimation) or from EDGE’s proposed modifications.

Authors:

Thank you for raising this important comparison. GIT-BO was developed for continuous optimization and does not directly handle mixed-variable domains, which motivated its omission from the original baseline set. We agree, however, that an adapted comparison is valuable for assessing EDGE’s contributions beyond the shared TabPFN surrogate and gradient-informed search.

1. Benchmarked against GIT-BO
To address this, we adapted GIT-BO using its 95% explained-variance rank-selection rule and evaluated both GIT-BO and GIT-BO+OTPR across all 32 problems. We show the results in the following table and in Appendix B.7, Figures 11–12 of the revised submission. EDGE retains the best average rank overall and on the engineering and HPO subsets.

Average ranks with GIT-BO on all 32 problems:

MethodAllApplied Sci. & Eng.HPOSynthetic
EDGE2.371.861.894.00
Bounce3.073.643.331.57
SMAC3.934.432.005.43
BoDi4.254.684.443.14
Casmopolitan4.673.647.003.71
PR4.875.295.223.57
GIT-BO5.384.685.227.00
Random7.477.796.897.57

Average ranks with GIT-BO+OTPR on all 32 problems:

MethodAllApplied Sci. & Eng.HPOSynthetic
EDGE2.642.081.894.57
Bounce3.253.833.781.57
GIT-BO+OTPR3.663.463.564.14
SMAC4.214.502.226.29
BoDi4.434.754.893.29
Casmopolitan5.114.087.114.29
PR5.205.545.564.14
Random7.507.757.007.71
  • Note: GIT-BO uses no probabilistic refinement. GIT-BO+PR and GIT-BO+OTPR add standard PR and transport-aware refinement, respectively. The GIT-BO entry in this table is not GIT-BO+PR.

2. Additional Ablations:

We conducted two additional ablations for EDGE per request from the other reviewers. Here we presented the additional results:

a. Ablation on rank-selection of EDGE:

A separate rank-selection ablation also favors EDGE’s MP rule over the 95% explained-variance criterion (Appendix B.2, Figure 7).

Ablation on rank-selection of EDGE:

Method variantsAvg. normalized regret (mean ± SEM over problems)Avg. rank
EDGE0.0433 ± 0.01341.688
EDGE (without MP) fixed r=100.0714 ± 0.01702.094
EDGE (without MP) 95% variance0.0642 ± 0.01662.219

b. Ablation on ten additional problems than the original 32 problems:

Moreover, on ten additional problems, EDGE outperforms GIT-BO on all ten and GIT-BO+OTPR on eight (Appendix B.8, Figure 14–15). Combined, over 42 problems, EDGE ranks first among all baselines, including GIT-BO and GIT-BO+OTPR. These results support contributions from transport-aware refinement and EDGE’s remaining design choices beyond the shared GIT-BO foundation.

Average ranks of the ablation on ten additional problems:

MethodHeld-out ten problems
EDGE1.20
GIT-BO+OTPR3.80
Bounce4.00
SMAC4.50
BoDi4.60
Casmopolitan5.70
PR5.70
GIT-BO6.70
Random8.80

Average ranks with GIT-BO and GIT-BO+OTPR on all 42 problems (32-suite + 10 held-out):

MethodAllApplied Sci. & Eng.HPOSynthetic
EDGE2.271.891.753.60
Bounce3.624.333.752.20
GIT-BO+OTPR3.793.584.003.90
SMAC4.534.892.336.50
BoDi4.815.365.253.30
Casmopolitan5.454.507.584.60
PR5.666.036.004.60
GIT-BO6.315.696.177.60
Random8.558.728.178.70
  • Note: GIT-BO uses no probabilistic refinement. GIT-BO+PR and GIT-BO+OTPR add standard PR and transport-aware refinement, respectively. The GIT-BO entry in this table is not GIT-BO+PR.

We hope our responses and new experimental results clarify EDGE’s contributions to mixed-variable BO. If they address your concerns, we kindly ask you to consider raising your score. Thank you.

Dear Reviewer EU79,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Reviewer_CLm9OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses tabular foundation models to solve high-dimensional mixed-variable Bayesian optimization. EDGE uses a similar approach like GIT-BO, which uses TabPFN as surrogate model and extracts the acquisition gradient subspace to identify important update direction. Experiments on a wide range of tasks show the good performance of EDGE.

References

[1] GIT-BO: High-Dimensional Bayesian Optimization with Tabular Foundation Models. https://arxiv.org/abs/2505.20685

Critical strengths

The evaluation is at scale, covering a wide range of tasks, and EDGE is combined with multiple mixed-space BO, showing tranferrable empirical advantages. The ablation studies are adequate.

Critical weaknesses
  1. The thought behind EDGE is quite similar to GIT-BO. Though GIT-BO is developed for continuous domains, we can still compare GIT-BO by encoding discrete variables and differentiating a continuous relaxation, just like EDGE, right? Or we can combine GIT-BO style candidate generation with standard probabilistic reparameterization. Only this can strengthen the effectiveness of EDGE.
  2. Robustness to discrete representations remain limited.
Questions for authors
  1. How does MP-based selection compare with explained-variance selection, as in GIT-BO?
  2. What is the running time?
13 comments

Reviewer:

The evaluation is at scale, covering a wide range of tasks, and EDGE is combined with multiple mixed-space BO, showing transferable empirical advantages.

Authors:

Thank you for recognizing the breadth of our evaluation and the transferability of EDGE’s components. Our new ablations further strengthen this evidence: OTPR improves both EDGE and adapted GIT-BO, while EDGE retains the best overall average rank (Appendix B.7, Figures 11–13).

Reviewer:

The thought behind EDGE is quite similar to GIT-BO. Though GIT-BO is developed for continuous domains, we can still compare GIT-BO by encoding discrete variables and differentiating a continuous relaxation, just like EDGE, right? Or we can combine GIT-BO style candidate generation with standard probabilistic reparameterization. Only this can strengthen the effectiveness of EDGE.

Authors:

Yes, we agree that GIT-BO can be adapted in this way, and we thank the reviewer for suggesting this comparison. We have now evaluated adapted GIT-BO across all 32 problems, including combinations of GIT-BO-style candidate generation with standard PR and OTPR (Appendix B.7, Figures 11–13). EDGE retains the best average rank overall and on the engineering and HPO subsets, including against GIT-BO+OTPR. The refinement ablation shows that OTPR improves both frameworks, with EDGE+OTPR ranking best among the four configurations.

Average ranks of the ablation on probabilistic refinement within EDGE and the adapted GIT-BO on all 32 problems (EDGE (OTPR) ≡ EDGE):

MethodAverage ranks (All)
EDGE (OTPR)2.18
GIT-BO+OTPR2.53
GIT-BO+PR2.61
EDGE (PR)2.68

Average ranks with GIT-BO on all 32 problems:

MethodAllApplied Sci. & Eng.HPOSynthetic
EDGE2.371.861.894.00
Bounce3.073.643.331.57
SMAC3.934.432.005.43
BoDi4.254.684.443.14
Casmopolitan4.673.647.003.71
PR4.875.295.223.57
GIT-BO5.384.685.227.00
Random7.477.796.897.57

Average ranks with GIT-BO+OTPR on all 32 problems:

MethodAllApplied Sci. & Eng.HPOSynthetic
EDGE2.642.081.894.57
Bounce3.253.833.781.57
GIT-BO+OTPR3.663.463.564.14
SMAC4.214.502.226.29
BoDi4.434.754.893.29
Casmopolitan5.114.087.114.29
PR5.205.545.564.14
Random7.507.757.007.71

These comparisons support the benefit of OTPR and additional gains from EDGE’s remaining design choices beyond the shared TabPFN and gradient-informed search foundation.

Reviewer:

Robustness to discrete representations remain limited.

Authors:

We agree that robustness to equivalent discrete representations remains a limitation. Our relabeling experiments preserve the underlying optimization problem while changing its input encoding. On cluster expansion, EDGE’s mean final objective decreases from 1765 to 884 after relabeling, whereas Bounce retains 1765 on both encodings (Appendix E.1, Table 5).

The same table shows that this is a property of every method whose candidate generation reads the integer codes: PR degrades from 960 to 564 and Casmopolitan from 1566 to 1520, whereas the random-embedding methods Bounce and BODi retain 1765 on both encodings. We explicitly discuss this sensitivity in Section 5 and the limitations paragraph. These results do not isolate which pipeline component causes the degradation, so we do not claim representation invariance. Encoding-aware candidate generation, for EDGE and for the other encoding-dependent methods, remains an important direction for future work.

Reviewer:

How does MP-based selection compare with explained-variance selection, as in GIT-BO?

Authors:

Thank you for suggesting this direct comparison. We added an ablation comparing EDGE’s MP-based selection with GIT-BO’s 95% explained-variance rule and a fixed rank of ten across all 32 problems (Appendix B.2, Figure 7). MP-based selection achieves the lowest final average normalized regret and the best final average rank among the three configurations, supporting its use over variance-based adaptation on this suite.

Ablation on rank-selection of EDGE:

Method variantsAvg. normalized regret (mean ± SEM over problems)Avg. rank
EDGE0.0433 ± 0.01341.688
EDGE (without MP) fixed r=100.0714 ± 0.01702.094
EDGE (without MP) 95% variance0.0642 ± 0.01662.219

Reviewer:

What is the running time?

Authors:

We report the runtime comparison in Appendix C.2, Table 4. EDGE’s mean per-iteration runtime is 26.20 seconds, with a median of 27.05 seconds and a standard deviation of 4.61 seconds. The corresponding means are 3.92 seconds for Bounce and 37.84 seconds for BODi. As stated in the main-text limitations, whether EDGE’s additional computation is worthwhile depends on objective-evaluation cost and the resulting solution quality.

Reviewer:

The ablation studies are adequate.

Authors:

Thank you for acknowledging the ablation studies. Our goal was to assess EDGE’s individual design choices alongside its aggregate performance. In addition to the rank-selection and GIT-BO comparisons, per the request of other reviewers, we benchmark EDGE on ten additional problems to further validate its generalizability. We found that EDGE ranks the top and outperforms GIT-BO on all ten and GIT-BO+OTPR on eight (Appendix B.8, Figure 14–15). These results support contributions from transport-aware refinement and EDGE’s remaining design choices beyond the shared GIT-BO foundation. Combined, over 42 problems, EDGE ranks first among all baselines, including GIT-BO and GIT-BO+OTPR.

Average ranks of the ablation on ten additional problems:

MethodHeld-out ten problems
EDGE1.20
GIT-BO+OTPR3.80
Bounce4.00
SMAC4.50
BoDi4.60
Casmopolitan5.70
PR5.70
GIT-BO6.70
Random8.80

Average ranks with GIT-BO and GIT-BO+OTPR on all 42 problems (32-suite + 10 held-out):

MethodAllApplied Sci. & Eng.HPOSynthetic
EDGE2.271.891.753.60
Bounce3.624.333.752.20
GIT-BO+OTPR3.793.584.003.90
SMAC4.534.892.336.50
BoDi4.815.365.253.30
Casmopolitan5.454.507.584.60
PR5.666.036.004.60
GIT-BO6.315.696.177.60
Random8.558.728.178.70

We hope our responses and new experimental results clarify EDGE’s contributions to mixed-variable BO. If they address your concerns, we kindly ask you to consider raising your score. Thank you.

Dear Reviewer CLm9,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Reviewer_reozOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This manuscript introduces a Bayesian Optimization (BO) method called EDGE that aims to better optimize blackbox functions of high dimension. EDGE uses tabular foundation models (TFMs) as the BO surrogate function to identify which dimensions are the most important to optimize. EDGE supports mixed-variable Bayesian Optimization.

The method is evaluated on 32 problems with 3-450 dimensions under a fixed budget of number of evaluations and compared with several other approaches, EDGE achieves the best average ranking overall and across several subsets of the problems. A large number of additional experiments (ablations) are also presented.

Critical strengths
  1. The manuscript conducts experiments against a reasonable set of baselines. The number of problem data sets used is reasonable and includes a variety of relatively high (<100)-dimensional probles. The experiments also include reasonable set of ablation studies are performed to isolate fallback-dimension, optimal transport PR, adaptive rank selection, the choice of acquisition family and surrogate model (not “ablations”, just verification of an algorithm design choices), and other tests.

  2. The empirical results are quite good across the test set. While EDGE isn’t always the best, it very frequently does find the best solution, and the cases where it doesn’t are typically the synthetic problems.

Critical weaknesses
  1. The method and the discussion of the insights derived from the results are difficult to understand and unpack. It is not clear how TabPFN is used or what the overall methodological approach is. Algorithm 1 does give pseudocode, but this is not well explained. The novelty of various ideas is not identified. Variants are referred to by various names inconsistently. Many statements are so compressed as to not be easily decodable (to include one example just to be clear: “Finally, using fallback dimension d/4 improves the mean outcome on seven of ten ablation tasks, indicating scope for adaptive fallback settings (Appendix B.4).” [is “indicating scope” a way of saying “where it works?”] It is not easy to draw out clear, separatable ideas and insights.

  2. Discussion of algorithmic overfitting. There is no discussion about how tuned the algorithmic choices are to this suite of 32 problems. Was the algorithm optimized (by hand or otherwise) using this same set?

Questions for authors
  1. What problem is used in Figure 6?
6 comments

Reviewer:

The number of problem data sets used is reasonable and includes a variety of relatively high (<100)-dimensional problems. The experiments also include a reasonable set of ablation studies.

Authors:

Thank you for recognizing the breadth of our benchmark suite and the ablation studies. We have further strengthened the evaluation with direct comparisons against explained-variance rank selection and adapted GIT-BO, together with ten additional benchmark problems beyond the original suite (Appendices B.2, B.7, and B.8).

Reviewer:

The empirical results are quite good across the test set. While EDGE isn’t always the best, it very frequently does find the best solution, and the cases where it doesn’t are typically the synthetic problems.

Authors:

We appreciate the reviewer’s positive assessment of EDGE’s empirical results. We have also clearly stated its limitations, recognizing that strong overall performance does not imply that EDGE is the best method on every problem.

Reviewer:

The method and the discussion of the insights derived from the results are difficult to understand and unpack. It is not clear how TabPFN is used or what the overall methodological approach is. Algorithm 1 does give pseudocode, but this is not well explained. The novelty of various ideas is not identified. Variants are referred to by various names inconsistently.

Authors:

Thank you for identifying these presentation issues. We have edited the Abstract, Introduction, and Algorithm 1 to explain EDGE’s pipeline and distinguish its contributions from the shared GIT-BO foundation. TabPFN remains frozen and is conditioned on the observed evaluations in the original input space. Its predictions define the acquisition function, the acquisition gradients then guide candidate initialization, and OTPR refines discrete choices within the trust region.

The revised introduction separately identifies MP-based direction selection and transport-aware probabilistic refinement, while Algorithm 1 highlights the steps that differ from GIT-BO. We will also standardize variant names throughout the text and figures, using EDGE for the default configuration and explicitly naming each modified component in the Result and Discussion session upon acceptance.

Reviewer:

Many statements are so compressed as to not be easily decodable (to include one example just to be clear: “Finally, using fallback dimension d/4 improves the mean outcome on seven of ten ablation tasks, indicating scope for adaptive fallback settings (Appendix B.4).” [is “indicating scope” a way of saying “where it works?”]

Authors:

We agree that this statement was overly compressed as we allocated most space to having extensive benchmarking results in the main paper. To help with readability, we have moved some results to the appendix and expanded on the sentences. Here, d denotes the original input dimension, and the comparison increases the random-sketch fallback dimension from d/5 to d/4. This change improves the mean final objective on seven of ten problems, but no single tested dimension performs best throughout. We will state these observations directly and distinguish them from the untested possibility of choosing the fallback dimension adaptively.

Reviewer:

Discussion of algorithmic overfitting. Was the algorithm optimized (by hand or otherwise) using this same set?

Authors:

Thank you for raising this point. By creating a large collection of problems to benchmark our method (to the best of our knowledge, this is the largest set of problems that other papers have benchmarked on) and taking special care to follow good benchmarking practices, we were confident that the method was not overfitting. However, inspired by a reviewer comment, we collected ten more problems. Appendix B.8 now evaluates EDGE on ten additional benchmark problems beyond the original suite and ranks the first. EDGE outperforms all original baselines on nine of ten problems, adapted GIT-BO on all ten, and GIT-BO+OTPR on eight. Combined, over 42 problems, EDGE ranks first among all baselines, including GIT-BO and GIT-BO+OTPR. This supports generalization beyond the initial benchmark selection.

Average ranks of the ablation on ten additional problems:

MethodHeld-out ten problems
EDGE1.20
GIT-BO+OTPR3.80
Bounce4.00
SMAC4.50
BoDi4.60
Casmopolitan5.70
PR5.70
GIT-BO6.70
Random8.80

Average ranks with GIT-BO and GIT-BO+OTPR on all 42 problems (32-suite + 10 held-out):

MethodAllApplied Sci. & Eng.HPOSynthetic
EDGE2.271.891.753.60
Bounce3.624.333.752.20
GIT-BO+OTPR3.793.584.003.90
SMAC4.534.892.336.50
BoDi4.815.365.253.30
Casmopolitan5.454.507.584.60
PR5.666.036.004.60
GIT-BO6.315.696.177.60
Random8.558.728.178.70

Average ranks of the ablation on probabilistic refinement within EDGE and the adapted GIT-BO on all 32 problems (EDGE (OTPR) ≡ EDGE):

MethodAverage ranks (All)
EDGE (OTPR)2.18
GIT-BO+OTPR2.53
GIT-BO+PR2.61
EDGE (PR)2.68

Reviewer:

What problem is used in Figure 6?

Authors:

Thank you for pointing out this ambiguity in writing. Figure 6 aggregates results across all 32 benchmark problems. Panel (a) reports average normalized regret, and panel (b) reports average rank, with each problem weighted equally. The revised Appendix B.2 now explicitly states this scope. We also added Figure 7, comparing MP-based selection, fixed rank ten, and 95% explained-variance selection across the same suite. MP-based selection achieves the lowest final aggregate regret and best final average rank.

Ablation on rank-selection of EDGE:

Method variantsAvg. normalized regret (mean ± SEM over problems)Avg. rank
EDGE0.0433 ± 0.01341.688
EDGE (without MP) fixed r=100.0714 ± 0.01702.094
EDGE (without MP) 95% variance0.0642 ± 0.01662.219

We hope our responses and new experimental results clarify EDGE’s contributions to mixed-variable BO. If they address your concerns, we kindly ask you to consider raising your score. Thank you.

Dear Reviewer reoz,

As the discussion phase comes to an end, please acknowledge the author's response and consider leaving them a comment on whether it has resolved some or all of your concerns.

Thanks!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of our rebuttal

We thank all reviewers for their thoughtful feedback and for recognizing EDGE’s strong empirical performance, benchmark coverage, and supporting ablations. In response to their concerns, we have clarified the methodological presentation and added experiments assessing EDGE’s contributions beyond GIT-BO and its generalization beyond the original benchmark suite. Below, we summarize the principal updates.

  • Methodological clarity and contribution positioning. We revised the Abstract and Introduction around EDGE’s two main contributions: transport-aware probabilistic refinement OTPR and MP-based direction selection. The revised text explains how these components work within the TabPFN search framework, and Algorithm 1 highlights the differences from GIT-BO.

  • Direct comparison with GIT-BO (Appendix B.7, Figures 11–13). While the original GIT-BO was designed for continuous variables, as suggested by reviewers, we adapted GIT-BO to mixed-variable optimization and evaluated combinations with standard PR and OTPR across all 32 problems. EDGE retains the best average rank overall and on the engineering and HPO subsets, including against GIT-BO+OTPR. OTPR improves both frameworks, with EDGE+OTPR ranking best among the four refinement configurations.

  • MP-based versus explained-variance selection (Appendix B.2, Figure 7). We added a comparison against fixed rank ten and GIT-BO’s 95% explained-variance rule. MP-based selection achieves the lowest final average normalized regret and the best final average rank across the suite.

  • Generalization beyond the original suite (Appendix B.8, Figure 14–16). In addition to the 32 problems originally compared, we evaluated ten additional problems from established benchmarks. EDGE achieves lower final regret than all original baselines on nine of ten problems, outperforms GIT-BO on all ten, and outperforms GIT-BO+OTPR on eight. Combined, over 42 problems, EDGE ranks first and wins in 40 problems over GIT-BO (~95% win rate) and 30 problems over GIT-BO+OTPR (~70% win rate).

  • Scope and limitations. We explicitly acknowledge the tradeoff between evaluation efficiency and computational overhead, weaker performance on some synthetic tasks, and sensitivity to equivalent discrete encodings. The additional evaluation supports generalization beyond the original suite without establishing independence from development choices made on that suite.

Together, these updates clarify EDGE’s relationship to prior work and provide direct evidence for the benefits of its proposed components. We appreciate the reviewers’ suggestions, which have helped us strengthen both the presentation and empirical evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.