EDGE: High-Dimensional Mixed-Variable Bayesian Optimization with Eigen-Detected Gradient Embeddings

Hyperparameter optimization and physical-system design often require searching high-dimensional, mixed-variable spaces, yet expensive evaluations leave little room for wasted exploration. Bayesian optimization (BO) handles the expense, but existing subspace methods often rely on random embeddings or fix the number of gradient-informed directions without testing whether they carry signal. We propose EDGE, a high-dimensional and mixed-variable BO algorithm that uses a differentiable tabular foundation model (TFM) as the surrogate and thus allows us to ask the model itself which directions of the search space matter. EDGE combines parameter-free detection of informative search directions with optimal-transport probabilistic reparameterization for geometry-aware optimization of discrete variables. We evaluate EDGE on 52 mixed-variable problems spanning 3 450 dimensions under a budget of 300 evaluations after initialization. Across these diverse benchmarks, EDGE achieves the best average rank among eight methods. EDGE’s advantage is strongest on engineering problems, but does not extend to the separately evaluated purely combinatorial benchmarks.