Which Variables Matter? LLM-Informed Competitive Relevance Learning for High-Dimensional Bayesian Optimization

High-dimensional design problems are common in scientific discovery and formulation optimization, where effective designs must be identified under tight experimental budgets. However, existing high-dimensional BO methods often struggle to uncover useful structural biases from sparse observations alone. To address this challenge, we exploit the pronounced hierarchy of variable relevance and propose CoRe-BO, a framework for LLM-informed competitive relevance learning. Rather than asking the LLM for precise relevance scores or direct kernel modifications, CoRe-BO uses coarse judgments about variable importance to initialize a task-aware relevance prior, with variables competing for a shared sensitivity budget. As experiments accumulate, this structure is continually adapted through bounded LLM reflection and posterior inference, progressively concentrating modeling and search on influential variables. A regret analysis characterizes how the quality of LLM relevance guidance affects search complexity and the cost of imperfect priors. Experiments on six scientific optimization tasks with 29–150 variables show consistent gains in both convergence and final optimization performance over high-dimensional and LLM-assisted BO baselines. Our results suggest that CoRe-BO provides a practical and theoretically grounded approach to integrating LLMs into high-dimensional scientific optimization.