HER: Hierarchical Evolutionary Reasoning for LLM-driven Algorithmic Optimization

Large language model (LLM)-driven evolutionary agents hold strong promise for algorithmic optimization, yet most existing paradigms still rely on flat population structures and uniform compute allocation. This fundamentally limits their ability to coordinate broad exploration with sustained refinement. To address this, we present HER ($\text{H}$ierarchical $\text{E}$volutionary $\text{R}$easoning). By organizing candidate programs into interacting layers, the framework couples cross-layer population management with bellwether-guided transitions and layer-wise prompt-context selection. Under limited test-time budgets, such a structure enables adaptive compute allocation and more structured search dynamics, allowing promising candidates to guide higher-level refinement while preserving diversity in lower layers. Extensive evaluation across diverse benchmarks, spanning both mathematical and system tasks, shows that HER consistently outperforms state-of-the-art baselines in solution quality, achieving an average relative improvement of 6.67% on mathematical benchmarks and 12.72% on system benchmarks. These results suggest that hierarchical organization is not merely a structural modification, but a promising strategy for improving LLM-driven algorithmic optimization.