Towards LLM-Based Algorithm Design for Heterogeneous Datasets: Decoupled Configuration Policy and Parameterized Algorithm

Automated Algorithm Design (AAD) based on Large Language Models (LLMs) has shown promising results, but existing methods often struggle to generalize across heterogeneous problem instances. Most of these methods either learn a single static algorithm to handle all cases or generate a finite set of algorithms to cover different types of instances. While these strategies can improve performance on seen data, they remain limited to a finite set of learned behaviors and may fail when encountering unseen or mixed cases that do not clearly belong to any known type. In this work, we aim to overcome this limitation by moving beyond selecting from a finite set of algorithms. We propose the decoupled configuration-policy and parameterized-algorithm (DCPPA) structure, in which the algorithm encodes behavioral differences as parameters, and the policy maps observable instance or state features to these parameters. DCPPA represents algorithmic behavior in a continuous, parameterized space, and enables the algorithm to capture gradual variations among diverse instances and smoothly adjust its behavior for cases not well covered by the training data. To construct DCPPA-structured algorithms, we develop TS-DCPPA, a two-stage pipeline. The first stage discovers complementary source algorithms and fuses them into a unified structure. With this structure fixed, the second stage employs a warm-start mechanism and learns a continuous parameter-tuning policy. Experiments on real-world problem instances demonstrate that TS-DCPPA achieves greater robustness and cross-distribution generalization than existing methods.