Optimizing Set-Level Objectives through Structured Control

Many objectives for language model generation are defined over sets of outputs rather than individual responses. We formulate set-level control as two coupled problems: learning a repertoire of controllable behaviors and organizing those behaviors within an output collection. From a common set-level optimization objective, we derive structure-aware credit for sampled modes, selected subsets, and prescribed semantic roles. Using lightweight controls over frozen generators and solvers, we instantiate the framework on fiction distribution matching, demographic regard equalization, and Pareto-frontier coverage. Across all three settings, the learned controls improve the corresponding set-level objectives, showing that collective properties can be optimized without updating the underlying generator or solver.