PrefGeo: Preference Geometry for Multi-Preference Fusion of Flow Matching Models

Human preference alignment has emerged as an effective approach for improving flow-matching-based text-to-image generation. However, existing methods primarily focus on aligning models with a single preference, while extending alignment to multiple preferences, such as compositional fidelity, text rendering, and aesthetics, remains challenging. Recent training-based multi-preference alignment methods, including DiffusionOPD, DiffusionNFT, and MapReduce LoRA, require at least 80GB of GPU memory and 170 GPU hours of training, making personalized multi-preference alignment computationally expensive and difficult to scale. In this paper, we introduce PrefGeo, a training-free framework that exploits the preference geometry between preference-aligned models and their shared base model. PrefGeo fuses models aligned with different preferences into a unified model without additional training. The fusion involves only model-weight operations and can be completed on a single NVIDIA RTX 4090 24GB GPU in 18 minutes, using only 0.5GB of GPU memory. Extensive experiments demonstrate that models produced by PrefGeo retain more than $\text{95\%}$ of the performance of their corresponding single-preference models, while eliminating the substantial training costs of existing multi-preference alignment methods.