- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LEMMA for consolidating specialist text-to-image models into a single generator. Rather than applying a global merging weight, it learns layer-specific coefficients for combining specialist task vectors while keeping the base and specialist weights frozen. These coefficients are optimized through multi-objective DPO for reward specialists or pooled-data SFT for style specialists, without adding inference overhead after merging. Experiments with SDXL and SD3 on PartiPrompts and WikiArt report improvements over merging baselines and some individual specialists, complemented by merge-granularity ablations and human preference evaluations.
Critical strengths
- LEMMA offers a simple, parameter-efficient approach to consolidating specialist text-to-image models. By optimizing only layer-specific merging coefficients, it accommodates both reward alignment through DPO and multi-style generation through SFT. The coefficients can then be folded into a single checkpoint without additional inference overhead, making the framework practically useful for reusing existing specialists.
- The empirical gains are supported by complementary forms of evaluation. On SDXL, improvements extend beyond the training rewards to held-out metrics, while the blind user study favors LEMMA over TIES in both the DPO and SFT settings tested. The merge-granularity ablations further support the benefit of finer-grained coefficients over a single global weight per specialist.
Critical weaknesses
- The methodological contribution appears incremental relative to existing learnable merging methods. Equation (9) uses the same layer-wise task-vector combination as AdaMerging, while ZipLoRA already learns merging coefficients for text-to-image diffusion models with frozen source weights. Although ZipLoRA addresses subject–style LoRAs rather than full-checkpoint reward specialists, it contradicts the paper’s broader positioning of learnable merging as confined to discriminative or autoregressive models. LEMMA’s main distinction is applying DPO and pooled-style SFT to optimize these coefficients, with reward calibration and frontier-based pair selection inherited from CaPO. This is a potentially useful extension, but the paper does not sufficiently distinguish a new merging mechanism from applying established coefficient optimization to different training objectives. The first-of-its-kind framing therefore overstates the methodological advance.
- The evaluation does not sufficiently establish the claimed multi-objective advantage. The main merging baselines are static methods, whereas LEMMA receives additional DPO or SFT supervision. The Mixture baseline and merge-granularity ablations are useful, but they do not establish an advantage over competitive multi-reward optimization such as CaPO under comparable data and training budgets. The claim that LEMMA resolves objective conflicts is also stronger than the supporting analysis: Section 4.2 infers that Pareto-dominant training pairs make the gradient improve all rewards jointly, although optimizing their relative likelihood does not ensure improvement in every expected reward. The results likewise require a clear distinction between improving over the pretrained model and retaining specialist performance: LEMMA’s SDXL VILA score is 6.13 versus the specialist’s 6.21, while its SD3 PickScore is 22.67 versus TIES’s 22.70. These exceptions do not negate the reported gains, but stronger comparisons and a more precise treatment of the remaining trade-offs are needed to support the central multi-objective claim.
Questions for authors
- Novelty beyond existing learnable merging. Could the authors clarify what is new in LEMMA’s merging mechanism beyond AdaMerging’s layer-wise task-vector parameterization and ZipLoRA’s learned diffusion-model merging? In particular, is the main contribution the use of DPO/SFT supervision for full-checkpoint specialists, or is there an additional methodological distinction that is not clear from the manuscript?
- Comparisons under matched supervision. How does LEMMA compare with CaPO using the same backbone, preference data, and reward models under comparable optimization budgets? Beyond the existing granularity ablations, what evidence distinguishes the benefit of LEMMA’s merging design from the additional target-objective training unavailable to the static merging baselines? Please distinguish the resources required for merge optimization from those used to train the specialists.
- Justification of joint reward improvement. What justifies the statement in Section 4.2 that Pareto-dominant training pairs cause the gradient to reinforce all rewards jointly? Dominance between two sampled images does not by itself determine how every expected reward changes after updating the generator. Is this statement intended as a formal property under additional assumptions, or as an empirical motivation for the preference objective?
- Interpretation of the remaining trade-offs. Does improvement “without trading one capability for another” refer to the pretrained model, the individual specialists, or competing merged models? How should the lower SDXL VILA score relative to its specialist (6.13 versus 6.21) and lower SD3 PickScore relative to TIES (22.67 versus 22.70) be interpreted? Clarifying which differences are statistically meaningful, and distinguishing target-reward trade-offs from variation in held-out metrics, would help establish the scope of this claim.
8 comments
1. Our main contribution is learning the merge of full-checkpoint specialists on the generator's own DPO or SFT objective in text-to-image models, with signed, unbounded coefficients.
LEMMA learns its merge in a fundamentally different way from AdaMerging (Yang et al., ICLR 2024) and ZipLoRA (Shah et al., ECCV 2024), where LEMMA optimizes the generator's own multi-objective DPO or SFT loss in a text-to-image setting, with signed, unbounded coefficients learned on training data only. A shared component with AdaMerging is Eq. 9, layer-wise merge parameterization, which we will credit explicitly in the final version. We summarize the differences below:
- vs AdaMerging. AdaMerging is designed for discriminative models, where it learns layer-wise coefficients, initialized at 0.3 and clamped to [0, 1], by minimizing the entropy of class predictions on the unlabeled test data. Beyond the shared parameterization, LEMMA differs in three ways: (i) it optimizes the generator's own multi-objective DPO or SFT loss instead of a classification surrogate, (ii) its coefficients are signed and unbounded, so a layer can subtract a specialist's update to remove conflicting skills, and (iii) it learns the merge once on training data and never sees the test inputs, whereas AdaMerging performs the merge on the unlabeled test samples.
- vs ZipLoRA. ZipLoRA merges two LoRAs (one subject, one style) with column-wise coefficients, trained to reconstruct each source LoRA's outputs. LEMMA merges any number of full-checkpoint specialists on the target multi-objective, and the merge by LEMMA can exceed the specialists on their own rewards (SDXL: MPS 11.24 vs 10.99; VQAScore 0.8148 vs 0.8050).
- vs CaPO. CaPO is not a merging method. It fine-tunes the full model on multi-reward preferences, producing a multi-objective specialist. It is therefore a baseline for LEMMA rather than a competing merge method. Our Mixture baseline in the original submission follows its "DPO + FRS" variant, and LEMMA outperforms it (please see below, reported in point 2). LEMMA instead reuses existing specialists and trains only 1,680 scalars per specialist on SDXL (490 on SD3). Reward calibration and frontier-based pair selection come from CaPO, as Sec. 4.2 states ("Following CaPO (Lee et al., 2025)"), which we do not claim.
2. LEMMA outperforms CaPO and its gains come from the merge design.
Below, we first compare LEMMA with CaPO directly, then explain that its gains come from the merge design, and finally show budget comparisons.
- (a) Comparison with CaPO. First, CaPO is not a merging method. It fine-tunes the pretrained model on multi-reward preferences. LEMMA instead starts from the specialists. Each specialist holds knowledge unique to its objective on top of knowledge shared with the other specialists, and LEMMA's merge combines both. Meanwhile, CaPO, starting from the pretrained model, cannot use the specialists unique knowledge. Please note that the Mixture baseline in the original submission follows CaPO's "DPO + FRS" variant [Lee et al., 2025, Tab. 2]: the pretrained model fine-tuned with the DPO loss and frontier-based pair selection on the same backbone, data and rewards. LEMMA outperforms it on every metric of Tables 1-2 (HZ-WS 9.37 vs 2.81 on SDXL and 4.35 vs 0.13 on SD3).
- (b) Advantage over static merges. Static merge methods (e.g., Task Arithmetic, TIES) combine the specialists with a few global coefficients. LEMMA’s advantage over them does not only come from training on the objective, but also it remains when the alternatives receive the same target supervision. (i) Task Arithmetic and TIES with their coefficients carefully tuned on the validation set: LEMMA still scores higher than both (SDXL HZ-WS 9.37 vs 5.64 and 4.89; SD3 HZ-WS 4.35 vs 2.60 and 3.15; SFT LPIPS 0.6765 vs 0.6812 and 0.6920). (ii) The coarse variant of our granularity ablation (Tables 5-6), i.e., Task Arithmetic with one weight per specialist learned with exactly LEMMA’s objective, data, and steps. Here, full-depth LEMMA beats it on SDXL (+7.99 HZ-WS) and on SFT (LPIPS 0.6765 vs 0.6842; CLIP 0.8116 vs 0.8052), and also beats the intermediate variant with one weight per top-level module (+4.18 HZ-WS; LPIPS 0.6765 vs. 0.6818, CLIP 0.8116 vs. 0.8039). (iii) The adapted AdaMerging (point 1, reported above), learned layer-wise with the same training but with coefficients restricted to [0, 1] as in its paper: LEMMA achieves improved performance compared to it. Therefore, target supervision alone does not explain the gains. LEMMA's merge formulation with layer-wise, signed coefficients with use of unique knowledge from the specialists do.
- (c) Resources. On SDXL, LEMMA learns its coefficients from 160k preference pairs (5,000 updates of batch 32), a quarter of the 640k pairs used to train the multi-objective model or each specialist (2,500 updates of batch 256); on SD3, it uses 80k for LEMMA vs 320k for multi-objective model or each specialist. Since specialists are trained once and reused, LEMMA is the cheaper option whenever they already exist, which is the setting it targets.
We sincerely thank the reviewer for the positive evaluation of our paper, including recognizing LEMMA's practicality and the breadth of our evaluation, and for the careful questions. We address each of them below.