- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes MOMoE, a preference-conditioned mixture of LoRA experts for multi-objective LLM alignment. A shared frozen backbone is augmented with objective-specific and free experts, with layer-wise routers conditioned on the hidden state and user preference. The framework combines joint expert–router training, additional free experts, synergy-aware checkpoint selection, and coverage regularization. Experiments evaluate reward trade-offs and preference-conditioned utility across several tasks, with comparisons to seven baselines and targeted design ablations.
Critical strengths
- The framework is well motivated and coherently combines joint expert–router training with preference-aware initialization and regularization to improve the trade-offs served by a single controllable model.
- The method shows substantial gains across multiple tasks and baselines, supported by repeated runs and targeted ablations that provide useful insights into joint training and router warm-up.
Critical weaknesses
-
The contribution of free experts is not fully separated from additional capacity. The default construction adds free experts to objective-specific experts, whereas removing them halves the expert count. This comparison therefore changes both expert roles and adapter capacity. The matched-count comparison of base and SFT initialization is informative, but it does not establish whether free experts are preferable to an equal-sized pool containing additional objective-specific experts. Such a control would clarify whether the gains depend on the proposed free-expert role or primarily on a larger trainable pool.
-
Blend selection is not established as an effective independent contribution. In the ablations on pages 8–9, the default blend criterion does not consistently outperform simpler diagonal-only checkpoint selection, and the ordering varies across tasks and backbones. Appendix F also reports a stronger alternative selection criterion on Beaver, although that result is limited to one seed. These findings leave the practical benefit of the proposed criterion uncertain and weaken the claim that each of the four principles is independently validated. The overall framework remains useful, but the contribution attributed to blend selection should be more cautiously stated.
Questions for authors
- At matched capacity and training budget, how does adding objective-specific experts compare with adding free experts?
- Given the mixed ablations, what evidence supports blend selection as an independently effective component?
2 comments
Thanks for the response. I am keeping my original rating.
We thank the reviewer for the constructive comments and for finding the framework useful. We address each point below.
W1 and Q1: Isolating the free-expert role from capacity.
Removing free experts (m = 0) changes both expert roles and pool size, and results in the original submission already bear on this: at the same N, replacing all objective-specific experts with free experts collapses the front (HV 0.119 vs. 0.588; pure free experts in Appendix F), so a larger pool alone does not yield the gains.
To test the requested control directly, we have added an equal-sized pool on all four tasks with two seeds in the revised manuscript (Sec. 4.4, l. 442–445; Table 14), with raw results in results/additional_experiments of our anonymous repository: the free experts are replaced with additional objective-specific experts, initialized from another checkpoint of each single-objective run, keeping N, routing, and training budget unchanged. This lowers every metric in all eight task–seed runs, e.g., mean HV from 0.60 to 0.56 on Beaver, indicating that the gains depend on the free-expert role rather than a larger pool.
W2 and Q2: Evidence for blend selection.
We agree that blend selection is not established as an independent contribution, as noted in Limitation (1). Since submission, we have extended the own-objective variant in Appendix F to all tasks and seeds: it raises mean HV on Beaver and LFQA but lowers it on Assistant and Summarization, so its gains likewise vary across tasks. In the revised manuscript, we have stated the contribution of blend selection more cautiously and narrowed the original claim that each principle is independently validated (abstract, l. 26–37; Sec. 4.4, l. 450-451; Sec. 5, l. 494–496; Appendix F, l. 1507–1511). In the final version, we will change blend selection from one of the design principles to an observation, describing it instead as part of expert initialization and moving its analysis (Sec. 3.4 and Theorem 3) to the appendix. All experts remain initialized as before, so the experimental setup and main results are unchanged.
We hope the added control for free experts and the revised treatment of blend selection address the reviewer’s concerns, and we welcome further discussion if any remain.