- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper looks at personalization in multimodal LLMs at the level of user preferences rather than user identity. The authors point out that when a model is trained on many users together, it tends to give the popular answer instead of the one the specific user prefers, and they call this group preference collapse. To deal with it, PrefMoE keeps a learned state for each user, splits it into profile and preference parts, and models each preference as a shared prototype plus a personal residual. On top of that there are a few extra losses (an imbalance-aware contrastive loss, pseudo-user augmentation, decorrelation) and a MoE-LoRA router. Experiments are on a cleaned version of MMPB and cover several backbones of different sizes.
Critical strengths
- I think the problem is worth studying. Pointing out that personalized models can drift toward the majority answer, and that overall accuracy can hide this, is a useful observation.
- The improvements over full fine-tuning are consistent across quite a few backbones and model sizes in Table 1, so they don't seem to be specific to one model.
- There's a lot of analysis beyond the main table (routing variants, removing facet labels, number of pseudo users, an extra domain, sequential updates).
- The paper is fairly honest about its limitations, e.g. it reports that PrefMoE degrades more than a baseline under sequential updates.
Critical weaknesses
- My main concern is that the ablation only partly supports the story of the paper. The introduction argues that existing methods focus on who the user is and not what they prefer. But in Table 2, the single largest improvement, on both accuracy and collapse, comes from the step labeled profile factor learning. As far as I can tell this step mainly adds the profile encoders and losses, although it may also introduce the factorized user state; the paper should clarify what exactly it adds. Of the preference-specific components, pseudo-user augmentation and decorrelation do reduce collapse noticeably, but the imbalance-aware residual term barely changes it, and their effect on accuracy is small compared with the profile step. So a good part of the gain seems to come from better modeling of the user's identity, which doesn't quite match the motivation. The ablation is also only cumulative; leave-one-out results would make the contribution of each part much clearer.
- I'm not convinced the paper has identified where collapse comes from. It frames collapse as arising under joint multi-user learning, and the method mainly targets preference signals being suppressed during representation learning. But the models without any fine-tuning already collapse a lot. In Figure 1, fine-tuning actually seems to reduce collapse for every group, and Table 1 shows the same trend for almost all backbones. That suggests the base model's prior (or just a common default answer) may be a big part of the problem. The paper's second explanation, that the model may follow generic or population-dominant patterns during generation, partly covers this, but the two sources are never separated experimentally. There's also a confound between popularity and semantic similarity: Pilates is the least popular option but also the one closest to Yoga, and it's generally the most collapsed. It would help to compare base and fine-tuned models under the same prompt, or to change which preference is the majority and see whether the collapse direction follows.
- The comparison with baselines mixes different things. The test users are the same as the training users, PrefMoE uses a learned per-user state at inference, and the fine-tuning baselines only see the preferences as text in the prompt. It's also not clear how the PEFT baselines get user information. To be fair, the basic user-embedding row in Table 2 suggests an ID embedding alone isn't enough. Still, I'd like to see a simple LoRA + user-ID embedding baseline and some evaluation on held-out users.
- The collapse metric could use more support. The number of negative samples it's computed on isn't reported, and from the table values it looks fairly small. Since it only counts false "yes" answers on negatives, it could on its own be lowered by a more conservative model. The accuracy gains make this less likely to be the whole story, but reporting overall yes-rates or recall would make it clear. All results also seem to be from a single run without variance.
- Some closely related work isn't discussed, for example shared + individual user representations in personalized alignment (P-RLHF, PAL), MoE-LoRA user routing (iLoRA), and popularity bias in recommender systems. The term "preference collapse" has also been used before in the RLHF literature. This makes it hard to judge what is actually new here.
- The benchmark is small (50 users) and the preference labels were reconstructed by the authors, so it's hard to know how well the results generalize. The authors do acknowledge this.
- Minor:
- Several method details are missing from the paper: the profile encoders and losses, how the experts are conditioned on the user state, how pseudo users get residuals, and key hyperparameters such as the temperature and loss weights. These may be in the supplementary code, but they should be in the paper.
- Figure 1 is a bit confusing. The options aren't really comparable (Yoga and Pilates overlap, and essential oils is a product rather than an activity), and the Yoga bar, which counts correct predictions, sits next to the collapse bars, so it's easy to misread.
Questions for authors
- What exactly does the profile factor learning step in Table 2 add on top of the basic user embedding? Does it also introduce the separate preference factors?
- Do you have results on held-out users, and a LoRA + user-ID embedding baseline with a similar number of parameters?
- How many negative samples is the collapse rate computed on, and what are the overall yes-rates or false-acceptance rates for each method? Any variance across seeds?
- How are the PEFT baselines conditioned on the user, and how were they tuned?
- How was full fine-tuning done for the 72B models with the reported hardware? And why does fine-tuned Qwen2.5-VL-7B do worse than the untrained one on preference accuracy in the 0-turn setting?