- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MoEMB, which uses a pretrained sparse MoE to scale the encoder capacity of universal multimodal embeddings (UME) while retaining single-vector encoding without generating reasoning text. The authors compare different MoE construction and router adaptation schemes and investigate adaptive computation to further reduce encoding costs. Among the compared models adapted using only public MMEB-family data, the paper achieves leading results on MMEB-V2 and MRMR, demonstrating a favorable trade-off between performance and computational efficiency.
Critical strengths
- The problem is important, and the core rationale is clear. The paper addresses a central tension in UME: supporting multiple tasks and modalities requires greater model capacity, whereas large-scale retrieval demands low encoding costs. Expanding capacity through sparse experts without relying on longer reasoning processes is a practically valuable direction. Beyond improving benchmark scores, the work offers a model design direction worth considering for efficient UME.
- The study provides useful findings on adaptation and efficiency. The study finds that native MoE outperforms upcycling during downstream adaptation, and that explicit task-based routing does not outperform retaining pretrained routing. For compute reduction, token pruning translates more readily into actual throughput improvements than expert-level sparsity. These findings help explain which designs are effective and which added complexities are unnecessary, giving the work value beyond the results of a single model.
Critical weaknesses
- The methodological novelty is relatively limited. Although the authors explore conflict-aware and task-conditioned routing, these additions do not become the main source of the final performance gains. The main model still relies primarily on a pretrained native MoE and contrastive adaptation. The paper's novelty therefore lies more in validating a scaling direction and conducting a systematic study than in introducing a new expert mechanism tailored to UME. Adaptive computation provides additional technical contributions, but these are not sufficient to fully address this limitation.
- The central claims are stronger than the conclusions supported by the available evidence. The results strongly support the conclusion that a pretrained native MoE can serve as an efficient UME encoder. However, support remains limited for the general claim that expert scaling is more effective than reasoning scaling. For example, the expert-count curve is obtained by pruning the same pretrained model. Such results demonstrate the importance of learned expert capacity, but make it difficult to disentangle the respective contributions of the MoE architecture, total capacity, and the backbone's pretrained capabilities. This limits the paper's ability to explain the mechanisms behind scaling.
Questions for authors
- Based on the existing results, could the authors clarify which evidence supports attributing the performance gains to expert capacity scaling rather than merely adopting a stronger pretrained backbone? Accordingly, under what conditions should the conclusion that "expert scaling outperforms reasoning scaling" be considered valid?
- Given that frozen routing already performs well and task-conditioned routing yields no clear gains, is the more plausible explanation that "the pretrained division of labor among experts is already suitable for UME," or that "the current adaptation data are insufficient to effectively reshape this division of labor"? Can the existing analyses help distinguish between these two explanations?
please examine rebuttal response
Hello Reviewer KbNp,
As you are likely aware, the rebuttal engagement period is coming to a close. Thus, at your earliest convenience, please take a look at the rebuttal and comment and decide if it changes your thoughts regarding your current evaluation.
Thanks.