- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes sparsified expert modules and a representation fusion architecture to balance compression and generality in vector retrieval. It demonstrates the effectiveness of the approach across multiple backbone models and languages, and provides empirical guidance on choosing between individual sparsified experts and the fused model.
Critical strengths
A strength of the paper is the clear practical relevance of the problem it addresses. The proposed approach demonstrates tangible improvements on the evaluated tasks, and the experiments characterize the compression–performance trade-offs of individual sparsified experts and the fused model.
Critical weaknesses
The methodological novelty is somewhat limited. The sparsified experts and fusion architecture largely build on established techniques, including sparse autoencoders, adapters, knowledge distillation, and expert fusion. The main innovation lies in their combination and the design of the training objectives. Although the experiments provide extensive performance comparisons and some component ablations, the multi-expert approach and the single multilingual expert are not matched in training data volume or model capacity, making it difficult to attribute the performance gains solely to expert complementarity. Additionally, the manuscript contains multiple unresolved cross-references displayed as "??", including those in the methodology section, AI Use Statement, Reproducibility Statement, and appendix. These issues suggest insufficient proofreading and negatively affect the clarity and overall presentation of the paper.
Questions for authors
I recommend adding controlled comparisons with matched training data, model capacity, or computational budgets to better substantiate the necessity and innovative value of the proposed design.
2 comments
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the controlled comparison with matched training data volume and the correction of cross-reference issues.
The additional results provide useful evidence for the effectiveness of MORSE. However, my concerns regarding the limited methodological novelty and the lack of fully controlled comparisons remain only partially addressed.
Overall, while the rebuttal has clarified several aspects of the proposed approach, I do not find the additional evidence sufficient to change my original assessment. I therefore maintain my rating of 2 .
Response to Reviewer Lsm1: methodological contribution and controlled comparisons
Thank you for recognizing the practical relevance of the problem. We address the problem with the standalone
SEand its training objective, then combine these experts throughMORSE.Weakness: methodological novelty
Our methodological contribution is a weighted-top-k objective that jointly optimizes one
SEacross active dimensions to produce elastic sparse representations. The objective combines reconstruction, contrastive retrieval and distillation of the backbone's query–document similarity distribution, with greater weight on extremely sparse representations. The backbone remains frozen, the signed representation has dimension d, and the active coordinates depend on the input. This differs fromCSRandCSRv2, which project embeddings into a nonnegative, higher-dimensional latent space, and fromMRL, which retains the first k dimensions.At k=32, the English
SEgains 10.58 points innDCG@10and 8.20 points inRecall@10over nativeMRLusing its first 48 dimensions onQwen3-Embedding-4B. Table F.1 reports these results over five primary English datasets:SE, k=32MRL, k=48nDCG@10Recall@10This benefit requires only the standalone
SE. Table C.1 also tests top-k weighting in the finalBGE-M3expert: primaryRecall@10at k=16 rises from 0.6492 to 0.6872 with data, initialization, optimization and selection fixed.MORSEadds an alignment objective for combining specialized experts into one elastic sparse representation. At each k, its alignment loss penalizes a positive–negative retrieval margin below the strongest frozen expert's margin. The target expert can vary across triples, while inference produces one sparse representation.Question / weakness: training budget and model capacity
To match training-row totals, we increased
SE_multito125ktriples per language (500kpooled rows). This matchesMORSE's total of four100k-row expert-training sets and100kaggregation rows, before epoch repetition.Mean
nDCG@10differences (MORSEminusSE_multi), in percentage points, over 20 primary retrieval datasets (five per language). kmax=1024 forQwen3-0.6BandBGE-M3, and kmax=2560 forQwen3-4B.Qwen3-0.6BQwen3-4BBGE-M3MORSEgains 0.1–2.1 points across the three backbones at the five reported budgets k≥32.At d=1024,
MORSE's inference heads use about10.5Mparameters vs.2.10Mfor oneSE. Both use one frozen backbone and store one vector: 6k bytes at sparse k<d, or 4d bytes at k=d. The gains at matched training-row totals require no additional index storage.Weakness: cross-references
We corrected the cross-references throughout the manuscript and verified the compiled PDF. The methodology links to the architecture and implementation appendices. The Reproducibility Statement links to the model, training and evaluation records.
Thank you again for the feedback. We hope these results and clarifications help you reassess the contribution.