acceptodds
Under review as a conference paper at ICLR 2027

MORSE: Mixture of Representations from Sparsified Experts

Abstract

Recent work on quantization (e.g., TurboQuant) and dimensionality reduction (e.g., Matryoshka Representation Learning; MRL) demonstrates that text embeddings contain substantial representational redundancy. However, enforcing extreme sparsity (retaining only a small number of active dimensions) can reduce embedding generality and favor task-specific performance, as observed in work on Contrastive Sparse Representations (CSR) and CSRv2. We show that such task-specific, low-active dimension representations can be learned by training a light-weight adapter (namely, Sparsified Expert) over any base model using a novel weighted-top-k objective. Our approach departs from prior methods that either (1) concentrate information in predetermined leading dimensions (i.e., prefix-based), as in MRL and Matryoshka Adapters, or (2) project embeddings into a higher-dimensional space to identify sparse representational axes, as in CSR and CSRv2. We then introduce Mixture of Representations from Sparsified Experts (MORSE), which combines representations from multiple SEs to recover task-agnostic performance while maintaining sparsity in representations. MORSE occupies the previously underexplored regime between moderately sparse, task-agnostic representations and extremely sparse, task-specific ones, revealing a Pareto frontier between representational sparsity and generality. Across three base models, our SE outperforms other methods at ultra-low dimensions and MORSE improves over others on cross-lingual generality.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
efficient representation learning, text embeddings, information retrieval, embedding compression, sparse representations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Lsm1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes sparsified expert modules and a representation fusion architecture to balance compression and generality in vector retrieval. It demonstrates the effectiveness of the approach across multiple backbone models and languages, and provides empirical guidance on choosing between individual sparsified experts and the fused model.

Critical strengths

A strength of the paper is the clear practical relevance of the problem it addresses. The proposed approach demonstrates tangible improvements on the evaluated tasks, and the experiments characterize the compression–performance trade-offs of individual sparsified experts and the fused model.

Critical weaknesses

The methodological novelty is somewhat limited. The sparsified experts and fusion architecture largely build on established techniques, including sparse autoencoders, adapters, knowledge distillation, and expert fusion. The main innovation lies in their combination and the design of the training objectives. Although the experiments provide extensive performance comparisons and some component ablations, the multi-expert approach and the single multilingual expert are not matched in training data volume or model capacity, making it difficult to attribute the performance gains solely to expert complementarity. Additionally, the manuscript contains multiple unresolved cross-references displayed as "??", including those in the methodology section, AI Use Statement, Reproducibility Statement, and appendix. These issues suggest insufficient proofreading and negatively affect the clarity and overall presentation of the paper.

Questions for authors

I recommend adding controlled comparisons with matched training data, model capacity, or computational budgets to better substantiate the necessity and innovative value of the proposed design.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Lsm1: methodological contribution and controlled comparisons

Thank you for recognizing the practical relevance of the problem. We address the problem with the standalone SE and its training objective, then combine these experts through MORSE.

Weakness: methodological novelty

Our methodological contribution is a weighted-top-kk objective that jointly optimizes one SE across active dimensions to produce elastic sparse representations. The objective combines reconstruction, contrastive retrieval and distillation of the backbone's query–document similarity distribution, with greater weight on extremely sparse representations. The backbone remains frozen, the signed representation has dimension dd, and the active coordinates depend on the input. This differs from CSR and CSRv2, which project embeddings into a nonnegative, higher-dimensional latent space, and from MRL, which retains the first kk dimensions.

At k=32k = 32, the English SE gains 10.58 points in nDCG@10 and 8.20 points in Recall@10 over native MRL using its first 4848 dimensions on Qwen3-Embedding-4B. Table F.1 reports these results over five primary English datasets:

metricEnglish SE, k=32k = 32MRL, k=48k = 48
nDCG@100.77090.6651
Recall@100.77650.6945

This benefit requires only the standalone SE. Table C.1 also tests top-kk weighting in the final BGE-M3 expert: primary Recall@10 at k=16k = 16 rises from 0.6492 to 0.6872 with data, initialization, optimization and selection fixed.

MORSE adds an alignment objective for combining specialized experts into one elastic sparse representation. At each kk, its alignment loss penalizes a positive–negative retrieval margin below the strongest frozen expert's margin. The target expert can vary across triples, while inference produces one sparse representation.

Question / weakness: training budget and model capacity

To match training-row totals, we increased SE_multi to 125k triples per language (500k pooled rows). This matches MORSE's total of four 100k-row expert-training sets and 100k aggregation rows, before epoch repetition.

Mean nDCG@10 differences (MORSE minus SE_multi), in percentage points, over 20 primary retrieval datasets (five per language). kmax⁡=1024k_{\max} = 1024 for Qwen3-0.6B and BGE-M3, and kmax⁡=2560k_{\max} = 2560 for Qwen3-4B.

backbonek=8k = 8k=16k = 16k=32k = 32k=64k = 64k=128k = 128k=256k = 256kmax⁡k_{\max}
Qwen3-0.6B−2.7−1.00.10.10.70.91.1
Qwen3-4B0.00.61.71.00.90.91.2
BGE-M3−3.1−1.70.31.32.11.40.8

MORSE gains 0.1–2.1 points across the three backbones at the five reported budgets k≥32k \geq 32.

At d=1024d = 1024, MORSE's inference heads use about 10.5M parameters vs. 2.10M for one SE. Both use one frozen backbone and store one vector: 6k6k bytes at sparse k<dk < d, or 4d4d bytes at k=dk = d. The gains at matched training-row totals require no additional index storage.

Weakness: cross-references

We corrected the cross-references throughout the manuscript and verified the compiled PDF. The methodology links to the architecture and implementation appendices. The Reproducibility Statement links to the model, training and evaluation records.

Thank you again for the feedback. We hope these results and clarifications help you reassess the contribution.

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the controlled comparison with matched training data volume and the correction of cross-reference issues.

The additional results provide useful evidence for the effectiveness of MORSE. However, my concerns regarding the limited methodological novelty and the lack of fully controlled comparisons remain only partially addressed.

Overall, while the rebuttal has clarified several aspects of the proposed approach, I do not find the additional evidence sufficient to change my original assessment. I therefore maintain my rating of 2 .

Reviewer_Wf49OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to compress pretrained text embeddings while preserving retrieval performance across different languages. The authors first propose Sparsified Experts, which use a weighted top-k loss to learn sparse embeddings at different sizes. They then introduce MORSE to combine several language experts into a single sparse representation. The experiments show promising results at very small embedding sizes and improvements in multilingual retrieval.

Critical strengths
  1. The paper presents a useful way to learn highly sparse embeddings without retraining the backbone. The results at small embedding sizes are promising.
  2. The experiments cover three different embedding backbones, including models trained with and without MRL. I also found the analysis in Appendix D useful, as it examines how sparsity affects representation geometry and how the different experts contribute to MORSE.
Critical weaknesses
  1. The comparison with the multilingual SE does not fully separate the benefit of combining experts from differences in training. Appendix C shows that each language expert is trained on 100,000 triples, while the multilingual SE uses only 25,000 per language. MORSE also has an additional training stage with a different weighting over sparsity levels. Although the authors include a multilingual SE baseline, it is difficult to tell how much of MORSE's improvement comes from combining experts rather than the training setup.
  2. The generality of MORSE beyond multilingual retrieval is not sufficiently evaluated. Section 5.5 includes several non-retrieval tasks, but these experiments only evaluate the individual SE, not MORSE. Since MORSE is intended to combine specialized representations into a more general one, it would be useful to know whether this advantage also holds for other types of tasks.
Questions for authors
  1. Does MORSE still outperform the multilingual SE when both are trained with comparable amounts of data and similar training settings? This would help clarify the benefit of expert aggregation.
  2. How does MORSE perform on the non-retrieval tasks in Table F.2, compared with the English SE and MRL?
  3. I noticed that Tables F.1 and F.4 report different English retrieval results for Qwen3-embedding-4B. For example, at k=32, the English expert has an nDCG@10 of 0.7709 in Table F.1 but 0.5818 in Table F.4. Recall@10 also differs. Are these results based on different checkpoints or evaluation settings?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Wf49: training budget, non-retrieval evaluation, and table consistency

Thank you for the feedback on how MORSE builds on our SE experts. We are glad that you found our method useful, our results promising at small embedding sizes, and analysis examining geometry as useful.

SE experts improve retrieval at ultra-low dimensions, including on non-MRL backbones, while MORSE combines them to improve transfer.

We trained larger-data multilingual experts for the comparison below and added MORSE's per-task results to Table F.2.

Q1 / weakness 1: training-data budget

With 500k training rows for each method, MORSE has higher mean nDCG@10 than SE_multi at all five reported budgets k≥32k \geq 32 on all three backbones. We increased SE_multi from 25k to 125k triples per language (500k pooled rows). MORSE uses four 100k-row expert-training sets and 100k aggregation rows, also totaling 500k. We match training-row totals, with each method retaining its objectives, checkpoint selection and capacity.

Mean nDCG@10 differences (MORSE minus SE_multi), in percentage points, over 20 primary retrieval datasets (five per language). kmax⁡=1024k_{\max} = 1024 for Qwen3-0.6B and BGE-M3, and kmax⁡=2560k_{\max} = 2560 for Qwen3-4B.

backbonek=8k = 8k=16k = 16k=32k = 32k=64k = 64k=128k = 128k=256k = 256kmax⁡k_{\max}
Qwen3-0.6B−2.7−1.00.10.10.70.91.1
Qwen3-4B0.00.61.71.00.90.91.2
BGE-M3−3.1−1.70.31.32.11.40.8

Q2 / weakness 2: non-retrieval evaluation

On the non-retrieval tasks, MORSE improves the seven-task mean over the English SE in 11 of 12 comparisons at equal vector storage.

We use the F.2 datasets and primary metrics. Sparse SE and MORSE use 6k6k bytes, matching MRL with its first 1.5k1.5k dimensions. BGE-M3 uses top-k at the same kk.

Mean score differences on a 0–100 scale over seven task-specific metrics and four budgets, k∈{32,64,128,256}k \in \{32, 64, 128, 256\}. Positive values favor MORSE.

backbonevs. English SEvs. MRL / top-k
BGE-M30.743.66
Qwen3-0.6B0.87−0.11
Qwen3-4B0.40−0.04

Averaged over the four budgets, MORSE gains 3.66 points over the native BGE-M3 baseline and is within 0.11 points of native MRL on both Qwen backbones.

Classification improves over SE in all 12 comparisons and semantic similarity in 10 of 12. MORSE also outperforms MRL on the classification and bitext mining datasets, across both Qwen models and all four storage budgets. The revised Table F.2 provides the complete per-task and per-budget results.

Q3: consistency of Tables F.1 and F.4

We corrected F.4's English-expert results to use the same checkpoint as F.1. F.1 averages five primary English datasets, while F.4 also includes eight extended datasets. At k=32k = 32, the means on a common 0–1 scale are:

evaluationdatasetsnDCG@10Recall@10
primary English, F.150.77090.7765
extended English80.46860.5478
combined English, revised F.4130.58490.6358

F.4 weights the groups by 5/135/13 and 8/138/13. We updated the captions and corresponding plots.

Thank you for prompting these comparisons and consistency checks.

Thanks for the response. The matched-data comparison and non-retrieval results are useful, and the table differences are now clear. However, the gains over a single multilingual expert are still fairly small in several settings, while MORSE requires additional model capacity. So I'll keep my original score.

Reviewer_cN9zOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a lightweight adapter (called Sparsified Expert or SE) which can be tuned on top of any base model to create sparse representations. This methodology works quite well for < 128 dimensions and to improve it in large dimensions, they introduce MORSE which stands for Mixture of Representations from Sparsified Experts. As the name suggests, it combines representations from multiple language specific SEs to gain performance, while still maintaining sparsity. They evaluate on ultra-low dimensions as well as cross language transfer to validate their method. Lastly, they also provide discussion on storage and efficiency which provides insights into real-world usage of the same. While SEs seem as a good contribution, MORSE adds on top of SE providing limited gains while requiring additional training and infrastructure to serve. Given that the weaknesses and questions are addressed, the rating would be increased to acceptance.

Critical strengths
  1. SE method is quite strong and achieves good performance compared to existing works. The adapter is easy to use and train for a single expert scenario, requiring only the adapter to be trained while keeping the backbone frozen.
  2. Discussion on storage and efficiency is very much appreciated for adapting to real-world use cases. This also highlights the trade-off of MORSE in terms of memory and encoding time for accuracy, compared to other approaches like MRL.
Critical weaknesses
  1. Figure 6 and Figure 7, highlight that SEmultiSE_{multi} gets competitive performance across languages. This raises the question of using MORSE compared to SEmultiSE_{multi} since additional overhead is required to train as well as serve MoE structures.
  2. While the paper is quite well written and easy to follow, the line plots having less margins between different variations make it hard to compare different methods. Better presentation is needed on this front as well as improving missing references (for e.g. - L210, L490, L496, etc.)
  3. The paper proposed the weighted top-k as a key contribution. However, Table C.1, compares uniform and weighted variations which actually have quite close values. Moreover, no discussion is provided in terms of how they are determined and how sensitive is the result in respect to K.
Questions for authors
  1. How are the values for top-k determined?
  2. The reason for using language-specific experts is not clear. Do they offer a clear benefit over having a simple learnt gating? A simple baseline with a learnt gating will help clear up the difference.
  3. Since each language-specific expert is trained on 100k triples, MORSE would receive 400k triples across each expert and then 100k for aggregation training, while singular SEs just receive 100k triples. Can you clarify whether the benefit is from additional training or the proposed mechanism? Moreover, how many triples are used to train SEmultiSE_{multi}? Equal training data comparisons would strengthen the paper.
  4. While the paper evaluates cross-lingual transfer, how does MORSE perform in cross-language retrieval tasks? This is quite relevant to the paper's setting and might help strengthen their claims.
  5. Lastly, can the authors explain this statement: "The MTEB-trained expert is produced by five checkpoint-linked training stages lasting 10, 5, 2, 1, and 8 epochs"?
4 comments
AuthorsOpenReview ↗

Response to Reviewer cN9z (1/2): weighting and learned gating

Thank you for recognizing the strong performance of our SE experts, simplicity and ease of our training method, and the practical value of the storage analysis. We would like to emphasize that these experts improve retrieval at ultra-low dimensions on frozen backbones, including non-MRL models. MORSE builds on them to improve performance at low dimensions across languages.

We increased the multilingual baseline's training data from 100k to 500k rows and added a final-model weighting control, a learned-gating baseline, direct cross-language retrieval, and Chinese evaluation. We report the results below. We also improved plot readability and cross-references throughout the manuscript.

Q1 / weakness 3: choice of kk and loss weighting

Our weighted-top-kk objective improves retrieval at low active dimensions. At k=16k = 16, it improves primary nDCG@10 by 4.73 and Recall@10 by 3.80 percentage points.

The active dimension kk is the number of coordinates retained by magnitude for each input. For full embedding dimension d=1024d = 1024, we use the fixed grid K={2,22,…,210}\mathcal{K} = \{2, 2^2, \ldots, 2^{10}\}. We assign 70%70\% of the objective to k=16k = 16 and k=32k = 32, with weights 0.400.40 and 0.300.30, to prioritize ultra-sparse representations. At inference, kk follows the storage budget without retraining.

Table C.1 rows 2–3 test weighting in a linear expert with ℓ2\ell_2 distillation. Weighting improves primary Recall@10 at k=16k = 16 by 1.62 percentage points in that setting, compared with 3.80 for the final SE. We added the control for the final two-layer BGE-M3 English SE with KL distillation, using wk=0.1w_k = 0.1 at each of the same ten dimensions. Data, initialization, optimization and selection by validation Recall@10 at k=16k = 16 are unchanged.

Mean Recall@10 on primary (five datasets) and extended (eight) English retrieval.

evaluationweightingk=8k = 8k=16k = 16k=32k = 32k=64k = 64k=128k = 128
primaryuniform0.53390.64920.71560.75390.7656
primaryweighted0.54480.68720.73110.76560.7740
extendeduniform0.20650.31120.38730.43280.4620
extendedweighted0.23360.33120.39090.43160.4624

On primary retrieval, weighting improves both metrics at all five tested budgets from k=8k = 8 to k=128k = 128. The revised Table C.1 includes this control at fixed K\mathcal{K}.

Q2: language-specific experts and learned gating

MORSE improves Recall@10 over the learned gate at five of eight budgets, including gains of 0.4 and 0.6 percentage points at k=16k = 16 and k=32k = 32.

For this comparison, we keep the four frozen language experts and their projections, replacing the mixer with a linear softmax gate on the backbone embedding. Data, losses, initialization, the schedule over active dimensions and checkpoint selection are unchanged. The gate results average two seeds.

Mean score differences (MORSE minus gate), in percentage points, over 20 primary retrieval datasets on Qwen3-0.6B.

metrick=8k = 8k=16k = 16k=32k = 32k=64k = 64k=128k = 128k=256k = 256k=512k = 512k=1024k = 1024
Recall@10−0.50.40.60.20.40.0−0.10.1
nDCG@10−0.80.20.3−0.20.00.0−0.20.0

We use language-specific training to let each expert specialize within the same retrieval task, while keeping the experts usable independently of the fusion method.

AuthorsOpenReview ↗

Response to Reviewer cN9z (2/2): training budget, cross-language retrieval, and clarifications

Q3 / weakness 1: training budget and serving overhead

With 500k training rows for each method, MORSE has higher mean nDCG@10 at all five reported budgets k≥32k \geq 32 on all three backbones. We increased SE_multi from 25k to 125k rows per language (500k pooled rows). MORSE uses four frozen experts, each trained on 100k rows, and an aggregator trained on 100k rows, totaling 500k training rows.

Mean nDCG@10 differences (MORSE minus SE_multi), in percentage points, over 20 primary retrieval datasets (five per language). kmax⁡=1024k_{\max} = 1024 for Qwen3-0.6B and BGE-M3, and kmax⁡=2560k_{\max} = 2560 for Qwen3-4B.

backbonek=8k = 8k=16k = 16k=32k = 32k=64k = 64k=128k = 128k=256k = 256kmax⁡k_{\max}
Qwen3-0.6B−2.7−1.00.10.10.70.91.1
Qwen3-4B0.00.61.71.00.90.91.2
BGE-M3−3.1−1.70.31.32.11.40.8

On BGE-M3, MORSE also exceeds the larger-data SE_multi using half the vector storage: 0.713 at k=128k = 128 vs. 0.710 at k=256k = 256, requiring 768 vs. 1536 bytes per vector.

MORSE evaluates four expert heads and an aggregator after one shared backbone pass. A standalone SE uses one adapter head. Both store one vector in one index, using 6k6k bytes at sparse k<dk < d (fp32 values and uint16 indices). The gains at matched training-row totals therefore require no additional index storage.

Q4: cross-language retrieval and Chinese transfer

For direct cross-language retrieval on Qwen3-0.6B, MORSE exceeds every reported SE at the tested k≥64k \geq 64, and MORSE-100k does so at k≥32k \geq 32.

We evaluated 14 directed pairs with different query and document languages. Belebele and XPQA cover English ↔ Japanese/Italian/Hindi, and MLQA covers English ↔ Hindi. MORSE and MORSE-100k use 25k and 100k aggregation triples per language, respectively.

Mean nDCG@10 on a 0–100 scale. The best SE is selected at each kk across all reported language-specific and pooled experts. MRL uses the first kk dimensions, while sparse systems retain kk nonzeros.

methodk=32k = 32k=64k = 64k=128k = 128k=256k = 256k=512k = 512k=1024k = 1024
best reported SE62.9865.4267.1168.3569.0369.29
MORSE (unweighted)57.1161.4164.4766.5767.5967.83
MRL (dense)45.7055.3461.8565.5868.4070.14
MORSE62.8365.8768.0069.3670.2470.40
MORSE-100k63.2266.2868.2169.4770.2370.42

MORSE also transfers to Chinese without Chinese adapter training. Across five Chinese retrieval datasets (Belebele, MLQA, XPQA, WebFAQ, and LeCaRDv2), it exceeds the Chinese-trained SE_zh at every reported k≥32k \geq 32, by 0.37–1.68 points. At k=512k = 512, the scores are 77.12 vs. 75.44.

Q5: checkpoint-linked training stages

We train one Qwen3-Embedding-4B expert with the backbone frozen in multiple stages. In the first stage, we train it for 10 epochs, then continue training from its best validation checkpoint (epoch 9) for 5 epochs in the second stage, with fresh optimizers and a new learning-rate schedule. Similarly, we continue training for three more stages of 2, 1, and 8 epochs, respectively, each starting from the previous stage's best validation checkpoint.

The schedule reflects successive tuning for the MTEB mixture, a necessary step given how diverse the eight retrieval datasets are compared to mMARCO. We adjusted loss weights and learning rates across stages, keeping the architecture and data fixed. The main mMARCO experts use 10 epochs. Appendix C and Table C.2 report the settings and selected epochs.

Weakness 2: plot readability and cross-references

We redrew the comparison plots with consistent method colors, and Tables F.1–F.5 provide the numerical results for direct comparison. The dense reference uses a dashed line, while method curves use solid lines and circular markers. We also standardized the evaluation labels and clarified the dataset groups in Table A.1.

Thank you again for the feedback. These comparisons strengthen the evidence for MORSE's gains at matched training-data totals and its transfer to Chinese without Chinese adapter training. We will include these ablations and results in the camera-ready version. We hope these additions clarify our contributions and address your concerns.

Dear authors,

Thank you for the response, a few questions from above are still unresolved:

  1. In language-specific experts and learned gating, I find that MORSE is only clearly beneficial in 2/8 setups. Moreover, since the language specific experts are frozen, this suggests the gain comes from ensembling. Can you please comment on that? Moreover, if possible, a control with four experts trained on randomly split data might resolve this, although I understand if sufficient time is not available.

  2. For weighted top-k, there is still no explanation of how they were chosen and no sensitivity analysis.

  3. Additional overhead query is not answer. Extra effort in terms of infrastructure is still required to train MoE models as well as serve them during inference. If the advantage is limited (say <1 percentage points) and MORSE also requires more parameters, why would it be reasonable to choose MORSE over SE_multi?

I maintain my score.

AuthorsOpenReview ↗

Response to Reviewer cN9z: top-k weight and grid sensitivity

Q2: weight selection and sensitivity

Thank you again for taking the time to review our response and clarify your concerns. We provide the requested sensitivity analysis below.

Across five alternative weight profiles, the mean changes by −0.001 to +0.006. We chose the original weights manually to prioritize ultra-sparse retrieval, assigning 0.400.40 to k=16k = 16 and 0.300.30 to k=32k = 32.

nDCG@10 differences from the paper's checkpoint, with the mean over k=8,16,…,1024k = 8, 16, \ldots, 1024. Weights follow k=2,4,…,1024k = 2, 4, \ldots, 1024. The first row retrains the original profile, and the first three rows average two seeds.

weighting wkw_kk=8k = 8k=16k = 16k=32k = 32mean
(.02,.04,.08,.40,.30,.08,.04,.02,.01,.01)(.02, .04, .08, .40, .30, .08, .04, .02, .01, .01)+0.001+0.001+0.001+0.000
(.01,.03,.07,.14,.25,.25,.14,.07,.03,.01)(.01, .03, .07, .14, .25, .25, .14, .07, .03, .01)−0.008+0.008+0.010+0.006
(.01,.02,.08,.29,.21,.08,.04,.09,.09,.09)(.01, .02, .08, .29, .21, .08, .04, .09, .09, .09)−0.004+0.005+0.011+0.005
(.02,.04,.18,.30,.30,.08,.04,.02,.01,.01)(.02, .04, .18, .30, .30, .08, .04, .02, .01, .01)−0.002−0.002−0.005−0.001
(–,–,.09,.43,.32,.09,.04,.02,.01,.01)(\text{--}, \text{--}, .09, .43, .32, .09, .04, .02, .01, .01)−0.005+0.008+0.002+0.002
(.04,.04,.07,.36,.27,.07,.04,.04,.04,.04)(.04, .04, .07, .36, .27, .07, .04, .04, .04, .04)−0.008−0.001−0.001−0.001

Dropping k=2,4k = 2,4 from training changes the mean by +0.002.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.