acceptodds
Under review as a conference paper at ICLR 2027

MoE Mirage: Selection Noise and Routing-Weight Geometry in Token-Level Counterfactuals

Abstract

Token-level "best expert" headroom in sparse mixture-of-experts (MoE) models is a same-sample best-of- maximum: each candidate expert is installed, the downstream loss is scored, and the best is kept, on the same tokens. We show that, at the resolution we can test, this quantity behaves as selection noise whose scale is set by the intervention operator. Selected on one half of the continuation and scored on the other, no operator retains headroom above the experiment's resolution (identity-preserving swap: nats, 95% CI ; planted advantages detected with 80% power from ), and an exact matched-noise null closely reproduces the primary same-sample values and the checkpoint ordering of their ratio. Published operators change support, weight shape, total mass or normalization together with the expert; we introduce an identity-preserving swap that changes only the expert, and full-commitment forcing reports more same-sample headroom than it in of cells (), as the null predicts. The operator also changes a decision that takes no maximum: one-shot pruning by mean expert effect yields operator-dependent masks, forcing's ranking partly reproduces across document halves while the swap's does not, and on two OLMoE releases forcing-ranked masks fall below random masks in benchmark accuracy. We recommend that token-level routing maps state their operator and scoring horizon, cross-fit candidate selection and scoring, and report a matched-noise null to separate persistent opportunity from selection noise.

Primary area:
interpretability and explainable AI
Keywords:
mixture of experts, sparse MoE, expert routing, counterfactual evaluation, routing intervention, mechanistic interpretability, model evaluation, selection bias, cross-fitting

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_H2qYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper looks at the “best expert for each token” idea for MoE models, where each candidate expert is tried, and the best one is kept, all on the same tokens. The authors argue that the observed effects are mostly due to selection noise. To show this, they select the expert on one half of the text and score it on the other half, and they compare against a matched-noise baseline that reproduces the original numbers. They also introduce an identity-preserving swap as a control, and recommend separating selection from scoring in future works.

Critical strengths

S1: The paper is well-written, engages with related works, and analyses current strategies.

S2: The authors propose an identity-preserving swap that changes only which expert gets a routing weight and leaves everything else fixed.

Critical weaknesses

W1: The cross-fit tests whether the chosen expert helps consistently across different downstream tokens after the intervention. The cross-fit and its detection floor are reported for the whole suffix, where a real gain on only the first few tokens could be hidden. The paper’s own results show the forcing/swap ratio depends strongly on scoring horizon, so it is unclear to me how robust the null is at short horizons.

W2: The matched-noise null relies on the two halves being exchangeable. The paper states this assumption, and the interleaved halves with matched distance profiles make it plausible, but I did not see a direct test of it. The cross-fit uses the same split, so the two main pieces of evidence share this assumption and are not fully independent. If the halves differ, agreement between the null and the observed values could be partly an artifact.

Questions for authors

Q1: Do the results rule out token-specific headroom that a learned selector could exploit, or only persistent headroom that carries over to other tokens?

Q2: Which version of the ratio (5.48, 3.63, or the weighted range) is the main one? Why does it differ so much between models, and does it also hold within each model, not only when averaged?

2 comments
AuthorsOpenReview ↗

Response to Reviewer H2qY: Short-Horizon Effects, Matched-Noise Assumptions, and Headroom Interpretation

Thank you for the thoughtful review and for identifying two important limits of the original analysis: short-horizon effects and the assumptions behind the matched-noise reference. Both concerns led to substantive revisions and additional experiments.

W1 — Short-horizon effects. We now evaluate scoring horizons from 2 to 64 suffix tokens, alongside alternative block-interleaved and contiguous splits (Section 5.3; Appendix B.3, Tables 12–13). The partial trace collection does not show detectably positive pooled restricted-candidate cross-fitted headroom, but its sensitivity is substantially weaker at short horizons. We therefore do not interpret non-detection over a full suffix as evidence that short-lived effects are absent.

Importantly, Qwen1.5-MoE layer 10 shows a positive short-horizon forcing effect. An initial two-token estimate of +0.052 [+0.028, +0.075] nats/site was followed by a preregistered fresh-document replication of +0.0189 [+0.0002, +0.0375] across 1,837 sites and 56 documents. The replicated estimate is smaller, neither seed alone excludes zero, and the full-suffix effect is not detected. This establishes a setting-specific exception to an overly broad interpretation of the pooled result, not universal short-horizon transfer.

W2 — Exchangeability and matched-noise validity. We directly examined the assumptions of the matched-noise construction. The interleaved halves differ substantially in contrast variance: half-A/half-B ratios are 11.4 [8.9, 14.4] for forcing and 3.7 [2.2, 5.1] for the swap (Appendix B.1, Table 9). The reference also shares its split with the cross-fit, so the two are not independent confirmations.

We have consequently removed the original interpretation of the reference as an exact calibrated null. It is now explicitly an approximate, uncalibrated descriptive comparison. The observed paired split-half ratio is 3.63 versus a reference ratio of 3.97; this does not establish that all apparent headroom is noise (Section 5.2).

Independent-continuation validation. We additionally tested whether candidate effects transfer across independently model-sampled continuations of the same prefix. Candidates were selected on four continuations and evaluated on four others. Two of three preregistered cells passed the instrument gate, contributing 437 sites. Forcing retains +0.0087 [+0.0060, +0.0119] nats/site, driven by OLMoE-0924 layer 7, whereas the swap gives −0.0001 [−0.0003, +0.0001]. The excluded cell and post-hoc replication are separately disclosed (Section 5.4; Appendix B.2). The positive forcing result is relative to its intervention-specific reference, not necessarily the native route. It demonstrates why the original within-continuation non-detection cannot be generalized to every evaluation protocol.

Q1 — Can a learned selector exploit token-specific opportunity? Our results do not rule this out. The action-class ceiling applies only to the specified identity-preserving swap family, candidate population, and utility (Section 3). A tested frozen identity-only selector produced +0.00086 [+0.00025, +0.00152] nats/site on one sealed split, but two fresh-cohort evaluations (+0.00016 and −0.00003) did not replicate a detectable gain (Appendix D). These findings constrain that particular selector, not every learned routing policy. Moreover, an exploratory exhaustive-pool test finds a positive swap cross-fit of +0.0044 [+0.0003, +0.0088] in one Qwen3 layer with 128 candidates, although its uncertainty and limited document coverage prevent a general claim.

Q2 — Which ratio is primary, and does it hold within models? The primary estimate is 5.48, defined as the ratio of summed cell means across the 202-cell registry. The site-pooled ratio of 5.65 uses different weighting, while 3.63 comes from a separate 115-cell paired split-half collection. They are not interchangeable estimates (Section 5.1; Table 2; Appendix B.5).

Within the eight evaluated checkpoints, the primary forcing-to-swap ratio ranges from 1.12 to 11.90 and exceeds one for every checkpoint. Forcing exceeds the swap in 199/202 individual cells. The magnitude varies with architecture and routing semantics; Section 6 shows that changes in candidate-effect dispersion are strongly associated with the reported ratio (Spearman 0.92), without claiming a complete causal decomposition.

We appreciate these questions. The revised manuscript retains the central intervention-dependent measurement finding while explicitly recognizing persistent effects, statistical limitations, and the boundaries of what the experiments establish.

Thank you for your response and the newly added experiments, which resolved my concerns. I will maintain my positive score.

Reviewer_Jsb9OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the reliability of token-level counterfactual evaluations in Mixture-of-Experts (MoE) models. The authors argue that the apparent improvement obtained by selecting the best expert for each token can largely result from selection noise rather than a genuine, reproducible advantage.

To investigate this, the paper introduces an identity-preserving swap intervention that changes expert assignments while keeping the routing-weight structure fixed. The authors also use cross-fitted evaluation and a matched-noise null to distinguish persistent improvements from same-sample selection effects.

Experiments across seven MoE architectures show that full-commitment forcing reports substantially larger apparent headroom than the proposed swap, but neither provides detectable positive headroom under the main cross-fitted evaluation. The paper further examines how routing-weight geometry influences these measurements and shows that different intervention operators can produce different expert-pruning rankings.

Critical strengths

1- The paper addresses an important issue in MoE interpretability: the measured importance of an expert may depend heavily on how the intervention is performed. The proposed identity-preserving swap is a useful control because it separates changes in expert assignments from changes in routing-weight geometry. The combination of cross-fitting and a matched-noise null is particularly valuable for distinguishing apparent improvements from reproducible effects. I find this a meaningful methodological contribution beyond simply observing that selecting the maximum of noisy estimates introduces bias.

2-The authors evaluate several MoE architectures, routing conventions, checkpoints, and downstream tasks. The results show that full-commitment forcing reports more same-sample headroom than the swap in 199 of 202 matched cells, while the cross-fitted analysis does not identify a corresponding persistent advantage at the tested resolution. The experiments also include intervention validation, statistical uncertainty estimates, numerical precision checks, and pruning evaluations. These controls strengthen the credibility of the main findings, and I appreciate that the authors explicitly acknowledge cases where the conclusions are limited.

Critical weaknesses

1- The paper provides convincing evidence that same-sample headroom can be substantially inflated by selection noise. However, the broader interpretation of the results requires some caution. The cross-fitted evaluation selects experts on one interleaved half of a continuation and evaluates them on the other. Since both halves come from the same autoregressive continuation, their exchangeability and dependence structure deserve further examination, particularly because the matched-noise null relies on this assumption. Also, the main experiments primarily consider four candidates from the model's existing expert committee. Although the appendix includes additional candidate-pool experiments, some out-of-committee forcing interventions retain positive cross-fitted effects. This suggests that the conclusions may depend on the intervention class and candidate-selection setting. Additional independent validation would help clarify how broadly the reported absence of persistent routing opportunity applies.

2- The paper demonstrates that intervention choice affects expert rankings, but the practical implications remain less clear. The pruning experiments are limited to one-shot pruning without retraining, and none of the evaluated criteria consistently outperforms random pruning in accuracy. The paper also reports disagreements between accuracy and negative log-likelihood, making it difficult to determine which intervention produces more useful rankings for downstream applications. Furthermore, the numerical precision analysis shows substantial changes in the magnitude of reported headroom, while the full planned fp32 evaluation remains incomplete. Although these issues do not necessarily invalidate the main conclusions, they limit the practical interpretation of the results.

3- Several paragraphs, particularly in Sections 5, 6, and 7, are unnecessarily long and contain many numerical values, confidence intervals, and mathematical notations. This makes some of the results difficult to follow without repeatedly checking the figures and tables. However, the organization and figures are generally clear, and the writing is better structured than many similarly technical papers. Shorter paragraphs and clearer summaries of the main findings would significantly improve readability.

Questions for authors

1- How sensitive are the main conclusions to the assumption that the two interleaved continuation halves are exchangeable? Would the results remain consistent with alternative splitting strategies, such as contiguous non-overlapping windows or independently generated continuations? This would help establish whether the observed lack of persistent headroom depends on the particular cross-fitting design.

2- Most experiments consider a small set of experts, primarily from the existing routing committee. How do the conclusions change when a larger and more diverse set of experts is evaluated across additional architectures? The positive cross-fitted effects observed for some out-of-committee forcing interventions seem particularly important to investigate.

3- Can the authors evaluate whether the differences between intervention operators remain meaningful after pruning and recovery fine-tuning or when used to train a new router? The current pruning results establish that rankings differ, but they do not clearly demonstrate which intervention is preferable for practical model improvement.

4- The fp32 experiments show that precision can substantially change the measured headroom ratio. Can the authors complete the planned fp32 evaluation and clarify whether these numerical effects vary systematically across architectures or routing implementations? This would help establish the robustness of the reported quantitative magnitudes.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Jsb9: Statistical Validity, Candidate Diversity, FP32, and Practical Scope

Thank you for the careful review. Your concerns motivated new experiments and a narrower interpretation of the original findings. The revised manuscript now distinguishes intervention effects, evaluation protocols, and candidate populations explicitly.

W1/Q1 — Exchangeability and independent validation. The two interleaved halves are not exchangeable in scale: their contrast-variance ratios are 11.4 under forcing and 3.7 under the swap (Appendix B.1, Table 9). The matched-noise reference shares the split used for cross-fitting. We therefore describe it as approximate and uncalibrated, not an exact null or independent confirmation (Section 5.2). A separate partial trace study examines block-interleaved, contiguous, and first-token-versus-rest splits; none yields a detectably positive pooled restricted-candidate cross-fit, with limited sensitivity at short horizons (Appendix B.3, Tables 12–13).

More importantly, we tested independently sampled continuations: for each prefix, experts are selected on four model-sampled continuations and evaluated on four others. Two of three preregistered cells passed the instrument gate (437 sites). Forcing transfers at +0.0087 [+0.0060, +0.0119] nats/site, driven by OLMoE-0924 layer 7; swap is −0.0001 [−0.0003, +0.0001]. This is a different model-sampled-text estimand, relative to forcing's own reference, not necessarily native routing. The excluded cell and a later post-hoc replicate are disclosed separately (Section 5.4; Appendix B.2). Thus, the original within-continuation non-detection cannot support a universal absence of persistent opportunity.

W1/Q2 — Larger candidate sets. We added five exploratory full-pool studies using actual pretrained models, evaluating every routed expert at one layer (Table 4; Appendix C): Mixtral-8x7B (8 experts), DeepSeek-MoE-16B (64), OLMoE-0924 (64), Qwen3-30B-A3B-Instruct-2507 (128), and Qwen3.5-35B-A3B-Base (256). Qwen3's swap cross-fit is +0.0044 [+0.0003, +0.0088] nats/site against native routing, versus 0.0268 same-sample. The other four studies do not show a detectably positive full-pool swap estimate. DeepSeek's positive committee-to-full-pool contrast (+0.0023) is not a detected gain over native routing (full-pool +0.0012 [−0.0009, +0.0037]). Each study covers one layer, 8 documents (16 for Qwen3.5), and unadjusted exploratory intervals. Qwen3.5 has a disclosed cross-row numerical-monitoring limitation. These results contradict an all-noise interpretation but do not establish a general deployable gain.

W2/Q4 — FP32 robustness. The planned matched deterministic FP32 audit is complete: 21/21 WikiText and 21/21 code cells across seven checkpoints. On 5,246 matched WikiText sites, the forcing/swap ratio changes from 4.25 (BF16) to 8.36 (FP32), largely because swap headroom falls from 0.0075 to 0.0036 nats/site (forcing: 0.0319 to 0.0299). The ratio increases on all seven checkpoints. Neither pooled FP32 cross-fit is detectably positive (swap +0.0002 [−0.0002, +0.0006]; forcing +0.0012 [−0.0015, +0.0040]). Mixtral-8x22B was not run in FP32, and the numerical source of precision sensitivity remains unlocalized (Section 5.3; Appendix A.4).

W2/Q3 — Pruning and recovery. We agree that one-shot pruning cannot determine which ranking is better after recovery fine-tuning or router retraining. The supported finding is operator-dependent ranking: mean within-layer Spearman 0.23; swap-ranked masks have higher accuracy in 12/16 cells (4 Holm-significant), while forcing-ranked masks have better WikiText NLL in 11. No evaluated criterion beats the median random mask on accuracy in more than half its cells. A one-seed router-only pilot is insufficient for a recovery claim; no controlled expert-weight recovery or new router trained from these rankings was evaluated (Section 7; Appendix D, Tables 20–21).

W3 — Readability. We rewrote the abstract, simplified Section 5 into four finding-led subsections, moved key results into main-text Tables 2–4, and clarified the practical and statistical boundaries in Sections 6–9. The central 199/202 same-sample result remains, but the manuscript now explicitly reports positive forcing and swap exceptions and avoids universal claims.

We appreciate the review's emphasis on testing where the conclusions hold, rather than treating the original protocol as universally decisive.

AuthorsOpenReview ↗

Follow-up to Reviewer Jsb9 (Q3): Completed Router Experiments After Pruning

Thank you again for your thoughtful review. Your Question 3 asked whether the differences between intervention operators remain meaningful after pruning and recovery fine-tuning, or when used to train a new router. In our first response we could only say that the one-seed router-only pilot was insufficient and that no new router trained from these rankings had been evaluated. Your question deserved a direct test, so we ran one, and we report the outcome here in full, including the parts that favor neither operator. It does not alter any claim in the manuscript.

Experimental design. We evaluated OLMoE-1B-7B-0924 and Granite-3.0-3B-A800M, each pruned at 25% and 50% of the routed experts in three layers, using the same certified masks as the pruning comparison in Section 7: a swap-ranked mask, a forcing-ranked mask, and four random masks of equal size. We applied two router treatments, with expert weights frozen in both and 500 optimizer steps each: (a) recovery fine-tuning of the existing routers; and (b) discarding the routers, re-initializing them, and training new routers from scratch over the surviving experts. Each ranked mask was run with three seeds and each random mask with one, for 80 runs in total, all completed. The study is post-review, exploratory, and not preregistered.

Results. The table reports held-out WikiText loss in nats as the swap-ranked mask minus the forcing-ranked mask (negative: swap-ranked is better). Intervals are unadjusted paired bootstrap intervals over 1,000 windows, averaged over three seeds.

CheckpointRouter treatmentBudgetOne-shotAfter training [95% interval]Random masks outperformed (of 4): swap / forcing
OLMoE-0924fine-tuned25%+0.0176+0.0115 [+0.0080, +0.0149]1 / 2
OLMoE-0924fine-tuned50%+0.0237+0.0089 [+0.0042, +0.0136]2 / 2
OLMoE-0924new router25%+0.0179+0.0085 [+0.0058, +0.0113]0 / 1
OLMoE-0924new router50%+0.0237+0.0086 [+0.0045, +0.0127]1 / 2
Granite-3.0fine-tuned25%-0.0166-0.0128 [-0.0152, -0.0105]1 / 0
Granite-3.0fine-tuned50%-0.0173-0.0129 [-0.0158, -0.0101]0 / 0
Granite-3.0new router25%-0.0166-0.0075 [-0.0093, -0.0056]0 / 0
Granite-3.0new router50%-0.0173-0.0085 [-0.0105, -0.0065]0 / 0

Answer to your question.

  1. Yes, the difference between the operators remains measurable after both recovery fine-tuning and training a new router. Every interval excludes zero, and the three seeds agree in sign in every row. The difference is, however, smaller than in the one-shot comparison in every row.
  2. The direction of the difference is not consistent across architectures. The forcing-ranked mask attains lower loss on OLMoE-0924, whereas the swap-ranked mask attains lower loss on Granite-3.0.
  3. Neither ranking is reliably better than a random mask of the same size. On Granite-3.0 the ranked masks outperform at most one of the four random masks; on OLMoE-0924, at most two.

So the operators continue to yield measurably different pruned models after router training, but these experiments give no evidence that either operator produces the preferable ranking for practical model improvement. This is consistent with the manuscript's statement in Section 7 and in the limitations that the one-shot result does not establish which ranking is preferable after recovery, and with the one-shot finding that no evaluated criterion exceeds the median random mask on accuracy in more than half of its cells (Appendix D, Tables 20-21). Your caution on this point was well founded.

Limitations. The experiment covers two checkpoints with three pruned layers each; training is router-only with frozen experts and limited to 500 steps; each random mask has a single seed; and the intervals are unadjusted. The Granite-3.0 routers trained from scratch reach a higher loss than the fine-tuned routers (approximately 2.12 versus 2.06 at the 25% budget), so those two rows may be under-trained. We did not evaluate expert-weight recovery or a router trained on per-token operator labels.

Availability. Because this study is exploratory and was completed after the revised manuscript was prepared, we have not added it to the manuscript, whose text on this point is unchanged and remains accurate. The design notes, all 80 per-run records, the analyses, and the code are in the updated supplementary material under postprune_router/, with a summary in postprune_router/README.md.

We appreciate your time and your thoughtful questions, which have materially improved the rigor of this work. We would be glad to provide any further detail you would find helpful.

Reviewer_tbUjOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

Previous research suggested that Mixture-of-Experts (MoE) models had substantial untapped potential. However, earlier studies may have relied on flawed evaluation methods that modified the model’s routing weights during testing. To address this issue, the authors introduce a new swap test method, which replaces only the expert without changing the model’s routing structure. Using this test and a held-out evaluation, where the expert is chosen on one half of the text and scored on the other, they find no hidden potential above what their test can detect. They argue that earlier findings may have resulted from statistical noise caused by evaluating expert selection on the same data used to select the expert. The authors therefore recommend that future research clearly document evaluation methods and assess expert selection on separate, held-out data.

Critical strengths
  1. Novel insights and a very good literature review.
  2. The paper highlights methodological flaws in existing methods, which also affect the model structure and may distort the results. It also provides a systematic comparison of the limitations of existing methods.
  3. The swap is a useful control method: it changes only which expert is used, so the effect of expert identity can be separated from side effects of the intervention.
Critical weaknesses
  1. The paper is very dense with specialized terminology, acronyms, and results presented in the text, making it difficult to read and, at times, incomprehensible. The authors use bold formatting inconsistently and seemingly at random, along with a mixed writing style.
  2. The results are scattered throughout the text, making them difficult to compare. All tables containing results are placed in the appendix, without clear references to them in the main text.
  3. Some conclusions seem stronger than the evidence. The authors themselves say their test cannot detect small effects.
Questions for authors
  1. You focus exclusively on individual words (tokens). Would the “noise” you identify disappear if experts were selected based on entire sentences or documents?
  2. What is the practical use of the proposed method?
1 comment
AuthorsOpenReview ↗

Response to Reviewer tbUj: Presentation, Scientific Scope, and Practical Use

Thank you for identifying the presentation problems and for asking what the reported maximum can actually support. Both shaped the revision.

W1: dense terminology, inconsistent emphasis, mixed style. The paper is reorganized around three contributions: the intervention control, the measurement analysis, and the downstream consequence. The introduction now explains the experiment with one worked example (three operators on the same 3,072 OLMoE sites) before any notation, and the routing-state definition appears once, in Section 3. Bold is used only for run-in paragraph labels and for the primary row of Table 2. The former standalone "diagnostic" section and its checklist are gone; the main text now ends on page 9 and the appendices are consolidated into four sections (A–D).

W2: scattered results, tables only in the appendix. The main text now has three result tables next to the paragraphs that use them: Table 2 (the forcing-to-swap ratio, with one primary row and the other populations labeled), Table 3 (same-sample versus held-out headroom for each operator, with the detection sensitivity), and Table 4 (the exhaustive candidate-pool tests). Each results subsection in Section 5 states its finding in its first sentences.

W3: conclusions stronger than the evidence. We narrowed the summary. The non-detection is stated for the primary restricted-candidate collection under within-continuation evaluation, at its measured sensitivity (80% power at 0.0021 nats per site for the swap and 0.0068 for forcing); it is not a claim that routing opportunity is absent. The matched-noise comparison is described as approximate and uncalibrated. Both kinds of counterevidence are in the abstract and in Section 5.4: a forcing-class effect across independently sampled continuations (+0.0087 [+0.0060, +0.0119] nats per site, driven by one OLMoE cell), and an exploratory positive swap estimate when all 128 experts of one Qwen3 layer are candidates (+0.0044 [+0.0003, +0.0088]). Four further exhaustive-pool studies, including a 256-expert layer, show no detectable swap gain, and the paper states that beside the Qwen3 result (Table 4).

Q1: sentences or documents instead of tokens. Document-level selection is reported as a separate, policy-level test (Appendix B.4). An expert chosen for a cell on other documents and scored on a held-out document gains +0.0029 [+0.0021, +0.0035] nats per site under forcing and −0.0000 [−0.0002, +0.0001] under the swap, relative to a random candidate; on documents the registry never saw the values are +0.0015 [+0.0003, +0.0031] and +0.0000 [−0.0002, +0.0002]. So coarser selection does not make the swap's headroom persistent, while a forcing-class component does carry over, measured against forcing's own reference. We did not run a separate sentence-level experiment, and these results do not rule out every context-dependent selector.

Q2: practical use. The swap is a diagnostic control, not a new router or pruning method. Added to an existing audit, it shows how much of a reported headroom depends on the routing weights the intervention changed, and a held-out evaluation shows how much of the maximum is retained. The practical recommendation is a reporting one (Conclusion): state the operator and its reference, the candidate pool, the scoring horizon, the held-out protocol, numerical validity and detection resolution. The pruning experiment (Section 7) shows that operator choice changes expert rankings; it does not establish a consistently superior ranking criterion, and we have not validated the recommendation as a way to save engineering effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.