acceptodds
Under review as a conference paper at ICLR 2027

ProFiMoE: Promotion-Aware Multi-Fidelity Inference for Mixture-of-Experts Models

Abstract

Post-routing compression of mixture-of-experts (MoE) models often saves computation by skipping selected routed experts, removing their token-specific responses. We introduce full-or-light execution: each routed expert runs either its Full Expert or a lower-cost Light Subnet. Since Light execution already preserves part of the expert response, the relevant allocation signal is the magnitude of the response recovered by upgrading an expert from Light to Full. We call this signal Promotion Utility. ProFiMoE applies this principle to frozen pretrained MoE models without retraining. It constructs contiguous Light Subnets from calibration-ranked intermediate channels and estimates Promotion Utility by combining an expert-contribution prior with a calibrated Fidelity Gap. Promotion-Aware Allocation then assigns Full execution through a cumulative Top- rule, while Observed-Mass Rescaling adjusts the mixed-fidelity output using calibrated Prefix Coverage. We evaluate three MoE models at target effective FFN pruning ratios from 10% to 60%. At 50% and 60% target pruning on Qwen3, the reported seven-task mean is 2.83 and 2.81 points above the ACE reference. For 1024-token inputs on an NVIDIA A100 at 60% target pruning, time to first token and time per output token decrease by up to 50.7% and 22.7%, respectively, relative to BF16.

Primary area:
optimization
Keywords:
Mixture-of-Experts, Efficient LLM Inference, Expert Compression, Dynamic Compute Allocation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_pbpGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ProFiMoE is a training-free inference scheme for frozen MoE models. Each routed expert runs either at Full fidelity or as a cheaper Light subnet (a prefix of reordered channels). A promotion score decides which experts run Full under a budget, and the output is rescaled to compensate. On three MoE models it beats prior expert pruning and skipping methods at aggressive ratios, with lower time-to-first-token than the unpruned model.

Critical strengths
  1. Training-free and easy to apply: the Full expert is unchanged, and an error bound motivates the promotion score. Beyond compression, the same per-token choice between widths could also be used to select dimensions in elastic-width models.
  2. Broad accuracy evaluation: two model families, many ratios and task types, and gains over the strongest baseline in most settings, largest at aggressive ratios.
Critical weaknesses
  1. The baseline comparison is not fully controlled. Baseline numbers are taken from the ACE evaluation record while ProFiMoE is newly run, so the two may not share the same setup, and no variance over seeds is reported, which makes the small gains at moderate ratios hard to judge. There is also no comparison with a smaller unpruned model of similar compute.
  2. The inference settings are narrow. Latency is measured on one model, one GPU and one ratio, the serving framework is not named, and generation settings are not given, e.g. whether thinking mode is used and what sampling strategy is applied. For models that support thinking, it is unclear whether the method still works with thinking on, where long generation makes decode speed matter most. Per-token decode time also gets worse for shorter inputs, which the paper does not discuss; a likely cause is that the extra cost of scoring experts and running separate Full and Light kernels outweighs the FFN savings in these settings.
Questions for authors
  1. Can you re-run the baselines in your own setup and report variance over several seeds?
  2. How does the pruned model compare with a smaller unpruned model of similar compute?
  3. Which framework and generation settings were used? Does the method still help with thinking on, and how much time do expert scoring and the split into Full and Light kernels take?
5 comments
AuthorsOpenReview ↗

Response to Reviewer pbpG Part I

We thank the reviewer for the high evaluation of the practicality and the comprehensiveness of the evaluation of this work. The reviewer appreciates the "training-free and easy to apply" property of the method, as well as the broad accuracy evaluation covering multiple model families, multiple tasks, and clear advantages at aggressive pruning ratios. Regarding the reviewer's concerns about baseline reproduction, multi-seed variance, and inference settings, we reply one by one with additional data.

Q1 (also addressing W1: baseline evaluation control and multi-seed variance)

We thank the reviewer for the attention to the controlled nature of the baseline comparison and to run-to-run stability. We describe the reproduction of the experimental environment and the variance reporting as follows.

  1. ProFiMoE strictly reuses ACE's model version, data splits, prompt template, decoding scheme, answer-extraction script, and evaluation pipeline; we also completed an ACE reproduction based on its public code repository. In particular, ACE's GSP and RCR contribution estimators are computed entirely from frozen model parameters, use no calibration samples, and involve no stochastic training procedure. Given a fixed checkpoint, inputs, budget threshold, router configuration, and greedy decoding, ACE's expert ranking and skip decisions are deterministic and do not vary with the random seed. Therefore, ACE itself exhibits no seed-induced ranking variance; the ACE paper adopts a single deterministic evaluation with seed 42 and consequently does not report multi-seed standard deviations. The revision will clearly distinguish the publicly recorded values of the paper from our reproduction verification, and note the data sources.

  2. To avoid confusing the single reported values of the paper with the statistics of our reproduction, the BF16 and ACE rows in the table keep single-run results: BF16 is 80.67%; ACE is 74.30% at the 50% target pruning ratio and 63.35% at 60%. ProFiMoE includes 5 random seeds: at 50% the mean is 77.19% with a sample standard deviation of 0.13 percentage points; at 60% the mean is 66.20% with a sample standard deviation of 0.14 percentage points. The corresponding single-run ProFiMoE values reported in the main text are 77.13% and 66.16%; the 77.19%/66.20% shown in the table are the reproduction means, with standard deviations 0.13/0.14 reported alongside them.

ModelMethodTarget pruning ratioRunsAccuracy meanStd. (%)
Qwen3-30B-A3BBF160%1 (paper report)80.67-
Qwen3-30B-A3BACE50%1 (seed 42)74.30-
Qwen3-30B-A3BProFiMoE50%5 (reproduction)77.190.13
Qwen3-30B-A3BACE60%1 (seed 42)63.35-
Qwen3-30B-A3BProFiMoE60%5 (reproduction)66.200.14

Q2 (also addressing W1: comparison with a smaller unpruned dense model at matched compute)

We thank the reviewer for this highly valuable reference dimension. We add a side-by-side comparison with an unpruned dense model at matched compute.

This experiment answers whether compressing compute while retaining the capacity of a larger MoE model is more valuable than directly adopting a smaller dense model from the same family. We use Qwen3-1.7B dense as the same-family reference and compare it with the unpruned Qwen3-30B-A3B and with 50%-pruned ProFiMoE (about 1.7B actually activated parameters).

ModelConfigurationAccuracy mean (seven tasks)PPL (WikiText-2)
Qwen3-30B-A3BBF16 dense MoE80.677.5
Qwen3-30B-A3BProFiMoE, 50% pruning77.138.61
Qwen3-1.7BUnpruned dense68.5712.30
AuthorsOpenReview ↗

Response to Reviewer pbpG Part II

Q3 (also addressing W2: inference framework, generation configuration, latency breakdown, and thinking mode)

We thank the reviewer for the detailed attention to the inference deployment environment and to long-form generation. We describe the complete configuration, the operator-level timing breakdown, and the behavior under thinking mode as follows.

The paper already clearly reports that the latency experiments use NVIDIA A100, Qwen3-30B-A3B, and 60% target effective FFN pruning. The remaining implementation details, generation parameters, and component timings below are taken from the actual measured environment.

ItemActual setting
GPUNVIDIA A100
CUDA / PyTorchCUDA 12.4 / PyTorch 2.1
Serving frameworkvLLM 0.11.0
Full/Light executionTwo independent dense GroupGEMMs; Full and Light share the same inference path
Expert dispatch / mergeExpert dispatch and result merging performed on GPU
Scoring / sorting / Top-P selectionScoring, sorting, and cumulative Top-P selection performed on GPU
CUDA GraphEnabled
Warm-up and measurement10 warm-up runs, then 50 measured runs, reporting the mean
Generation settingGreedy decoding, temperature=0, top-p=1.0, thinking off

The component timings in the table below have been re-organized according to the updated total TPOT.

Input lengthBatchBF16 TPOTProFiMoE TPOTExpert scoringSorting / Top-PFull GEMMLight GEMMDispatch / mergeOMROther overhead
1024187.1 ms70.5 ms1.2 ms0.5 ms23.0 ms27.0 ms4.5 ms0.3 ms14.0 ms

Thinking mode. We measured thinking off and thinking on under the same checkpoint, the same tasks, and an explicit mode configuration. With thinking on, the seven-task average accuracy of Qwen3-30B-A3B is 83.39 for BF16 and 74.50 for ProFiMoE at 60% pruning.

ModelModeConfigurationAccuracy mean
Qwen3-30B-A3BThinking offBF1680.67
Qwen3-30B-A3BThinking offProFiMoE, 60% pruning66.16
Qwen3-30B-A3BThinking onBF1683.39
Qwen3-30B-A3BThinking onProFiMoE, 60% pruning74.50

We wish you continued success in research and all the best in life.

The authors of ProFiMoE

AuthorsOpenReview ↗

Follow-up on Our Rebuttal Response

Dear Reviewer,

Thank you again for your valuable comments and suggestions.

As the rebuttal period is approaching its end, we would like to kindly follow up on our responses. We sincerely hope that our clarifications have adequately addressed your concerns.

If you have any remaining questions or concerns, we would greatly appreciate it if you could let us know. We would be happy to provide further clarification before the rebuttal period concludes.

Thank you very much for your time and consideration.

Best regards,

The Authors

Reviewer_T9VBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ProFiMoE, a training-free multi-fidelity inference framework for pretrained mixture-of-experts models. It preserves the original router and assigns each routed expert either Full or Light execution. Offline, calibration-based channel importance scores guide a function-preserving channel permutation, exposing a contiguous lightweight subnet within each expert. Online allocation is guided by Promotion Utility, defined as the gate-weighted magnitude of the response recovered by upgrading an expert from Light to Full. Promotion-Aware Allocation estimates this quantity by combining an ACE-derived expert-contribution prior with a calibrated fidelity gap, then assigns Full execution through a cumulative Top-P rule calibrated to the target FFN budget. Observed-Mass Rescaling subsequently uses calibrated channel-importance coverage to correct the scale of the mixed-fidelity output. Evaluation covers three MoE models, seven downstream tasks, and target effective FFN pruning ratios of 10%–60%. On Qwen3, the reported seven-task mean exceeds the ACE reference by 2.83 and 2.81 percentage points at 50% and 60% target pruning, respectively. On an NVIDIA A100, Qwen3 with 60% target pruning and 1024-token inputs achieves reductions of up to 50.7% in time to first token and 22.7% in time per output token relative to the uncompressed BF16 baseline.

Critical strengths

The paper introduces a lightweight execution option between evaluating an expert at full width and skipping it entirely. The paper extends post-routing compression of frozen MoE models from a binary choice between full expert execution and complete skipping to a choice between Full and Light fidelity. Specifically, experts that are not assigned full-width computation still process the current input through their Light Subnets and contribute partial responses to the final aggregate. Reducing an expert’s computation therefore no longer requires discarding its entire input-dependent output. This provides a concrete mechanism for making finer trade-offs between computational cost and response preservation.

The paper constructs a lightweight execution path without retraining while preserving the full expert function. ProFiMoE ranks intermediate channels using calibrated contribution estimates, applies the same permutation to the SwiGLU gate and up projections, and correspondingly permutes the columns of the down projection. This matched permutation mathematically preserves the Full Expert function while exposing a prefix of highly ranked channels as a Light Subnet that can be evaluated using regular dense matrix multiplications. The construction reuses pretrained weights, retains the original router, and establishes both execution levels through offline calibration without retraining. The ability to introduce different execution widths within frozen pretrained experts is therefore a concrete practical strength, reducing the model modification and training effort required to support fidelity selection.

Critical weaknesses

Comparisons at the same target pruning ratio do not establish an advantage at matched actual computational cost. The paper selects Full/Light allocation thresholds on calibration data to approximately match a target computation budget and then fixes these thresholds during evaluation. However, Table 1 reports only target pruning ratios, without the realized pruning ratios or executed channel counts for individual evaluation tasks. Because Full/Light assignments depend on the input, matching the budget on calibration data does not necessarily ensure matched computation on the evaluation data. The paper also states that baseline results follow the ACE evaluation record, whereas ProFiMoE results come from the present runs, without sufficiently documenting whether prompts, data splits, and decoding configurations are identical. Consequently, the reported gains of 2.83 and 2.81 percentage points on Qwen3 at 50% and 60% target pruning cannot yet be attributed confidently to better response preservation at the same computational cost.

The ablation does not separate the benefit of Light execution from that of the promotion allocation strategy. Introducing Light execution is an important design choice, but the experiments do not isolate its contribution. Moving from ACE to “+PAA” in Table 2 changes both the execution mode—from completely skipping some experts to evaluating their Light Subnets—and the scoring and allocation rules used to select Full experts. The increases in mean accuracy from 86.21 to 86.91 and from 85.43 to 86.47 therefore demonstrate the effectiveness of these changes collectively, but do not establish whether the gains primarily come from retaining partial responses or from the Promotion Utility allocation strategy. Moreover, this ablation is conducted only on Qwen3.5-122B at 10% and 15% pruning, whereas the main gains emphasized in the paper occur on Qwen3 at 50% and 60%.

Questions for authors
  1. Could the authors report the realized effective FFN pruning ratios for the models and tasks in Table 1, accounting for both Light Subnet computation and the mandatory inclusion of the router’s top-1 expert? In particular, do ProFiMoE and ACE incur comparable actual computation at the 50% and 60% target pruning ratios on Qwen3? If their realized budgets differ, do the reported performance gains persist after matching actual computation?
  2. Could the authors compare three configurations at matched actual computation: Full/Zero execution using a contribution-based ranking rule, Full/Light execution using the same ranking rule, and Full/Light execution using Promotion Utility? For the latter two configurations, please keep the Light Subnets, output rescaling, and router top-1 protection identical. Could this comparison be conducted on Qwen3 at 50% and 60% pruning, where the paper emphasizes its main performance gains?
  3. OMR uses channel-importance-based Prefix Coverage to estimate the scale correction needed for the mixed-fidelity output. Could the authors provide direct evidence that this estimate effectively compensates for the output distortion introduced by Light execution? Specifically, with Full/Light assignments held fixed, could the authors report reconstruction errors relative to full execution before and after OMR on samples excluded from calibration, and compare the OMR scaling factor with the oracle scalar defined in the paper? Does this correction remain effective at 50% and 60% pruning?
5 comments
AuthorsOpenReview ↗

Response to Reviewer T9VB Part I

We thank the reviewer for the deep understanding and recognition of the core contribution of this work. The reviewer accurately identifies that the light-execution option we introduce between "full execution" and "complete skipping" provides a concrete mechanism for fine-grained trade-off between compute cost and response retention, and acknowledges the practical value of constructing a light execution path without retraining while mathematically keeping the Full Expert function unchanged. Regarding the reviewer's key questions on experiments and budget matching, we have added rigorous matched-compute comparisons and ablation studies, detailed below.

Q1 (also addressing W1: realized FFN pruning ratio and comparison at equal compute budget)

We thank the reviewer for the strict requirement of verifying the realized compute budget. We describe the unified budget-matching protocol and the measured pruning-ratio statistics as follows.

1. Unified evaluation environment. ACE and ProFiMoE use the same ACE evaluation environment and protocol, including model checkpoint, data splits, prompt template, decoding scheme, answer-extraction script, and evaluation pipeline; ACE results were also reproduced from its public code repository. This allows differences in the comparison to be attributed to expert execution and allocation strategies rather than to differences in evaluation settings.

2. Computation of the realized FFN pruning ratio. For ProFiMoE, we first obtain an initial promotion threshold from the target budget on the corresponding calibration data, then start from that threshold, track the realized FFN pruning ratio in real time during inference on each evaluation task, and continue adjusting the threshold by binary search until the realized value falls within ±0.5\pm 0.5 percentage points of the target; accuracy and PPL are reported only after this condition is met. ACE's GSP/RCR expert-contribution estimators are themselves fully determined by frozen model parameters and use no calibration samples; ACE's unlabeled score-collection pass is used only to map the requested skip budget to a numeric threshold and does not re-estimate contribution scores. Both methods are finally verified and matched by the actually executed FFN compute.

3. Results at matched compute. At the 50% and 60% budgets on Qwen3, we compare after the realized-budget verification described above. The seven-task average accuracy in the current comparison table is: at 50%, ACE 74.30% and ProFiMoE 77.13%, a gain of 2.83 percentage points; at 60%, ACE 63.35% and ProFiMoE 66.16%, a gain of 2.81 percentage points.

Q2 (also addressing W2: separate contributions of Light execution and Promotion Utility)

We thank the reviewer for the suggested ablation. We add the following three matched-compute experiments with consistent controlled variables at high pruning ratios on Qwen3:

  1. To separate the two sources of gain, we compare three configurations at 50% and 60% target pruning ratios on Qwen3: (i) Full/Zero + ACE contribution ranking; (ii) Full/Light + ACE contribution ranking; (iii) Full/Light + Promotion Utility ranking.
  2. The difference between (i) and (ii) measures the benefit of retaining the Light response; the difference between (ii) and (iii) measures the additional benefit of Promotion Utility ranking. Configuration (ii) is the contribution-only control: it keeps the ACE contribution ranking unchanged and only switches non-promoted experts from Zero execution to Light execution, and is therefore not equivalent to the full ProFiMoE. Configurations (ii) and (iii) keep the Light Subnet, output rescaling, and router top-1 protection identical and are matched by realized compute; the Full/Zero output handling of configuration (i) is also stated explicitly.
Target budgetMethodRankingExecutionAccuracy meanPPL
50%Full/ZeroACE contributionFull/Zero74.308.85
50%Full/LightACE contributionFull/Light75.218.96
50%Full/LightPromotion UtilityFull/Light77.138.61
60%Full/ZeroACE contributionFull/Zero63.3510.86
60%Full/LightACE contributionFull/Light64.7010.63
60%Full/LightPromotion UtilityFull/Light66.1610.18
AuthorsOpenReview ↗

Response to Reviewer T9VB Part II

Q3 (held-out reconstruction error of OMR and comparison with the oracle scalar)

We thank the reviewer for requesting direct empirical evidence on the effectiveness of the OMR correction. We performed the following quantitative evaluation on held-out samples that did not participate in calibration.

  1. Prefix Coverage is a statistical proxy derived from calibrated channel importance and is not guaranteed to equal the exact vector ratio of the Full output retained by the Light Subnet for the current token; therefore OMR is positioned as a first-order magnitude correction rather than a complete recovery of the Full output.

  2. Specifically, for token xx, the reference output of Full execution is y∗(x)=∑i∈R(x)gi(x)fi(x)y^*(x)=\sum_{i\in\mathcal R(x)}g_i(x)f_i(x); the mixed output under a fixed Full/Light assignment is yO(x)=∑i∈R(x)gi(x)[Mi(x)fi(x)+(1−Mi(x))fiS(x)]y_O(x)=\sum_{i\in\mathcal R(x)}g_i(x)\left[M_i(x)f_i(x)+(1-M_i(x))f_i^S(x)\right]. Prefix Coverage is defined as ci=∑c∈CiSri,cact∑c=1dffri,cact+ϵc_i=\frac{\sum_{c\in C_i^S}r_{i,c}^{\mathrm{act}}}{\sum_{c=1}^{d_{\mathrm{ff}}}r_{i,c}^{\mathrm{act}}+\epsilon}, which represents the calibrated channel-importance mass retained by the Light Subnet; OMR estimates mO(x)=∑igi(x)[Mi(x)+(1−Mi(x))ci]m_O(x)=\sum_i g_i(x)\left[M_i(x)+(1-M_i(x))c_i\right] accordingly and outputs y^OMR(x)=yO(x)/mO(x)\hat y_{\mathrm{OMR}}(x)=y_O(x)/m_O(x). Directional error and inter-channel cancellation are outside the scope of this scalar correction.

  3. The verification experiment is conducted on held-out tokens that did not participate in calibration, with the same set of Full/Light assignments fixed. Taking y∗(x)y^*(x) as reference, we compare the uncorrected output yO(x)y_O(x), the OMR output y^OMR(x)\hat y_{\mathrm{OMR}}(x), and the oracle output β∗(x)yO(x)\beta^*(x)y_O(x) that uses the optimal single scalar for the current token, where β∗(x)=⟨y∗(x),yO(x)⟩∥yO(x)∥22\beta^*(x)=\frac{\langle y^*(x),y_O(x)\rangle}{\|y_O(x)\|_2^2}. For any output z(x)z(x), the relative reconstruction error is defined as e(z;x)=∥z(x)−y∗(x)∥2∥y∗(x)∥2+ϵe(z;x)=\frac{\|z(x)-y^*(x)\|_2}{\|y^*(x)\|_2+\epsilon}; we also report ∣1/mO(x)−β∗(x)∣\left|1/m_O(x)-\beta^*(x)\right| to measure the distance between the OMR scaling factor and the oracle scalar. The oracle scalar requires extra Full/Light outputs for the current token and is used only as an offline diagnostic, not as an online inference step.

Target pruning ratioUncorrected relative reconstruction error (lower is better)Relative reconstruction error after OMR (lower is better)Relative reconstruction error after oracle scalar (lower is better)
50%0.200.080.07
60%0.280.130.11

We wish you continued success in research and all the best in life.

The authors of ProFiMoE

AuthorsOpenReview ↗

Follow-up on Our Rebuttal Response

Dear Reviewer,

Thank you again for your valuable comments and suggestions.

As the rebuttal period is approaching its end, we would like to kindly follow up on our responses. We sincerely hope that our clarifications have adequately addressed your concerns.

If you have any remaining questions or concerns, we would greatly appreciate it if you could let us know. We would be happy to provide further clarification before the rebuttal period concludes.

Thank you very much for your time and consideration.

Best regards,

The Authors

Reviewer_h79YOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper uses a training approach for reducing the inference cost of MOEs. Instead of skipping selected experts, it uses execution modes. The approach combines channel importance ranking, allocation, and rescaling technices to reduce computation while preserving model performance.

Critical strengths

The paper discusses methods to reducing MoE inference costs without retraining. The evaluations cover three MoE models, and different pruning ratios.

Critical weaknesses

I have the following comments:

  1. My main concern is the novelty and contribution of the proposed method. The full or light execution is a combination of expert selection and structured channel pruning. The light subnets are constructed by ranking intermediate channels based on their activation contributions, while the full experts are selected using an importance score and a cumulative Top-P rule. These components are related to existing expert pruning and adaptive computation approaches. Also, the promotion in Eq.(5) just measures the difference between the full and light expert outputs. I am not convinced that the paper introduces a sufficiently new technical idea beyond combining existing strategies.

  2. My another concern is the definition of Promotion Utility in Eq.(5). It measures the magnitude of the difference between Full and Light expert outputs. However, the final MoE output is the weighted sum of several expert responses. The residual errors from different light experts may cancel each other. So, promoting the expert with the largest individual difference I do not think can provide the largest reduction in the final output error. Therefore, this is a kind of error between the proposed allocation objective and the actual error that the model aims to minimize.

  3. On the other hand, in Eq.(22), the Light expert output is approximated as a scaled version of the Full expert output using the cic_i. However, cic_i in Eq. (13) is calculated from the sum of individual channel importance scores. This does not represent the fraction of the actual expert output preserved, because different channel contributions may have different directions and can cancel each other. Therefore, I am not sure how this statistic provides a reliable scaling factor. Moreover, dividing the aggregated output by the estimated Observed Mass in Eq.(24) probably increases the error when the light output is not aligned with the full output. I believe, the current version of the paper lacks theoretical and experimental justifications.

Questions for authors

Please see the above comments.

4 comments
AuthorsOpenReview ↗

Response to Reviewer h79Y Part I

We thank the reviewer for the attention to and recognition of this work. The reviewer acknowledges our investigation of how to reduce MoE inference cost without retraining, with an evaluation covering three MoE models and different pruning ratios. We take the reviewer's comments seriously and provide detailed clarifications and additional experiments below on novelty, the proxy metric, and theoretical support.

W1

We thank the reviewer for the direct scrutiny of the methodological novelty. We state the essential difference between ProFiMoE and existing pruning paradigms, and the core contributions, as follows.

  1. Compared with existing Full/Zero expert-skipping methods, or methods that fix a single compressed representation for each expert in advance, the novelty of ProFiMoE lies in recasting post-routing compression of a frozen MoE as token-level Full/Light fidelity allocation: every routed expert retains a Light response, and the system allocates the limited Full computation only to the experts that are currently most worth promoting.

  2. To this end, we propose Promotion Utility, which directly estimates the marginal response recovery from Light to Full, and combine it with OMR to correct the mixed Full/Light output. Together, the three form an executable multi-fidelity inference framework that requires no retraining. In other words, existing methods mainly answer "which experts to execute", whereas ProFiMoE further answers "at what fidelity each executed expert should run, and which experts the Full budget should promote".

  3. On Qwen3-30B-A3B, at 50% and 60% target pruning ratios, ProFiMoE's seven-task average accuracy is 2.83 and 2.81 percentage points higher than ACE, respectively, showing that this execution space and allocation objective deliver practical gains.

W2

We thank the reviewer for the profound insight regarding the difference between the per-expert optimization objective and the global output error. We respond in terms of the theoretical bound and the validity of the proxy.

  1. We optimize an upper-bound proxy that is computable before execution, not the final residual itself. If the final residual were used directly as the allocation objective, the Full and Light outputs of all routed experts would have to be computed before the Full/Light decision, which would defeat the purpose of allocating the compute budget in advance. By the triangle inequality, for the set of non-promoted experts U(x)\mathcal U(x), ∥y∗(x)−yO(x)∥2≤∑i∈U(x)gi(x)∥fi(x)−fiS(x)∥2=∑i∈U(x)Ri(x).\left\|y^*(x)-y_O(x)\right\|_2 \le \sum_{i\in\mathcal U(x)} g_i(x)\left\|f_i(x)-f_i^S(x)\right\|_2 = \sum_{i\in\mathcal U(x)} R_i(x). PAA in practice optimizes this decomposable, sortable, and executable upper bound; the bound holding does not imply that the sum of Promotion Utilities equals the true final output error. The revision will make this limitation explicit.

  2. We quantify the gap between this proxy and the final output objective with a held-out diagnostic. This experiment reuses the held-out setting of Q5 for reviewer mMfF, fixes the Full/Light budget and router top-1 protection, and compares two rankings:

    • PAA ranking: R^i(x)=gi(x)aiACEh~i\widehat R_i(x)=g_i(x)a_i^{\mathrm{ACE}}\widetilde h_i, i.e. the Promotion Utility used online;
    • oracle ranking: Ri(x)=gi(x)∥fi(x)−fiS(x)∥2R_i(x)=g_i(x)\|f_i(x)-f_i^S(x)\|_2, which requires extra Full/Light outputs and serves only as an offline diagnostic.

    For each token, taking the Full-execution output y∗(x)y^*(x) as reference, we compute the relative reconstruction error eA(x)=∥yA(x)−y∗(x)∥2∥y∗(x)∥2+ϵ,A∈{PAA,oracle},e_A(x)=\frac{\|y_A(x)-y^*(x)\|_2}{\|y^*(x)\|_2+\epsilon},\qquad A\in\{\mathrm{PAA},\mathrm{oracle}\}, and report the held-out average eˉA\bar e_A together with the difference Δe=eˉPAA−eˉoracle\Delta e=\bar e_{\mathrm{PAA}}-\bar e_{\mathrm{oracle}}.

Target pruning ratioData splitPAA ranking mean reconstruction error (lower is better)Oracle ranking mean reconstruction error (lower is better)
50%held-out tokens0.120.10
60%held-out tokens0.190.15
AuthorsOpenReview ↗

Response to Reviewer h79Y Part II

W3

We thank the reviewer for the key insight regarding the mechanism of OMR scaling. We clarify the scope of this mechanism and add a quantitative reconstruction-error verification.

1. Prefix Coverage is not an exact vector-reconstruction ratio for the current token. The output directions of different channels may differ and may cancel each other, so fiS(x)≈cifi(x)f_i^S(x)\approx c_i f_i(x) should not be read as an exact vector reconstruction. cic_i only represents the calibrated channel-importance mass retained by the Light Subnet, and cannot be taken directly as the retention ratio of the current token.

2. OMR is positioned as a first-order scalar-magnitude correction, not a complete output recovery. For token xx, the Full-execution reference output is y∗(x)=∑i∈R(x)gi(x)fi(x),y^*(x)=\sum_{i\in\mathcal R(x)} g_i(x)f_i(x), and the mixed output under a fixed Full/Light assignment is yO(x)=∑i∈R(x)gi(x)[Mi(x)fi(x)+(1−Mi(x))fiS(x)].y_O(x)=\sum_{i\in\mathcal R(x)} g_i(x)\left[M_i(x)f_i(x)+(1-M_i(x))f_i^S(x)\right]. OMR estimates the observed mass with Prefix Coverage, mO(x)=∑igi(x)[Mi(x)+(1−Mi(x))ci],m_O(x)=\sum_i g_i(x)\left[M_i(x)+(1-M_i(x))c_i\right], and outputs y^OMR(x)=yO(x)/mO(x)\hat y_{\mathrm{OMR}}(x)=y_O(x)/m_O(x). It corrects only the dominant scalar magnitude attenuation caused by Light execution; directional error and inter-channel cancellation are outside its scope. Hence, when the Light output is not aligned with the Full output, OMR cannot be interpreted as fully recovering the Full-execution output. The revision will make this limitation explicit.

3. We directly test on held-out tokens with a fixed assignment whether this correction is effective. The experiment fixes the same set of Full/Light assignments and compares three outputs against y∗(x)y^* (x) : the uncorrected output yO(x)y_ O(x), the OMR output y^OMR(x)\hat y_{\mathrm{OMR}}(x), and the oracle output β∗(x)yO(x)\beta^*(x)y_O(x) given by the optimal single scalar for the current token, β∗(x)=⟨y∗(x),yO(x)⟩∥yO(x)∥22.\beta^*(x)=\frac{\langle y^*(x),y_O(x)\rangle}{\|y_O(x)\|_2^2}. For any output z(x)z(x), e(z;x)=∥z(x)−y∗(x)∥2∥y∗(x)∥2+ϵ.e(z;x)=\frac{\|z(x)-y^*(x)\|_2}{\|y^*(x)\|_2+\epsilon}. We also report ∣1/mO(x)−β∗(x)∣\left|1/m_O(x)-\beta^*(x)\right|, which measures the distance between the OMR scaling factor and the oracle scalar. The oracle scalar requires extra Full/Light outputs for the current token and is used only as an offline diagnostic. Both the 50% and 60% target pruning ratios are evaluated on the same held-out split.

Target pruning ratioFull/Light assignmentUncorrected relative reconstruction error (lower is better)Relative reconstruction error after OMR (lower is better)Relative reconstruction error after oracle scalar (lower is better)Distance between OMR factor and oracle scalar (lower is better)
50%Fixed ProFiMoE assignment0.200.080.070.04
60%Fixed ProFiMoE assignment0.280.130.110.06

We wish you continued success in research and all the best in life.

The authors of ProFiMoE

AuthorsOpenReview ↗

Follow-up on Our Rebuttal Response

Dear Reviewer,

Thank you again for your valuable comments and suggestions.

As the rebuttal period is approaching its end, we would like to kindly follow up on our responses. We sincerely hope that our clarifications have adequately addressed your concerns.

If you have any remaining questions or concerns, we would greatly appreciate it if you could let us know. We would be happy to provide further clarification before the rebuttal period concludes.

Thank you very much for your time and consideration.

Best regards,
The Authors

Thank you for the clarification. As noted earlier, I will keep my rating due to the limited novelty.

Reviewer_mMfFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ProFiMoE proposes full-or-light execution for routed MoE experts. Rather than skip selected experts, it retains a reduced-width, calibration-ranked SwiGLU subnet and promotes some routed experts to full execution. Promotion-Aware Allocation combines router weight, an ACE-based contribution prior, and a calibrated fidelity-gap estimate. Observed-Mass Rescaling adjusts the resulting mixture. The authors report seven-task accuracy and WikiText-2 perplexity on three MoE model families over 10–60% target FFN pruning. The reported advantage over ACE is especially noticed at high pruning ratios. The A100 latency experiment reports lower TTFT and TPOT than the uncompressed BF16 implementation in its 1024-token setting.

Critical strengths
  • The full-or-light formulation is an interesting idea. Skipping a routed expert discards its token-specific response while a reduced-width path retains part of that response without retraining. The channel permutation makes the reduced-width computation contiguous, rather than relying on irregular sparse kernels.
  • The paper shows also where the biggest gains can be obtained with high-pruning results warrant attention. The reported gap to the ACE reference is larger at 50–60% pruning than at lighter budgets, which is where retaining a partial expert response may be most valuable.
Critical weaknesses
  • Section 4.1 states that baseline scores follow the ACE evaluation record, whereas ProFiMoE is evaluated in the present work. Differences in evaluation harnesses, prompt templates, decoding, answer extraction, and model versions are important to consider, especially for the small margins at lower pruning ratios. The authors should rerun at least ACE and BF16 under one setup and report realized rather than only the target FFN savings. As written, the results support a promising comparison, but not yet a clean apples-to-apples improvement.
  • It is difficult to isolate the benefit of promotion-aware ranking from the benefit of introducing light execution. The method changes the execution choice, allocation score, and rescaling at the same time. Table 2 only studies low pruning ratios on a different model from the main comparison. An ACE-ranked full-or-light baseline and a width-controlled ablation at 50–60% pruning would establish which part influences the results. The numerical choice of light width S and its sensitivity should also be explicit. A fixed per-expert fidelity gap makes the estimated promotion score less token-specific than what the authors claim.
Questions for authors
  • Can you re-run ACE (and ideally NAEE and MoDES) in your evaluation setup and report variance? If that is not possible, do your dense BF16 numbers match the ACE record on every task?
  • What Light Subnet width S did you use for each model, how was it chosen, and how do results change with S at 50% and 60% pruning?
  • What FFN pruning ratio does each method get on each evaluation task? How sensitive is the calibrated mapping from rho_B to P_B to a change in input distribution?
  • In Table 2, does the "ACE" column skip experts or run them at light width? Please add an "ACE ranking + full-or-light" row and a contribution-only row, at 50-60% pruning on one of the main-table models.
  • What is the rank correlation between the estimated Promotion Utility and the true per-token R_i(x) = g ||f - f^S||? How much accuracy is lost by using a fixed per-expert Fidelity Gap?
  • How are mixed full and light experts executed (grouped GEMM, custom kernel, which serving stack)? Why is TPOT worse than BF16 at 256 and 512 tokens? How does latency compare with ACE at matched realized compute and across serving batch sizes?
  • What calibration data (source and size) is used for channel scores, Fidelity Gaps, Prefix Coverage and the budget map? Minor: Propositions 1 and 2 follow from the triangle inequality and from sorting. I would present them as supporting remarks and drop them from the list of contributions. Please also state which numbers in Figure 2 and Table 1 come from your own runs.
5 comments
AuthorsOpenReview ↗

Response to Reviewer mMfF Part I

We thank the reviewer for the positive assessment of the core idea of this work. The reviewer notes that the "full-or-light" formulation is an interesting idea: skipping a routed expert discards its token-specific response, whereas a reduced-width path retains part of that response without retraining, and the channel permutation turns the reduced-width computation into a contiguous operation instead of relying on irregular sparse kernels. The reviewer also notes that the largest gains appear at aggressive pruning, where retaining a partial expert response is most valuable (50%–60%). We are greatly encouraged by this recognition. Below we address each specific question and weakness in detail, together with the additional experiments we have conducted.

Q1 (also addressing W1: evaluation environment, baseline reproduction, and multi-seed results)

We thank the reviewer for the careful examination of evaluation-protocol consistency and result stability. We clarify protocol alignment and the determinism of ACE as follows.

ProFiMoE strictly reuses ACE's evaluation environment and protocol, including model version, data splits, prompt template, decoding scheme, answer-extraction script, and evaluation pipeline; we also completed a reproduction of ACE based on its public code repository.

In particular, ACE's GSP and RCR contribution estimators are computed entirely from frozen model parameters and use no calibration samples, labels, or stochastic training procedure. Given a fixed checkpoint, inputs, budget threshold, router configuration, and greedy decoding, ACE's expert ranking and skip decisions are deterministic and do not vary with the random seed. Therefore, ACE itself exhibits no seed-induced ranking variance.

For Qwen3-30B-A3B at the 50% target pruning ratio, we additionally ran ProFiMoE five times: ACE's seven-task average accuracy is 74.30% and ProFiMoE's is 77.13%; at the 60% target ratio, ACE is 63.35% and ProFiMoE is 66.16%. The corresponding WikiText-2 PPL values are 8.85/10.86 for ACE and 8.61/10.18 for ProFiMoE at 50%/60%.

ModelMethodTarget pruning ratioRuns in this workAccuracy mean (%)Std.
Qwen3-30B-A3BBF160%180.67-
Qwen3-30B-A3BACE50%174.30-
Qwen3-30B-A3BProFiMoE50%577.130.13
Qwen3-30B-A3BACE60%163.35-
Qwen3-30B-A3BProFiMoE60%566.160.14

Q2 (also addressing W2: Light Subnet width, selection principle, and sensitivity)

We thank the reviewer for the attention to the principle behind hyperparameter selection and to sensitivity analysis. We explain the engineering setting of width SS and the iso-compute sensitivity study as follows.

  1. In the current version, the Light Subnet of all three models uses a fixed S=64S=64. This setting is mainly driven by kernel-implementation constraints: a uniform subnet width allows us to reuse regular dense GEMM/kernel shapes and avoids the additional kernel-scheduling and implementation overhead of dynamically varying the width across tokens, experts, or compute budgets. Hence S=64S=64 is a uniform and reusable engineering configuration, not a value tuned on any particular test set. Here SS denotes the intermediate channel width of the Light Subnet, whereas the intermediate width of the full expert is denoted dffd_{\mathrm{ff}}; these are distinct parameters. The revision will report the exact value of dffd_{\mathrm{ff}} from each model configuration.

  2. To test whether fixing S=64S=64 significantly affects performance, we vary the Light Subnet width on Qwen3-30B-A3B, re-match the realized compute budget at 50% and 60% target FFN pruning ratios, and report the realized FFN pruning ratio, accuracy, PPL, and end-to-end latency for different SS. Restricting the main variable in the comparison to the subnet width, we compare S=32S=32, S=64S=64, and S=128S=128.

Target pruning ratioLight width SSAccuracy meanPPL
50%3277.248.83
50%6477.138.61
50%12875.479.76
60%3265.3310.35
60%6466.1610.18
60%12864.2711.47
  1. Accuracy mean in the table is the unweighted average over seven downstream tasks: PIQA, ARC-Easy, ARC-Challenge, MATH-500, GPQA-Diamond, HumanEval, and LiveCodeBench. PPL is the language-model perplexity measured on WikiText-2 with sequence length 2048, and constitutes an independent language-modeling evaluation.
AuthorsOpenReview ↗

Response to Reviewer mMfF Part II

Q3 (also addressing W1: realized FFN pruning ratio, budget mapping, and input-distribution shift)

We thank the reviewer for pointing out the importance of aligning the realized compute budget. The question has two sub-questions, which we answer in turn.

  1. Realized FFN pruning ratio of each method on each evaluation task. The paper reports the realized FFN pruning ratio on each test set, rather than treating the target pruning ratio directly as the realized value. For each test set, we first obtain an initial promotion threshold with the calibration method of this paper, then start from that initial value and perform a binary search, tracking the realized FFN pruning ratio during inference and adjusting the threshold until the realized ratio falls within ±0.5\pm 0.5 percentage points of the target, i.e. [ρB−0.5,ρB+0.5][\rho_B-0.5,\rho_B+0.5]; accuracy and PPL for that test set are reported only after this range is reached. Therefore, the reported results of every method on every task are already matched to the target pruning ratio, with deviations no larger than 0.5 percentage points.

  2. Sensitivity of the calibrated mapping from ρB\rho_B to PBP_B under input-distribution shift. ρB→PB\rho_B \rightarrow P_B is a budget-initialization mapping that depends on the input distribution and cannot guarantee exactly the same realized pruning ratio on every test distribution. We use precisely the per-test-set binary calibration described above to remove the budget deviation caused by distribution shift; the revision will state this explicitly.

Q4 (also addressing W2: ACE execution mode and the separate contributions of Light and PAA)

We thank the reviewer for the constructive ablation suggestions. To cleanly disentangle the respective contributions of Light execution and Promotion Utility allocation, we have added the following three-way matched-compute comparison:

  1. The ACE entry in Table 2 is the Full/Zero configuration: skipped experts do not execute the Light Subnet. For the 50% and 60% budgets on Qwen3, we add the following three-way matched-compute ablation:
    • Full/Zero + ACE contribution ranking: the standard ACE execution mode;
    • Full/Light + ACE contribution ranking: the ACE contribution ranking is kept unchanged and only non-promoted experts are switched from Zero execution to Light execution;
    • Full/Light + Promotion Utility ranking: under the same Full/Light execution framework as the second setting, the ranking criterion is replaced by Promotion Utility.
Target budgetMethodRankingExecutionAccuracy meanPPL
50%ACE baselineACE contributionFull/Zero74.308.85
50%ACE-ranking Full/Light baselineACE contributionFull/Light75.218.96
50%ProFiMoEPromotion UtilityFull/Light77.138.61
60%ACE baselineACE contributionFull/Zero63.3510.86
60%ACE-ranking Full/Light baselineACE contributionFull/Light64.7010.63
60%ProFiMoEPromotion UtilityFull/Light66.1610.18

The two deltas in the table correspond to two independent factors: the difference between Full/Zero ACE and Full/Light ACE ranking measures the benefit of retaining Light execution, and the difference between the two Full/Light configurations measures the additional benefit of switching the ranking from ACE contribution to Promotion Utility. On Qwen3-30B-A3B, at the 50%/60% target budgets the former yields accuracy gains of 0.91/1.35 percentage points and the latter adds a further 1.92/1.46 percentage points, respectively.

AuthorsOpenReview ↗

Response to Reviewer mMfF Part III

Q5 (also addressing W2: rank correlation of Promotion Utility and the effect of a fixed Fidelity Gap)

We thank the reviewer for the insightful point about the validity of the online proxy. We provide a quantitative verification through held-out rank correlation and an oracle-reference experiment.

The reviewer identified two questions that need separate verification: first, whether the Promotion Utility used online can correctly rank the experts that are most worth promoting from Light to Full; second, how much accuracy is lost by using an offline fixed per-expert Fidelity Gap instead of the true gap of the current token. We design the following two auxiliary experiments accordingly.

(a) Rank correlation between Promotion Utility and the true token-level promotion gain.

Promotion Utility estimates the additional response that can be recovered by promoting a routed expert. For each held-out token xx, within the set of experts already selected by the original router R(x)\mathcal R(x), we compare the ranking induced by the online estimated score with the ranking induced by the true Full/Light output difference.

Computation. The online estimated score is R^i(x)=gi(x)aiACEh~i\widehat R_i(x)=g_i(x)a_i^{\mathrm{ACE}}\widetilde h_i, where gi(x)g_i(x) is the router weight, aiACEa_i^{\mathrm{ACE}} is the contribution prior, and h~i\widetilde h_i is the calibrated, stored per-expert Fidelity Gap. The true reference value is Ri(x)=gi(x)∥fi(x)−fiS(x)∥2R_i(x)=g_i(x)\|f_i(x)-f_i^S(x)\|_2, where fi(x)f_i(x) and fiS(x)f_i^S(x) are the Full and Light expert outputs for that token. The reference value requires computing paired Full/Light outputs and is used only for offline diagnosis; it does not change the online inference procedure.

Metric. For each token, we rank R^i(x)\widehat R_i(x) and Ri(x)R_i(x) separately within R(x)\mathcal R(x) and compute the Spearman rank correlation ρx=Spearman⁡({R^i(x)}i∈R(x),{Ri(x)}i∈R(x))\rho_x=\operatorname{Spearman}\left(\{\widehat R_i(x)\}_{i\in\mathcal R(x)},\{R_i(x)\}_{i\in\mathcal R(x)}\right). We report the average over all valid held-out tokens, ρˉ=1∣V∣∑x∈Vρx\bar\rho=\frac{1}{|\mathcal V|}\sum_{x\in\mathcal V}\rho_x. The closer ρˉ\bar\rho is to 1, the better the online estimate reproduces the expert ranking of the true promotion gain; a value close to 0 indicates a weak ranking relationship.

Control. To test whether the Fidelity Gap improves the ranking, we also report the Contribution-only score gi(x)aiACEg_i(x)a_i^{\mathrm{ACE}}, which uses only the contribution prior, and compare it with the full ProFiMoE score. The held-out data come from 256 ×\times 2048 tokens sampled from the WikiText-2 training set.

ModelRanking methodMean Spearman rank correlation (higher is better)
Qwen3-30B-A3BContribution-only0.65
Qwen3-30B-A3BProFiMoE0.73

(b) How much accuracy is lost by a fixed per-expert Fidelity Gap.

We replace only the source of the Gap while keeping the contribution prior, Light Subnet, OMR, router top-1 protection, evaluation data, and realized compute budget identical, so that the effect of the fixed per-expert Gap is observed in isolation.

Two allocation schemes. Actual ProFiMoE uses a fixed per-expert Gap: sifixed(x)=gi(x)aiACEh~is_i^{\mathrm{fixed}}(x)=g_i(x)a_i^{\mathrm{ACE}}\widetilde h_i. The offline oracle reference uses the true Gap of the current token, ηi(x)=∥fi(x)−fiS(x)∥2∥fi(x)∥2+ϵ\eta_i(x)=\frac{\|f_i(x)-f_i^S(x)\|_2}{\|f_i(x)\|_2+\epsilon}, and ranks by sioracle(x)=gi(x)aiACEηi(x)s_i^{\mathrm{oracle}}(x)=g_i(x)a_i^{\mathrm{ACE}}\eta_i(x). The oracle reference requires computing the Full/Light output of the token before allocation and therefore cannot serve as an online deployment method; it is used only to quantify the accuracy gap caused by the fixed-Gap approximation and is not interpreted as an accuracy ceiling.

Setup. At the 50% and 60% target pruning ratios on Qwen3-30B-A3B, we re-match the realized FFN budget of the two allocation schemes and report the realized pruning ratio, seven-task average accuracy, and WikiText-2 PPL.

Pruning ratioConfigurationSeven-task accuracy mean (%) (higher is better)WikiText-2 PPL (lower is better)
50%Fixed per-expert Gap (ProFiMoE)77.138.61
50%True token-level Gap (oracle reference)77.408.53
60%Fixed per-expert Gap (ProFiMoE)66.1610.18
60%True token-level Gap (oracle reference)66.7210.06

Comments on the paper

5 comments
AuthorsOpenReview ↗

Summary Of Rebuttal

Dear Area Chair,

Thank you and the four reviewers for evaluating our paper. ProFiMoE gives already-routed experts a training-free Light path and uses Promotion Utility to allocate the Full budget. Matching channel permutations preserve the Full expert function and keep Light execution compatible with dense matrix multiplication. At the 50% and 60% FFN pruning targets, switching from standard Full/Zero ACE to Full/Light with ACE ranking improved the seven-task mean by 0.94 and 1.36 percentage points. Under the same Full/Light setup, using Promotion Utility added 1.89 and 1.45 points.

Our replies report realized FFN budgets and same-run results for all three configurations on seven tasks and WikiText-2. The accounting includes Light execution and top-1 protection. We will clarify the per-test-set threshold search and fixed thresholds during scoring in Sections 3.2.3 and 4.1 and in the notes to Tables 1 and 4. With expert assignments held fixed, OMR reduced held-out reconstruction error from 0.20 to 0.08 at 50% pruning and from 0.28 to 0.13 at 60%, compared with oracle-scalar errors of 0.07 and 0.11.

The table summarizes all four reviews and our replies, including the second-round questions from T9VB and pbpG. In their follow-up, T9VB said that we had answered some of their questions and retained a score of 2.

ReviewerScoreStrengths identifiedMain questions and concernsOur response
mMfF2Full/Light is an interesting idea. Skipping an expert discards its token-specific response, while a narrower execution path preserves part of that response without retraining. Channel permutation makes the reduced-width computation contiguous and avoids the need for irregular sparse kernels. The gains over ACE are larger at high pruning ratios (50%–60%).1. Did the baselines follow the same protocol as ACE, and how much do results vary across runs?<br>2. How was the Light width SS chosen, and how sensitive are the results to it?<br>3. What FFN pruning ratio did each task actually achieve, and how sensitive is the mapping ρB→PB\rho_B\rightarrow P_B to distribution shift?<br>4. Does ACE in Table 2 skip experts or run them in Light mode? Can the contributions of Light execution and Promotion Utility be isolated?<br>5. How well does Promotion Utility correlate with the ranking under the true Ri(x)R_i(x), and how much performance does a fixed Fidelity Gap cost?<br>6. How are Full and Light executed, what explains the short-input TPOT, and how does latency compare with ACE?<br>7. Where do the calibration data come from? Should Propositions 1 and 2 be presented as supporting analysis?1. We followed the ACE evaluation protocol and reproduced ACE using its public repository. Under fixed conditions, ACE’s GSP/RCR scores are deterministic across seeds. In the three-configuration ablation on Qwen3-30B-A3B, the seven-task means at 50% and 60% pruning were 74.30% and 63.35% for ACE, and 77.13% and 66.16% for ProFiMoE. The standard deviations over five ProFiMoE runs were 0.13 and 0.14 percentage points.<br>2. We used S=64S=64 for all three models so that the kernels could reuse the same GEMM shapes. We selected this width independently of the test sets. At matched budgets, we evaluated S=32,64,128S=32,64,128, in that order. At 50% pruning, the accuracies were 77.24%, 77.13%, and 75.47%, with PPLs of 8.83, 8.61, and 9.76. At 60%, the accuracies were 65.33%, 66.16%, and 64.27%, with PPLs of 10.35, 10.18, and 11.47.<br>3. For the ProFiMoE scores in Table 1 and perplexities in Table 4, we matched the budget to the realized FFN pruning ratio on each test set. We held the threshold fixed within each pass and reported the score from the last pass, whose realized pruning ratio was within ±0.5 percentage points of the target.<br>4. Table 2 uses Full/Zero ACE. In the second-round per-task runs, the seven-task means for standard ACE, Full/Light with ACE ranking, and ProFiMoE were 74.30%, 75.24%, and 77.13% at 50% pruning, and 63.35%, 64.71%, and 66.16% at 60%.<br>5. On held-out data, the Spearman rank correlation was 0.65 for contribution-only ranking and 0.73 for ProFiMoE. Comparing the fixed expert-level Gap with a token-level Gap, the accuracies at 50% were 77.13% and 77.40%, with PPLs of 8.61 and 8.53. At 60%, the accuracies were 66.16% and 66.72%, with PPLs of 10.18 and 10.06.<br>6. The Full and Light paths run in vLLM as two independent dense GroupGEMM operations. The previous table inadvertently used short-input measurements from an earlier serial kernel. For 256- and 512-token inputs, the current implementation measured 26.4 and 46.2 ms/token, respectively, versus 34.8 and 55.8 ms/token for BF16.<br>7. Calibration used 256×2048256\times 2048 tokens from the WikiText-2 training set. We present Propositions 1 and 2 as supporting analysis and reserve the main contribution claims for the method.
ReviewerScoreStrengths identifiedMain questions and concernsOur response
h79Y1The paper addresses how to lower MoE inference costs without retraining. The evaluation covers three models and multiple pruning ratios.1. The reviewer questioned the novelty, describing the approach as a combination of expert selection and channel pruning.<br>2. Residuals can cancel, so ranking experts by their individual Ri(x)R_i(x) need not minimize the final MoE output error.<br>3. The reviewer questioned the reliability of Prefix Coverage as an OMR scaling factor, since misalignment could amplify error.1. We reformulate post-routing compression. Full/Zero asks whether a routed expert will execute. Full/Light asks at what fidelity it should execute and which experts should receive the Full budget. On Qwen3-30B-A3B, the gains over ACE were 2.83 and 2.81 percentage points at 50% and 60% pruning.<br>2. We agree that the individual residuals do not determine the final error. PAA optimizes a triangle-inequality upper bound that can be computed before execution. In the held-out evaluation, the mean reconstruction errors for PAA and the oracle were 0.12 and 0.10 at 50% pruning, and 0.19 and 0.15 at 60%.<br>3. We agree that Prefix Coverage does not give the exact fraction of the current token’s output vector retained. OMR applies a first-order magnitude correction. With assignments fixed on held-out data, the reconstruction errors without correction, with OMR, and with the oracle scalar were 0.20, 0.08, and 0.07 at 50%, and 0.28, 0.13, and 0.11 at 60%.
T9VB2Light execution adds an option between Full execution and skipping, so a reduction in compute can preserve part of the input-dependent output. The Light path requires no retraining. Applying matching channel permutations preserves the Full Expert function exactly and allows the use of ordinary dense matrix multiplication.1. Equal target ratios alone do not establish equal realized compute. Table 1 omits the realized ratios, and the reviewer questioned whether the baselines followed the same protocol.<br>2. The cumulative ablation in Table 2 changes both execution and allocation, and uses a different setting from Qwen3 at 50% and 60% pruning.<br>3. Does OMR with Prefix Coverage compensate for distortion in the Light output? The reviewer requested held-out reconstruction errors and an oracle scalar comparison.<br>In round two, T9VB asked whether the threshold was fixed when computing the final scores in Tables 1 and 4, and requested the realized budget and result for each task under all three configurations.1. For the ProFiMoE scores in Table 1 and perplexities in Table 4, we started at the calibration threshold and ran a binary search on each test set. We held the threshold fixed within each pass and reported the result of the last pass, whose realized FFN pruning ratio was within ±0.5 percentage points of the target.<br>2. Our second-round reply lists each configuration’s realized pruning ratio and score or PPL from the same run for each of the seven downstream tasks and for WikiText-2. The pruning calculation includes both Light channels and the Full channels executed under top-1 protection. At 50% and 60% pruning, ProFiMoE’s seven-task mean exceeded standard ACE in the same ablation by 2.83 and 2.81 percentage points.<br>3. The held-out OMR reconstruction experiment used the same fixed assignments as in our response to h79Y W3. Its measured errors appear in the row above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.