acceptodds
Under review as a conference paper at ICLR 2027

QAttrib-MoE: Runtime-Controlled Attribution of Cross-Precision Trajectory Divergence in Mixture-of-Experts Inference

Abstract

Low-precision mixture-of-experts inference is usually compared by endpoint accuracy, yet similar accuracy can conceal substantial changes in the autoregressive trajectory, while runtime variability can confound attribution to numerical precision. We introduce QAttrib-MoE, a pre-frozen, runtime-controlled, noise-floor-aware protocol for attributing cross-precision trajectory divergence. Same-path fresh-server variability defines an empirical control floor, cross-path divergence is the observed contrast, and excess divergence measures the contrast remaining above the two path-specific floors. The protocol freezes the evaluation cohort and decision rule before generation, counterbalances execution order, and reconstructs multiple decoding budgets by censoring a single stored trajectory rather than regenerating requests. We instantiate QAttrib-MoE on a shared GLM-5.2 checkpoint executed through FP8 and online-NVFP4 routed-expert paths. At the pre-frozen 256-token prefix, both paths exhibit 0.00%/0.00% within-path mismatch on 64 controls, while cross-path mismatch and excess divergence reach 98.44% (95% CI: 95.31-100.00%); the effect is positive on both GSM8K and MMLU-Pro and grows from 50.00% at 32 tokens to 98.44% at 256 tokens. In contrast, strict 1024-token accuracy is 64.45% versus 66.41% (difference 1.95 pp, 95% CI: -1.95 to 5.86 pp; McNemar p = 0.442). A gated teacher-forced diagnostic preserves token-ID alignment yet finds top-1 prediction flips in all 48 anchored inputs, with median first flip position 2. The results show that individually reproducible precision paths can be strongly non-equivalent at the reasoning-trajectory level without a statistically significant endpoint-accuracy shift in the evaluated setting.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model inference, quantization, mixture-of-experts, reasoning trajectories, inference reproducibility, FP8/NVFP4

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_PcEGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a controlled protocol for measuring cross-precision trajectory divergence in MoE inference and shows that FP8 and NVFP4 can produce highly different token trajectories despite similar endpoint accuracy.

Critical strengths
  1. The paper uses a controlled experimental setup, including fresh-server replicas and pre-specified decision rules. This detailed control setup helps separate cross-precision effects from runtime variability.
  2. The experiments show a clear difference between trajectory-level divergence and endpoint accuracy. FP8 and FP4 produce substantially different generation trajectories but they have similar task accuracy.
Critical weaknesses
  1. The practical significance of trajectory divergence is limited. The metric marks two trajectories as different if any token in the prefix is different, but such token-level difference does not imply a significant difference in reasoning or accuracy. So I do not know whether this
    difference is important for practical usage. If the author can provide more information, I will consider to increase my score.
  2. The empirical setup only include one model, GLM 5.2. It is unclear whether the observed conclusion can generalize to other models.
Questions for authors
  1. Besides GLM 5.2, please use more models to evaluate the performance.
  2. Use more reasoning benchmarks to evaluate whether the result is consistent.
  3. Can the authors demonstrate whether the measured trajectory divergence leads to meaningful downstream consequences, such as differences in tool use or agent behavior beyond token-level disagreement?
Reviewer_1pnXOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the trajectory divergence between different low-precision methods.

The motivation is clear, but the method and the experiment are incomplete.

Critical strengths

The paper raises an important question: matching accuracy does not necessarily imply matching inference behavior. The experimental setup is reasonably controlled, although the evaluation scope is limited.

Critical weaknesses

This paper identifies an interesting phenomenon, but lacks mechanism analysis and adequate experiments.

Questions for authors

The paper claims to run on NVIDIA B200 GPUs.

Given such a substantial hardware resource, it is unclear why the evaluation is limited to a single model and one precision pair without deeper analysis or broader experiments.

Could the authors clarify the experimental setup and provide more evidence for reproducibility?

Reviewer_JCLrOpenReview ↗
Rating
1(−1)
Confidence
1
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents a protocol to measure trajectory divergence across models of different precision. It reruns each precision on fresh servers to show that each is deterministic on its own, then compares the two using the same prompts and decision thresholds. The two models generate different text on 98% of the prompts within 256 tokens, while final accuracy is statistically the same, and a teacher-forced check shows their next-token predictions diverge within the first few tokens.

Critical strengths
  • Well-controlled experiments.
Critical weaknesses
  • The paper presents a well-designed protocol for measuring cross-precision trajectory divergence, but the authors never explain in what practical setting this divergence actually matters. Quantization methods are typically designed and evaluated to preserve accuracy and output quality, not to reproduce the original model's exact output.
  • The abstract and introduction rely heavily on specialized or coined terms (e.g., "pre-frozen," "attribution contract," "noise-floor-aware," "runtime-controlled"), which makes the paper hard to follow.
Questions for authors

n/a

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.