- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents a controlled protocol for measuring cross-precision trajectory divergence in MoE inference and shows that FP8 and NVFP4 can produce highly different token trajectories despite similar endpoint accuracy.
Critical strengths
- The paper uses a controlled experimental setup, including fresh-server replicas and pre-specified decision rules. This detailed control setup helps separate cross-precision effects from runtime variability.
- The experiments show a clear difference between trajectory-level divergence and endpoint accuracy. FP8 and FP4 produce substantially different generation trajectories but they have similar task accuracy.
Critical weaknesses
- The practical significance of trajectory divergence is limited. The metric marks two trajectories as different if any token in the prefix is different, but such token-level difference does not imply a significant difference in reasoning or accuracy. So I do not know whether this
difference is important for practical usage. If the author can provide more information, I will consider to increase my score. - The empirical setup only include one model, GLM 5.2. It is unclear whether the observed conclusion can generalize to other models.
Questions for authors
- Besides GLM 5.2, please use more models to evaluate the performance.
- Use more reasoning benchmarks to evaluate whether the result is consistent.
- Can the authors demonstrate whether the measured trajectory divergence leads to meaningful downstream consequences, such as differences in tool use or agent behavior beyond token-level disagreement?