- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper asks when routing dynamics observed in a learned multi-expert MoE can be meaningfully compared with a reduced two-expert stochastic model. It derives an exact endpoint probability for a specified E=2 score-gap chain, proves that the historical aggregate E=8 logging schema cannot identify the required hard state, one-step transition, or endpoint, and proposes a logging contract for a valid dynamics test. An imposed E=2 control validates the analysis procedure, while experiments on a learned E=8 router show that a predefined aggregate-soft predictor does not outperform a learning-rate-only baseline. A narrower gain-dependent soft-share movement nevertheless replicates across Shakespeare and enwik8.
Critical strengths
- The paper clearly shows that matching aggregate curves is insufficient when state, step, score-change, and measurement maps are not identified.
- The authors do not overclaim the E=8 results and explicitly distinguish a reproducible local routing phenomenon from validation of the E=2 transfer law.
Critical weaknesses
- The paper primarily establishes that a particular historical logging schema is insufficient, rather than demonstrating a new general law of learned MoE routing dynamics.
- The imposed-chain control verifies the analysis pipeline, but the actual learned-router experiment does not establish the state or transition correspondence required by the theory.
Questions for authors
Q1. Have the authors collected any new runs with the proposed logging contract that would directly test the E=2 transition law on a learned router? Q2. How sensitive is the non-identifiability result to adding one or two additional practical logging fields, such as hard pair counts or tokenwise score gaps? Q3. Beyond the reported gain-dependent soft-share movement, is there evidence that the learned E=8 dynamics are well approximated by any low-dimensional stochastic process?