Coherent Overlap: Token-Specific Expert Committees in Sparse Mixture-of-Experts Routing

Why do sparse mixture-of-experts (MoE) language models route a token to several experts? The standard explanation is geometric complementarity: co-selected experts cover different directions of the token representation. This view has shaped non-overlapping expert designs, diversity objectives, and similarity-based compression, but it has a sharper contextual implication: a selected expert should become more valuable with the experts selected beside it. We test this implication with a prefix-controlled 2×2 factorial that crosses the selected expert with its highest-scoring unselected rival and the actual co-selected context with load-matched alternatives. A positive interaction supports complementarity, whereas a negative interaction supports relevance-driven selection, where the router forms a token-specific committee of individually relevant experts from a shared geometric neighborhood. Across six open MoEs, routed-input subspaces share a large layer-wide core and actual routes fit tokens better than matched alternatives. The selected expert beats its rival in 100 of 102 geometric cells, yet the actual context narrows this advantage in all 102. In OLMoE, Mixtral, and DeepSeek-MoE, the negative interaction persists under order-free contexts, sensitivity analyses, and next-token loss, which fits no subspaces. The overlapping experts remain functionally valuable: their outputs separate more than their inputs, later experts beat unselected substitutes in next-token loss, Top-2 beats Top-1 at matched compute, and explicit complementarity regularization does not consistently improve validation loss. We call this pattern **coherent overlap**: MoE routers select useful expert committees from overlapping neighborhoods, so geometric similarity alone is a poor guide for analyzing, pruning, or designing experts. Code is available in the supplementary material.