Reconstructing the Routing Space: Retraining-Free Compression of Vision-Language Mixture-of-Experts

Vision-language Mixture-of-Experts models (VLM-MoEs) hold most of their parameters in expert weights, yet their expert pools are hard to compress because visual tokens use experts far more diffusely than text tokens. Our experiments show that covering $90\%$ of visual routing mass takes up to $1.8\times$ as many experts as for text, and that at $50\%$ expert retention frequency-based pruning keeps only $51$–$53\%$ of dense performance. This exposes a limitation of existing compressors: they rank experts alone or fuse parameters, ignoring the function expressiveness of the routing space. We propose \method, which selects a well-conditioned expert subset by pivoting on the eigenbasis of a routing-aware expert-output Gram matrix and, from the same basis, reconstructs the discarded routing coordinates in closed form. The procedure is one-shot and retraining-free, with no learned parameters. On Kimi-VL-A3B and Qwen3-VL-30B-A3B, \method delivers the strongest overall performance across compression budgets, and its advantage widens as compression becomes more aggressive, reaching up to $3.6$ points over the strongest baseline at $50\%$ expert retention.