RouteLedger: Routing-Aware Answer Selection with Task-Gated Visual Verification

Mixture-of-experts vision-language models expose input-dependent routing sig- nals, but directly intervening on expert routes is unreliable and can replace correct answers with plausible errors. We propose RouteLedger, a certificate-based resid- ual selection framework that uses routing evidence conservatively. RouteLedger records expert assignments under text-only, visual, and full multimodal views, and converts their differences into route-aware candidate proposals. A beam-first selec- tor accepts a route residual only when it receives sufficient independent support and passes explicit fidelity constraints. We further introduce SafeSwitch-Margin, a task-family-gated CLIP-L/336 verifier that evaluates a fixed non-route candi- date pool and preserves the Beam-based answer when no residual is certified. On 45,857 examples from 12 VQA-style datasets, RouteLedger obtains 23,419 cor- rect answers, improving Beam by 277 predictions and significantly outperforming the verifier-free selector. A matched Beam+Anchor system with the same veri- fier reaches 23,397 correct answers, leaving a small and non-significant residual for routing; visual verification therefore explains most of the gain. These results support MoE routing as supplementary evidence within a conservative candidate recipe.