Lens-Router: Progressive Fine-Grained Routing for Large Language Models

The LLM ecosystem is undergoing rapid model updates and intense tiered competition, requiring LLM routers to support efficient model expansion and fine-grained routing. However, existing routing paradigms face a dilemma between scalability and fine-grained routing: finer-grained decisions require more historical supervision, making new model incorporation less scalable. Motivated by this dilemma, we propose Lens-Router, a progressive fine-grained router that abstracts reusable, cross-model resource demand features from queries. These features provide effective supervision for both capability filtering and realized cost prediction, enabling Lens-Router to efficiently incorporate new models and select the lowest realized cost model that can fulfill the query. Our experiments show that with only 700 historical supervision for model adaptation, Lens-Router improves routing accuracy while reducing realized cost by up to 82.8% compared with SOTA LLM routers.