acceptodds
Under review as a conference paper at ICLR 2027

GAME-THEORETIC DATA ALLOCATION FOR EXPERT SPECIALIZATION AND ROUTING

Abstract

Can the allocation used to train experts also predict which expert will perform best on a prompt? Such a connection would make the intended division of labor inspectable before training and allow the allocation itself to guide inference. We introduce INFLENS, a mixture of LoRA experts whose training weights and inference routes come from the same location game in a fixed prompt-trait space. Players choose positions to maximize their expected shares of the prompt distribution, without using expert losses. We derive a kernel-general criterion for when co-located positions become locally unstable to separation, with an explicit Gaussian boundary determined by player count, kernel geometry, and trait covariance. In a transductive evaluation on a corpus assembled from seven safety-related benchmarks, the highest-share expert has the lowest held-out response negative log-likelihood (NLL) on approximately 90% of prompts across four kernels under a sharpened simplex representation, compared with approximately 15% after permuting allocations across prompts. In the principal seven-adapter configuration, token-level probability mixing improves NLL by 0.105 nats over a pooled adapter. Across four data splits, game shares outperform validation-trained linear gates in expected NLL when sampling one expert, while completed-sequence mixtures perform similarly. Controls show that the correspondence between allocations and expert identities matters, and that fixing solved positions retains performance at the principal operating point. These results connect an analyzable allocation of training data to useful expert specialization and routing without fitting a gate to expert performance.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.