A Dynamic Bradley-Terry Approach to Learning Time-Varying Leaderboards from Sparse Comparisons in Sports and LLM Evaluation

State-of-the-art rankings of large language models, such as those of Arena.AI, are summarized in leaderboards that are continually updated over time based on a steady stream of head-to-head comparisons between models. Such time-varying leaderboards must learn from sparse and uneven data while accommodating temporary model absences. This challenge also arises in international sports, where, unlike in league competitions, teams play sporadically and with highly imbalanced schedules in mostly bilateral series. We develop a dynamic Bradley–Terry (BT) estimator for jointly estimating smooth, item-specific ability trajectories and time-varying contextual effects using a temporal basis representation and penalized logistic likelihood. By pooling information across time, the method can recover latent abilities during temporary gaps in an item’s comparison history when the observations on either side provide sufficient information to identify the chosen temporal representation. We establish recovery guarantees and estimation rates that depend on the ability-curves’ smoothness and the conditioning of the joint time–comparison design, thereby accounting for uneven comparison information across items and time. Extensive simulations examine performance across diverse temporal patterns, imbalanced comparison schedules, and temporary item absences, and demonstrate the benefits of temporal information pooling. Applications to LLM evaluations from Arena.AI (previously Chatbot Arena) and international T20 cricket further illustrate the methodology, yielding lower held-out errors than static and kernel BT methods under constructed interpolation holdouts.