Contextual agent evaluation with orthogonal equilibrium learning

Contextual agent evaluation seeks to identify the best agents for each context from offline, selectively observed relative feedback. Existing score-based methods, such as Bradley–Terry, impose a transitive preference ordering, which fails to reflect collective preferences when human judgements are heterogeneous. Inspired by social choice theory, we frame evaluation as a contextual game between two players, each selecting a distribution over agents as the strategy to receive greater collective preference than the other. Then, the support of the Nash equilibrium defines a context-specific set of winners. However, learning context-specific equilibria from offline logs is difficult because each context reveals human feedback on only a subset of agents, and na\"ive plug-in estimators can therefore be biased. To address these challenges, we propose NashEval, a general framework for robust contextual equilibrium learning. NashEval first constructs debiased estimates of the contextual payoff matrix that characterizes the game. It then learns the context-to-equilibrium mapping with a tailored orthogonal loss, which avoids the need to solve a separate game for each context. We show theoretically that errors in estimating the nuisance functions underlying the payoff matrix affect the regret of the learned equilibrium (i.e., exploitability) only through higher-order terms. Empirically, NashEval improves robustness of equilibrium learning and adaptively identifies the set of top-performing agents across contexts. In summary, our framework enables context-aware agent evaluation that reflects diverse preferences across the population.