Zero-Shot Evaluation of Strategies by Playing Imitation Policies

How a strategy handles an opponent it has never met, or a condition it has never seen, is not settled by running it against the opponents at hand, since the one that defeats it need not be among them. The strategies that most need an answer often cannot be run at will, leaving only a short record of past matches. Ratings and outcome predictors fit the results of those matches but take no condition as input, while a policy cloned from the record knows only the situations the record reached. The match can still be played, with a policy inferred from the record in place of the original. The simulator supplies the opponent, the condition and the dynamics, so the only unknown is how the strategy responds. We introduce PlayRead, an in-context imitation policy with a retrieval memory, trained on a population of runnable strategies. At each step of a new match, it retrieves recorded moments that resemble the current situation and the responses that followed them, while the match in progress is kept in its context. Training pairs the record of one strategy with its actions against an opponent absent from that record, and a new strategy is evaluated zero-shot by placing its record in the memory, whose values may then be tuned on that record. We evaluate PlayRead on a continuous-control game, Leduc poker, Overcooked and the iterated prisoner's dilemma. In the continuous-control game, playing the inferred policy is substantially more accurate than predicting payoff from the same memory when the record is short or the condition is unseen. In an unseen condition, PlayRead with one recorded episode predicts matchups more accurately than behavioral cloning fitted to eighteen. For most strategies the inferred policy also names the condition in which the strategy is weakest, and it yields an opponent that defeats it. With six open language models in the prisoner's dilemma, PlayRead predicted how they would play one another before they met.