CLOSER: Characteristic Locations for Efficient Geospatial Model Evaluation

Geospatial prediction tasks estimate outcomes such as house prices or disease incidence for specific locations or regions. Candidate pipelines for such tasks may select different options for representing locations, additional input features, and prediction models. Comparing these pipelines can require repeated training and evaluation across thousands of locations, often for multiple outcomes or time periods. This paper addresses the challenge of efficient candidate pipeline selection: Can we identify a small weighted subset of characteristic locations to approximate the performance of a candidate pipeline over all locations? We introduce CLOSER, which uses knowledge about task-aware location representations and solutions to the facility location problem to select complementary representative locations, where a weight associates the count of locations that are assigned to each representative location. The subset size is determined using predictive scores on the candidate subsets, without requiring full-location pipeline scores. The selected subset and weights is then applied across all candidate pipelines for that task. Based on evaluations over outcomes including house prices, health statistics, COVID-19 incidence, and point-of-interest spending, we demonstrate that the selected subsets achieve higher Spearman rank correlation with all location rankings, in comparison to matched-size uniform sampling, while retaining less than $2\%$ of locations on most tasks. Paired timing measurements show $6.3\times$–$84.2\times$ speedups for the subset, when compared to evaluation over all locations, over the suite of outcomes and candidate pipelines.