Look Before You Stop: How Language Models Respond to Changes in Search Costs

Recent developments in the capabilities of large language models (LLMs) have allowed for their increasingly effective deployment in situations where they are delegated to make decisions. In many of these economically relevant settings, LLMs must make dynamic choices where information acquisition is costly, which requires them to understand the value of their own future decision-making behavior. We evaluate how LLMs trade off the value of continued search against its cost across various dynamic economic contexts and cost regimes. In a sequential search game where exploration is costly, we find that meaningful sensitivity of exploration to changes in search costs, an essential feature of payoff-optimal Bayesian decision-makers, is a capability exhibited by only the most recent reasoning LLMs. We additionally allow this exploration cost to be charged in terms of token consumption rather than reward observations to more closely reflect the costs of deploying LLMs in real-world settings and measure LLMs' ability to internalize search costs more directly. We find that LLMs use this cost regime to adapt their thinking in order to observe more states, but that the extent to which they do so varies across models. We extend our game to a real-world shopping environment and find that reasoning LLMs' cost sensitivity generalizes to this setting but with less consistent results, indicating that this reasoning is key to dynamic decision-making in deployment but that models must possess broader capabilities in order to fully succeed in these settings.