Organizing LLM Inference for Aggregate Predictions from Behavioral Data

When using a large language model (LLM) to simulate a user population, how should we combine individual behavioral histories to predict the fraction who take a particular action? We study this question in e-commerce, retrospectively predicting product-level conversion rates from browsing sessions without adapting the model to the task. We vary two design choices, each involving a bias–variance tradeoff. Summarizing a product's sessions before inference is economical but can discard predictive detail, adding bias, whereas judging sessions one at a time and averaging afterward keeps the detail but requires many calls to reduce variance. Evaluating products together in one prompt can refine the model's prior about what is typical in the setting, reducing bias, but also makes each judgment depend on which other products are sampled to share the prompt, adding noise. We evaluate both choices across five LLMs and two datasets. Our main finding is that judging individual sessions outperforms judging summaries in isolation, but once both are compared within a prompt, summaries lead: comparative summaries achieve the highest Spearman correlation with observed rates in 12 of 15 settings, with no alternative significantly better. Comparison thus helps summaries far more than session judgments, overcoming the advantage of behavioral detail. We interpret this disparity through a Bayesian model of the second tradeoff, which suggests that comparison helps more when the inputs in a prompt differ more across products, consistent with what we observe across four kinds of comparative context. Comparative summaries also require a small fraction of the inference cost of the session methods, and with the best model for each outcome they attain 66–89% of the correlation of the best supervised benchmark trained on thousands of labeled products.