Closing the Loop on Human Behavior Prediction
Abstract
LLM digital twins are usually scored against how closely people reproduce their own answers on a retest, as if retest agreement were a ceiling on individual accuracy. It is a floor: the accuracy of copying the earlier answer, hence a lower bound on the best accuracy attainable from information that includes that answer, which the nearest-neighbor inequality of Cover and Hart bounds from above under stable propensities. We score LLM and non-LLM predictors of individual decisions on identical rows in the closed loop, where a predictor sees the person's earlier behavior and observes each decision after predicting it, on the Twin-2K-500 retest wave, three waves of the General Social Survey, Psych-101 and held-out Psych-201 experiments. Most of the individual signal our predictors extract lies in the person's most recent relevant behavior, and they differ in whether they anticipate a departure from it. On Twin-2K-500, dividing by retest agreement credits the item mode with more than the dataset's default twin under the metric of the dataset's authors, every released twin falls below the item mode, and an LLM given the earlier answer matches copying only by repeating it. On cognitive tasks, conditioning on the previous choice is the largest gain for statistical predictors, which then predict repeats, and a recurrent network trained on other persons' choices and outcomes reaches the accuracy of the best LLM on Psych-101. Reading the person's raw transcript, gemini-3.6-flash detects more switches than CAOP-sel and the transition predictors at the same false-alarm rate, but on held-out Psych-201 experiments it is right on switches only slightly more often than the item mode. Predicting when and how a person departs from recent behavior remains open.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.