A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

Large language models can shift human beliefs across high-stakes domains, but most persuasion studies rely on pre/post belief change. These endpoint measures identify whether persuasion occurred, yet miss where and how beliefs moved within a dialogue. We present PERSUASIONTRACE, a framework for studying persuasion in human-LLM interaction. Built on a web-based experimental platform, PERSUASIONTRACE contributes a tool for multi-turn persuasion studies and a process-level evaluation protocol: it records multi-turn belief reports from human or simulated targets of persuasion and supports comparisons of how their beliefs move during dialogue. Using this framework, we find that human targets exhibit heterogeneous patterns of multi-turn belief updates, including low movement and early movement followed by partial regression, and that LLMs are persuasive across generic and personalized topics and multi-turn interactions. Prior work has chiefly used vanilla-prompted LLMs to simulate human targets, but we show that simulator choice materially affects apparent persuader quality. We introduce a Bayesian-network simulated target that maintains an explicit latent belief state over time so each persuader message yields a structured belief update. In blinded pairwise human-likeness evaluation with two annotators (76.1% agreement), real human targets score higher than our Bayesian target in 96.7% of matched comparisons, while the Bayesian target scores higher than the two baseline LLM targets in 77.5% of those head-to-head comparisons on average. PERSUASIONTRACE reframes persuasion evaluation from endpoint movement alone to turn-level measurement, providing a basis for studying when beliefs move and auditing simulator-based evaluations.