CITP: Context-Induced Tilt Posterior

Predictive distributions of large language models (LLMs) remain sensitive even to semantically equivalent prompts. However, the variation in predictions across these semantically equivalent prompts can also provide useful information for predictive refinement. We propose **CITP (Context-Induced Tilt Posterior)**, a Bayesian method for refining a frozen LLM's predictive distribution using its logit responses across semantically equivalent prompts. Using a small calibration set of labeled examples from the target task, CITP learns a posterior over how to refine the model's predictions using its responses to semantically equivalent prompts. It then averages predictive distributions over this posterior to account for uncertainty in the learned refinement. Across three model families and nine tasks spanning fixed-label classification and multiple-choice question answering (MCQA), CITP achieves the best macro-averaged normalized Brier score and normalized negative log-likelihood in the main comparison, with relative reductions of 9.5% and 20.2%, respectively, compared with the strongest baseline for each metric. Further analyses show that learning from how predictions change across semantically equivalent prompts can improve predictive performance, and that averaging over the learned posterior provides additional gains.