Right Number, Wrong Question: LLM Probability Substitution in Incomplete Searches

Suppose an agent fails to find the target in a partial search and must report the chance the target still exists in the unsearched share. How do language models respond to a required probability field when the prompt does not contain the base rate for the search target? In this setting, when the prompt states the search's coverage as the share of records each searched location holds, we find that 22 of 46 substitute a different probability event for the requested posterior. Supplying the base rate removes the substitution in most models, and asking the model for its assumed prior removes the substitution in half of the models scored. Even when the prompt asks for the prior and for the event the number describes, relabeling the probability field as “the number you report” brings the substitution back. The substitution is a default rather than a misunderstanding. When asked in separate inference calls for the probability that the target exists and for the probability that the search missed it, 18 of 30 models answer the same number to both questions. This two-call test detects 19 of 20 substituting models and none of the seven that report the posterior, so a pipeline can screen a model before trusting its probabilities. The substitution persists even when some models run the search themselves with tools. A calibration score cannot tell which event this default probability refers to, and no recalibration of the reported probability can correct to the posterior at every base rate. We simulate a thresholding pipeline on two datasets and show how thresholding on these LLM-reported probabilities leads to silent failures in downstream applications at low base rates. As self-reported model probabilities are commonly used to make decisions and orchestrate agent pipelines, careful evaluations should test what events those probabilities are actually estimating.