Prompt Conditions Can Change Measured Intervention Effects in LLM Evaluation

Prompted language model evaluations often report the effect of an intervention under a single prompt condition. We test whether that reported effect changes when the surrounding prompt condition changes. On MMLU-Pro, a silent self-check increases exact-format accuracy for Gemma by $29.33\text{pp}$ under a standard prompt condition, $14.00\text{pp}$ under a masked condition, and $20.00\text{pp}$ under a token-matched metadata condition. A separate, pre-specified 300-task replication gives gains of $30.33\text{pp}$, $10.33\text{pp}$, and $22.67\text{pp}$. The self-check gain is therefore $20.00\text{pp}$ larger under Standard than under Masked, with a simultaneous 95% confidence interval of $[11.82,28.18]\text{pp}$. DeepSeek shows little observed difference across the same prompt conditions and Llama is inconclusive. The result also depends on scoring. On the replication set, an automated semantic answer scorer gives a Standard-minus-Masked difference of $15.67\text{pp}$, while a separate constrained-decoding evaluation gives $-1.00\text{pp}$. Supporting experiments further show that average accuracy can remain almost unchanged even when many individual outcomes flip. These findings motivate reporting intervention gains under the prompt and scoring conditions that define them, rather than treating those choices as incidental details. Code: https://anonymous.4open.science/r/PCCMIE_ICLR27-BCA1/README.md