Measure Before They Debate: Auditing LLM Response Composition

Response probes measure how an LLM reacts to individual inputs, but using those measurements to predict responses to joint inputs requires a separate composition assumption. We study this assumption through a finite-input audit that holds the interface fixed, excludes combinations from calibration, and separates prediction error from response noise. In a joint study of $5,184$ responses, summed single-sender secants fall below common-input secants by $.34915$ and $.45782$; both approximate simultaneous $95\%$ intervals exclude zero. One configuration also answers all $768$ nonzero-target arithmetic queries correctly, although the prespecified global screen fails. A prospective test on eight new, separately calibrated propositions replaces numerical cards with prose comments. Across $4,352$ responses, $11$ of $16$ composition contrasts have approximate simultaneous intervals below zero; five remain unresolved. Addition retains the lowest observed overall held-out error, while averaging measured increments improves mixed-input prediction. Further tests show how calibration support redistributes error and how remeasuring $216$ archived replies changes a gain ordering. The audit distinguishes failed composition identities from useful predictions and makes sensitivity to input families and readouts explicit. The evidence covers numerical cards and controlled prose substitutions; it does not establish internal mechanisms or performance in unrestricted discussion.