Beyond PID Synergy: Fusion Value and Pairing Value for Multimodal Bayes Risk

Multimodal fusion is commonly motivated by partial information decomposition (PID) synergy, and a growing diagnostic literature treats this atom as the reason to combine two views. In this work, we show that this identification does not hold at Bayes optimality. The logarithmic loss gap between the better unimodal Bayes predictor and the joint Bayes predictor is not synergy alone, since it also carries the weaker unique atom. We call this gap fusion value. A second gap separates early fusion from the stacking of two calibrated unimodal posteriors, and this gap is not a linear function of any Williams and Beer quadruple. We call it pairing value. Pairing value lies between zero and fusion value, with AND attaining the lower bound under a strictly positive synergy atom and XOR attaining the upper bound with one bit of both. Adding a named view and deleting it produce the same Bayes gap, so the empirical missing modality drop reads the same quantity under two names. For trained models, the observed fusion gain decomposes into fusion value plus a signed excess risk remainder. An operational protocol built from held out logarithmic loss keeps this remainder near zero whenever a single view already saturates the label. On language saturated sentiment, the remainder is near zero, because the better singleton already captures the label. On caption matching, both views are necessary and late fusion stays at chance, since the label lives only in the pair. The two indices therefore separate the decision to retain a weak view from the decision to interact early.