Sharp Rates for Information Composition under Source Uncertainty

Suppose each view of an observation is compressed on its own before the views are combined for prediction. Even when each compressed view loses very little task information, the combined representation can lose much more. We study when that can happen for a fixed finite source and deterministic encoders. For finite positive failure depth d, the sharp relation between local and joint information loss has exponent 1 - 2^(-d). Infinite depth gives a linear relation. At depth zero, zero local loss can still come with positive joint loss. For rational inputs, the depth can be computed in polynomial time, and two views are enough to realize every finite depth. We then allow the source distribution to move within total variation e. For a depth-two eight-atom source, with local-loss budget u and relative posterior floor tau, the sharp joint rate is minu^(3/4), tau^(-1/3)u + e log(1/e). Small changes in source mass can change the exact depth without changing the support. Under log loss, the same exponent sets the sample-size scale where raw and compressed decoders trade places, while the crossing point depends on the smoothing rule. Existing semialgebraic and conic results cover part of the general machinery. The result here gives the exact exponent from the source/encoder structure and quantitative bounds when the source is uncertain.