Synoptic Information: A Variational Principle for Joint Representation Learning

How should two high-dimensional noisy views of the same system be jointly compressed so that their representations capture structure common to both while flexibly discarding view-specific detail? Learning such a joint reduced representation by exploiting shared internal structure can extract interesting features from the input data without the need for an external teacher. Here, an information-theoretic description of the problem is introduced and formalized as a two-temperature variational principle for the $\text{synoptic information}$, the optimal number of bits shared between the two compressed views that maximally retain what each representation predicts about the other. Functional optimization yields exact coupled self-consistent equations for both encoders, readily solved by a generalized iterative algorithm. For multivariate Gaussian inputs, the synoptic information decomposes as a sum over canonical correlation modes, admitting closed-form analytical solutions. The phase diagram of optimal solutions in the temperature plane reveals phase transitions through nested regions of active modes and interpolates between differing representation learning frameworks, such as canonical correlation analyses and the information bottleneck, which appear as limiting cases.