A Nonlinear Kernel Spectral Perspective on Multi-View Contrastive Learning

Multi-view contrastive learning has become a powerful paradigm for exploiting cross-view consistency, yet most existing methods treat contrastive objectives primarily as representation-level regularizers within larger learning pipelines. The spectral structure induced by these objectives remains less understood. We present a nonlinear kernel spectral perspective on multi-view contrastive learning and derive MVCL-KS, a direct spectral realization of this perspective. Starting from bidirectional InfoNCE, we show that its exact dot-product gradient induces a signed cross-view relation operator; aggregating all view pairs yields a unified multi-view block spectral system, which is further lifted to view-specific reproducing kernel Hilbert spaces. Because the contrastive probabilities depend on the current representation, the resulting formulation is a nonlinear self-consistent spectral system. This formulation leads to dominant-eigenspace spectral refinement with basis-invariant cycle-aware stabilization, together with low-rank and anchor-based implementations for scalability. Under the same evaluation setting, MVCL-KS is $1.7\times$ to $22.1\times$ faster than the fastest competing multi-view baseline across all benchmarks, while maintaining competitive or superior clustering quality. Experiments with multiple kernel families further show that the same spectral formulation works with different nonlinear view geometries.