Learning Before Predicting: Physics-Guided Joint-Embedding Learning for Scientific Systems

Scientific systems are heterogeneous twice: they live in different spatial dimensions, and their states contain quantities with different geometric laws. Density is a scalar, velocity is a vector, and orientational order is a tensor, yet learning pipelines often flatten them into interchangeable channels and ask a downstream predictor to recover both dynamics and structure from scarce labels. We ask a different question: can the physical organization of a state be learned before forecasting? We introduce UNIJEPA, a physics-guided joint-embedding framework that keeps field type explicit at the interface while a shared predictive core learns coupled spatiotemporal dynamics. During pretraining, lightweight field decoders expose predicted streams to system-specific equations, differential relations, realizability constraints, and transformation laws; the decoders are discarded before downstream prediction. Across 1D–3D scalar PDEs, 2D coupled flow, and scalar–vector–tensor Active Matter, the same parameterization principle improves few-shot transfer. On Active Matter with 16 labeled trajectories, UNIJEPA reduces five-seed full-state VRMSE and velocity error by 13.8% and 20.4%, respectively, and remains 10.1% better than a parameter-matched Scratch FNO. Frozen probes reduce mean physical readout error from 0.955 to 0.389, while typed-interface and exact-$C_4$ controls attribute these gains to physically organized representations.