GSJEPA: Self-Supervised Approach to Learning Gaussian Splat Representations Using Joint Embedding Predictive Architecture

We introduce a self-supervised representation learning framework for 3D objects represented as Gaussian Splats based on Joint Embedding Predictive Architectures (JEPA). GS-JEPA represents individual Gaussian primitives as nodes in a local graph and learns their representations through latent prediction, explicitly modeling geometric and appearance-aware relationships between primitives. To isolate the effect of relational modeling, we additionally consider a point-based formulation that treats Gaussian primitives as an unordered set and serves as a baseline. Both formulations learn representations directly from Gaussian splat objects without relying on reconstruction objectives. Across classification and segmentation benchmarks, predictive learning outperforms the reconstruction-based Gaussian Masked Autoencoder. Notably, despite being pretrained exclusively on synthetic objects, the proposed JEPA-based approaches generalize effectively to real reconstructed Gaussian splat data. On MACGS classification, the point-based and graph-based approaches achieve $95.76\%$ and $94.60\%$ accuracy, respectively. On ShapeNetPart segmentation, the graph-based model achieves an mIoU of $85.15\%$, demonstrating the potential benefits of explicit relational modeling. These results establish JEPA-based predictive learning as an effective approach for transferable Gaussian splat representations and highlight relational modeling as a promising direction for 3D Gaussian representation learning.