Can A Model Learn From Arbitrary Pairs?

Contrastive learning based on global embeddings has traditionally followed a simple rule (behind most supervised and self-supervised methods): semantically similar or identical samples should be pulled together, while distinct samples should be pushed apart. Despite its success, this learning principle does not use a pair of distinct classes as a positive. A snake and a cable can share an elongated outline, and a dog and a cow can share fur and a four-legged shape, but depicting that attribute is not trivial. This raises the question: can a model learn from arbitrary class pairs without explicit guidance or attribute annotations? The challenge is to pull a distinct pair together while preserving class discrimination. We address it with a discriminative subspace for every pair of classes, using a learning principle called SimLAP (Simple framework for Representation Learning from Arbitrary Pairs). SimLAP introduces a feature filter that activates and deactivates pair-specific feature dimensions. An arbitrary class pair conditions a shared feature gate for all samples. The gate weights coordinates so that the contrastive loss imposes similarity for that pair. The positive is another view of the partner class, which is a random sample in the batch, so it may be the same class or a distinct class. With one shared gate, this objective matches a SupCon trained with the same recipe on ImageNet (75.8 vs. 76.1 linear probe) and CIFAR-100 (76.3 vs. 72.1 kNN). The gate is what makes that training succeed: without it, distinct and arbitrary positives collapse. Arbitrary pairs can provide useful representation learning signals for cross-class features, but their value depends on the semantic structure of the dataset. SimLAP makes such pairs trainable, providing a robust framework to exploit them.