A Geometric Theory of Semantic Relation Operators

Analogies such as man : woman :: king : queen demonstrate that two relations can be applied in either order with the same result. This property is formalized as the flatness of a discrete connection, where tokens are represented as vertices in a graph and edges are labeled by relation types, each acting on embeddings via a specific affine operator. An analogy square corresponds to a closed walk whose holonomy is defined by the commutator of the two operators. Exactly fitted squares enforce flatness if their base points affinely span the space. Flat squares ensure path independence of composite analogies when they generate the fundamental group. For contextual representations, operators may depend on context through softmax mixtures of Lie-algebra generators, such that curvature is uniformly bounded by their brackets. In an evaluation of 1,154 mined morphological analogy squares in GPT-2, affine operators outperform vector offsets by ten points on two-hop analogies, but display path dependence in 11–25% of held-out compositions. Application of curvature regularization reduces path dependence to 3–6% across all relation pairs, and ensures consistency in unpenalized four-hop detours, validating our theoretical predictions. When syntactic frames shift, static operators lose 8–15 points, while contextual operators recover to within two points of per-context oracles and exhibit context dependence.