Beyond Continuity Or Discontinuity: Zigzag Algebra for Compositional Relations in Low-Rank Adaptation

Modern neural networks continuously update parameters in a shared space, while existing methods provide only limited forms of relational organization. We introduce a framework that partitions the trainable parameter space into groups, infers their relations from the model’s own training signals, and organizes them through zigzag algebra. Zigzag algebra is a path-based algebra in which vertices are connected by directional arrows and relations constrain how these arrows can compose. We map parameter groups to vertices and their interactions to arrows, allowing strongly related groups to remain closely connected, weakly related groups to interact less, and unrelated groups to become effectively separated. Unlike orthogonality, expert routing, or heuristically assigned group labels, our approach learns this graded relational structure directly from training dynamics while using algebraic paths to constrain higher-order interactions. We instantiate the framework within LoRA by partitioning the low-rank space and imposing these algebraic relations during optimization, without additional inference-time parameters, routing modules, or architectural branches.