Which Pairs Should Agree? Triangular Consistency for Adversarial Training

Consistency regularization has become a standard component of adversarial train- ing (AT), encouraging agreement between predictions on related inputs. We ar- gue that existing formulations are structurally incomplete. Given two augmented views of the same sample, each subjected to an adversarial perturbation, there are two distinct consistency constraints: one connecting each clean view to its corresponding adversarial example, and another connecting the adversarial exam- ples across views. Existing objectives enforce only the former constraint, leav- ing part of the predictive geometry unconstrained. We jointly impose both con- straints, forming a triangular structure among the predictive distributions, which we term triangular consistency (TC). We provide a formal analysis showing why this combination is important. Specifically, the consistency terms contribute to a Rademacher-based bound on the robust generalization gap, while a triangle- inequality argument demonstrates that bounding the divergence between a clean prediction and an adversarial prediction from the other view requires both con- straints. Empirically, TC achieves the strongest robustness among the evaluated methods under ℓ∞ attacks. In addition, a term-count-matched ablation indicates that the improvement stems from the structural arrangement of the consistency constraints rather than simply from introducing additional loss terms. Finally, the robustness induced by TC transfers beyond the training setting, improving perfor- mance under larger unseen perturbation budgets as well as non-ℓp attacks.