Purifying Supervised Representations by Folding Selected Difference Subspaces

We design a stronger form of domain alignment and bring it inside supervised data, addressing generalization itself — far from its classical habitat of unsupervised domain adaptation. The logic: first select a difference worth erasing — the choice dictated by the generalization we want and by what the data supports; then partition the data accordingly (multiple appearance variants of the same supervision semantics); then align — a discriminator drives the representation through the corresponding transform, so that the representation's semantic coverage extends over the whole region swept by that transform, which can be far larger than the support of the two aligned domain distributions. No additional data is needed; the product is a purified representation that no longer carries the selected difference, together with real generalization gains. We instantiate this as co-manifold alignment (CM): the reference domain keeps its label and keeps the discriminator at work, while variant domains are pushed toward the reference with an anti-label objective. On colored digits with a strong color–label shortcut, CM escapes the shortcut that ERM and DANN consolidate (rule-reversed unseen accuracy 91.5% vs. 80.1%/81.3%); on a channel combination that never appeared in training (pure-blue digits), both baselines degrade — ERM from overfitting, DANN from unanchored alignment — while CM stays accurate. On monocular depth under weather variants — the evaluation scene held out entirely, three weather types never trained, supervision a plain log-depth loss — the objective improves d1_6 from 73.3% to 78.8% and closes the train–held-out generalization gap from −18 points to ≈0 or beyond. These results provide behavioral support for our coverage account: aligning a selected relation inside supervised data improves generalization along the folded difference direction; the gain comes from the space swept by the alignment transform, not from the two aligned domains.