acceptodds
Under review as a conference paper at ICLR 2027

Joint Holonomy Matching: Complete Supervision of Finite Representation Connections

Abstract

What information must a student preserve to reproduce a teacher’s representation connection up to local coordinate changes? We introduce Joint Holonomy Matching (JHM), which aligns a fundamental loop family under one shared root gauge. Using the standard spanning-tree classification of graph connections, we establish that zero JHM is equivalent to gauge equivalence of the complete observed connections. Among supervision schemes retaining a subset of a fixed fundamental generator family, the full family is deletion-minimal for a uniform completeness guarantee: omitting any generator can leave inequivalent connections indistinguishable. Independent per-loop alignment is also incomplete in general, as matching individual conjugacy classes can miss incompatible joint compositions. We prove that every finite orthogonal connection on a simple graph admits an exact realization by local orthonormal frames and their Procrustes transports. A non-Abelian realization exhibits , , and a group-commutator discrepancy of . A quantitative bound controls errors on new compositions of observed edges. In a protocol-frozen three-seed neural pilot, Joint reduced composition error relative to Separate by 1.16 percent on the common evaluable subset of the designated test set, with the same direction across all seeds. In this implementation, Separate required 3.35 times as much loss-and-alignment time and 1.36 times as much full-update time as Joint. The structural effect was not independently established on validation, and unequal effective geometric regularization prevents attribution to the shared-gauge constraint alone. These results establish a complete connection-level supervision target, identify compositional information lost by independent alignment, and provide preliminary neural evidence for its optimization.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
knowledge distillation, representation learning, graph connections, holonomy, gauge invariance, geometric deep learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_NaLrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Joint Holonomy Matching (JHM), a teacher–student supervision objective that aligns fundamental-loop holonomies under a shared orthogonal transformation. The paper establishes completeness up to gauge equivalence, deletion-minimality within a fixed loop family, and strict separation from independent loop alignment. A preliminary neural experiment shows a small improvement in composition error.

Critical strengths
  1. The paper presents a mathematically well-defined structural supervision objective with clear theoretical guarantees.

  2. The separation examples and Procrustes realization theorem provide a coherent justification for joint alignment.

Critical weaknesses
  1. The main completeness result relies on classical spanning-tree gauge theory, while the distinction between independent and simultaneous conjugation follows naturally from established group-theoretic principles. The novel contribution beyond applying these ideas to teacher–student supervision is not sufficiently clear.

  2. Joint and Separate have substantially different effective geometric gradient loads, making it difficult to attribute the observed improvement to the shared-gauge constraint itself.

  3. The reported improvement is small (1.16%). Moreover, the evaluation focuses on a structural metric closely related to the training objective, without demonstrating benefits on independent downstream tasks.

Questions for authors
  1. What is the key novel theoretical contribution beyond existing gauge-theoretic and simultaneous-conjugation results?

  2. Does JHM still outperform Separate when their effective geometric gradient loads are matched?

  3. Can the authors demonstrate that preserving joint holonomy structure benefits downstream representation learning?

Reviewer_Boe9OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduce Joint Holonomy Matching (JHM), a supervision target for teacher-student representation matching. This supervision target aims to force that the student to reproduce the same local geometries along loops as of the teacher. They prove theoretical results about the Joint Holonomy Matching objective:

  • Ljoint=0L_{joint} = 0 iff the teacher and student connections on the observed graph are gauge-equivalent.
  • Deletion minimality: if you remove any loop in the graph, the loss can be equal to zero even if the connections on the observed graph are not gauge-equivalent
  • that Lsep≤LjointL_{sep} ≤ L_{joint} , showing that minimizing LsepL_sep does not guarantee a low LjointL_{joint}

They also include a neural pilot, showing that JHM is trainable.

Critical strengths

The paper theoretical part is well written and their definition of the JHM objective is sound. They do prove that JHM is the right objective in their context.

Critical weaknesses

1- While you do prove that JHM is the right objective for the specific task you defined, your theoretical result mainly say that having a shared root is better than treating each loop independently. It does not validate that the task is interesting or useful.

2- While you state "The endpoint itself is structural, the effective geometric loads are unequal, and no downstream or external criterion was measured", your paper would be a lot more interesting if you gave experimental result showing that the task improve knowledge distillation and that training with LjointL_{joint} rather than training with LsepL_{sep} do really improve the results.

In summary: You introduce a task and you give the right algorithm to compute it, yet you do not justify anywhere that the task is sound. The experiments do not show any meaningful improvement at a task where the goal is explicitly crafted for your objective and you do not show that on a real benchmark using the LjointL_{joint} objective improve the performances.

Questions for authors

1- Can you run knowledge distillation experiments where we see that adding your objective improve performances ?

2- In Eq. 12 you give a deterministic formula for computing Q if R is a proper rotation. In O(3) you can also find a deterministic formula if it is improper (because if R is improper there exists a proper R' such that R = -R'). Can you use this formula in your experiments for the separate case ? It will be a lot quicker than using Newton and i expect that computing LsepL_{sep} will be quicker than LjointL_{joint}.

Reviewer_XG2ROpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies the teacher-student superivision target, in the context of knowledge distillation. The authors introduce Joint Holonomy Matching and show that shared alignment is complete on the observed graph and that independent loop alignment is incomplete in general.

Critical strengths

The source code is included in the submission, which increases the validity of the proposed work.

Critical weaknesses

The paper is not well-written, and it is very hard to follow. Many terms are used before being introduced (e.g. Separate in the abstract). There is almost no appendix to provide background knowledge needed to understand the paper and the theory, and the bibliography is almost non-existent. Many terms used in the equations in section 3 are not explained.

Questions for authors

Clarification questions

  • The presented work seems almost detached from all the machine learning literature. Can the authors elaborate on this?
  • There are limited details provided on the Neural Pilot in section 7. Can the authors provided further details on the data, the task, and the student/teacher architecture specifics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.