acceptodds
Under review as a conference paper at ICLR 2027

A Constrained Kuramoto Gradient-Flow System Can Perform High-Accuracy Finite-Time Inference

Abstract

A central question in physical inference is whether strongly constrained dynamical systems can realize accurate input–output maps through their own finite-time evolution. We study this question in Kuramoto phase networks, whose deterministic dynamics form an input-conditioned gradient flow and whose predictions are read directly from output oscillators. As a constructive training approach, we develop a two-stage teacher–student procedure. A neural teacher is first converted into an explicit phase trajectory whose terminal oscillator activations reproduce the teacher outputs, and the Kuramoto parameters are trained by matching the student vector field along this prescribed path. Because accurate teacher-forced path matching does not ensure accurate autonomous inference, we then differentiate through the autonomous finite-time rollout and directly align its terminal output with the neural target. The resulting oscillator system, with oscillators, reaches mean test accuracies of on MNIST and on Fashion-MNIST. This capability persists across neural-teacher architectures, matched system sizes, thermal perturbations, and integration-grid refinement. Together, these results provide a constructive demonstration that a strongly constrained, small-sized Kuramoto gradient-flow system can be trained for high-accuracy finite-time inference through a direct oscillator readout.

Primary area:
learning on time series and dynamical systems
Keywords:
Kuramoto model, physical neural networks, dynamical systems, finite-time inference, gradient flow, coupled oscillators

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Xhv1OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a trainable constrained Kuramoto model and an accompanying two-stage training method. It demonstrates that the model can perform classification on MNIST and Fashion-MNIST, achieving accuracies of 96.711% and 86.399%, respectively, and evaluates the robustness of its performance under different parameter settings and perturbation conditions.

Critical strengths

The strength of this work lies in its relatively comprehensive demonstration of the effectiveness and robustness of the proposed approach within this particular Kuramoto architecture. The experimental results are fairly complete.

Critical weaknesses

(1) The paper claims that its model achieves “high accuracy” on MNIST and Fashion-MNIST. However, since both tasks are relatively simple, merely reporting accuracies of 96.711% and 86.399% does not establish that these constitute “high accuracy”; a comparison standard is necessary. Compared with the performance reported in related physical computing work by Wright et al. (Nature, 2022), the accuracies reported here are lower on both datasets. The authors need to explain how they define “high accuracy” and the source of this gap.

(2) According to the paper’s presentation, one of its core innovations lies in the two-stage training method. However, highly similar training methods already exist in prior work, such as Wan et al. (ICLR 2023), yet the paper lacks a discussion of such work. The authors need to clarify whether this work merely transfers a training paradigm previously applied to other models to a Kuramoto model, or whether the similarity is only conceptual and there is fundamental innovation at the implementation level. Otherwise, this core innovation cannot be substantiated, leaving the contribution of this work as merely a model evaluation report.

Questions for authors
  1. The model imposes several restrictions, including symmetric coupling, gradient-flow dynamics, no direct input-to-output connections, and no additional trainable readout layer. Are these restrictions primarily motivated by practical physical implementation requirements, or are they modeling choices made for this study? What does successful classification on MNIST and Fashion-MNIST under these restrictions teach us about the computational capabilities of physical dynamical systems?

  2. Why does the direct endpoint-training baseline use an all-zero initialization? Could this initialization cause the hidden oscillators to remain in identical states throughout training, limiting the baseline’s performance? Have the authors tried random initialization followed by direct BPTT optimization of the endpoint output? If so, how does it perform?

3 comments

Thank you for your detailed response and for conducting the additional experiments. I appreciate the authors’ acknowledgment of the limitations of the original endpoint-training baseline and their clarification of the relationship to prior work. However, I remain concerned about the novelty and significance of the paper’s principal contributions.

First, the response leaves me uncertain about how to assess the significance of the proposed architecture. The comparison with the same-size teacher MLP and a linear classifier provides useful context, but I do not think these relatively simple baselines alone establish that the model achieves “high accuracy” in the broader context of physical computing. The authors discuss architectural differences that may explain the performance gap with Wright et al. and acknowledge that they do not claim superiority. This clarification is helpful, but it also leaves me uncertain about the significance of the proposed model relative to existing physical computing approaches. Separately, the comparison with Wright et al. focuses on MNIST and does not address Fashion-MNIST.

More broadly, it remains unclear to me whether the main goal is to develop a physically realizable model with competitive performance, or to study the computational capability of this particular constrained oscillator architecture. If the latter is the intended contribution, I would like to see a stronger justification for why these specific constraints are of general interest and what insight the results provide beyond this particular model and task.

Second, I appreciate the authors’ clarification of the differences from Wan et al. However, my understanding is that the general two-stage training principle is already present in prior work, while the differences identified here primarily concern its adaptation to the Kuramoto model and the classification task. The convex quadratic structure of the first stage is an interesting property of the chosen parameterization, but it remains unclear to me whether these differences constitute a significant methodological innovation.

Moreover, the new experiments show that direct BPTT with an appropriate random initialization can outperform the proposed two-stage method without a teacher. While the authors report greater robustness to larger initialization scales under the tested protocols, these results make it less clear when the two-stage procedure is necessary or offers a practically significant advantage.

Overall, the response has clarified several technical issues, but it has also made the novelty and significance of the paper’s central contributions less clear to me. I will therefore retain my original rating.

Thank you for your detailed follow-up response and for the additional experiments. Your clarifications and revisions have addressed some of my concerns and made the paper’s claims more precise. However, at the same time, the revised positioning narrows the scope of the paper’s novelty and contributions. In my assessment, the resulting contribution does not warrant raising my rating to 3. I will therefore maintain my original rating.

We thank the reviewer for the follow-up comments. We are pleased that our clarifications and revisions have addressed some of your concerns, although we regret that we could not fully resolve all of them. We respect your assessment.

Regardless of the final outcome, we sincerely appreciate the time and effort you have devoted to reviewing our manuscript. Your constructive feedback has been valuable in helping us clarify and improve our work.

Reviewer_WG2uOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper trains a Kuramoto model-based model using a teacher-student approach. Specifically, the teacher is a tanh network trained with an end2end classification loss. Time-dependent trajectories are then constructed post hoc from its activations, and the student is trained to match the dynamics along these trajectories. The student is then fine-tuned to match the teacher’s outputs after a rollout. Performance is evaluated on MNIST and Fashion-MNIST.

Critical strengths

The proposed teacher-student training can train a student model stably even though the teacher model does not share the same topology, which could be an interesting insight in its own right.

Critical weaknesses

The experimental scale is quite limited. Only very small networks are trained on two very small datasets (MNIST, FashionMNIST). The framework should be evaluated on more meaningful tasks and larger networks (e,g. ImageNet with CNNs, language modeling with RNN/Transformers)

Also, I believe that to show the efficacy of the proposed framework, the authors need to do at least the following two experiments:

  • Training the student model with a classification loss + more sophisticated optimizers, without a teacher model. The authors only experimented with teacher endpoint loss + zero weight init for direct endpoint-only training. I would like to see the performance with e.g. classification loss + Adam or Muon + random init.
  • Comparison with conventional equilibrium propagation apparoch for training the Kuramoto model under similar architecture sizes and formulations.
Questions for authors
  • Regarding (1), what is this term: b_i(u)sin θi? This term does not usually appear in the conventional Kuramoto model literature. Why is the term introduced here?

  • I believe the teacher network’s trajectories do not minimize the same form of energy function as the student model does. Can the authors show whether the student model learns a meaningful trajectory in this case? For example, do you see performance degradation if you remove the interaction term between hidden oscillators? The model is still trainable with this removal, as there is a path from input to output through b(u) and the hidden-to-output connections.

Reviewer_2R7MOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper trains small oscillators(<100 nodes) under the constraints of symmetric coupling and gradient flow dynamics to classify MNIST and Fashion-MNIST. For this task they show that naive training does not work and they suggest a two-stage training. In the first step a teacher non-physical MLP is trained and used to derive a time-varying target for their student physical oscillator. In the second step backpropagation is used to train to align the final output of the system. The authors then provide ablation and tests on different teacher architectures.

Critical strengths

The paper is well structured and easy to follow.

The paper shows positive results on a new training method for oscillatory networks.

Critical weaknesses

Motivation. The work is not clearly motivated. It is justified as a follow-up work to Whitelam (2026) but the relevance for the ML community is not motivated. Why should we care about Kuramoto models to do ML? Is it because it can lead to energy efficient implementation? Then what's the purpose of their learning algorithm? To be directly implemented physically like Equilibrium Propagation? Or to train a physical system based on a digital twin? More motivation for work is needed. I did not understand the motivation for "constrained dynamical systems”: why focus on symmetric systems and gradient flow systems? For instance in bio-plausible or physics-based learning methods, researchers often treat symmetry as a constraint we want to remove (for instance the weight transport problem). For the gradient flow constraint, I'm also not understanding the appeal. It forces the author to use a very unusual kuramoto model without natural frequency (l.931–933), for instance this model does not display the classical phase-difference synchronization phenomena.

Why does simple endpoint training fail? The authors build on the negative results that classical methods fail, but in my experience, teacher forcing and backpropagation can be made to work on these systems. The argument would be stronger if the authors analyzed why their baseline method fails.

Lack of theoretical insight. The authors validate their training methodology only based on empirical evidence.

Single seeds where repeats are cheap. The system has 74 oscillators integrated over 200 Euler steps on MNIST-scale data, so runs are inexpensive. Yet the ablation (Table 9), the 15-teacher sweep (Table 8), the endpoint-weight sweep and the trajectory analysis are single runs with no standard deviation.

Small scale experiments. MNIST is a very low empirical bar for a training method that fully uses automatic differentiation. I would have expected at least CIFAR-10 results.

Questions for authors

Future work: Could you scale your experiment to CIFAR-10?

Clarification: Could you clarify the motivation behind the two constraints of your work: symmetric connectivity and gradient flow dynamics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.