- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a trainable constrained Kuramoto model and an accompanying two-stage training method. It demonstrates that the model can perform classification on MNIST and Fashion-MNIST, achieving accuracies of 96.711% and 86.399%, respectively, and evaluates the robustness of its performance under different parameter settings and perturbation conditions.
Critical strengths
The strength of this work lies in its relatively comprehensive demonstration of the effectiveness and robustness of the proposed approach within this particular Kuramoto architecture. The experimental results are fairly complete.
Critical weaknesses
(1) The paper claims that its model achieves “high accuracy” on MNIST and Fashion-MNIST. However, since both tasks are relatively simple, merely reporting accuracies of 96.711% and 86.399% does not establish that these constitute “high accuracy”; a comparison standard is necessary. Compared with the performance reported in related physical computing work by Wright et al. (Nature, 2022), the accuracies reported here are lower on both datasets. The authors need to explain how they define “high accuracy” and the source of this gap.
(2) According to the paper’s presentation, one of its core innovations lies in the two-stage training method. However, highly similar training methods already exist in prior work, such as Wan et al. (ICLR 2023), yet the paper lacks a discussion of such work. The authors need to clarify whether this work merely transfers a training paradigm previously applied to other models to a Kuramoto model, or whether the similarity is only conceptual and there is fundamental innovation at the implementation level. Otherwise, this core innovation cannot be substantiated, leaving the contribution of this work as merely a model evaluation report.
Questions for authors
-
The model imposes several restrictions, including symmetric coupling, gradient-flow dynamics, no direct input-to-output connections, and no additional trainable readout layer. Are these restrictions primarily motivated by practical physical implementation requirements, or are they modeling choices made for this study? What does successful classification on MNIST and Fashion-MNIST under these restrictions teach us about the computational capabilities of physical dynamical systems?
-
Why does the direct endpoint-training baseline use an all-zero initialization? Could this initialization cause the hidden oscillators to remain in identical states throughout training, limiting the baseline’s performance? Have the authors tried random initialization followed by direct BPTT optimization of the endpoint output? If so, how does it perform?
3 comments
Thank you for your detailed follow-up response and for the additional experiments. Your clarifications and revisions have addressed some of my concerns and made the paper’s claims more precise. However, at the same time, the revised positioning narrows the scope of the paper’s novelty and contributions. In my assessment, the resulting contribution does not warrant raising my rating to 3. I will therefore maintain my original rating.
We thank the reviewer for the follow-up comments. We are pleased that our clarifications and revisions have addressed some of your concerns, although we regret that we could not fully resolve all of them. We respect your assessment.
Regardless of the final outcome, we sincerely appreciate the time and effort you have devoted to reviewing our manuscript. Your constructive feedback has been valuable in helping us clarify and improve our work.
Thank you for your detailed response and for conducting the additional experiments. I appreciate the authors’ acknowledgment of the limitations of the original endpoint-training baseline and their clarification of the relationship to prior work. However, I remain concerned about the novelty and significance of the paper’s principal contributions.
First, the response leaves me uncertain about how to assess the significance of the proposed architecture. The comparison with the same-size teacher MLP and a linear classifier provides useful context, but I do not think these relatively simple baselines alone establish that the model achieves “high accuracy” in the broader context of physical computing. The authors discuss architectural differences that may explain the performance gap with Wright et al. and acknowledge that they do not claim superiority. This clarification is helpful, but it also leaves me uncertain about the significance of the proposed model relative to existing physical computing approaches. Separately, the comparison with Wright et al. focuses on MNIST and does not address Fashion-MNIST.
More broadly, it remains unclear to me whether the main goal is to develop a physically realizable model with competitive performance, or to study the computational capability of this particular constrained oscillator architecture. If the latter is the intended contribution, I would like to see a stronger justification for why these specific constraints are of general interest and what insight the results provide beyond this particular model and task.
Second, I appreciate the authors’ clarification of the differences from Wan et al. However, my understanding is that the general two-stage training principle is already present in prior work, while the differences identified here primarily concern its adaptation to the Kuramoto model and the classification task. The convex quadratic structure of the first stage is an interesting property of the chosen parameterization, but it remains unclear to me whether these differences constitute a significant methodological innovation.
Moreover, the new experiments show that direct BPTT with an appropriate random initialization can outperform the proposed two-stage method without a teacher. While the authors report greater robustness to larger initialization scales under the tested protocols, these results make it less clear when the two-stage procedure is necessary or offers a practically significant advantage.
Overall, the response has clarified several technical issues, but it has also made the novelty and significance of the paper’s central contributions less clear to me. I will therefore retain my original rating.