- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper gives an ablation analysis of untrained coupled oscillator networks, which are basically used as reservoirs for the task of spoken digit recognition on the AudioMNIST data. The reservoir is connected to a linear readout for the output computation, and the results are analyzed and compared in different setups, including: with the uncoupled network, with the input alone, with leaky integrators, and with small fully trainable baselines. The paper studies the performance by varying several aspects of the methodology, among which are the coupling function, natural frequencies, lattice geometry, input gain, and pathway. The main result is that the reservoir-based network is able to classify the digits almost as well as its own input, and that the untrained reservoir is mainly adding memory.
Critical strengths
- I appreciated the design of the experimental analysis, which I could find pretty careful and transparent. In particular, the authors included controls that are often left out in most contributions in the area, such as the ablation vs the input alone, the uncoupled network, and the leaky integrators.
- Systematic ablations of oscillatory networks are largely missing in the related literature, and this paper can be considered as a reference for the future works in this sense.
Critical weaknesses
-
Lack of model selection for the oscillatory networks risks to reduce the soundness of the reported comparisons. In fact, from my understainding, the reference configuration is fixed, the input gain is also set to 1 or 2 in the main experiments.
-
I was surprised to notice that some natural baselines are missing in the comparison, including standard echo state networks, and previously proposed architectures in the area of coupled oscillators networks (see also the questions section).
Questions for authors
- How did you choose the reference configuration anf the input gains? Please report the results achieved after careful selection of the hyperparameters on a validation set.
- Please include among the baselines to test also other relevant models from the reservoir computing literature, including the conventional echo state networks, and the reservoir oscillator network RON (Ceni et al., 2024), again after propser tuning of the hyperparameters through rrigorous model selection.
- It would be interesting to add a further control to assess the effect of the dynamics against the effect of the dimensionality. For example, using the same readout, but applying it to a random nonlinear expansion of the input alone.