- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces MARS, a deep reservoir architecture combining memristive-inspired state dynamics with parallel scans and subtractive skip connections. All feature-producing weights remain fixed after random initialization, and only a linear readout is fitted using ridge regression. Experiments on seven short-sequence and six long-sequence classification benchmarks report competitive accuracy and substantial reductions in fitting time.
Critical strengths
The effort to connect device-inspired dynamics with efficient parallel computation is a critical strength
Critical weaknesses
I believe equation 13 is wrong. Let's do it for simplicity in 1D We have the recurrence: h(t) = a(t) h (t-1) + b(t) If we solve this we will find h(t) = product of (h(0) a(1) a(2)...a(t)) + sum from i=1 to t ( b(i) a(i+1) a(i+2) ..a(t)) So h(t) is a sum and there is no way that this sum gives rise to equation (13)
The authors should correct the derivation and demonstrate numerical agreement between sequential and parallel implementations across representative inputs and sequence lengths. This is essential to validating the central methodological contribution.
Questions for authors
The related-work discussion could consider ROCKET (Dempster, Petitjean, and Webb, 2020) and MiniRocket (Dempster, Schmidt, and Webb, 2021). These methods use fixed convolutional features and linear classifiers for efficient time-series classification, making them particularly relevant to MARS-TC and its claims about fast fitting with untrained features. Direct comparisons would help establish the method’s novelty and practical advantages. The authors should also clarify how their contribution differs from existing parallel recurrent scans and reservoir architectures.
Overall, MARS is a potentially interesting contribution, but the mathematical issue in the scan derivation is a fundamental concern. If you can fix this error and follow up with stronger ablations and more comparable computational evaluations, you may have a good paper.
2 comments
Relationship to parallel scans and reservoir architectures
We agree that these distinctions should be clarified. We have expanded Section 2 into “Background and related work” to position MARS more explicitly.
Associative parallel scans and the log-space construction are established methods; we do not claim these algorithms as new. Our contribution is to reformulate the memristive-inspired reservoir update into a compatible input-dependent affine recurrence and integrate it into a deep architecture with fixed feature-producing weights and readout-only training.
We now distinguish MARS from DeepESNs, which already establish stacked untrained reservoirs; MF-ESN, whose dense recurrent coupling makes its update sequential; and ParalESN, which uses a fixed complex-valued diagonal transition matrix. MARS instead uses real-valued transition coefficients determined by the input through the memristive potentiation–depression rates, while enabling parallel evaluation.
Additional ablations and computational evaluation
We also strengthened the experimental evaluation. Appendix A now compares subtractive, additive, and no-skip variants under fixed configurations. Section 4.1 additionally compares sequential and parallel implementations of the same memristive recurrence, isolating the contribution of parallel execution and measuring approximately 26–99× speedups across the tested sequence lengths. We also clarified the hardware and the exclusion of hyperparameter-search costs from the reported training times in Section 4.2.
Closing remarks
We would like to close by clarifying the motivation of this study. Walker et al. (2024) introduced Log-NCDEs to improve the efficiency of neural controlled differential equations, while Rusch and Rus (2025) demonstrated the effectiveness of oscillatory state-space models for long-sequence learning. All these models are gradient-based requiring many training iterations to converge. Building on this benchmark setting, we investigate whether competitive classification accuracy can also be achieved with fixed, device-inspired recurrent features and a linear readout.
Our results show that, on the evaluated tasks, competitive performance is possible without gradient-based training of the feature-producing layers. Our primary aim is to demonstrate this accuracy–cost trade-off and the potential of lightweight memristive-inspired reservoirs. The physical memristive-based reservoir demonstrated by Milano et al. (2022b) indicates a potential for future hardware implementations. Honestly, achieving such competitive results in the tested benchmarks was more of a byproduct of our aim, showcasing a stable (physical) model development. We have updated the conclusion section to state our aim more explicitly.
We hope that the additional experiments, analysis, presentation improvements, and the fact that the Equation (13) is correct address the reviewer’s concerns and we actually now "have a good paper". If so, we would be grateful if the reviewer would reconsider their assessment in light of these revisions. We would be happy to discuss further if needed.
Sincerely, The authors
We thank the reviewer for recognizing the potential of MARS and highlighting the connection between device-inspired dynamics and efficient parallel computation as a critical strength. This connection is central to our study. We investigate whether fixed, memristive-inspired recurrent features can support competitive long-sequence classification with low training cost. The reviewer’s constructive feedback helped us make this contribution clearer and strengthen its validation. We have added an explicit scan derivation and numerical equivalence tests, expanded the related-work discussion, and strengthened the ablations and runtime analysis. We also revisited the manuscript throughout to sharpen the explanations, clarify the assumptions and experimental protocols, and make the presentation easier to follow. Below, we address each concern in detail.
Associative scan derivation
This is an important observation and we agree that the derivation of Eq. (13) should have been given explicitly. Equation (13) is the log-space form used to evaluate the affine recurrence by associative scan and it follows the Equation 1 in Heinsen (2023). The given unrolled scalar recurrence in the review is exactly the starting point of the derivation. We provide the full derivation in the Appendix E and pointed to the derivation from the main text. We believe this contribution further developed the robustness of our manuscript. With the included derivation, we hope to make it crystal clear that the Equation (13) is indeed correct.
As suggested by the reviewer, we also compared our parallel implementation with the sequential recurrence on synthetic inputs. The table below reports absolute errors across the full state trajectories for different sequence lengths up to 20,000t. We include a short Jupyter notebook file in the supplementary material which produced the following values:
Relationship to ROCKET and MiniRocket
We thank the reviewer for suggesting new baselines. Our study focuses on whether fixed, memristive-inspired recurrent dynamics can be a viable option for sequential classification tasks in the realm of recurrent neural networks. The small convolutional filters in MARS-TC serve as an input encoder within this recurrent architecture, particularly developed for very long sequences (e.g., EigenWorms with ~18k steps). ROCKET and MiniRocket instead construct high-dimensional representations through fixed convolutional feature transforms (for instance ROCKET applies the linear mapping from a 20,000 dimensional space). The proposed convolutional architectures can access the full sequence at once, without stateful memories, which makes them --in our view-- an entirely different model family. Therefore, we have not added that comparison in this revision. We believe, even if convolutional feature methods achieved higher accuracy, this would not invalidate our finding that fixed, memristive-inspired recurrent dynamics can provide competitive classification with low fitting cost.
We agree that our naming was misleading. Our model is not a classical temporal convolution model. We use convolution as a tool to generate local temporal features. Hence, we decided to change our naming to MARS-LTF (local temporal features). Also, it can be seen from the results that the benefit of these local features varies across datasets. The encoder provides a way to extract high-frequency patterns with potentially low-frequency recurrent dynamics.
Following this suggestion, we realized that the manuscript did not sufficiently position MARS within the existing literature. Therefore, we have expanded Section 2, also discussing the fixed-kernel convolutional approaches.
We consequently retain the recurrent-model focus of our experimental evaluation and clarify this scope in the revised related-work discussion.