acceptodds
Withdrawn from ICLR 2027

: An Interpretable State Space Model via an End-to-End Closed-Form Operator

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The linear dynamics of state-space models are – in principle – mathematically solvable, yet the nonlinear readouts required for expressivity render the full computation inaccessible to mathematical analysis. We introduce , a state-space model derived from a system of complex nonlinear oscillator networks in which the entire computation – both dynamics and readout – admits an exact, end-to-end mathematical description. The key is a nonlinear coordinate transformation that simultaneously linearises the recurrent dynamics and uniquely determines the readout nonlinearity, which in turn enables an exact analytical expression for the end-to-end computation done by the network. A broad class of connectivity patterns gives rise to traveling wave modes in the recurrent layer, and classification emerges from nonlinear interactions between wave modes, which we illustrate in an exact manner. We validate on EEG classification and sequential MNIST, providing a first-principles account of classification that no existing sequence model can offer, while also achieving competitive performance on benchmarks. The same mathematical framework also enables principled control: specific eigenmodes can be selectively amplified or suppressed to causally redirect model classifications. These results demonstrate a new SSM architecture that achieves competitive performance while adding precise interpretability and control.

Primary area:
interpretability and explainable AI
Keywords:
state space models, sequence modeling, mechanistic interpretability, closed-form analysis, coupled oscillators, traveling waves, model control, time series classification, activation steering, Kuramoto oscillators

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Q6J7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this paper, the authors introduce an SSM-like architecture designed to perform sequence classification in an interpretable manner. To do so, they replace the standard nonlinearity in SSMs (the feedforward MLP) with a nonlinearity based on the phase of each complex coordinate. Using experiments across multiple datasets, they argue that their proposed model has better analytical tractable at no cost to empirical performance. Finally, they use perturbations to show that the model is amenable to interventions that result in switching classification predictions.

Critical strengths
  • The proposed architecture is to my knowledge novel, and finds a way to replace the standard feedforward MLP nonlinearity in SSMs without appreciably damaging performance in sequence classification tasks.
Critical weaknesses
  • I think the central claim of this paper regarding the unique advantages in interpretability or "naturalness" of the proposed model is not sufficiently established. For instance, the phrase "grounded in first-principles" comes up repeatedly but is unjustified; why is the standard SSM nonlinearity any less "principled"? I think I somewhat grasp what the authors are trying to get at here, in that the nonlinearity they devise is somehow more recurrent-linear-flavored, but why is oscillator phase the canonical nonlinearity? Why is being able to visualize the phasors more informative/interpretable than being able to visualize MLP activation histograms in standard SSMs in PCA space? Regarding causal control of network predictions, as the paper states, the network output couples eigenmodes in a complicated way, so perturbations along one eigenmode do not cleanly decompose in a predictable fashion, much as is the case in standard SSMs. Maybe it is the case that these perturbations are "cleaner" and easier to find in this model compared to standard SSMs, but no comparative evidence is actually provided here.

  • The organization of the paper is quite confusing; section 2 introduces a bunch of objects and notation for which the shapes and interpretations are not defined, and only in section 3 do some of these things get clarified. It's as if the sections and subsections of 2 and 3 were written in reverse natural order?

Questions for authors
  • Is there any reason why an average over time in particular is used in the output in Eq. 6, as opposed to something more generic like an arbitrary linear function?
  • Why is fixing the connectivity to be circulant not a major restriction on expressivity?
Reviewer_AH3TOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces S^1 as an interpretable state space model which is derived from complex nonlinear oscillator networks and admits an exact, closed form operator description. The authors show that S1 has performance comparable to existing state space models while arguing it provides interpretability benefits.

Critical strengths

This work provides an interesting connection between nonlinear oscillator dynamics and state space modelling.

S^1 is a well motivated method with results competitive with the included baselines.

Critical weaknesses

Since the main contribution of the paper is the interpretability benefit of S^1, the paper would be strengthened by a more detailed comparison to a relevant baseline like gradient based attribution with existing state space models.

It would also be useful to know the sizes and speeds of different methods compared in table 1. Is there a computational overhead of S^1 compared to the other methods in the table?

Questions for authors

The main concerns that stop me from recommending acceptance are not being sure about the interpretability benefit and not being sure about the computational cost that is paid to achieve them.

Regarding the interpretability benefit it could be due to my lack of expertise in interpretability of time series models, but I would benefit from a clearer explanation of why the analysis in Figures 2 and 3 is more informative of the inner workings of the model than other post hoc methods like gradient based attribution for similarly simple models.

Reviewer_cfRhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

a state-space model derived from a system of complex nonlinear oscillator networks in which the entire computation -- both dynamics and readout -- admits an exact, end-to-end mathematical description.

Critical strengths

A principled design idea. The readout nonlinearity is chosen as the inverse of the coordinate transform that linearizes a (complex) oscillator network. It therefore has a dynamical meaning — oscillator phase — rather than being an arbitrary activation. The circulant/DFT parametrization exposes the modal structure without numerical diagonalization, and the resulting single-layer model is simple and cheap. This idea is the paper's main merit and could seed useful follow-up work, which is why informativeness is not rated lowest.

Critical weaknesses
  1. The central claim — the first SSM whose full linear–nonlinear cascade has an exact closed-form end-to-end description — does not hold, and the interpretive content of that closed form is limited.
    • The closed form is generic. Eq. 12 is obtained by substituting the modal expansion of yky_k into y/∣y∣y/|y|; the denominator is just ∣ynk∣|y_{nk}| written as a double sum. The same substitution gives a closed form for any single-layer diagonal SSM (e.g., S4D/LRU) followed by any pointwise nonlinearity, temporal pooling and a linear readout. Those nonlinearities equally "couple all eigenmodes". What makes S¹'s formula tractable is that it is a single layer with a pointwise nonlinearity, not the oscillator derivation.
    • The oscillator is a linear system in disguise. From Eq. 2 with y=eiψy=e^{i\psi}, one obtains y˙=(CKC−1)y+Bu\dot y=(CKC^{-1})y+Bu directly, and ψ\psi is complex-valued (Eq. 5). The real-phase oscillator picture therefore applies only to Re ψ\mathrm{Re}\,\psi.
    • Missing prior work. Earlier work on the exactly solvable complex-valued Kuramoto model (e.g., Muller et al., Phys. Rev. E 2021, and follow-ups) is not discussed.
    • The "traveling wave" interpretation has no functional content. CC is a learned dense complex N×NN\times N matrix, so the eigenvectors seen by the readout, CvℓCv_\ell, are arbitrary: CVCV can be any invertible matrix. The traveling-wave structure of the xx-system therefore has no functional consequence. The model is equivalent to a fixed-spectrum diagonal LTI system, followed by an arbitrary linear mixing, a phase nonlinearity and a linear readout.
    • The analyses are descriptive. The SCP1 and sMNIST analyses (class-wise modal power, phase rose plots) are plots one could produce for any linear SSM. They give no domain-level explanation (e.g., which EEG channels or frequency bands drive the decision) and no quantitative faithfulness evaluation. "A first-principles account of classification that no existing sequence model can offer" is therefore not substantiated.
  2. Empirical support for "competitive performance" and for the causal steering claim is weak.
    • UEA ranking. On the six UEA datasets, S¹ averages 59.8% and ranks 8th of the 10 methods in Table 1. It is below LinOSS-IM (67.8), LRU (63.1), S6 (62.2), S5 (61.7) and NRDE (60.2), and above only Mamba (58.1) and NCDE (55.5).
    • Per-dataset gaps. There are large deficits on EigenWorms (72.8 vs 95.0) and Heartbeat (68.4 vs 78.1), and the result on MotorImagery (50.5%, binary task) is at chance level.
    • Other benchmarks. S4 is named as a comparator in the abstract and introduction but does not appear in the table. sMNIST 97% and sCIFAR 63% are well below standard SSM results, and the asserted parity with single-layer S4 is not supported by any reported number.
    • Steering lacks controls. The steering experiment flips 8 samples by optimizing per-mode controls against a hinge loss, with WW and CC frozen. This is an optimized hidden-state perturbation, and small optimized perturbations can flip almost any classifier. By itself it does not show that the selected modes "carry the discriminative information". Same-budget perturbations on random or low-score modes, and more samples, are needed.
    • Inconsistent steering numbers.
      • Section 6 says five modes ℓ∈{1,5,22,32,37}\ell\in\{1,5,22,32,37\} carry non-zero controls of magnitude ≲5×10⁻³, and the mean margin moves from ≈−5 to ≈+5.
      • Fig. 4 reports top-kk=16 modes, γ\gamma changes on six modes including ℓ=60\ell=60, a broadly active δθ\delta\theta, an η\eta change of ≈−10⁻² at ℓ=1\ell=1, and a mean final margin of +3.47. Minor (not decisive):
  • Sec. 3 says the eigenvalues are "free learnable parameters", but the "Learnable parameters" paragraph says they are not trained. Their values, NN, the initial state x0x_0 and the training hyperparameters are not given.
  • Appendix B is truncated mid-sentence ("We optimise").
  • Cross-references are broken ("Eq. equation 12", "(q. equation 12", "gives . . . gives", and Fig. 1 refers to "equation 16").
  • Sec. 5.1 and 5.2 have duplicated "CASE STUDY" headings.
  • Eq. 15 drops the 1/T1/T.
  • Λ\Lambda denotes both continuous and discrete eigenvalues.
  • How y=0y=0 is handled in y/∣y∣y/|y| is not stated.
Questions for authors
  1. Concretely, what interpretability does S¹ offer that a single-layer S4D/LRU with a pointwise nonlinearity, mean pooling and a linear readout does not? Could the same modal and phase analyses be done there, and if so, how does S¹'s explanation differ in kind? This bears directly on the main contribution.
  2. Given that CC is learned and dense, in what sense are the modes seen by the readout traveling waves? Please report the structure of CvℓCv_\ell after training (e.g., is CC close to identity or diagonal?), or constrain CC and show the effect on accuracy and interpretability.
  3. Which eigenvalues (αℓ,ωℓ)(\alpha_\ell,\omega_\ell) are used, are they trained in any reported experiment, and how sensitive is accuracy to them? Are any modes non-decaying? Fig. 7 shows SCP1 modal amplitudes increasing monotonically over the trial.
  4. For steering, please provide:
    • (i) same-sparsity, same-magnitude controls on random or low-score modes;
    • (ii) results on all correctly classified source samples;
    • (iii) a reconciliation of the numbers in Sec. 6 and Fig. 4.
  5. Please position the work relative to prior results on the solvability of complex-valued Kuramoto models and relative to oscillatory sequence models (e.g., LinOSS, coRNN).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.