acceptodds
Under review as a conference paper at ICLR 2027

CPTP-SSM: Expanding State-Tracking to Continuous Groups with Uncertainty

Abstract

Recent selective state-space models (SSMs) have increased transition expressivity to support the state tracking of finite-state automata (FSA). Recently, SSMs are increasingly applied to long sequences from the real world beyond language modeling. We extend state tracking to continuous groups with uncertainty, requiring input-selective transitions that can be either conservative on the continuous state or dissipative under uncertainty. Rather than enlarging the transition, we constrain the state space and admit only transitions that map it into itself. We introduce CPTP-SSM, a selective SSM on the compact convex set of qudit density matrices. The transitions are completely positive trace-preserving (CPTP) maps, a single family that admits both conservative and dissipative transitions. Expressing the Kraus representation in the generalized Gell-Mann basis turns each transition into a real affine map, enabling the recurrence to run as a parallel associative scan. We establish expressivity and stability guarantees for CPTP-SSM and further characterize how the qudit dimension governs the representable groups, while the Kraus rank governs the capacity for dissipation. Experimentally, CPTP-SSM achieves strong length extrapolation on both finite and continuous state tracking, with competitive performance in long time-series classification and continuous sensor prediction.

Primary area:
learning on time series and dynamical systems
Keywords:
State-Space Models, Linear RNNs, State Tracking, Expressivity, Length Generalization, Sequence Modeling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_z33KOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Focusing on enhancing the expressiveness of state-space models (SSMs), this paper proposes quantum-channel-inspired transition matrices for SSMs whose structures admit both conservative and dissipative transitions with stability. Experimental results show superior state tracking across synthetic and real-world benchmarks, especially under noisy measurements/observations, and competitive time-series classification performance.

Critical strengths
  1. The paper clearly provides a novel lens on improving recurrent models by observing the fact that the permutation and reset characterization of finite-state computation suggests two different roles for a recurrent transition. In effect, it motivates input-selective transitions in which a single update can preserve some state distinctions while contracting others.

  2. Providing new experimental setups and results demonstrating improved state tracking capability under noisy observations/measurements in synthetic and real-world benchmarks.

Critical weaknesses
  1. The paper has not provided any computational complexity analysis of the proposed CPTP-SSM (and a comparison with other SSMs). What are the memory and computational complexities with respect to state size and input length?

  2. Although it was not motivated by use cases in language modeling, it would be useful to elaborate/comment the potential advantages of the CPTP-SSM for LLMs, if any.

Questions for authors
  1. Noisy measurement experiments are very interesting. Could they be systematically expanded with more synthetic tasks or, better, with more real-world benchmarks? This is a very nice feature to be thoroughly evaluated, which can unlock truly new use cases. Also, PD-SSM is missing from Tables 6 and 3; it would be good to add it for the sake of a complete comparison.

  2. For time-series classification, it would be great to provide a more comprehensive comparison with other time-series/long-sequence tasks that LTI systems (like LinOSS) are normally benchmarked.

2.1) In Table 5, it would be great to cross-compare with Flash PD-SSM and benchmark with, e.g., time-series forecasting tasks reported in Flash PD-SSM paper.

2 comments
AuthorsOpenReview ↗

Response to Reviewer z33K, part 1 of 2. Complexity and language models

Thank you for the encouraging assessment and the concrete requests. Section, table and appendix numbers below refer to the revised PDF.

Computational and memory complexity

Section 3.2 now states the complexity, and Table 8 in App. B.4 gives the per-token costs. The table below summarizes it for sequence length LL, width DD, and nqn_q heads of dimension dd and Kraus rank rr, with state size N=nq(d2−1)N=n_q(d^2-1).

General headQubit head, d=2d=2
TimeO(L nq(rd2D+d6))O(L\,n_q(rd^2D+d^6))O(LNrD)O(LNrD)
Training memoryO(L nq(rd2+d4))O(L\,n_q(rd^2+d^4))O(LNr)O(LNr)
State at inferencenq(d2−1)n_q(d^2-1)NN

Adding heads is linear in NN, while raising dd costs d6d^6 per head. The backward pass is the same scan run in reverse on transposed transitions, as App. B.3 describes, and costs 1.81 times the forward pass according to Table 7.

Comparison with other SSMs, Table 9 in App. B.4. One layer of width 1,024 with about 6.7M parameters, batch 4, one A100 80GB, three seeds. PD-SSM runs in FP32 and the others in BF16.

LayerParametersForward + backward in ms at LL = 512 / 2,048 / 16,384Peak memory in GiB at LL = 512 / 2,048 / 16,384
Mamba-36.69M4.0 / 6.6 / 48.60.2 / 0.7 / 5.7
Gated DeltaProduct, nh=4n_h=4, 3 heads of size 1286.72M7.6 / 7.7 / 41.60.2 / 0.8 / 6.5
PD-SSM, dictionary size 32, state size 3316.69M2,256 / 34,767 / OOM8.7 / 34.6 / OOM
CPTP-SSM, 187 qubit heads, r=4r=46.71M2.6 / 9.4 / 75.00.3 / 1.1 / 8.5

At L=16,384L=16{,}384 our layer takes 1.5 times the time of Mamba-3 and 1.8 times that of Gated DeltaProduct, and at L=512L=512 it is the fastest. PD-SSM runs out of the 80 GB of memory at L=16,384L=16{,}384. Flash PD-SSM is not included because its repository had no model code when we checked on 11 November.

Potential advantages for language models

We expect the advantage where language meets physical data, as in vision-language-action models. The layer tracks both finite automata and continuous sensor states, and such models need both at once. Section 5 now says so. Whether qubit heads have enough capacity at the scale of a language model and kernels that make them cheap enough there are future work.

AuthorsOpenReview ↗

Response to Reviewer z33K, part 2 of 2. Noisy measurements and long sequences

We agree that tracking under noisy measurements deserves a test on real data, and the pointer to Flash PD-SSM was useful. Both led to new results in the revision.

Q1. More noisy-measurement experiments, and PD-SSM in Tables 3 and 6

Two new real-world benchmarks in Section 4.2. We add orientation tracking from a real gyroscope on the EuRoC drone flights and on the hand-held TUM-VI sequences, with the split and the open-loop protocol of Brossard et al. 2020. The input is a rotation increment per 50 ms, as in Table 3, but measured by a real sensor. Models are trained on windows of at most 2 s. Each whole test sequence, 98 to 150 s long on EuRoC and 2 to 2.5 min on TUM-VI, is then run once from start to end without reset. The table gives the absolute orientation error in degrees, mean ± SD over three seeds.

ModelEuRoCTUM-VI
LSTM33.34 ± 8.3910.37 ± 0.36
PD-SSM65.99 ± 10.72104.09 ± 10.87
Gated DeltaProduct61.50 ± 4.9899.55 ± 6.19
Mamba-3109.82 ± 6.49124.46 ± 4.60
CPTP-SSM, r=1r=19.82 ± 0.716.03 ± 5.79
CPTP-SSM, r=4r=46.35 ± 0.241.98 ± 0.27
Gyroscope integration with calibrated bias5.751.76

First, CPTP-SSM has by far the lowest error among the learned models, and it is on par with the classical solution of this problem, gyroscope integration with a calibrated bias, although the bias is never given to it. Second, length extrapolation carries over from discrete tasks to continuous real data. Every model is within 1.4° at the training length, and beyond it only CPTP-SSM keeps tracking. Trained on windows of 76.8 s instead, CPTP-SSM with r=4r=4 stays the most accurate on EuRoC, with 5.21 ± 0.03° against 14.49 ± 1.16° for the LSTM and 20.75 ± 7.56° for Mamba-3, so the result is not an effect of short training windows. The higher rank helps on real sensors as it does in Table 3.

PD-SSM. It is a baseline in the table above. For Table 6 its released implementation is too slow, as App. C.2 explains. CSP-Bench sequences have up to 12,000 steps, and one training step on JoystickControl took 93 s. Flash PD-SSM removes this cost, but its repository had no model code when we checked on 11 November.

Q2 and Q2.1. Long-sequence benchmarks and Flash PD-SSM

The revision adds Flash PD-SSM to Tables 4 and 5. It has the highest average on the six UEA datasets, 69.1 against 67.6 for CPTP-SSM, which keeps the highest mean on EigenWorms with 96.1 and on Heartbeat with 81.3.

PPG-DaLiA. The table gives the test MSE in units of 10−210^{-2} on sequences of 49,920 steps. Our row is the mean over five seeds, with 27 of the 162 grid configurations searched. The other rows are from Rusch & Rus 2025 and Terzić et al. 2026.

ModelMSE
LinOSS-IM6.4 ± 0.23
Flash PD-SSM6.5 ± 0.35
LinOSS-IMEX7.5 ± 0.46
CPTP-SSM9.22 ± 0.80
Log-NCDE9.56 ± 0.59
NRDE9.90 ± 0.97
Mamba10.65 ± 2.20
LRU12.17 ± 0.49
S512.63 ± 1.25
S612.88 ± 2.05
NCDE13.54 ± 0.69

We hope this addresses the weaknesses and questions, and we would be grateful if you would reconsider the rating.

Reviewer_uMcVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the state-tracking problem, i.e., the capacity of a class of state-space models with inputs to represent a (semi-)group as a semi-group of its state-transition maps. In this paper, the class of state-space models is a particular class of SSM, state-space of which is define on positive definite matrices whose trace is 11, and whose affine state-space transformations (indexed by inputs) have a specific form. It is shown that such models can represent 1) finite-state automata 2) various groups.

Critical strengths

The paper shows a property of a class of SSM which perhaps could be useful.

Critical weaknesses

The weaknesses of the paper is motivation, presentation, depth of the contribution.

  1. Motivation: Neither the proposed class of SSMs nor the state-tracking problem are very well motivated. It is not clear why the considered sub-class of SSMs is useful. The fact that it can be used to represent certain groups is not very compelling. So what ? From the numerical examples it is not clear if the proposed class of SSMs outperforms other classes on relevant benchmarks.

The same goes for state-tracking problem: being able to simulate finite state machines is not a very surprising property, for state-affine systems, i.e. general SSMs of the form x(t+1)=A(u(t))x(t)+b(u(t)), y(t)=Cx(t) this was shown in Salomaa A., Soittola M. Automata-theoretic aspects of formal power series, 1978, Theorem 5.1, page 36, if one uses the relatively well-known correspondence between such systems and recognizable linear representations: E. Sontag. Realization theory of discrete-time nonlinear systems: Part I-The bounded case, IEEE Transactions on Circuits and Systems, 1979, which was also done for switched systems (SSMs with a finite input space) Mihály Petreczky, Laurent Bako, Jan H. van Schuppen, Realization theory of discrete-time linear switched systems, Automatica, Volume 49, Issue 11, 2013. The ability to represent groups of rotation matrices, etc. is perhaps more difficult to derive.

However, in both cases, the question is how to use these results to say something meaningful about SSMs.

  1. Presentation: The presentation is not always precise and sometimes uses terminology which is not necessary. Some examples: -- Section 2.4: why to talk about qudits, Kraus representations etc., when the quantum interpretation of these concepts are not relevant for the paper. One could just define CPTP-SSMs by defining the state-space as as the set Dd\mathcal{D}_d from eq. (2), and the state-transition matrices and vectors as in eq. (7). A remark in the appendix on the relationship with qudits would have been sufficient. At the end of the day, CPTP-SSMs are just SSM with a specific choice of matrices A,b and a specific state-space.

    -- Definition 2: what is the topology of the non-abelian group GG ? Do you mean a topological group here, i.e., the group operations have to be continuous, or do you have further restrictions on the topology.

    Why Definition 3, Definition 4, Definition 5 are relevant properties ? What can we prove using them ?

    -- Propostion 1: 'the regular representation acts by permutation unitaries' - I don"t know what the authors are trying to say, what a regular representation is, what a permutation of unitaries is. I could not find the definition of these concepts in the paper. -- Proposition 2; Define PU(d). Consider not using terms such qudit head, etc., just show the corresponding equation. It will be more readable.

-- What exactly do the authors want to demonstrate in the numerical example section: what is the hypothesis the experiments are supposed to prove/refute ?

-- Depth of contribution I found the results relatively straightforward, with an unclear use for the wider field.

Questions for authors
  1. Why to study state-tracking ? What is the use of CPTP-SSMs beyond proving that it has some state-tracking properties ?
  2. What is the hypothesis which you wanted to prove using numerical examples ?
8 comments
AuthorsOpenReview ↗

Response to Reviewer uMcV, part 1 of 3. Why state tracking, and what we are trying to do

Thank you for your careful review and the systems-theory perspective. Your comments helped us clarify the motivation for state tracking and the problem CPTP-SSM addresses. We have revised the introduction to make both explicit, and we explain them here in more detail.

Why state tracking

Transformers attend to past tokens, so their computation and memory costs grow with sequence length. Deep SSMs such as Mamba instead carry a fixed-size state, which makes long sequences cheaper to process. Much of the work on deep SSMs has focused on language modelling. A key theoretical question has been whether their state can retain the information needed and update it correctly with each input. State-tracking tasks test this, for example whether a model can keep track of a variable as a program changes its value.

This ability depends on the structure of the transition matrix A(u)A(u). Mamba takes it diagonal, and diagonal matrices commute, so such a transition cannot follow updates whose order matters. Prior work identified this limitation and developed more expressive transition structures. These theoretical results have led to gains on language tasks, lowering perplexity on code and mathematics in Grazzi et al. 2025 and improving entity tracking in Terzić et al. 2026.

Expanding state tracking to noisy, continuous real-world sequences

Much of the work on state tracking has been motivated by language modelling and has focused on finite automata and finite groups. Deep SSMs are now also used in vision, robotics, and inertial and tactile sensing, where states are continuous and measurements are noisy. Our goal is to identify the properties needed to track these states and to introduce a family of SSMs that satisfies them.

In finite-state tracking a transition has two roles, permutation and merge, as in Definitions 1 and 3. Every automaton transition can be composed from these two types of update. We consider how these roles extend to continuous states and uncertain inputs.

Orientation gives a concrete example. It evolves through an ordered sequence of rotations, each measured with noise. This motivates three requirements.

  • Rotate. Represent continuous rotations, which take the role of permutations in finite-state tracking. This is Definition 2.
  • Average. Represent averages over uncertain rotations, so that uncertainty reduces the distinctions between states. This is Definition 4.
  • Stay bounded. Keep the state in a fixed bounded set for every input sequence, however long. This is Definition 5, state invariance. Bounding the norm of A(u)A(u) alone is insufficient, because the offsets b(u)b(u) can accumulate. The table places them next to their finite-state counterparts.
Role of a transitionFinite states, prior workContinuous states and uncertain inputs, this paper
Keep distinctionsPermutation, Def. 1Rotation, Def. 2
Remove distinctionsMerge, Def. 3Mixing, Def. 4
Stay stable at every lengthBound on the norm of A(u)A(u)State invariance, Def. 5

Propositions 1 to 3 show that our family meets all of these requirements, and none of the existing families in Table 1 does. Figure 2 tests tracking beyond the training lengths, and the noisy rotation experiments test the ability to rotate and average.

AuthorsOpenReview ↗

Response to Reviewer uMcV, part 2 of 3. Why not a general state-affine system, and what the paper contributes

The references to realization theory were very helpful. We now cite them, and they let us state our question more precisely.

Why not a general state-affine system

We agree that with unconstrained A(u)A(u) and b(u)b(u) every finite automaton is realized, and that this is classical. Section 2.2 of the revision cites Salomaa and Soittola 1978, Sontag 1979 and Petreczky et al. 2013. The automaton part of Proposition 1 is that construction inside our family, and we do not claim it as new.

You ask how such results can say something meaningful about SSMs. Our answer is that the meaningful question is a constrained one, because a deep SSM cannot use unconstrained A(u)A(u) and b(u)b(u).

  • Cost. A layer is evaluated in parallel by composing transitions. Composing two dense N×NN\times N matrices costs N3N^3 operations, against NN for diagonal ones. This is why every practical deep SSM restricts the structure of A(u)A(u). Ours composes 3×33\times3 blocks, so its cost is linear in NN.
  • Stability. Products of input-selected matrices have no uniform bound. Our comparison includes BD-SLiCE, a layer with unconstrained block-diagonal transitions that can represent every transition we consider. Its accuracy on A5A_5 falls to 0.15 at length 5,120. We also put unconstrained 3×33\times3 matrices and offsets into our own layer. All three trained models produce non-finite outputs, where ours reaches 1.000, as Table 16 in App. C.3 shows. So the question is which cheap and stable families can still represent the system, in the spirit of positive realization. An answer tells which structure and state size a layer needs for a given class of dynamics, and its predictions can be tested, as part 3 describes.

Our answer, choosing the state set first

You describe the model as an SSM "with a specific choice of matrices A, b and a specific state-space". That is accurate, and the state space is where we start. Existing deep SSMs choose a structure for A(u)A(u) and then bound its norm. We choose a bounded convex set and admit the transitions that map it into itself, so that state invariance holds by construction. In systems terms the layer is a state-affine system, or a switched affine system when the input alphabet is finite, whose transitions share one invariant set and are non-expansive in one common norm. This is Proposition 3.

Which set? It must be convex for mixing, bounded for stability, and it must admit rotations. Lemma 1, which follows from standard facts on compact groups, says that such a set has dimension at least three and that in dimension three it is a ball. This is why the heads we use have a three-dimensional state, the unit ball of R3\mathbb R^3. On the ball, the transitions of our family with zero offset are exactly the rotations and their averages, a classical fact due to Landau and Streater 1993, and in general dimension the family contains the merges that automata need by Proposition 1.

We do not propose a new orientation filter. The point is a trainable layer that contains the exact filter for noisy rotations as a special case and is bounded by construction, so that it can be trained on raw sensor streams for which no measurement model is given.

What the paper contributes

On the depth of the contribution, we agree that the automaton result is straightforward. The paper is a theoretical contribution in the line of the works cited in part 1. Each of them placed a known mathematical structure in the transition and characterized what the model then tracks, and those results later improved language models. Our contribution does not rest on a single proposition. It is the following.

  1. The formulation. The two roles of a transition in finite-state tracking are carried to continuous states with uncertain inputs and stated as requirements that can be checked for any family. Table 1 checks them for the existing families.
  2. State invariance as a requirement for deep SSMs, with evidence in Fig. 2 that the norm bounds used so far do not suffice at long lengths.
  3. One family that meets all requirements, computed by the same parallel scan as other deep SSMs. Its two parameters have a precise meaning. The dimension dd decides which groups a head tracks, and the number of terms rr decides whether it can contract. This includes an impossibility. No number of 2×22\times2 heads tracks S5S_5 in one layer, while one 4×44\times4 head does, by Proposition 6.
  4. Experiments that test these statements, including noisy rotation tracking against the exact posterior mean and real sensor data, described in part 3.

For the wider field, the use is a design rule for deep SSMs on physical data. Decide what the state must be able to do, choose a state set that admits it, and admit only the transitions that preserve the set.

AuthorsOpenReview ↗

Response to Reviewer uMcV, part 3 of 3. What the experiments test, and a new experiment on real sensors

We agree with your comments on the experiments and the presentation. We used several terms without defining them, and we answer each point below.

What the experiments test

Section 4.1 tests the theory, one statement per experiment.

  • A5A_5, Fig. 2. One invariant state set keeps tracking beyond the training length. Trained on lengths up to 40, ours stays at 1.000 at length 5,120, where the best parallel baseline is at 0.15.
  • S5S_5, Table 2. d=2d=2 cannot track S5S_5 and d=4d=4 can, by Proposition 6. Accuracy is 0.083 against 1.000.
  • Noisy rotations, Table 3. Exact rotations need only r=1r=1, noisy ones need r>1r>1. With noise, R2R^2 is 0.618 at r=1r=1 and 0.993 at r=4r=4.
  • Automata, Table 4. A reset needs r>1r>1. Modular Arithmetic is at 26.3% with r=1r=1 and 99.7% with r=4r=4.

Section 4.2 then tests whether the model works in practice on real data. It is competitive on long time-series classification in Table 5 and has the lowest error on four of the six sensor tasks of Table 6. The revision adds orientation tracking from a real gyroscope, described next.

What the model is useful for

You note that it was not clear whether the model outperforms other SSMs on a relevant benchmark. The example that motivates the paper, orientation tracking from an inertial sensor, was not tested in the submission. Section 4.2 of the revision now tests it on two real datasets, the EuRoC drone flights and the hand-held TUM-VI sequences, with the protocol of Brossard et al. 2020. The input is the gyroscope alone. Models are trained on windows of at most 2 s and then run once over each whole test sequence, 98 to 150 s long on EuRoC and 2 to 2.5 min on TUM-VI, without reset. The table gives the absolute orientation error in degrees, mean ± SD over three seeds.

ModelEuRoCTUM-VI
LSTM33.34 ± 8.3910.37 ± 0.36
PD-SSM65.99 ± 10.72104.09 ± 10.87
Gated DeltaProduct61.50 ± 4.9899.55 ± 6.19
Mamba-3109.82 ± 6.49124.46 ± 4.60
CPTP-SSM, r=4r=46.35 ± 0.241.98 ± 0.27
Gyroscope integration with calibrated bias5.751.76

First, CPTP-SSM has by far the lowest error among the learned models, and it is on par with the classical solution of this problem, gyroscope integration with a calibrated bias, although the bias is never given to it. Second, length extrapolation carries over from discrete tasks to continuous real data. Every model is within 1.4° at the training length, and beyond it only CPTP-SSM keeps tracking. This is what state invariance gives.

Presentation

  • Section 2.4. The quantum reading is not needed. The state set is the set in eq. 2, and the transition is the affine map whose matrix and offset are given in eq. 7. "Qubit head" and "qudit head" are only names for heads with state in D2\mathcal D_2 and Dd\mathcal D_d.
  • Definition 2. Yes, GG is a topological group and φ\varphi is an injective continuous homomorphism. No further restriction is needed. The definition now says so.
  • Definitions 3 to 5. Part 1 explains why they are needed, and Propositions 1 to 3 prove that our family meets them.
  • Proposition 1. PgP_g is the permutation matrix that sends ehe_h to eghe_{gh}. The map g↦Pgg\mapsto P_g is the regular representation, and a permutation matrix is unitary. The transition ρ↦PgρPg†\rho\mapsto P_g\rho P_g^\dagger sends the encoding of hh to that of ghgh.
  • Proposition 2. PU(d)=U(d)/{eiθI}PU(d)=U(d)/\{e^{i\theta}I\} is the group of maps ρ↦UρU†\rho\mapsto U\rho U^\dagger with UU unitary. For d=2d=2 it is SO(3)SO(3) acting on the ball. The proposition now names it. We hope this explains why the problem and the model matter, and we would be grateful if you would reconsider the rating.
Reviewer_iZi7OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Proposes CPTP-based selective SSMs with density-matrix states, affine parallel scans, and theoretical guarantees for group tracking, dissipation, and stability.

Strength

  • CPTP transitions guarantee state invariance and trace-distance contraction; (|A_t|_2\leq1) for qubits.
  • Unified framework for reversible rotations, irreversible merges, and uncertainty-induced contraction.

Weakness

  • Limited novelty: CPTP recurrence and Kraus normalization already explored in DM-RNN; main novelty is scan integration and group-tracking analysis.
  • Composition closure: (\mathcal C_{d,r}) is not generally composition-closed for (r<d^2), conflicting with Section 3.1's assumption.
  • State minimality motivates the Bloch ball, not complete positivity; no comparison with other invariant affine dynamics.

Question Q1 What is fundamentally new beyond DM-RNN's Kraus normalization and existing scan-based SSMs?

Q2 How is composition closure justified for bounded Kraus rank?

Q3 Can the affine Kraus generator approximate continuous group actions with guarantees?

Q4 How does CPTP compare with parameter-matched, Bloch-ball-preserving affine models?

Q5 How frequently does eigenvalue flooring activate, and what are the trace-preservation errors?

Q6 Does state-distribution overlap persist at lengths approaching 327,680?

Q7 What is the posterior-reference approximation error, and how are state-dependent noise effects handled?

Q8 Can parameter-matched comparisons and independent CSP-Bench test evaluations be provided?

Q9 What are the end-to-end throughput, memory, and inference costs versus Mamba-3 and DeltaProduct?

Critical strengths
  • CPTP transitions guarantee state invariance and trace-distance contraction; (|A_t|_2\leq1) for qubits.
  • Unified framework for reversible rotations, irreversible merges, and uncertainty-induced contraction.
Critical weaknesses
  • Limited novelty: CPTP recurrence and Kraus normalization already explored in DM-RNN; main novelty is scan integration and group-tracking analysis.
  • Composition closure: (\mathcal C_{d,r}) is not generally composition-closed for (r<d^2), conflicting with Section 3.1's assumption.
  • State minimality motivates the Bloch ball, not complete positivity; no comparison with other invariant affine dynamics.
Questions for authors

Q1 What is fundamentally new beyond DM-RNN's Kraus normalization and existing scan-based SSMs?

Q2 How is composition closure justified for bounded Kraus rank?

Q3 Can the affine Kraus generator approximate continuous group actions with guarantees?

Q4 How does CPTP compare with parameter-matched, Bloch-ball-preserving affine models?

Q5 How frequently does eigenvalue flooring activate, and what are the trace-preservation errors?

Q6 Does state-distribution overlap persist at lengths approaching 327,680?

Q7 What is the posterior-reference approximation error, and how are state-dependent noise effects handled?

Q8 Can parameter-matched comparisons and independent CSP-Bench test evaluations be provided?

Q9 What are the end-to-end throughput, memory, and inference costs versus Mamba-3 and DeltaProduct?

4 comments
AuthorsOpenReview ↗

Response to Reviewer iZi7, part 1 of 2. Novelty, composition closure and complete positivity

Thank you for a review that went through the appendix. Section, table and appendix numbers below refer to the revised PDF.

Q1. What is new beyond DM-RNN and existing scan-based SSMs

We do not claim the CPTP recurrence or the normalization K=ZS−1/2K=ZS^{-1/2}. Channel-valued recurrences go back to hidden quantum Markov models, and the normalization is the standard polar factor. The DM-RNN of Seo and Montiel 2026 is a sequential cell presented as a theoretical architecture, without a scan, an analysis of what it tracks, or an experiment. Our contributions, now listed at the end of the introduction, are these.

  • We extend the two roles of a transition in finite-state tracking to continuous states with uncertain inputs and formulate them as requirements that any model can be checked against.
  • We introduce state invariance, a stability criterion stronger than a bound on the transition norm, under which the state never leaves a fixed bounded set.
  • Rather than enlarging the transition, we constrain the state space and admit only the transitions that map it into itself, computed by a parallel associative scan.
  • We establish expressivity and stability guarantees, and show that the qudit dimension governs the representable groups while the Kraus rank governs the capacity for dissipation.
  • CPTP-SSM shows strong length extrapolation on group word problems, tracks noisy SO(3)SO(3) dynamics and orientation from a real gyroscope, and is competitive on long time-series classification and sensor prediction.

The scan and the group-tracking analysis that you identify are the third and fourth. The first two are in neither DM-RNN nor the existing scan-based SSMs. A norm bound controls the linear part of a transition but not its offset, and the existing structures that rotate and mix have no invariant set, as Table 1 and App. A.7 show. The new experiment on real inertial sensors in Section 4.2 shows what this buys. Every model is accurate at its training length of 2 s, and over whole drone flights of 98 to 150 s only CPTP-SSM keeps tracking, at 6.35° against 33° to 110° for the LSTM, PD-SSM, Gated DeltaProduct and Mamba-3. It is on par with the classical solution of this problem, gyroscope integration with a calibrated bias at 5.75°, although the bias is never given to it.

Q2. Composition closure for bounded Kraus rank

You are right. Cd,r\mathcal C_{d,r} is not closed under composition for 1<r<d21<r<d^2, and we used the symbol F\mathcal F for it. The composition-closed family of Section 3.1 is Cd\mathcal C_d, all CPTP maps, and Cd,r\mathcal C_{d,r} is the set a single token can select. No result uses closure of Cd,r\mathcal C_{d,r}.

  • The homomorphisms of Definitions 1 and 2 land in the unitary maps, the case r=1r=1, which form a group.
  • Proposition 2 claims closure under mixing only at r=d2r=d^2, where Cd,r=Cd\mathcal C_{d,r}=\mathcal C_d.
  • Definitions 3 and 5 and Proposition 3 concern single transitions, and bounds for products follow step by step.
  • The scan composes affine pairs on Rd2−1\mathbb R^{d^2-1} and never re-expresses a product with rr Kraus operators.

Section 3.3 now states this distinction.

Q4. The ball versus complete positivity, and ball-preserving affine models

We agree that Lemma 1 selects the ball and not complete positivity, and Proposition 3 holds for every positive trace-preserving map, PTP for short. Complete positivity gives the parameterization. The Kraus form maps unconstrained parameters onto the whole family in every dimension dd, while no such parameterization of positive maps is known for d≥3d\ge3, and we need d>2d>2 by Proposition 6.

For the comparison you ask for, we replaced our head on A5A_5 by two alternatives on the same three-dimensional state, with 32 heads and the same protocol, trained on lengths 3 to 40. The PTP head uses ball-preserving affine maps with ∥A∥2+∥b∥2≤1\|A\|_2+\|b\|_2\le1, and the TP head uses unconstrained affine maps. Table 16 in App. C.3 reports the accuracy, mean ± SD over three seeds.

TransitionsParametersL=128L=128L=1,024L=1{,}024L=5,120L=5{,}120L=40,960L=40{,}960
TP94knon-finitenon-finitenon-finitenon-finite
PTP102k0.584 ± 0.2970.372 ± 0.4400.325 ± 0.4230.181 ± 0.232
CPTP177k1.000 ± 0.0001.000 ± 0.0001.000 ± 0.0000.990 ± 0.011

The CPTP head at r=2r=2, with 111k parameters, also reaches 1.000 at length 5,120 on every seed. Section 3.1 now refers to this comparison.

AuthorsOpenReview ↗

Response to Reviewer iZi7, part 2 of 2. Questions Q3 and Q5 to Q9

Each of the remaining questions was a useful check, and we ran all of them.

Q3. Affine Kraus generator and continuous group actions

For rotations given as matrices, as in our SO(3)SO(3) experiments, one affine input projection realizes every rotation exactly at r≥2r\ge2. This is the new Proposition 5 in App. A.3. If a learned generator is within ε\varepsilon of the intended rotation at each step, the state after TT steps is within TεT\varepsilon, because every transition is non-expansive by Proposition 3.

Q5. Eigenvalue flooring and trace-preservation error

The floor was not active on any token of any model we audited. We instrumented the saved models on their evaluation inputs, namely the group tasks, the automata, SO(3)SO(3), and r=2r=2 models of CSP-Bench and UEA, with up to 2.9×1092.9\times10^{9} readings per group of models. The largest value of ∥∑ℓKℓ†Kℓ−I∥F\|\sum_\ell K_\ell^\dagger K_\ell-I\|_F is 3.4×10−53.4\times10^{-5} on the group tasks, 9.3×10−59.3\times10^{-5} on the automata, 1.8×10−41.8\times10^{-4} on SO(3)SO(3), 7.4×10−47.4\times10^{-4} on CSP-Bench and 1.6×10−31.6\times10^{-3} on UEA. If the floor did act, the map would be trace non-increasing and the state would stay in the ball, as App. B.2 explains.

Q6. State overlap near length 327,680

It persists. For the model of Fig. 2b, 95.0% of the states at length 327,680 lie inside the region that contains 95% of the training states in the projection of the figure, measured on 200 sequences. The states contract slowly, with a median norm of 0.89 of its training value, which goes with the decline in accuracy at the longest lengths noted in the Limitations.

Q7. Posterior reference and state-dependent noise

For Haar increments the reference is the closed form of eq. 25, with no approximation. For small increments it is a Gauss–Hermite quadrature of order 8. Order 12 changes it by at most 1.4×10−81.4\times10^{-8} per step, and a simulation of 10610^6 samples agrees within sampling error. State-dependent noise is outside the exactness statement. Eq. 16 requires increments independent of the past, and otherwise no linear recurrence, ours included, propagates the conditional mean exactly in one layer.

Q8. Parameter-matched comparisons and CSP-Bench test evaluation

Table 13 in App. C.2 lists the parameters behind every CSP-Bench cell. The models are swept over grids of the same shape, and at r=2r=2 ours is smaller than Gated DeltaProduct on all six tasks and than Mamba-3 on two. CSP-Bench has no test split. To keep the comparison with its baselines fair, we follow its protocol faithfully and select and report on the validation split for every row.

Q9. Throughput, memory and inference cost

Table 9 in App. B.4 compares one layer of width 1,024 at about 6.7M parameters, with batch 4, one A100 80GB and three seeds.

LayerParametersForward + backward in ms at LL = 512 / 2,048 / 16,384Forward in ms at LL = 16,384Peak memory in GiB at LL = 16,384
Mamba-36.69M4.0 / 6.6 / 48.616.85.7
Gated DeltaProduct, nh=4n_h=46.72M7.6 / 7.7 / 41.610.96.5
PD-SSM6.69M2,256 / 34,767 / OOMOOMOOM
CPTP-SSM, 187 qubit heads, r=4r=46.71M2.6 / 9.4 / 75.026.18.5

At L=16,384L=16{,}384 our layer takes 1.5 times the training time of Mamba-3 and 1.8 times that of Gated DeltaProduct, with 1.5 and 1.3 times the memory. At L=512L=512 it is the fastest. Section 3.2 now states the complexity.

We hope these answers resolve the concerns on novelty and soundness, and we would be grateful if you would reconsider the rating.

I thank the authors for their detailed rebuttal and revisions. The clarification of Kraus-rank composition closure, the constructive SO(3) realizability result (Proposition 5), and the additional numerical stability and runtime analyses adequately address several of my technical concerns. The ball-preserving affine ablations and real-world gyroscope experiments further strengthen the empirical evaluation, particularly regarding length extrapolation.

My remaining concern is the attribution of these gains specifically to complete positivity, as the PTP comparison does not fully disentangle representational constraints from parameterization and optimization effects. Additionally, the methodological novelty beyond existing CPTP recurrences remains somewhat incremental, with non-negligible computational overhead.

Nevertheless, the rebuttal has substantially improved the paper's technical justification and empirical support. I therefore revise my rating from 1 to 2.

We appreciate the updated assessment. We would like to clarify the three remaining concerns regarding complete positivity, novelty, and computational overhead. We believe the answers below resolve each of them.

Complete positivity. You are right that Table 16 compares two parameterizations at d=2d=2 and cannot isolate the constraint itself. The decisive reason for complete positivity is dimension, which our first answer mentioned only in passing. At d=2d=2 the state set is a ball, so positivity can be enforced geometrically and a PTP head can be built. For d≥3d\geq3 the state set is no longer a ball. Although particular families of positive maps can be constructed, general positivity is a block-positivity constraint, for which no comparably simple parameterization is known. Complete positivity instead reduces to a positive-semidefinite condition on the Choi matrix, and the normalized Kraus form satisfies it by construction in every dimension. This matters in our experiments and not only in principle. By Proposition 6, no number of qubit heads can track S5S_5, and the S5S_5 results in Table 2 use heads with d=4d=4. Thus, the role of complete positivity is not to explain the accuracy gap in Table 16, but to provide a tractable construction of the higher-dimensional transitions needed for our tracking tasks. We will revise the sentence in Section 3.1 that cites this ablation accordingly.

Novelty. Our starting point was the state-tracking problem, not the recurrence. We first extend state-tracking from finite automata to continuous states with uncertain inputs, where permutation becomes rotation and merging becomes mixing. We then study the mathematical requirements of this extension. Lemma 1 establishes geometric constraints on bounded convex state spaces that admit rotations. We also formulate state invariance as a stability requirement for SSMs, since bounding the transition norm alone does not prevent affine offsets from accumulating. State invariance also helps explain the length extrapolation that you highlight, as it keeps the state in a fixed bounded set at every sequence length. These requirements lead us to CPTP maps, which admit both rotations and mixing while preserving a bounded state space. We build CPTP-SSM from this structure, establish its expressivity and state-invariance guarantees, and implement it through a parallel scan. We do not claim novelty for CPTP recurrences or Kraus normalization themselves. Our contribution lies in the extended tracking problem, its mathematical analysis, and the resulting SSM design.

Computational overhead. We agree that the overhead is real and comes from the structure. A qubit head composes 3×33\times3 blocks where a diagonal layer multiplies scalars. The parallel scan retains the same asymptotic scaling in sequence length, but with a larger constant factor. Table 9 quantifies this trade-off. Our implementation has not yet benefited from the kernel optimizations available to established SSM architectures, but we recognize the current overhead as a limitation rather than assuming that future optimizations will eliminate it. We will make this trade-off and its structural source explicit in the final paper, while leaving dedicated kernel optimization to future work.

We hope these clarifications address the remaining concerns and help establish that they do not undermine the paper's main contributions. We would be grateful if you would reconsider the rating once more.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for careful and constructive reviews. We have uploaded a revised PDF with all changes in blue. Detailed answers are under each review.

What the paper is about

The reviews showed us that the paper was hard to follow without background in this area. The revised introduction now states the motivation and ends with an explicit list of contributions. We summarize both here.

We intended the paper as a theoretical contribution in the line of recent work on negative eigenvalues, DeltaProduct and PD-SSM, which studies what the state of an SSM can track. Each introduced a new transition structure and characterized, in theory and in experiments, what the state can then track. However, that line of work was aimed at language modeling, so its theory covers finite automata and finite groups.

SSMs are now also applied to physical systems, where the state is continuous and the measurements are noisy, such as an orientation measured by a gyroscope. We refer to this as state tracking on continuous groups under uncertainty. Some work, such as HiSS on CSP-Bench, has shown experimentally that SSMs can work on physical systems, but what the state and the transition need in order to track such states has been little studied in theory. We believe this must be understood before new families of SSMs can be designed for real-world data.

To find what tracking them requires, we start from a concrete case, the rotations SO(3)SO(3). Tracking a continuous rotation under uncertainty places three requirements on the transition. It must rotate the state, average over uncertain rotations, and stay in a bounded set at every length. CPTP maps meet all three, and no existing transition structure does, as Table 1 shows. The idea is that, instead of enlarging the transition, we fix the state set first and admit only the maps that preserve it. The layer runs in the same parallel scan as other SSMs.

The experiments of the submission serve three purposes.

  1. Extended state tracking and length extrapolation, on A5A_5, S5S_5 and noisy SO(3)SO(3).
  2. Finite automata, still solved at 99.9% on average, so CPTP-SSM extends what SSMs can track and does not trade the earlier ability for a new one.
  3. Real data, with classification on UEA and sensor prediction on CSP-Bench.

New experiment on real sensors

Beyond the theoretical contribution, the reviews asked how this new structure is useful in practice. We have added an orientation tracking experiment on real gyroscope data from EuRoC and TUM-VI, two benchmarks widely used in robotics, with the split and the open-loop protocol of Brossard et al. 2020.

The key result is that CPTP-SSM is trained only on windows of at most 2 s, yet it tracks whole test sequences of up to 150 s without reset. Its error is far below that of every other learned model and comparable to gyroscope integration with a calibrated bias, which CPTP-SSM is never given. The table gives the orientation error in degrees over three seeds.

ModelEuRoCTUM-VI
LSTM33.34 ± 8.3910.37 ± 0.36
PD-SSM65.99 ± 10.72104.09 ± 10.87
Gated DeltaProduct61.50 ± 4.9899.55 ± 6.19
Mamba-3109.82 ± 6.49124.46 ± 4.60
CPTP-SSM, r=4r=46.35 ± 0.241.98 ± 0.27
Gyro integration, calibrated bias5.751.76

Every model is within 1.4° at the training length of 2 s, so all of them learn the task, but CPTP-SSM is the only model we tested that keeps tracking beyond the training length. We believe this benchmark shows how CPTP-SSM extends the capabilities of SSMs to continuous groups under uncertainty, with strong length extrapolation.

Other main changes

ConcernChangeWhere
Composition closure, iZi7Corrected. The set a single token can select is not closed for 1<r<d21<r<d^2, and compositions are taken in the set of all CPTP maps. No result depended on closureSec. 2.1, 3.1, 3.3
Complete positivity, iZi7TP, PTP and CPTP heads compared on A5A_5Sec. 3.1, App. C.3
Cost, z33K and iZi7Complexity, and time and memory against Mamba-3, Gated DeltaProduct and PD-SSMSec. 3.2, App. B.4
Baselines, z33KFlash PD-SSM in Tables 4 and 5Sec. 4
Realization theory and definitions, uMcVClassical results cited, and Def. 2 and Prop. 2 made preciseSec. 2.2, 3

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.