acceptodds
Under review as a conference paper at ICLR 2027

Can We Find the Optimal Variable Ordering for Mamba in Large-Scale Systems?

Abstract

When Mamba reads variable histories one by one, which variables should it read together? Large physical systems contain many pieces of equipment, each described by different signals. A tank's pressure, level, and flow, for example, measure different aspects of the same process. We hypothesize that keeping such heterogeneous signals together can improve prediction. We test a policy that groups signals by component while randomizing the sequence of groups. In a simulated nuclear power system, this policy gives lower mean test error than unrestricted random orders and a history-similarity order derived from GPS-Mamba. We then investigate the learned computation in a diagnostic comparison of a fixed component order with random orders. Removing an estimated output interaction between a target history and the other histories reverses the fixed order's advantage on shared diagnostic windows. An internal control, however, does not support attributing that advantage to the immediately preceding variable's write. Together, these experiments connect variable organization to prediction and distinguish joint use of histories from a simple local-write explanation.

Primary area:
learning on time series and dynamical systems
Keywords:
Mamba, variable ordering, multivariate time-series forecasting, physical systems, state-space models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_N953OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how the order in which Mamba reads variable histories affects multivariate forecasting. The authors hypothesize that keeping the heterogeneous signals of the same physical component together improves prediction and propose a component-grouped ordering policy Across 100 matched seeds, the grouped policy yields lower test MSE than random orders.

Critical strengths

The paper studies how component-grouped variable orders influence Mamba-3 and evaluates whether they improve prediction on a nuclear power system.

The experiments on the nuclear system evaluate the proposed ordering policy under well-controlled conditions.

Critical weaknesses

The paper’s technical proposal is a single ordering variable by documented physical components, which is applied to a fixed Mamba-3 architecture.

Evaluation of a single simulated system. All results come from one simulated nuclear plant dataset. It is unclear whether the conclusion generalizes to other systems, and the title’s claim about “large-scale systems” is not supported.

Presentation. The figures are somewhat hard to interpret: Figures 2 and 4 pack many symbols and annotations into small panels.

Questions for authors

Several abbreviations are used without definition, e.g., GPS (in GPS-Mamba), NPPAD and SISO. Please define them at first appreances.

The Mamba-3 architecture is used without modification, so the contribution is limited to the input order. Does the benefit of component grouping also hold for other variable-token architectures (e.g., S-Mamba or Mamba-2)?

2 comments
AuthorsOpenReview ↗

[W1] Scope of the contribution

Scope of the contribution

The input order is a simple initial setting that can be changed without modifying the architecture. We did not stop at this result: we questioned where it comes from and tried to explain it with ablations. In the initial version that explanation was wrong, and Reviewer JC4u pointed it out, so we corrected it. Of course, the cause of the ordering advantage is not 'fully' resolved. Still, by presenting our hypotheses, results, and many experiments, we hope the question will eventually be resolved, and we would like this work to give the community that wants to use Mamba-type models on such systems something that "can bring a gain, while the change itself is easy."

Added systems and "large-scale"

We added four systems, and to support "large-scale", we included two small systems as a contrast group (table in the general response; Section 2.1, Table 1, Appendix A). Grouping lowers the error in the three systems with many variables and signal types, with both cores; in the two 17-variable systems, seven of the eight comparisons are not statistically significant. How the effect changes at sizes in between was not examined, and the limitations say so.

Mamba-2 and other architectures

We ran additional experiments with Mamba-2. We exchanged only the core of the same bidirectional predictor, kept the sizes and training budget the same (Table A3), and trained all five systems with the same 100 seeds per policy. As with Mamba-3, grouping helps in the three larger systems and not in the two small ones.

System (Mamba-2)vs Random (%)Holm pvs Similarity (%)Holm p
Nuclear plant−4.92×10⁻⁶−4.77×10⁻⁶
Simulated chiller plant−4.60.034−13.09×10⁻¹¹
Industrial testbed−1.68×10⁻³³−0.20.047
Multiphase flow rig−0.91.000−1.31.000
Building chiller plant−0.40.467+2.04×10⁻⁴

The pattern is the same as with Mamba-3, and the reductions are larger. Mamba-2 is more sensitive to order in four of the five systems (Appendix C.3, Figure C1), which is consistent with its short convolution that mixes each token with the three before it (Figure 2(b)). The output removal test on the nuclear test records gives the same result with Mamba-2 (Figure 4(c), Table D3). In addition(We would not say our contribution but still would like to report..), Mamba-3 has the lower error in all fifteen system-policy combinations (Table B2).

AuthorsOpenReview ↗

[W3] Presentation, Figure, Abbreviations

Figures

Thank you for your contribution and sorry about it. We agree that Figures 2 and 4 packed too many symbols into small panels. We redrew both figures and added a new Figure 5 for the test in Section 5.2.

+Section 3.2 now explains the state update of the Mamba-2 and Mamba-3 item by item against Figure 2.

Abbreviations

We used the names given in the original papers and data sets and thought their names were not that important(especially NPPAD..), but the first explanation for readers unfamiliar with them was missing. Each is now explained where it first appears, with its source.

AbbreviationMeaning
GPS-Mambagraph permutation scanning Mamba (Yao et al., 2026)
NPPADNuclear Power Plant Accident Data (Qi et al., 2022)
SISOsingle-input, single-output form of Mamba-3

We do appreciate your time!

Reviewer_JC4uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates whether ordering multivariate time-series tokens by physical component membership improves Mamba-3 forecasting on the 96-channel NPPAD nuclear power plant simulation dataset. Using a shared bidirectional Mamba-3 architecture where each token represents one variable's 24-step history, the authors compare a Grouped policy (fixing within-component order across 18 equipment blocks while randomizing block order per seed and placing 8 constant channels at the tail) against unrestricted Random permutations and a GPS-derived Laplace Scanning order. The authors then probe a separate Fixed grouped cohort on validation windows via reference-based output-interaction removal and an internal preceding-token write intervention.

Critical strengths
  • Matched-seed experimental control and self-contained verification artifacts
  • Thoughtful negative control for internal state-write attribution
Critical weaknesses
  • Headline gap reversal in Section 5.1 rests on 2-draw reference disagreement VV, not the pre-specified BB. When μ1=μ2\mu_1 = \mu_2 in Eq. (D3), the pre-specified statistic B=⟨r,J1+J2⟩+⟨J1,J2⟩B = \langle r, J_1+J_2\rangle + \langle J_1, J_2\rangle is unbiased for interaction removal cost, while Dˉ=B+V\bar{D} = B + V adds 2-sample variance V=14∥J1−J2∥2V = \frac{1}{4}\|J_1-J_2\|^2. In (Table D1), ΔB=1.375×10−3<g=1.514×10−3\Delta B = 1.375 \times 10^{-3} < g = 1.514 \times 10^{-3}, leaving g−ΔB=+0.138×10−3>0g - \Delta B = +0.138 \times 10^{-3} > 0 (no reversal). The claimed reversal to −1.166×10−3-1.166 \times 10^{-3} (Figure 4c) comes from 2-sample noise ΔV=1.305×10−3\Delta V = 1.305 \times 10^{-3} on checkpoint-selection validation windows. Sec. 5.1 concedes VV "contributes", yet the abstract still claims reversal.
Questions for authors
  • Since BB in Eq. (D3) is the unbiased estimator of interaction removal cost when μ1=μ2\mu_1=\mu_2 while V=14∥J1−J2∥2V = \frac{1}{4}\|J_1-J_2\|^2 is 2-sample Monte Carlo variance, and gbefore−ΔB=+0.138×10−3>0g_{\text{before}} - \Delta B = +0.138 \times 10^{-3} > 0, how can you claim interaction removal reverses Fixed grouped's advantage rather than reflecting higher 2-sample estimation noise ΔV=1.305×10−3\Delta V = 1.305 \times 10^{-3}?

  • list raw p-values for two unreported systems (IBAL and HAI). Why were IBAL, HAI, and the test-set MSE of the Fixed grouped cohort omitted from the paper?

5 comments
AuthorsOpenReview ↗

[W1, Q1] The reversal and the reference noise

We do thank Reviewer JC4u for the contribution and the time spent on this point.

The point is exactly right: the reported reversal came from noise in estimating the interaction. The numbers below are for the cohort you analysed, the Fixed grouped and Random models of the submission's Section 5.1. We revised the paper (Section 5.1; all steps in Appendix D.2–D.3), and below we go through, line by line, where the noise enters and how we correct it. Values are in 10−310^{-3} MSE; brackets are paired-seed 95% intervals.

Step 1. What is removed. The interaction between the target history xx and the other histories zz, its mean, and the increase in error when that mean is removed:

J=f(x,z)−f(x,Z′)−f(X′,z)+f(X′,Z′)J=f(x,z)-f(x,Z')-f(X',z)+f(X',Z')

I=E[J],C=MSE(f−I)−MSE(f)\mathcal I=\mathbb E[J],\qquad C=\mathrm{MSE}(f-\mathcal I)-\mathrm{MSE}(f)

Step 2. How the error changes when any output is subtracted. With the residual r=Y−fr=Y-f, subtracting any output aa gives

MSE(f−a)−MSE(f)=∥Y−f+a∥2−∥Y−f∥2\mathrm{MSE}(f-a)-\mathrm{MSE}(f)=\lVert Y-f+a\rVert^2-\lVert Y-f\rVert^2

=∥r+a∥2−∥r∥2=\lVert r+a\rVert^2-\lVert r\rVert^2

=2⟨r,a⟩+∥a∥2=2\langle r,a\rangle+\lVert a\rVert^2

so the target is C=2⟨r,I⟩+∥I∥2C=2\langle r,\mathcal I\rangle+\lVert\mathcal I\rVert^2.

Step 3. What we actually subtract. I\mathcal I is not known exactly, so we subtract I^=I+e\hat{\mathcal I}=\mathcal I+e, the average of KK draws from each of two reference sets. The error ee changes from draw to draw and has mean zero. Putting a=I+ea=\mathcal I+e into Step 2:

D^K=2⟨r,I+e⟩+∥I+e∥2\hat D_K=2\langle r,\mathcal I+e\rangle+\lVert\mathcal I+e\rVert^2

=(2⟨r,I⟩+∥I∥2)+2⟨r+I,e⟩+∥e∥2=\bigl(2\langle r,\mathcal I\rangle+\lVert\mathcal I\rVert^2\bigr)+2\langle r+\mathcal I,e\rangle+\lVert e\rVert^2

=C+2⟨r+I,e⟩+∥e∥2=C+2\langle r+\mathcal I,e\rangle+\lVert e\rVert^2

Step 4. Taking the mean. The middle term vanishes because E[e]=0\mathbb E[e]=0. The last term is a square, so it stays, and its mean is the variance of the estimate:

E[D^K]=C+N1K,N1=14(σ12+σ22)\boxed{\mathbb E[\hat D_K]=C+\frac{N_1}{K}},\qquad N_1=\tfrac14\bigl(\sigma_1^2+\sigma_2^2\bigr)

where σb2\sigma_b^2 is the variance of one draw from set bb. For the gap between the two models, g=MSER−MSEFg=\mathrm{MSE}_R-\mathrm{MSE}_F, and the gap after removal, g−=g−ΔCg^-=g-\Delta C (with Δ\Delta meaning Fixed grouped minus Random):

E[g−ΔD^K]=g−−ΔN1K\mathbb E\bigl[g-\Delta\hat D_K\bigr]=g^- -\frac{\Delta N_1}{K}

Step 5. Its size with one draw per set. The noise is larger for Fixed grouped: ΔN1=1.290\Delta N_1=1.290 [1.031, 1.549]. With K=1K=1 this predicts 0.174−1.290=−1.1160.174-1.290=-1.116; the measured value was −1.166 [−1.681, −0.652]. The reported reversal was this noise term.

Step 6. The correction. N1N_1 is a variance, so with two or more draws from the same set it can be estimated from their spread: the sample variance sb2s_b^2 has E[sb2]=σb2\mathbb E[s_b^2]=\sigma_b^2. Subtracting that amount,

C^=D^K−s12+s224K\hat C=\hat D_K-\frac{s_1^2+s_2^2}{4K}

E[C^]=C+N1K−σ12+σ224K=C\mathbb E[\hat C]=C+\frac{N_1}{K}-\frac{\sigma_1^2+\sigma_2^2}{4K}=\boxed{C}

(Eq. 6, Proposition 2). This needs no assumption of equal set means, on the distribution, on large KK, or on the model. With one draw (K=1K=1) the spread cannot be seen, and no statistic can be unbiased (Proposition 1).

Step 7. Relation to your BB. The pre-specified statistic BB you point to is unbiased when the two set means are equal. Here the difference between the sets is negligible, so ΔB=1.375\Delta B=1.375 (submitted Table D1) and the corrected ΔC=1.340\Delta C=1.340 nearly coincide. Your g−ΔB=+0.138g-\Delta B=+0.138 and our corrected g−=0.174g^-=0.174 [−0.188, 0.535] reach the same conclusion: no reversal.

Step 8. Without and with the correction, on the same draws (your cohort; the revised Table D2 and Figure 4(b) show the same analysis for Grouped):

Draws per setGap after removal, uncorrectedGap after removal, corrected
1−1.166 [−1.681, −0.652]-
2−0.483 [−0.909, −0.057]0.166 [−0.200, 0.532]
2−0.447 [−0.865, −0.029]0.196 [−0.163, 0.556]
2−0.476 [−0.896, −0.056]0.166 [−0.194, 0.527]
2−0.481 [−0.904, −0.058]0.164 [−0.201, 0.529]
4−0.139 [−0.526, 0.248]0.185 [−0.177, 0.547]
4−0.159 [−0.546, 0.228]0.162 [−0.200, 0.524]

The gap before removal is g=1.514g=1.514 [1.331, 1.696]. Without the correction, the reversal remains even with two draws per set, with intervals that exclude zero, and the values follow the curve 0.174−1.290K0.174-\frac{1.290}{K}. With the correction, the estimates at two and four draws agree; the mean of the two four-draw rows is 0.174 [−0.188, 0.535]. The four two-draw rows are independent estimates; each four-draw row pools two of them.

With Grouped, which the revised paper uses, the gap is 1.294 before removal, −0.529 after uncorrected removal with one draw, and 0.150 [−0.251, 0.552] after corrected removal (revised Table D1).

AuthorsOpenReview ↗

[W1] The reversal claim in the abstract, Validation windows that also selected the checkpoints

The reversal claim in the abstract

You are right on this point as well. With the correction above, removing the mean interaction raises the error of Fixed grouped more than that of Random, and the gap closes from 1.514 to 0.174 [−0.188, 0.535]. The remaining gap cannot be distinguished from zero, so we no longer argue a reversal effect. The abstract now reads "raises the error of the grouped-order models more and removes most of their advantage", and the introduction, Section 5.1, Figure 4(b), and the conclusion say the same.

Validation windows that also selected the checkpoints

You are also right that the validation windows were used to select the checkpoints. We repeated the corrected removal on the 12 test records of the nuclear plant, with Mamba-3 and with Mamba-2 (Figure 4(c), Appendix D.4, Table D3). What the paper states is that models trained with grouped orders rely more on the interaction than random-order models do, with both cores.

AuthorsOpenReview ↗

[Q2] IBAL, HAI, and the test MSE of Fixed grouped, not reported in the submission

Here, we may lose some brevity, sorry about it.

As you can see in the supplementary files of the submission, every result you asked about carries "Fixed", as in "Fixed grouped". Fixed means that, for example, with five components A, B, C, D, and E, we fixed one of the 5! possible block orders, such as B–C–A–D–E, and ran the experiment with that order only. HAI is the data of the industrial testbed in Table 1, and IBAL is the data of the building chiller plant in Table 1 (17 variables). At first we saw an advantage of Grouped with such fixed orders and wrote the paper on that basis.

However, if Grouped is a single fixed order, there is no way to answer the objection "this may not be the effect of grouping, but of that particular order being good by chance." We judged that the experimental design had been wrong. So that the advantage of grouping can be claimed honestly, we retrained with Grouped drawing a new block order for each training run. Because rerunning all 100-seed experiments was costly in time, we started with the nuclear datasets, and we removed the fixed-order forecasting comparisons (nuclear plant, HAI, IBAL) from the main report, since they carry no or little scientific meaning about grouping. In fact, on the IBAL building data, where the earlier fixed order had shown a small advantage, Grouped with shuffled block orders shows no significant difference from Random (building chiller plant in Table 1). Had we kept the fixed design, we might have confidently reported an advantage that came from one particular order.

In short, we experimented with grouping "held fixed", and the result looked good, but we judged that this alone was not enough to claim an advantage of grouping, so we ran the experiments again.

All forecasting comparisons of the revision (Table 1) use Grouped with drawn block orders(shuffled). The building chiller plant of Table 1 (17 variables in two groups) is a newly trained experiment and differs from the earlier IBAL cohort above. Industrial testbed above is HAI.

Reviewer_Qa9WOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper asks a very simple question: in a mamba model, what is the ideal variable order? The authors hypothesize that order matters quite a bit, and demonstrate that this is indeed the case. The domain is NPPAD simulation of a nuclear power system. The authors propose an ordering method, and compare it both with history-similarity ordering, and random ordering, showing superior performance. In their method, they “form one block for each documented component or process group and randomly arrange the blocks for each training run”. The results are positive but modest.

Critical strengths

The central idea is interesting, and the proper variable ordering has not been explored.

Each ordering is trained 100 times, so although the setting itself is very limited (see weaknesses), within that setting, the method is well evaluated.

Critical weaknesses

The entire paper is evaluated on a single setting. This makes it impossible to determine if the results are in any way robust. Many more settings are needed to determine if the results are significant or not. This significantly undermines the paper, especially since the improvments on this one setting were very minor to begin with.

More significantly, the paper itself has very little novelty. The method presented seems to work, but there is little theoretical motivation, and the gains themselves are small over the baseline (see table 1).

The paper also overclaims. The word “Optimal” is in the title, but no attempt at any form of optimality is made in the paper.

Questions for authors

Can you remove “Optimal” from the title? Can you evaluate on at least 4 more settings?

3 comments
AuthorsOpenReview ↗

[W1, Q2] Four more settings

You are right. We understand that one data set cannot support the claim or contribution well enough. At least in an ML paper, a claim should be checked both statistically (many seeds) and in data coverage (several data sets that fit the claim). How and why this happened is explained in detail in our reply to Reviewer JC4u (3/3).

In the revision, to make up for this, we added four systems whose (1) data are public and (2) components are documented, ran them at the same scale of 100 seeds, and also ran Mamba-2. The results are below; every comparison with its p-value is in the general response and in the revised paper.

SystemVariablesGroupsSignal typesChange in test MSE with GroupedSignificant after Holm adjustment
Nuclear plant961813−1.4% to −4.9%4 of 4
Simulated chiller plant73198−3.0% to −13.0%3 of 4
Industrial testbed661213−0.2% to −1.6%4 of 4
Multiphase flow rig1785−2.6% to +3.1%0 of 4
Building chiller plant1724−0.5% to +2.0%1 of 4 (in favor of Similarity)

Grouping lowers the error in all three systems with many variables and signal types, and in the two small systems, seven of the eight comparisons are not statistically significant.

From this, we would like to argue that grouping has the advantage in what may be a somewhat vague group, "large-scale systems", and that in small systems one may not need to care much about the arrangement of its input.

AuthorsOpenReview ↗

[W2] Novelty, theoretical motivation, and size of the gains

First, we agree that the gains in the current table may seem small. From an already low error, they are 1.4–4.9% in the nuclear plant, 0.2–1.6% in the industrial testbed, and 3.0–13.0% in the simulated chiller plant. We think, however, that these gains mean two things.

  1. They repeat. Over 100 paired seeds, all twelve reductions in these three systems have 95% intervals that exclude zero, and eleven remain significant after the Holm adjustment.
  2. They cost almost nothing. Nothing is added to the model; the same model only reads its variable tokens in a different order (the architecture and training settings are the same for all orders.). Grouping needs only the component each variable belongs to, which the variable descriptions and equipment documents of the data already contain.

On novelty, we see it a little differently: the size of a gain and the novelty of a study should be judged separately. Your review lists as a strength that "the proper variable ordering has not been explored". Existing methods learn or derive the order from data (MambaTS, GPS-Mamba) or make the model robust to the order (SOR-Mamba). We ask whether physical membership known before training, that is, which signals belong to the same equipment, is useful ordering information. We did not stop at the result: we formed hypotheses about where the difference comes from and tested them.

We accept that the theoretical motivation was thin. In the revision we state it as follows. In a Mamba that reads variable tokens, the order decides which histories enter the state one after another. Section 3.2 and Figure 2 show the two routes by which the order enters the two cores: state transport through the tokens in between, and a direct route for neighbors. The direct route is an extra write of the preceding token in Mamba-3, and a short convolution over each token and the three before it in Mamba-2.

We tested the direct route of Mamba-3 against a control matched in size, and it does not explain the advantage (T=−2.133×10−3T=-2.133\times10^{-3} MSE, 95% interval [−2.805, −1.462]×10−3\times10^{-3}; Section 5.2, Figure 5).

In the output test, models trained with grouped orders use the histories more jointly, also with Mamba-2, which has no preceding-token write. We could not fully identify the mechanism, but we separate the explanation that was supported from the candidate that was not, and name the way information from several sources accumulates in the state and is read as the next candidate.

AuthorsOpenReview ↗

[W3, Q1] On the title

To be honest, this was the hardest question to answer.. It is a simple question but we may be wordy, sorry about it.

The title is the question we asked when we started this work. We understand that a title phrased as a question obliges the author to answer it clearly, and that the word "optimal" is used with care in mathematics and machine learning, or only in narrowly defined settings. The title was accordingly the first thing that colleagues and an AI reviewing tool pointed out. The tool suggested a title such as "Component-Grouped Variable Orders for Mamba Forecasting of Physical Systems". Still, we were stubborn about keeping our first question as the title. We also tried words such as "Proper" in place of "Optimal". Honestly, even now we would like to keep this title and state in the conclusion that the question is not fully answered; the limitations of the revised paper now say so.

We do not deny, however, that such a title can look like click bait or overselling. The ICLR 2027 Author Guidelines state that no changes to the title are permitted after the submission deadline, so we will revise it once the meta-review is out, at the stage where a change is permitted. We think we were being stubborn.. Sorry and we appreciate it.

Again, thank you for your time!

Comments on the paper

4 comments
AuthorsOpenReview ↗

General response to all reviewers

Before our answers, we thank all reviewers for time spent reviewing our paper. Following the guidance, we tried to answer as briefly as possible. So as not to waste the reviewers' time on an AI-written, unreviewed rebuttal, we responded with our own careful review and additional experiments, and all answers were written by our own hands (except for grammar correction, the formatting of tables, and the typesetting of equations, for which we used AI tools).

Our paper asks two questions:

  1. When Mamba reads variable histories one by one, does reading the variables of the same component next to one another improve prediction? (Specifically Large-Scale)
  2. If so, what causes the improvement?

To answer them, we compare Grouped, which keeps the variables of each component together, with Random, which shuffles all variables, and with Similarity, the history-similarity order of graph permutation scanning Mamba (GPS-Mamba, from prior work).

The changes we would like to share with all reviewers are as follows.

1. Four more systems and additional experiments with Mamba-2

For four systems whose data and component structures are published together, we trained Grouped, Random, and Similarity with 100 seeds each, and we ran all five systems with Mamba-2 as well as Mamba-3.

Here, Grouped draws a new block order for each training run (Grouped Shuffling, why this matters is explained in detail in our reply to Reviewer JC4u.)

Two of the four addtional systems have fewer variables than the nuclear plant but still many groups and signal types; the other two have far fewer variables and signal types. (Bold: Holm-p < 0.05)

SystemVariablesGroupsSignal typesCorevs Random (%)Holm pvs Similarity (%)Holm p
Nuclear plant961813Mamba-2−4.92×10⁻⁶−4.77×10⁻⁶
Mamba-3−2.22×10⁻⁵−1.40.001
Simulated chiller plant73198Mamba-2−4.60.034−13.09×10⁻¹¹
Mamba-3−3.00.079−5.55×10⁻⁴
Industrial testbed661213Mamba-2−1.68×10⁻³³−0.20.047
Mamba-3−0.46×10⁻¹⁷−0.36×10⁻¹¹
Multiphase flow rig1785Mamba-2−0.91.000−1.31.000
Mamba-3+3.10.198−2.60.198
Building chiller plant1724Mamba-2−0.40.467+2.04×10⁻⁴
Mamba-3−0.50.384−0.20.649

First, in the three systems with many variables and signal types, Grouped has the lower error in all twelve comparisons, and all but one remain significant after the Holm adjustment. In that one case the Holm-adjusted p is 0.079.

Second, in the two small systems, seven of the eight comparisons are not statistically significant. We find this fairly intuitive: in a system with few variables and few signal types, there may be little reason for such an arrangement.

We would also like to mention that we tried to collect more public systems as diverse as the nuclear plant in variables, components, and signal types, but data of similar scale and purpose are scarce.

2. Correction of the "reversal" in the component-removal study

We are especially grateful to Reviewer JC4u for pointing this out. The details are in our reply to Reviewer JC4u. In short, the "reversal" we reported in the submission came from noise in the estimate of the interaction. With the noise corrected, removing the interaction removes most of the advantage but does not reverse it. In the revised paper, this analysis is run on Grouped itself (a new block order for each run), on the validation windows, and on the test records with both cores. We revised the paper throughout, and it now includes the uncorrected estimate, the corrected result, and the derivation of the correction. We sincerely thank the reviewer for this contribution.

AuthorsOpenReview ↗

Summary of last updates

Dear AC and reviewers,

Thank you for the reviews and the discussion; they have taken the paper well beyond the submitted version.

Our question was which variables a Mamba forecaster should read together when it reads the variables of a physical system one after another. A tank's pressure, level and flow look different as time series, yet they describe the same piece of equipment. Our answer is to keep the variables of each component next to each other. The components are known from the design of the system, so the gain comes at no cost: the model and its training stay exactly the same.

During the discussion, we tested this answer as thoroughly as we could:

What we testedWhat we found
Three large systems, with Mamba-3, Mamba-2 and S-MambaBetter than random orders in all nine comparisons
The same systems, against the order of GPS-Mamba (prior work)Better in all six comparisons
Two small systems, with both Mamba coresNever significantly worse than random orders
More and more variables kept in their componentsThe error falls with every step
Components replaced by arbitrary groups of the same sizesEvery arbitrary grouping is worse
Retraining with routes of the scan cut, two large systemsOwn component as the only source: the advantage grows. Exactly that route removed: it disappears

The last row convinces us most: we changed only what a variable may receive, wrote down both predictions before training, and they held in both systems. The gain comes from the information a variable receives from the other variables of its own component.

Every request in the three reviews has been carried out. The next comment lists them, as Reviewers Qa9W and N953 have not commented since the revision. The last comment clears up what we believe are misunderstandings behind Reviewer JC4u's four reasons for keeping the score.

We hope the paper helps anyone who forecasts physical systems with Mamba, and we would be grateful if the AC and the reviewers looked carefully at the experiments and at how each point was answered.

AuthorsOpenReview ↗

Each review point and what we did

ReviewerPointWhat we did
Qa9WA single setting; "evaluate on at least 4 more settings"Five systems and three models, 100 paired seeds per comparison
Qa9W"Optimal" in the titleIt will become "Proper" once the author guidelines allow a title change
Qa9WLittle theoretical motivationThe paper shows where the order enters each core, and the cause is now tested by changing the architecture
Qa9WLittle novelty, small gainsEarlier ordering methods use the data alone; which variables belong together is known before training. The gain holds in all nine large-system comparisons
N953One simulated system; "large-scale" not supportedThree large systems, with two small systems for contrast
N953Other variable-token architectures, e.g. S-Mamba or Mamba-2Both tested; the gain holds with both in all three large systems
N953Figures 2 and 4; undefined abbreviationsFigures redrawn; abbreviations defined at first use
JC4uThe "reversal" of Section 5.1Traced to a noise term in the estimate and corrected, with a derivation and a proof
JC4uValidation windows also used to select checkpointsThe corrected test repeated on the test records with both cores
JC4uIBAL, HAI and the fixed grouped cohortRetrained with orders drawn anew for every seed, and reported with the other systems

The results are in the revised paper and the supplementary material.

AuthorsOpenReview ↗

Regarding reviewer JC4u's last review.

We thank Reviewer JC4u for the close reading and for calling our response "thorough, transparent, and mathematically rigorous". We believe the four reasons for keeping the score rest partly on misunderstandings, which we would like to clear up.

  1. "a straightforward heuristic with modest-to-negligible gains on large systems". The method is deliberately simple: it needs only the component list and changes nothing in the model. But we did not stop there. To find out why it works, we retrained the models with routes of the scan cut, with order-dependent parts switched off, and with the memory length set by hand. We also retrained with partly grouped orders, and with arbitrary groups of the same sizes to separate the component information from the shape of the order. As for the size of the gain, what matters for a method that costs nothing is that it is reliable: it repeats in every large system with every model, and it grows step by step as more variables are grouped.

  2. "significant degradation on at least one smaller system". This refers to one cell, the building chiller plant with Mamba-2, where the component order is worse than Similarity, a single fixed order from prior work. A single fixed order is a noisy reference: in that cell, ten fixed orders differ from each other by up to 7.7%, and Similarity lies within that range. The fair control is Random. Against Random, that cell shows no significant difference, and neither small system is significantly worse with either core. Against its control, grouping degrades no system.

  3. "an unresolved core mechanism". We wrote this limitation into the revision ourselves: fully explaining why a trained network prefers one input order is hard, and we did not want to claim more than we had shown. But we did not leave the result as an unexplained heuristic; we formed hypotheses and tested them by changing the architecture. If the gain came from anywhere other than a variable's own component, it would survive when that route is cut. It does not, in the industrial testbed or in the nuclear plant, as we predicted before training, and it grows when that route is the only one left. The switch-off and memory-length retrainings point in the same direction.

  4. "a title/framing that still overpromises". The reviews point to two words in the title: "Optimal" (Reviewer Qa9W) will become "Proper", and "large-scale" (Reviewer N953) is now backed by three large systems. Beyond the title, the paper claims only what its tables show; the abstract itself limits the claim to systems with many variables and signal types and says that small systems show almost no significant difference.

The follow-up also mentions "overclaimed findings", referring to the "reversal" of Section 5.1. That claim came from an error in one diagnostic estimate. The reviewer's point led us to correct it with a proof, and the correction made the contribution clearer: models trained with grouped orders rely more on the joint use of the histories, and the route cuts show where this comes from. The paper's main claim rests on five systems and three models, with its cause tested inside the architecture. We are grateful to the reviewer, whose questions made this possible.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.