acceptodds
Under review as a conference paper at ICLR 2027

PALM: Physically-Anchored Latent-timescale Mamba for Stable Representations under Distribution Shift

Abstract

Scientific foundation models are built to reuse representations across datasets and acquisition protocols, yet their representations degrade under distribution shift. We trace one source of this failure to selective state-space models (SSMs). Their memory depends on the step ∆ and the state matrix A only through their product, so data do not identify the physical timescale of memory, and under a shift the resulting memory no longer matches the signal’s timescales. We call this failure memory-timescale misalignment. To resolve it, the split between ∆ and A must be tied to physical units. We therefore propose Physically-Anchored Latent-timescale Mamba (PALM), which disentangles A from ∆ by anchoring both to acquisition units: ∆ through spectrum-estimated knee frequencies and A through a physical memory range. We evaluate PALM against parameter-matched baselines in two domains, under temporal-context truncation, low-resource transfer, and brain-state shift on fMRI and under held-out-station forecasting on Weather-5K. Under fMRI context truncation with run-level spectral calibration, PALM keeps its latent timescales in physical units and retains 11% higher representation similarity than Mamba2. At geographically isolated weather stations never seen during training, PALM achieves the lowest error on every variable and horizon, 15.2% below the best baseline on average, and outperforms zero-shot forecasting foundation models. Across both domains, the gains come from where the backbone places memory in physical time rather than from model size. We therefore argue that physically anchored latent timescales are a design requirement for scientific foundation models that must transfer across acquisition conditions.

Primary area:
learning on time series and dynamical systems
Keywords:
Mamba, State Space Models, Physical Timescale Alignment, Physical Inductive Bias, Out-of-Distribution Generalization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jDF7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present PALM, which builds on Mamba and uses memory timescales of multiple scales from the frequency spectrum of the inputs. The intent is to correct for an ambiguity between Δ\Delta and A (which the authors term "memory-timescale misalignment"), particularly for scientific modeling. This appaers as a kind of inductive bias, where the input modality is assumed to have distinct regimes in the frequency spectrum and some heads are allocated to each regime. Specifically, the regime-specific heads have their fade rates set appropriately. The paper evaluates the model for fMRI brain imaging and weather forecasting, with mild improvements over included baselines for the former and stronger ones for the latter.

Critical strengths
  1. The paper presents an interesting inductive bias that - to my knowledge - is novel and quite unique, appears a good candidate for the appropriate domains. As such, it appears to potentially open a new design space, although this is conditional on the utility of the research direction (see below).
Critical weaknesses
  1. The evaluations - in particular for fMRI - are oddly chosen. In brain imaging, distribution shifts are common and a serious problem in need of better solutions. These can concern shifts between sites, scanners/hardware, and age (spectral shape changes across lifespan!). So the obvious choice to me would have been to do here do also leave-one-site-out, cross-dataset with different TRs, and age-transfer; they're common and have issues w.r.t. 'acquisition units'. Instead, the authors use mostly contrived examples of which I do not see practical relevance, except the rest/task comparison. This means the primary evidence comes from weather, which seems to be missing relevant baselines.
Questions for authors
  1. Would the authors be able to explain why they chose these fMRI evaluations, and what their practical relevance is? Why were the more 'obvious' options relying on site and hardware not chosen?
  2. Would it be possible to add standard forecasting baselines (the weather-5k paper itself seems to include them..)?
  3. I'm uncertain what the Gauge-theory contributes to the paper. Would it be possible for the authors to concretely tie the identified 'mislaignment' to failure behaviour of SSMs in practice?
10 comments
AuthorsOpenReview ↗

General response (to all reviewers)

We thank the reviewers for careful and constructive reviews.

  • The question. The submission was built around one theoretical question: the data identify only the product Δ|A|, so what fixes the physical timescale of an SSM's memory, and what happens to that memory when the sampling interval (e.g., the fMRI TR) changes?
  • What did not change. The question, the theory, the method, the datasets, and the evaluation protocols are those of the submission. What changed is one implementation fix in the fMRI runs, RevIN on every weather baseline, and the controls the reviews asked for; each claim these change is listed under Withdrawn, next to what replaces it.
  • What the reviews asked. All three reviews evaluate this question (R1: the reading of Proposition 1; R2: whether both anchors are needed; R3: how the gauge argument connects to failures in practice). The revision answers it with the controls they asked for. While running them we corrected an implementation issue in the fMRI runs: a step-size limit inherited from the Mamba default had overridden the spectrum-derived Δ.
  • Scope. The central contribution of PALM is not the discovery of the Δ–A scaling symmetry (Gu et al., 2022), but the formulation of its consequences for memory-timescale parameterization under changes in acquisition units, and controlled tests of the prediction that follows.
  • The prediction, and how PALM is built for it. PALM expresses Δ in seconds (tied to the TR) and anchors A in 1/s. When the sampling interval changes by a factor r, such a model can be given the new interval and simply rescale Δ → rΔ with A fixed; this is the exact discretization of the same continuous-time system at the new step, so its memory in seconds is unchanged. A model whose Δ has no unit has nothing to rescale. Our zero-shot cross-TR experiments, which apply one trained encoder to the same data resampled to a coarser interval, test exactly this. The prediction concerns how memory responds to a change of sampling interval; accuracy gains are expected only where the task depends on that memory.
  • Supported, with the evidence. Weather-5K: hourly weather at geographically held-out stations, 168 h forecast from a 48 h input, two station splits. "Anchored" variants set A in physical units (τ = 1/|A| ∈ [1, 100] h; PALM, a log-spaced Δ ladder, and A initialization without penalties); "unanchored" ones keep Mamba's unit-free A (spectral, learnable, or random Δ). HCP: motor-task decoding from fMRI at TR = 0.72 s.
What the controls showEvidence
A mismatch between the unit of Δ and the scale of A collapses memoryWeather-5K, Δ in seconds with a unit-free A: memory 0 and switching their memory off changes the error by 0% (3 variants, 2 splits); anchored variants keep a 3.8–50 h median memory and lose 15–34% without it
Only models with acquisition-unit Δ and an anchored A use a given change of sampling intervalHCP, one full-block encoder, whole blocks resampled from TR 0.72 to 2.16 s (10 seeds): PALM 0.943 → 0.973, log-spaced Δ 0.845 → 0.963, learnable Δ 0.851 → 0.851. Weather-5K, r = 2: PALM's degradation +10.6% → +5.7%, unanchored change < 10⁻⁴
The memory buys robustness where the task depends on itWeather-5K, every anchored vs. every unanchored variant, both splits: 3 h resampling +9–14% vs. +20–24%; 12 h truncation +19–28% vs. +35–42%. On unchanged inputs the unanchored ones are 3–4% better
Accuracy at matched size is competitiveWeather-5K: PALM 0.3104 vs. best 0.2893 (24 h, PatchTST), 0.4590 vs. 0.4456 (168 h); split 2: 0.3757 vs. 0.3535, 0.6556 vs. 0.6251. HCP, 2 TRs: 0.923 vs. 0.939. Below every zero-shot foundation model at both horizons, in both splits
  • Withdrawn. That conditioning Δ on each input's spectrum is necessary; the +0.014 CKA gain on fMRI; the lowest error at every weather endpoint.
AuthorsOpenReview ↗

W1 & Q1 & Q2

We thank the reviewer for the perspective from brain imaging; the suggestion to test shifts that change acquisition units led to the paper's most direct test. The inductive bias the reviewer found novel, memory timescales tied to physical units, is exactly what that test isolates: only models whose A is anchored in acquisition units keep a memory and can use a new TR (Q3).

W1 / Q1. Why these fMRI evaluations; why not site, scanner, cross-dataset with different TRs, age? In short: we added both acquisition shifts the reviewer suggests, a change of TR and a change of site and scanner; age transfer is left to future work.

  • Why the original axes. Each one changes a single, controllable factor while subjects, labels, and acquisition stay fixed, so any gain can be attributed to how the model handles time; site, scanner, and age shifts change many factors at once. Each also comes from a practical need: (i) decoding from the first few TRs after onset is the calibration problem of fMRI brain–computer interfaces and real-time fMRI, where only a few volumes are available; (ii) fewer labeled subjects is the usual situation in fMRI, where labeled data are scarce; (iii) resting-state to task transfer is what a brain foundation model is meant to do, pretraining on abundant resting-state data and serving task decoding (Section 3).
  • Change of TR: zero-shot cross-TR (Section 6.4). One encoder trained at TR = 0.72 s, applied without retraining to the same HCP runs resampled to 1.44 s and 2.16 s (whole blocks; a different protocol from the 3-TR early-window accuracy, where separate models are trained on the first 3 TRs). Over ten seeds at 2.16 s, among the models we compared, only those with an anchored A use the new TR (PALM 0.943 → 0.973, log grid 0.845 → 0.963); the unanchored ones barely change (learnable Δ 0.851 → 0.851, random Δ 0.958 → 0.962). Given the TR, PALM is the most robust SSM (0.973 vs. log grid 0.963, random Δ 0.962, Mamba2 0.887, learnable Δ 0.851). On Weather-5K, after 3 h resampling, PALM has the lowest error of all variants in both splits (0.510 vs. 0.541–0.559; 0.719 vs. 0.755–0.767).
  • Change of site and scanner: held-out ABCD sites (Appendix). 22 sites, each scanned on its own GE, Siemens, or Philips scanners (TR = 0.8 s); train on 17 sites, test on the other 5. By device serial number (ABCD release 5.1), the test sites used seven scanners, none of which scanned a training subject; the same scanner models do occur in training. Three seeds, all models at the main-text size: PALM 0.728, Mamba2 0.700, 1D-CNN 0.597, Transformer 0.548. Every PALM-family Δ variant is also above Mamba2 (log-spaced 0.733, random 0.731, learnable 0.726).

Q2. Standard forecasting baselines. In short: PatchTST, which the Weather-5K paper benchmarks, is added with RevIN at a matched budget (17.99–18.31M parameters), together with the three zero-shot forecasting foundation models. PALM is within 3–5% of PatchTST at 168 h and 6–7% at 24 h (split 1 / split 2), and below every foundation model at both horizons in both splits. The gap is largest at 24 h, where the most recent values matter most: PALM uses the same head as for fMRI classification (temporal mean pooling, then a linear map), whereas PatchTST keeps every patch token and maps their concatenation to the forecast. Results:

HorizonModel (Weather-5K)Test MSE, split 1 / split 2
24 hPatchTST0.2893 / 0.3535
PALM0.3104 / 0.3757
Chronos-Bolt (zero-shot)0.3417 / 0.4175
TimesFM-2.5 (zero-shot)0.3435 / 0.4167
Moirai-1.1 (zero-shot)0.3629 / 0.4417
168 hPatchTST0.4456 / 0.6251
PALM0.4590 / 0.6556
TimesFM-2.5 (zero-shot)0.5455 / 0.7559
Chronos-Bolt (zero-shot)0.5607 / 0.8021
Moirai-1.1 (zero-shot)0.5706 / 0.8138

Q3. What the gauge theory contributes; tie misalignment to failure in practice. In short: the theory turns a change of sampling interval into a known transformation of a single parameter, and it predicts exactly when a selective SSM loses its memory. Both predictions are confirmed on two data sets.

  • The symmetry. A selective SSM discretizes h_t = exp(Δ_t A) h_{t−1} + Δ_t B_t x_t, so its function is unchanged under (Δ, A, B) → (cΔ, A/c, B/c): the data identify only the products ΔA and ΔB. The physical timescale of memory, τ = 1/|A|, is therefore a gauge freedom, not something training can determine (credited to Gu et al., 2022).
  • Why it matters for a selective SSM. Δ_t is recomputed from each input while A is shared, so the gauge decides what a given Δ means physically. Once the gauge is fixed in acquisition units (Δ in seconds, A in 1/s), a change of sampling interval T → rT acts on the model as Δ → rΔ with A, B, and every learned weight unchanged; this is the exact discretization of the same continuous-time system at the new step, so the decay per second, |A|Δ/T, is preserved. Without units on Δ, there is no principled way to know which parameters to rescale.
  • Prediction 1: an inconsistent gauge erases memory. The state keeps a fraction e^{−|A|Δ} per step. With Δ in seconds but A at Mamba's unit-free scale, that is e^{−1.67} ≈ 19% per step on fMRI (one sample = 0.72 s) but e^{−10⁴} ≈ 0 on hourly weather data (one sample = 3,600 s): the same mismatch is harmless in one acquisition unit and fatal in another. Measured on Weather-5K, the unanchored models indeed keep no memory: switching their memory off changes the error by 0% in both station splits, while it costs the anchored models 15–34%.
  • Why Mamba's defaults give this. A large Δ resets the state (Gu & Dao, 2023, §3.5), and Mamba2 initializes |A| ∈ [1, 16] and Δ ∈ [0.001, 0.1] without units (Dao & Gu, 2024), assuming Δ is a small per-step quantity; the Δ range sets the timescales the model can capture (Gu et al., 2022). Δ in seconds breaks that assumption on hourly data. Anchoring A in the same units restores memory with the same learnable Δ (median 30–35 h; Table 5; see below), so the memory loss is a unit mismatch, not a limit of the architecture.
  • Prediction 2: only an anchored model can use a new interval. Given the new TR, the anchored models improve (HCP, whole blocks resampled to TR 2.16 s, ten seeds: PALM 0.943 → 0.973, log grid 0.845 → 0.963; weather: PALM's degradation at r = 2 drops from +10.6% to +5.7%), whereas the unanchored ones barely change (weather: by less than 10⁻⁴; HCP: learnable Δ 0.851 → 0.851, random Δ 0.958 → 0.962), since their Δ carries no unit or there is no memory for the rescaling to act on. Switching the memory off shows that it is what uses the TR (PALM 0.973 vs. 0.956 without memory).
  • Where it pays off. Accuracy gains are expected only where the task depends on that memory. On Weather-5K, under 3 h resampling or 12 h truncation, every anchored variant degrades less than every unanchored one in both splits, and after 3 h resampling PALM has the lowest error of all variants, although it is not the most accurate on unchanged inputs.

Table 5

VariantA anchoredTest MSEMedian memoryError increase, memory off3 h resampling12 h truncation
PALM✓0.4590 / 0.65566.0 / 7.4 h+18% / +17%+11.8% / +9.7%+27.1% / +22.1%
Log-spaced Δ✓0.4670 / 0.681531 / 50 h+26% / +34%+13.8% / +11.6%+25.9% / +19.0%
No coupling penalties✓ (init only)0.4702 / 0.65793.8 / 4.7 h+15% / +32%+9.2% / +10.6%+21.5% / +22.9%
Learnable Δ, A anchored✓0.4510 / 0.645035 / 30 h+15% / +20%+13.5% / +13.6%+28.3% / +27.4%
Spectral Δ, no A anchor–0.4447 / 0.62960 / 00% / 0%+21.2% / +20.8%+38.1% / +35.0%
Learnable Δ (Mamba2's Δ and A, Δ in seconds)–0.4441 / 0.62700 / 00% / 0%+22.6% / +21.5%+41.9% / +37.9%
Random Δ–0.4479 / 0.63250 / 00% / 0%+24.0% / +19.8%+42.3% / +35.7%
Reviewer_tATPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that selective state-space models identify the step Δ and the state matrix A only through the product ΔA, so the physical memory timescale τ = 1/|A| is a gauge freedom; under shift Δ changes with the input while A does not, and memory no longer matches the signal’s timescales. PALM anchors Δ to spectrum-estimated knee frequencies (in TRs or hours) and A to a physical memory range, with soft coupling penalties, a SpectralHyperNet and multi-scale SSD heads. It is evaluated on HCP motor fMRI under context truncation, label scarcity and resting-to-task shift, and on Weather-5K held-out-station forecasting.

Critical strengths
  1. The Δ–A gauge diagnosis is crisp and testable. The scaling invariance (Δ, A, B) → (cΔ, c⁻¹A, c⁻¹B) is correct and explains why Δ is a free hyperparameter mistaken for a sampling interval; the fix changes only the parameterization, so it is architecture-complementary.
  2. The Weather-5K result is large and systematic. PALM has the lowest MAE at all 24 variable-horizon endpoints, 15.2% below the best baseline, with the gain growing from 11.4% to 17.9% with horizon.
Critical weaknesses
  1. The paper’s own decomposition does not support its central claim. A fixed log-spaced grid with no spectral prior reaches 0.928 against 0.932 for full PALM, with all pairwise confidence intervals including zero; the anchoring adds 0.014 CKA, so the accuracy gain is mostly multi-scale capacity.
  2. A generic regularizer reproduces the stability claim. Mamba2 with a drift penalty reaches CKA 0.837 and dCor 0.854, at least as stable as PALM (0.812 / 0.846), so physical anchoring is not shown to be required.
Questions for authors

Could you ablate the anchoring on Weather-5K, with a log-spaced grid, learnable Δ, no A-prior and no coupling penalty, so that the 15.2% can be attributed? What is the variance of the MAE reduction across several held-out-station splits? What is the CKA between two seeds of the same model, as a baseline for the stability metric?

9 comments
AuthorsOpenReview ↗

General response (to all reviewers)

We thank the reviewers for careful and constructive reviews.

  • The question. The submission was built around one theoretical question: the data identify only the product Δ|A|, so what fixes the physical timescale of an SSM's memory, and what happens to that memory when the sampling interval (e.g., the fMRI TR) changes?
  • What did not change. The question, the theory, the method, the datasets, and the evaluation protocols are those of the submission. What changed is one implementation fix in the fMRI runs, RevIN on every weather baseline, and the controls the reviews asked for; each claim these change is listed under Withdrawn, next to what replaces it.
  • What the reviews asked. All three reviews evaluate this question (R1: the reading of Proposition 1; R2: whether both anchors are needed; R3: how the gauge argument connects to failures in practice). The revision answers it with the controls they asked for. While running them we corrected an implementation issue in the fMRI runs: a step-size limit inherited from the Mamba default had overridden the spectrum-derived Δ.
  • Scope. The central contribution of PALM is not the discovery of the Δ–A scaling symmetry (Gu et al., 2022), but the formulation of its consequences for memory-timescale parameterization under changes in acquisition units, and controlled tests of the prediction that follows.
  • The prediction, and how PALM is built for it. PALM expresses Δ in seconds (tied to the TR) and anchors A in 1/s. When the sampling interval changes by a factor r, such a model can be given the new interval and simply rescale Δ → rΔ with A fixed; this is the exact discretization of the same continuous-time system at the new step, so its memory in seconds is unchanged. A model whose Δ has no unit has nothing to rescale. Our zero-shot cross-TR experiments, which apply one trained encoder to the same data resampled to a coarser interval, test exactly this. The prediction concerns how memory responds to a change of sampling interval; accuracy gains are expected only where the task depends on that memory.
  • Supported, with the evidence. Weather-5K: hourly weather at geographically held-out stations, 168 h forecast from a 48 h input, two station splits. "Anchored" variants set A in physical units (τ = 1/|A| ∈ [1, 100] h; PALM, a log-spaced Δ ladder, and A initialization without penalties); "unanchored" ones keep Mamba's unit-free A (spectral, learnable, or random Δ). HCP: motor-task decoding from fMRI at TR = 0.72 s.
What the controls showEvidence
A mismatch between the unit of Δ and the scale of A collapses memoryWeather-5K, Δ in seconds with a unit-free A: memory 0 and switching their memory off changes the error by 0% (3 variants, 2 splits); anchored variants keep a 3.8–50 h median memory and lose 15–34% without it
Only models with acquisition-unit Δ and an anchored A use a given change of sampling intervalHCP, one full-block encoder, whole blocks resampled from TR 0.72 to 2.16 s (10 seeds): PALM 0.943 → 0.973, log-spaced Δ 0.845 → 0.963, learnable Δ 0.851 → 0.851. Weather-5K, r = 2: PALM's degradation +10.6% → +5.7%, unanchored change < 10⁻⁴
The memory buys robustness where the task depends on itWeather-5K, every anchored vs. every unanchored variant, both splits: 3 h resampling +9–14% vs. +20–24%; 12 h truncation +19–28% vs. +35–42%. On unchanged inputs the unanchored ones are 3–4% better
Accuracy at matched size is competitiveWeather-5K: PALM 0.3104 vs. best 0.2893 (24 h, PatchTST), 0.4590 vs. 0.4456 (168 h); split 2: 0.3757 vs. 0.3535, 0.6556 vs. 0.6251. HCP, 2 TRs: 0.923 vs. 0.939. Below every zero-shot foundation model at both horizons, in both splits
  • Withdrawn. That conditioning Δ on each input's spectrum is necessary; the +0.014 CKA gain on fMRI; the lowest error at every weather endpoint.
AuthorsOpenReview ↗

General Response to Reviewer 2

We thank the reviewer for a focused review; we agree with the reading of our own decomposition. In short: on HCP the gain is not from spectral placement, and the corrected experiments confirm this (W1, W2); the anchoring is tested instead on Weather-5K, where the target depends on memory (W1, Q1, Q2).

What the work contributes, after these corrections.

  • A failure mode of selective SSMs that we have not seen reported. Giving Δ physical units, the natural step for scientific data, while A keeps Mamba's unit-free initialization silently erases the memory: on hourly data every such model applies |A|Δ ≈ 10⁴ per step and keeps nothing, yet its accuracy on unchanged inputs looks normal (0.444–0.448 vs. PALM 0.459). The standard benchmark cannot see the failure; it appears only when the input changes, and then these models degrade about twice as much under resampling (+20–24% vs. +9–14%).
  • A fix at the level of the parameterization. Anchoring A to the same unit restores the memory for every Δ we tried, spectral, log-spaced, or learnable (median 6–50 h vs. 0), without changing the architecture, so it applies to existing selective SSMs. Every anchored model degrades less than every unanchored one under resampling and truncation (table in Q1).
  • A testable prediction, confirmed on two domains. With Δ in acquisition units, a change of sampling interval maps onto Δ → rΔ, and only the anchored models can use it (HCP, one full-block encoder applied to whole blocks resampled to TR 2.16 s: PALM 0.943 → 0.973; Weather-5K: degradation at r = 2 from +10.6% to +5.7%); the unanchored ones do not change.
  • Not a self-inflicted mismatch. Default Mamba2, with no unit on Δ or A, keeps its memory (12 h on weather) but has nothing to rescale when the sampling interval changes (HCP, TR 0.72 → 2.16 s: 0.887, against 0.973 for PALM given the new TR). Using a new interval requires Δ in physical units, as in S4's test-time rescaling of Δ, and that is precisely the setting in which a unit-free A erases the memory. A selective SSM thus has either a memory or a usable sampling interval, not both, unless A is anchored in the same unit as Δ; PALM is that configuration.
  • The in-distribution cost is small. With a learnable Δ, the anchored model is within 1.5–3% of the best unanchored one on unchanged inputs (0.4510 vs. 0.4441; 0.6450 vs. 0.6270) while keeping a 30–35 h memory and degrading far less under 3 h resampling (+13.5% vs. +22.6%; +13.6% vs. +21.5%).
  • The spectral placement is the smaller part. It keeps the memory within the data's time range and is ahead of a same-capacity log-spaced ladder on Weather-5K in both splits (0.4590 vs. 0.4670; 0.6556 vs. 0.6815); the robustness comes from anchoring A.

W1. A fixed log-spaced grid reaches 0.928 vs. 0.932; the anchoring adds 0.014 CKA, so the accuracy gain is mostly multi-scale capacity. In short: HCP motor decoding is saturated, so on HCP the reviewer is right: a spatial-only classifier already reaches 0.987 and every Δ placement ties. On Weather-5K, where the forecast depends on temporal structure, it is not mostly capacity: PALM beats a log-spaced ladder of the same capacity, and the anchoring separates the variants sharply.

  • Why Weather-5K, not HCP motor, tests the theory. Weather-5K was already one of the submission's two domains, not chosen after the fact. (1) The theory is about memory timescales, so it can only show where the target depends on the signal's past. An HCP motor block is identified from its spatial pattern alone (a classifier without any temporal model reaches 0.987), whereas a weather forecast is the future of the same signal, and the anchored models lose 15–34% when their memory is switched off. (2) The consequence of the unit choice scales with the sampling interval: with Δ in seconds and a unit-free A, one step decays by |A|Δ ≈ 1.7 at TR = 0.72 s but ≈ 10⁴ at 1 h, so the failure the theory predicts is mild on fMRI and total on hourly data. (3) Changing the sampling interval is natural there (many stations report every 3 h), and 100 stations over ten years with geographically held-out test stations give enough data to separate variants that differ by a few percent.
  • HCP: we agree. After the correction (table below), the four Δ placements tie: a fixed log-spaced ladder matches the spectrum-derived Δ, and a learnable or random Δ is at least as good. HCP motor decoding, which a spatial-only classifier solves at 0.987, cannot separate Δ placements. The fMRI gain over the Transformer and 1D-CNN comes from the selective SSM backbone, which PALM-family models share with Mamba2.
  • Withdrawn accordingly. The +0.014 CKA claim and the stability claim (abstract, contributions, Section 6.1, discussion), and the claim that the spectrum-derived Δ is necessary or adapts to each input.
  • The correction behind this. In the submitted fMRI runs, a step-size limit inherited from the Mamba default, dt ≤ 0.1, overrode the spectrum-derived Δ, so every head used the same step (weather runs were not affected). We fixed it and reran every main-text fMRI comparison with five seeds, with all baselines retrained at PALM's size. We also found that SpectralHyperNet's encoder saturated during fMRI training (its first Tanh output was identical for every input, so all subjects received the same knees); the fMRI PALM runs now place a LayerNorm in front of that encoder, which leaves accuracy unchanged. On Weather-5K the encoder was not saturated in this way, and those runs are unchanged.
HCP motor (5 seeds, matched size)Accuracy, 3 TRsAccuracy, 2 TRsCKA, full vs. 2 TRs
PALM (spectrum-derived Δ)0.9450.9230.814
Log-spaced Δ0.9430.9310.830
Learnable Δ0.9470.9350.853
Random Δ0.9440.9390.858
Mamba20.9340.9240.830
Transformer0.9060.8630.752
1D-CNN0.8890.8540.692

Accuracy, 3 / 2 TRs: separate models trained and tested on the first 3 or 2 TRs of each block. This differs from the zero-shot cross-TR test, in which one full-block encoder is applied to whole blocks resampled to a new TR.

  • Weather-5K: not mostly capacity. The log-spaced ladder has exactly PALM's multi-scale capacity and A anchor; only the placement of Δ differs. PALM is ahead in both splits on unchanged inputs (0.4590 vs. 0.4670; 0.6556 vs. 0.6815) and after 3 h resampling (0.510 vs. 0.535; 0.719 vs. 0.746). Given the new interval, PALM improves (+10.6% → +5.7% at r = 2), whereas the log-spaced ladder gets worse at r = 3 (+13.8% → +16.4%). Where the ladder sits therefore matters on data whose target depends on time, even though we no longer claim that Δ adapts to each input.
  • Weather-5K: the anchoring matters. Accuracy on held-out stations is close (0.444–0.470 MSE in split 1 and 0.627–0.682 in split 2, unanchored slightly ahead), but the variants differ in whether they have memory at all: the unanchored ones have none, the anchored ones use it (switching it off costs 15–34%). Under 3 h resampling or 12 h truncation of one fixed encoder, every anchored variant degrades less than every unanchored one in both splits, and under 3 h resampling PALM, given the new interval, has the lowest absolute error of all variants (table in Q1).
Reviewer_Xes6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies why representations of scientific foundation models degrade under distribution shift, and traces one source of this failure to selective state-space models: their memory depends on the step Δ\Delta and the state matrix AA only through their product, so the data do not identify the physical timescale of memory. The authors call this memory-timescale misalignment and propose PALM, which anchors Δ\Delta through spectrum-estimated knee frequencies (via a spectral hypernetwork) and AA through a physical memory range, with coupling penalties and a multi-scale state-space block. The method is evaluated on fMRI (context truncation, low-resource transfer, and resting-to-task brain-state shift) and on Weather-5K forecasting at held-out stations. PALM retains about 11% higher representation similarity than Mamba2 under context truncation, achieves the lowest error on every weather variable and horizon (15.2% below the best baseline on average), and outperforms the zero-shot forecasting foundation models Chronos-Bolt, TimesFM-2.5, and Moirai.

Critical strengths
  1. The paper is well-motivated and states a concrete, testable hypothesis about where selective SSM backbones fail under shift.
  2. Connecting the spectral structure of scientific signals (spectral knees, scale-free dynamics) to the discretization of SSMs is interesting and could be useful more broadly for scientific machine learning backbones.
  3. The evaluation spans two different domains (fMRI and weather) and several types of shift.
  4. The weather results are consistent: PALM has the lowest mean error at all 24 variable–horizon combinations.
  5. The authors include informative controls and are candid about them, e.g., Table 2 shows that the accuracy difference between a fixed grid and spectrum-conditioned steps is not significant, and the paper clearly distinguishes target-trained PALM from zero-shot foundation models.
Critical weaknesses
  1. The novelty of the gauge argument is overstated: that discretized SSMs depend only on (ΔA,ΔB)(\Delta A, \Delta B), so Δ\Delta and AA can be rescaled jointly, is already discussed in [1] (not cited). As Section 4.1 itself admits, gauge-equivalent models compute the same function on shifted inputs, so Proposition 1 alone does not explain the degradation. Also, for a constant step the memory in acquisition time is δobs/(Δ∣A∣)\delta_{\text{obs}}/(\Delta|A|), so calling 1/∣A∣1/|A| a physical timescale is a convention, not a result.
  2. My main concern is that the experiments do not show that dual anchoring drives the gains. In Table 2, a fixed log-grid reaches 0.928 vs. ~0.932 for PALM with overlapping intervals; the learnable- and random-Δ\Delta variants also drop the physical initialization of AA; and there is no ablation on weather, where the gains are largest. Moreover, a single soft anchor on Δ\Delta already fixes the gauge, so the need for two anchors is not justified.
  3. The stability results compare separately trained models per context length (with the spectral branch seeing the full run), not one fixed encoder under truncation; with a frozen backbone (Table 10), PALM's CKA is slightly lower than Mamba2's.
  4. The baselines are called parameter-matched, but PALM has 5.98M parameters vs. 7.56-7.72M. Forecasting-specific models (e.g., PatchTST [2], already benchmarked on Weather-5K) and multi-scale SSMs (e.g., MS-SSM [3]) are missing, and the zero-shot results lack checkpoints, configurations, and a full MSE table.
  5. Proposition 3 does not hold as stated: for a single exponential on [1,1+ϵ][1, 1+\epsilon], the best approximation error is of order ϵ2\epsilon^2, whereas the claimed bound decays faster than any polynomial in ϵ\epsilon.
  6. The mechanistic diagnostics are unconvincing: the Jacobian-kernel distance (Appendix O) is not scale-invariant and is smaller for the CNN than for PALM, and reading WeightWatcher α≈2\alpha \approx 2 as saturation contradicts the cited source [4].
  7. Key data details are missing (HCP subject counts and pretraining overlap; held-out station IDs, date ranges, normalization), and the ~140-hour knees in Figure 4 cannot be resolved from the default 48-hour context.

[1] A. Gu, I. Johnson, A. Timalsina, A. Rudra, and C. Ré. How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections. arXiv:2206.12037, 2022.

[2] Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. ICLR, 2023.

[3] M. Karami, A. Behrouz, P. Zhong, R. Pascanu, and V. Mirrokni. MS-SSM: A Multi-Scale State Space Model for Efficient Sequence Modeling. arXiv:2512.23824, 2025.

[4] C. H. Martin, T. Peng, and M. W. Mahoney. Predicting Trends in the Quality of State-of-the-art Neural Networks without Access to Training or Testing Data. Nature Communications, 12:4122, 2021.

Questions for authors
  1. What is the exact mapping from the estimated knee frequencies to Δ\Delta, including units and bounds, and how is it computed from a 48-hour weather context?
  2. Can authors provide a factorial ablation (spectral placement, AA initialization, each penalty) with matched parameter counts, in particular on the weather benchmark?
  3. Can authors include at least one forecasting-specific baseline (e.g., PatchTST [2]) and one multi-scale SSM baseline (e.g., MS-SSM [3])?
  4. Can authors report the full zero-shot foundation-model results, together with the checkpoints and configurations used?
  5. What is the additional training and inference cost of the spectral hypernetwork and the multi-scale block compared with Mamba2?

Any discussion around these will help improve the paper over its current state.

7 comments
AuthorsOpenReview ↗

General response (to all reviewers)

We thank the reviewers for careful and constructive reviews.

  • The question. The submission was built around one theoretical question: the data identify only the product Δ|A|, so what fixes the physical timescale of an SSM's memory, and what happens to that memory when the sampling interval (e.g., the fMRI TR) changes?
  • What did not change. The question, the theory, the method, the datasets, and the evaluation protocols are those of the submission. What changed is one implementation fix in the fMRI runs, RevIN on every weather baseline, and the controls the reviews asked for; each claim these change is listed under Withdrawn, next to what replaces it.
  • What the reviews asked. All three reviews evaluate this question (R1: the reading of Proposition 1; R2: whether both anchors are needed; R3: how the gauge argument connects to failures in practice). The revision answers it with the controls they asked for. While running them we corrected an implementation issue in the fMRI runs: a step-size limit inherited from the Mamba default had overridden the spectrum-derived Δ.
  • Scope. The central contribution of PALM is not the discovery of the Δ–A scaling symmetry (Gu et al., 2022), but the formulation of its consequences for memory-timescale parameterization under changes in acquisition units, and controlled tests of the prediction that follows.
  • The prediction, and how PALM is built for it. PALM expresses Δ in seconds (tied to the TR) and anchors A in 1/s. When the sampling interval changes by a factor r, such a model can be given the new interval and simply rescale Δ → rΔ with A fixed; this is the exact discretization of the same continuous-time system at the new step, so its memory in seconds is unchanged. A model whose Δ has no unit has nothing to rescale. Our zero-shot cross-TR experiments, which apply one trained encoder to the same data resampled to a coarser interval, test exactly this. The prediction concerns how memory responds to a change of sampling interval; accuracy gains are expected only where the task depends on that memory.
  • Supported, with the evidence. Weather-5K: hourly weather at geographically held-out stations, 168 h forecast from a 48 h input, two station splits. "Anchored" variants set A in physical units (τ = 1/|A| ∈ [1, 100] h; PALM, a log-spaced Δ ladder, and A initialization without penalties); "unanchored" ones keep Mamba's unit-free A (spectral, learnable, or random Δ). HCP: motor-task decoding from fMRI at TR = 0.72 s.
What the controls showEvidence
A mismatch between the unit of Δ and the scale of A collapses memoryWeather-5K, Δ in seconds with a unit-free A: memory 0 and switching their memory off changes the error by 0% (3 variants, 2 splits); anchored variants keep a 3.8–50 h median memory and lose 15–34% without it
Only models with acquisition-unit Δ and an anchored A use a given change of sampling intervalHCP, one full-block encoder, whole blocks resampled from TR 0.72 to 2.16 s (10 seeds): PALM 0.943 → 0.973, log-spaced Δ 0.845 → 0.963, learnable Δ 0.851 → 0.851. Weather-5K, r = 2: PALM's degradation +10.6% → +5.7%, unanchored change < 10⁻⁴
The memory buys robustness where the task depends on itWeather-5K, every anchored vs. every unanchored variant, both splits: 3 h resampling +9–14% vs. +20–24%; 12 h truncation +19–28% vs. +35–42%. On unchanged inputs the unanchored ones are 3–4% better
Accuracy at matched size is competitiveWeather-5K: PALM 0.3104 vs. best 0.2893 (24 h, PatchTST), 0.4590 vs. 0.4456 (168 h); split 2: 0.3757 vs. 0.3535, 0.6556 vs. 0.6251. HCP, 2 TRs: 0.923 vs. 0.939. Below every zero-shot foundation model at both horizons, in both splits
  • Withdrawn. That conditioning Δ on each input's spectrum is necessary; the +0.014 CKA gain on fMRI; the lowest error at every weather endpoint.

We thank the reviewer for a detailed and technically precise review. Of the two strengths the reviewer named, the first, a crisp and testable gauge diagnosis whose fix changes only the parameterization, is what the new experiments test, and it holds: anchoring A restores the memory for every Δ we tried, including a learnable one, and every anchored model degrades less than every unanchored one (W2). The second, the per-endpoint weather gain, came from baselines without RevIN; we withdraw it, and the controlled comparison in W2 replaces it.

W1. Novelty of the gauge argument; Proposition 1 alone does not explain degradation; τ as a physical timescale is a convention. In short: we agree on all three points and revised Section 4 accordingly.

  • (a) Novelty. We now cite Gu et al. (2022) and S4's test-time rescaling of Δ, and no longer claim the symmetry as new.
  • (b) Proposition 1. Agreed: gauge-equivalent models compute the same function, which depends only on |A|Δ. The split matters once Δ is given in physical units: the same Δ in seconds then gives a different memory |A|Δ depending on A's scale, and with a unit-free A the memory is zero (table in W2).
  • (c) Convention. Agreed, and now stated (Section 4.1). The convention yields a testable prediction: when the sampling interval T becomes rT, Δ → rΔ with A fixed is the exact discretization of the same continuous-time system, so the decay per second, |A|Δ/T, is unchanged. The new cross-TR experiment confirms it (Section 6.4): among the models we compared, only those with Δ in acquisition units and an anchored A respond to the given interval, and their memory then helps (HCP: 0.973 with memory vs. 0.956 without; weather: degradation at r = 2 drops from +10.6% to +5.7%).

W2. The experiments do not show that dual anchoring drives the gains; a single soft anchor already fixes the gauge. In short: the A anchor carries the effect; the coupling penalties only keep Δ in acquisition units. New factorial ablation on Weather-5K (Table 5; 168 h horizon, 48 h input, 17.99–18.31M parameters; split 1 / split 2, two held-out station splits; a third is running; "memory off" resets the SSM's recurrent memory at every step, everything else unchanged):

VariantA anchoredTest MSEMedian memoryError increase, memory off3 h resampling12 h truncation
PALM✓0.4590 / 0.65566.0 / 7.4 h+18% / +17%+11.8% / +9.7%+27.1% / +22.1%
Log-spaced Δ✓0.4670 / 0.681531 / 50 h+26% / +34%+13.8% / +11.6%+25.9% / +19.0%
No coupling penalties✓ (init only)0.4702 / 0.65793.8 / 4.7 h+15% / +32%+9.2% / +10.6%+21.5% / +22.9%
Learnable Δ, A anchored✓0.4510 / 0.645035 / 30 h+15% / +20%+13.5% / +13.6%+28.3% / +27.4%
Spectral Δ, no A anchor–0.4447 / 0.62960 / 00% / 0%+21.2% / +20.8%+38.1% / +35.0%
Learnable Δ (Mamba2's Δ and A, Δ in seconds)–0.4441 / 0.62700 / 00% / 0%+22.6% / +21.5%+41.9% / +37.9%
Random Δ–0.4479 / 0.63250 / 00% / 0%+24.0% / +19.8%+42.3% / +35.7%
  • Without the A anchor, memory is zero, however Δ is set. The state keeps e^{−|A|Δ} per step, and a large Δ resets it (Gu & Dao, 2023, §3.5). Mamba2 initializes |A| ∈ [1, 16] and Δ ∈ [0.001, 0.1] without units (Dao & Gu, 2024); with Δ in seconds, one hourly step gives |A|Δ ≈ 10⁴, so the state is reset at every step. Default Mamba2, with no units on either factor, keeps 12 h, and the A initialization alone keeps 3.8–4.7 h.
  • With it, the models are more robust. Under 3 h resampling and 12 h truncation of one fixed encoder (a coarser sampling interval and a shorter history), every anchored variant degrades less than every unanchored one, in both splits. The memoryless variants are 3–4% more accurate only on unchanged inputs.
  • One anchor suffices, as the reviewer says. Without the penalties memory survives (third row), so we now present them as keeping Δ in acquisition units, not as a second source of robustness ("Why anchor both factors", rewritten). A learnable Δ with the same A anchor (fourth row) recovers memory (median 35 / 30 h, vs. 0 without the anchor) and degrades like the other anchored variants in both splits (+13.5% / +13.6% under resampling, +28.3% / +27.4% under truncation), so the A anchor carries the effect. The spectral Δ keeps memory within the data's time range (≤ 24 h vs. up to 1,600–2,700 h) and gives the lowest error after 3 h resampling in both splits, by a small margin (0.5097 vs. 0.5107; 0.719 vs. 0.727).
  • HCP. After the corrections in W4, the Δ variants tie (0.923–0.939 at 2 TRs), and we withdrew the +0.014 CKA claim.

W3. Stability compares separately trained models; with a frozen backbone PALM's CKA is slightly lower. In short: agreed. As one fixed encoder on shorter inputs, PALM is not more stable on HCP; on Weather-5K, the anchored models degrade less.

  • HCP (Table 9 (Appendix K)). Each full-block encoder is applied unchanged to the first 2 or 3 TRs. Every encoder with a causal sequential state, PALM included, drops to chance or below (0.13–0.26), and all Δ placements fail alike, so the cause is the sequential state facing inputs far shorter than in training, not Δ placement. We state this as a limitation (Discussion).
  • Weather-5K. The same protocol does not collapse, and anchored variants degrade less than unanchored ones in both splits (12 h truncation: +19–28% vs. +35–42%; W2).
  • Frozen backbone. Agreed and stated in the Appendix; we claim no stability advantage for frozen features.

W4. "Parameter-matched" is inaccurate; missing PatchTST, MS-SSM; zero-shot details. In short: fixed by new experiments.

  • Parameter matching. We retrained Mamba2 (6.14M), Transformer (5.98M), and 1D-CNN (5.96M) at PALM's size (5.98M) with PALM's recipe, five seeds each, for every main-text fMRI table (Weather-5K models already had 17.99–18.31M). In the same reruns we also fixed a step-size limit inherited from the Mamba default (dt ≤ 0.1) that had overridden the spectrum-derived Δ in the submitted fMRI runs, and added a LayerNorm in front of SpectralHyperNet's encoder, whose first Tanh had saturated during fMRI training so that every input received the same knees (accuracy unchanged; the knees vary across subjects again):
HCP motor (5 seeds, matched size)Accuracy, 3 TRsAccuracy, 2 TRsCKA, full vs. 2 TRs
PALM (spectrum-derived Δ)0.9450.9230.814
Log-spaced Δ0.9430.9310.830
Learnable Δ0.9470.9350.853
Random Δ0.9440.9390.858
Mamba20.9340.9240.830
Transformer0.9060.8630.752
1D-CNN0.8890.8540.692

Accuracy, 3 / 2 TRs: separate models trained and tested on the first 3 or 2 TRs of each block. This differs from the zero-shot cross-TR test, in which one full-block encoder is applied to whole blocks resampled to a new TR.

  • Weather-5K. All trained models now have 17.99–18.31M parameters and use RevIN, PatchTST is added, and the zero-shot foundation models are evaluated on the same held-out stations. PALM is within 3–5% of the best model at 168 h (split 1 / split 2) and 6–7% at 24 h, and below every foundation model at both horizons in both splits; we withdraw the claim of lowest error at every endpoint. The gap is largest at 24 h, where the most recent values matter most: PALM uses the same head as for fMRI classification (temporal mean pooling, then a linear map), whereas PatchTST keeps every patch token and maps their concatenation to the forecast.
HorizonModel (Weather-5K)ParametersTest MSE, split 1 / split 2
24 hPatchTST18.09M0.2893 / 0.3535
PALM17.99M0.3104 / 0.3757
Chronos-Bolt (zero-shot)pretrained0.3417 / 0.4175
TimesFM-2.5 (zero-shot)pretrained0.3435 / 0.4167
Moirai-1.1 (zero-shot)pretrained0.3629 / 0.4417
168 hPatchTST18.16M0.4456 / 0.6251
PALM18.27M0.4590 / 0.6556
TimesFM-2.5 (zero-shot)pretrained0.5455 / 0.7559
Chronos-Bolt (zero-shot)pretrained0.5607 / 0.8021
Moirai-1.1 (zero-shot)pretrained0.5706 / 0.8138
  • MS-SSM. Its code has not been released, and a re-implementation would be our own model; the fixed log-spaced Δ ladder serves as the controlled multi-scale stand-in in Table 5 (W2).
  • Zero-shot settings. Checkpoints Chronos-Bolt base, TimesFM-2.5 200M, Moirai-1.1-R small; the same 48 h context, each variable forecast univariately, median forecast. The full table with every horizon and split is in the Appendix.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for the Area Chair

We thank all reviewers. Reviewers 1 and 2 confirmed that their major concerns are resolved. New evidence added during the discussion, all in the revised paper or the threads:

  • Memory collapse and its fix (Table 5, three held-out station splits). A selective SSM whose Δ is given physical units while A keeps its unit-free initialization has zero memory; anchoring A in the same unit restores it, and every anchored variant degrades less than every unanchored one under 3 h resampling and 12 h truncation, in all three splits.
  • Cross-TR on fMRI (Table 4, ten seeds). Given the new TR, the anchored models reach 0.973 / 0.963; unit-free models rescaled as in S4 reach 0.899 / 0.884.
  • ABCD (Tables 15–16). Leave-one-site-out over all 22 sites: PALM 0.758, re-tuned Transformer 0.734, Mamba2 0.710; in the original site split, the held-out scanners were verified by device serial number never to have scanned a training subject.
  • Baselines. PatchTST, three zero-shot forecasting foundation models, a re-tuned Transformer, matched parameter counts, and cost.

The central claim is the one we submitted. The submitted abstract already stated that the memory of a selective SSM is not identified in physical units, that "the split between Δ and A must be tied to physical units," and that PALM anchors A "through a physical memory range"; it already framed PALM as a general mechanism for scientific foundation models, evaluated in two domains (fMRI and weather). The discussion changed the evidence, not the claim: it showed concretely what fails without the A anchor (zero memory) and what the anchor buys. One claim was withdrawn: with RevIN applied to every baseline, PALM no longer has the lowest in-distribution weather error, and we report this openly as a trade-off.

Reviewer 3's remaining concern is positioning rather than correctness: that the paper should be organized around the memory-collapse finding and its accuracy–robustness trade-off. We agree, and for a final version we would restructure the introduction and contributions around it. This requires no new claims; the supporting results are already in the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.