PALM: Physically-Anchored Latent-timescale Mamba for Stable Representations under Distribution Shift

Scientific foundation models are built to reuse representations across datasets and acquisition protocols, yet their representations degrade under distribution shift. We trace one source of this failure to selective state-space models (SSMs). Their memory depends on the step ∆ and the state matrix A only through their product, so data do not identify the physical timescale of memory, and under a shift the resulting memory no longer matches the signal’s timescales. We call this failure memory-timescale misalignment. To resolve it, the split between ∆ and A must be tied to physical units. We therefore propose Physically-Anchored Latent-timescale Mamba (PALM), which disentangles A from ∆ by anchoring both to acquisition units: ∆ through spectrum-estimated knee frequencies and A through a physical memory range. We evaluate PALM against parameter-matched baselines in two domains, under temporal-context truncation, low-resource transfer, and brain-state shift on fMRI and under held-out-station forecasting on Weather-5K. Under fMRI context truncation with run-level spectral calibration, PALM keeps its latent timescales in physical units and retains 11% higher representation similarity than Mamba2. At geographically isolated weather stations never seen during training, PALM achieves the lowest error on every variable and horizon, 15.2% below the best baseline on average, and outperforms zero-shot forecasting foundation models. Across both domains, the gains come from where the backbone places memory in physical time rather than from model size. We therefore argue that physically anchored latent timescales are a design requirement for scientific foundation models that must transfer across acquisition conditions.