Can We Find the Optimal Variable Ordering for Mamba in Large-Scale Systems?
Abstract
When Mamba reads variable histories one by one, which variables should it read together? Large physical systems contain many pieces of equipment, each described by different signals. A tank's pressure, level, and flow, for example, measure different aspects of the same process. We hypothesize that keeping such heterogeneous signals together can improve prediction. We test a policy that groups signals by component while randomizing the sequence of groups. In a simulated nuclear power system, this policy gives lower mean test error than unrestricted random orders and a history-similarity order derived from GPS-Mamba. We then investigate the learned computation in a diagnostic comparison of a fixed component order with random orders. Removing an estimated output interaction between a target history and the other histories reverses the fixed order's advantage on shared diagnostic windows. An internal control, however, does not support attributing that advantage to the immediately preceding variable's write. Together, these experiments connect variable organization to prediction and distinguish joint use of histories from a simple local-write explanation.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.