STRUM: A Dyadic Multimodal Benchmark for Comparison of Physiological Modalities

Representation learning for physiological signals has focused on scale, prioritizing longer recordings and larger cohorts. How the signals compare to one another is harder to establish: cross-modality comparisons are typically drawn from separate studies, each with its own cohort and instrumentation, so any apparent difference carries everything else that varied along with it. Isolating it requires recording every candidate modality from the same participants simultaneously, which no EEG benchmark provides. We present STRUM (Small Team Reconnaissance Urban Missions), a corpus of 66 people recorded as 33 pairs playing a cooperative mission game, with both partners instrumented at once across eleven stream families on a single clock: 205-channel scalp EEG each, muscle, cardiac, ocular, electrodermal and respiratory channels, eye tracking, motion capture, the in-game avatar, a force plate and the controller, over roughly 87 h, with the two participants' neural recordings aligned to a median of 0.208 ms. We define two continuous, event-free regression tasks on the corpus, predicting a person's phasic electrodermal arousal and mean heart rate from the remaining simultaneously acquired channels, evaluated under leave-one-dyad-out cross-validation, released with fixed splits and a reference loader. Across both targets, modality choice affects decoding more than model choice: the neck muscle band alone reaches $r=0.4449$ on heart rate, ahead of every other modality in the same comparison, and pretrained encoders on EEG do not surpass a small network trained directly on muscle on matched windows. Multimodal fusion improves further even with encoders reused off the shelf: the full suite reaches $0.5072$ against the best single modality's $0.4449$, a gain of $0.062$. Simultaneous acquisition separates an apparent neural result into the muscular, ocular and cardiac signal inside it, and recording both partners makes pair-level targets measurable. For continuous autonomic decoding, the instrumentation chosen at acquisition constrains performance more than the encoder chosen at training.