acceptodds
Under review as a conference paper at ICLR 2027

BioLT: Streaming Biosignal Modeling

Abstract

Clinical biosignals such as ECG, PPG, and EEG are increasingly acquired as hours-long streams from bedside monitors and wearables, yet existing analysis pipelines almost universally operate on isolated short windows of around 10 seconds and cannot process the long, irregular, frequently interrupted signals. We introduce BioLT, a streaming decoder-only framework for long-context biosignal modeling that turns any window-level feature extractor or pretrained biosignal foundation model into a streaming model. BioLT combines (i) a discontinuity-aware encoding that exposes the inter-window elapsed time as an explicit input channel, so that the decoder can learn to model these gaps as clinically meaningful information, and (ii) a fixed-size compressed memory bank supervised with cumulative summary statistics, which keeps both training and inference linear in the recording length while preserving long-range context. We evaluate on CPSC-2021, VitalDB, MC-MED, and CHB-MIT for arrhythmia, sepsis, and seizure detection and for forecasting up to 20 minutes ahead, using domain features and embeddings from six biosignal foundation models. Across 21 dataset-feature settings, BioLT achieves the highest detection AUROC in 15 and the highest 20-minute forecasting AUROC in 13 settings, and shows consistent performance gains over state-of-the-art snapshot foundation-model embeddings. These highlight the importance of modeling how streaming biosignal representations evolve over time, beyond improving window-level representations alone.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Biosignal Modeling, Physiological Time Series, Continuous Clinical Monitoring, AI for Health

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_s9YqOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Physiological signals are often continuously streamed, with different problems such as missing data. To handle this, the authors propose BioLT. They evaluate BioLT on classification/detection and forecasting across many datasets and achieve strong performances when compared with baselines.

Critical strengths
  • they do a lot of experiments
  • the experimental setting is well described.
  • Lots of details in appendix
Critical weaknesses
  • The main baseline comparisons for both detection and forecasting are not too convincing. BioLT is often extremely close to Trans XL, which the authors mention is essentially the same model. This hurts the author's contributions and makes the readers wonder why one would use BioLT in particular.
  • I think the paper's scope is kind of all over the place and it seems like many portions should have been expanded upon. For example, the ablation of core components before the conclusion is I think one of the more interesting experiments but the authors provide 2 sentences and the table, without any interpretation of the results.

Overall, I think this is a cool paper, but the authors contributions do not convince me why I would want to use their model over existing ones.

Questions for authors
  • Although the authors stress how most baseline models evaluate in 10s windows, if I am not mistaken, all of the inputs are still 10s. I think the memory bank is supposed to give the model access to more "time" then 10s but BioLT is still only evaluated with 10s inputs. Why did the authors not try longer inputs?
  • What is BioLT’s performance if the autoregressive stage is removed and the decoder is trained only with the task loss?
  • what does 10 binary dataset settings mean in Figure 6? Would you be able to report the per dataset Sens@1FA/h?
  • For Figure 8, it seems BioLT actually decreases in performance for Beyond 500 windows where trans-xl does not. However the authors state "In contrast, BioLT achieves the strongest performance both before and after the boundary, with no evident degradation beyond the training length." I am a bit confused by this, and would appreciate any clarifications.
  • For Table 4, would the w/o both be equivalent to the trans-xl? If so, for this task, why is the results so much worse than in tables 2 and 3? It sees trans xl is quite competitive with BioLT across all datasets.
5 comments
AuthorsOpenReview ↗

Response to Reviewer s9Yq (1/3)

Thank you for the detailed questions.

W1. BioLT is often extremely close to Transformer-XL, which the authors describe as essentially the same model.

Thank you for pointing this out. Our wording “architecturally identical” was confusing, apologies. We meant that Transformer-XL was deliberately matched to BioLT in its basic Transformer backbone, so that it serves as a strong and fair long-context baseline; we did not mean that the complete models are essentially the same. We will revise this wording accordingly.

Table 9. Memory probe, VitalDB ECG AF (H=20H{=}20), mean over 3 seeds (std ≤0.03 for R2R^2); 30–118 positive chunks per distance.

How far back30 min60 min90 min120 min180 min
R2R^2, mean features of the past chunk
BioLT memory0.980.770.660.520.30
Transformer-XL cache0.860.630.520.450.27
Snapshot (current window)0.500.440.410.370.30
AUROC, did an AF event occur in that chunk
BioLT memory0.680.680.680.660.70
Transformer-XL cache0.600.500.560.580.48
Snapshot (current window)0.520.490.480.600.48

Transformer-XL is intentionally a stringent comparison in this setting. Despite the close matching, across the 21 dataset–feature settings BioLT is significantly ahead at 5 of 6 evaluated horizons (Table 10), with median AUROC gains of +1.52+1.52, +1.76+1.76, and +1.53+1.53 points at H=10,20,30H{=}10,20,30, respectively. The difference is also visible when we directly probe what the learned state retains: the frozen BioLT memory reconstructs the preceding chunk at R2=0.98R^2{=}0.98, compared with 0.86 for the Transformer-XL cache, with the advantage persisting over longer temporal distances (Table 9).

Especially, our key novelty and also the main contribution is that if we these established sequential modelling (such as Transformer-XL) established in LLMs or somewhere else, what they cannot do? As highlighted in Figure 1 and throughout our paper, there are two main challenges, (1) discontinuity, where valid biosignal windows are separated by irregular gaps that may themselves carry information; and (2) open-ended context, where monitoring streams continue to grow and require efficient retention of long-term physiological history.

More broadly, as also clarified in response to the other reviewers, we do not intend to claim that BioLT is an entirely new architecture for generic sequence modeling, or that its individual building blocks are novel in isolation. Our motivation comes from a different problem: biosignal modeling has largely focused on short “snapshot” windows, while how to extend established window-level representations toward long, irregular, continuously streaming monitoring remains much less clear. Our contribution is therefore in identifying this problem, adapting appropriate and domain-informed approaches to address it, and empirically validating the need for these design choices.

We will therefore revise the paper in two ways: first, to distinguish clearly between a matched Transformer backbone and an identical model; and second, to position the contribution more precisely as a practical streaming-biosignal formulation, rather than novelty of any individual sequence-modeling primitive.

Table 10 is a two-sided paired Wilcoxon test over the 21 dataset–feature settings (three-seed means), as in App. Table A.16. Entry: median AUROC difference (BioLT − baseline) and pp; bold: p<0.05p<0.05.

Table 10. BioLT minus baseline, median over 21 settings (pp).

HH (min)Transformer-XLMamba-3
0+0.62 (0.001)+2.54 ($<$0.001)
1+0.27 (0.43)+0.53 (0.10)
5+0.39 (0.029)+1.15 (0.019)
10+1.52 (0.001)+0.91 (0.014)
20+1.76 ($<$0.001)+2.22 (<\0.001)
30+1.53 ($<$0.001)+1.42 (0.001)

Q5. Is “w/o both” in Table 4 equivalent to Transformer-XL? If so, why is it much worse than in Tables 2/3?

No. “W/o mem. bank” removes all cross-chunk memory, so “w/o both” is a chunked causal decoder that sees only the current chunk and no elapsed time; Transformer-XL keeps a 180-window cache and Δs\Delta s. Table 4 is also at H=10H{=}10, Tables 2/3 at H=0H{=}0 and 2020. At H=10H{=}10 Transformer-XL is close to BioLT, as in Tables 2/3: 60.46±3.38 vs 60.29±0.64 on ECG and 63.91±0.39 vs 63.19±0.48 on PPG, within the seed variation.

AuthorsOpenReview ↗

Response to Reviewer s9Yq (2/3)

W2. The scope is broad; the ablation of core components gets two sentences and no interpretation.

Agreed.

Table 11. Ablation of core components (revised Table 4). Δ\Delta is relative to BioLT.

ConfigurationWhat the model seesECGΔ\DeltaPPGΔ\Delta
BioLTcurrent chunk + 32-slot memory + Δs\Delta s60.29±0.64–63.19±0.48–
w/o Δs\Delta sno elapsed-time channel58.02±1.71−2.2760.61±1.43−2.58
w/o mem. bankcurrent chunk only (≤30 min)56.68±0.23−3.6157.83±0.34−5.36
w/o bothcurrent chunk only, no Δs\Delta s53.82±0.17−6.4753.74±0.95−9.45

The revised ablation gives two clear findings. First, removing cross-chunk memory causes the largest single drop in performance (−3.6-3.6 AUROC for ECG and −5.4-5.4 for PPG), showing that information beyond the current 30-min chunk contributes to sepsis forecasting. Second, removing the elapsed-time input (Δs\Delta s) reduces AUROC by a further 2.3–2.6 points, showing that treating separated valid windows as if they were uniformly spaced loses useful information. Removing both components produces the largest degradation (−6.5-6.5/−9.5-9.5 points), suggesting that long-term context and recording discontinuity provide complementary information.

We will therefore revise Section 5.3 so that the ablation is explicitly linked back to the two problems introduced in Figure 1, infinitely-long context and discontinuity, rather than presenting it as an isolated table.

Overall: why would I use BioLT over existing models?

Thanks. We see three practical gaps in existing approaches that motivate BioLT.

(1) Limited longitudinal context. Snapshot-based biosignal models operate on individual short windows and therefore cannot model how physiological information evolves over longer periods. Most findings in our paper and also in the existing literature have demonstrated the importance of modeling long-term information for biosignals.

(2) Limited handling of recording gaps. Generic sequential models can aggregate temporal context, but they typically do not explicitly represent the irregular interruptions that occur in real biosignal streams. In our ablation, removing the elapsed-time input reduces AUROC by 2.3–2.6 points on MC-MED (Table 11).

(3) Limited streaming beyond a fixed training context. Many sequence models are trained with bounded contexts and are not specifically designed to maintain useful long-term state as the recording continues to grow. In our experiments, several baselines show variable performance beyond the 500-window training context, whereas BioLT maintains continuously updated inference with fixed per-update computation and memory (Table 12).

Table 12. Streaming inference cost of BioLT (App. Table A.17; RTX 4090, batch size 1, median over repeated runs).

HistoryLatencyThroughputIncremental GPUFLOPs/update
0.5 h (180 windows)36.6 ms4,913 win/s3.47 MB386 M
2 h (720 windows)37.3 ms4,822 win/s3.62 MB398 M
8 h (2,880 windows)36.8 ms4,895 win/s3.62 MB398 M
32 h (11,520 windows)37.1 ms4,856 win/s3.62 MB398 M

BioLT is designed to address these three requirements together, while remaining agnostic to the window-level representation. It can therefore be placed on top of existing domain features or pretrained biosignal foundation models rather than replacing them.

Empirically, across the 21 dataset–feature settings, BioLT is significantly ahead of Transformer-XL and Mamba-3 at 5 of 6 evaluated horizons (Table 10, previous post). Its learned state also retains recoverable long-term information: a linear probe reconstructs the preceding chunk at R2=0.98R^2{=}0.98, compared with 0.86 for the Transformer-XL cache (Table 13).

Table 13. Memory probe, VitalDB ECG AF (H=20H{=}20), mean over 3 seeds (std ≤0.03 for R2R^2); 30–118 positive chunks per distance.

How far back30 min60 min90 min120 min180 min
R2R^2, mean features of the past chunk
BioLT memory0.980.770.660.520.30
Transformer-XL cache0.860.630.520.450.27
Snapshot (current window)0.500.440.410.370.30
AUROC, did an AF event occur in that chunk
BioLT memory0.680.680.680.660.70
Transformer-XL cache0.600.500.560.580.48
Snapshot (current window)0.520.490.480.600.48

We therefore do not argue that BioLT should replace every existing sequential model. Rather, it is most useful when biosignal monitoring requires the combination of long-term context, explicit handling of interrupted recordings, and continuously updated prediction over an extending stream.

AuthorsOpenReview ↗

Response to Reviewer s9Yq (3/3)

Q1. All inputs are still 10 s; why not try longer inputs?

(Un)fortunately, we believe this point reflects a misunderstanding, but one that can be resolved quickly by distinguishing token length from context length.

Each token represents one 10 s window, but the context extends over the whole recording. At test time, BioLT streams each recording from its first window to its last, reaching up to 5,907 windows (around 16.4 h) in the original experiments, while carrying the memory throughout.

During fine-tuning, the task loss is applied to segments of at most 500 windows, but the memory entering each segment is initialized from the preceding recording history. The 10 s token length is intentionally retained because BioLT is designed to operate on top of existing window-level feature extractors and foundation models, many of which use fixed short-window inputs.

Therefore, increasing the duration of the individual input window would test a different question: how much local waveform context should be given to the upstream encoder. Our question is instead how these established short-window representations can be modeled over hours-long streaming context. We will make this token/context distinction explicit in Section 3.

Q2. Performance without the autoregressive stage?

Table 14 reports the change in AUROC when AR pretraining is removed: same architecture and fine-tuning, random initialisation, task loss only, against BioLT with pretraining trained in the same code version.

Table 14. Effect of removing AR pretraining: Δ\Delta AUROC (points) relative to BioLT, means over 3 seeds.

DatasetHHw/o AR pretraining (Δ\Delta)
CHB-MIT0−0.8
CHB-MIT20−2.1
MC-MED ECG0−1.7
MC-MED ECG20−3.1
MC-MED PPG0−0.6
MC-MED PPG20−0.5
VitalDB ECG0−0.6
VitalDB ECG20−1.9

Without AR pretraining AUROC drops in all six AF and sepsis settings, by 0.5–3.1 points. It highights the importance of the autoregressive pretraining.

On the other hand, we did experiments to see whether the autoregressive pretraining did actually learn “trajectory dynamics”, rather than just learning to copy paste the last-token values. The results shown below demonstrates the effectiveness of our method compared to other trivial predictors.

Table 15. Trivial-predictor controls for the autoregressive objective. Validation MSE on the Δz\Delta\mathbf{z} target (lower is better); mean over 3 seeds.

PredictorCPSC-2021MC-MED ECGVitalDB ECG AF
Trained AR (ours)0.6633\mathbf{0.6633}0.5572\mathbf{0.5572}0.4211\mathbf{0.4211}
Always-zero (copy previous window)0.79790.70190.5583
Mean predictor0.79790.70190.5583
Persistence2.37652.08381.6396

Q3. What does “10 binary dataset settings” mean in Figure 6? Would you report the per-dataset Sens@1FA/h?

Thanks for your question. A setting is one detection task paired with one window representation. Figure 6 averages Sens@1FA/h over these 10:

  • CHB-MIT seizure detection (EEG): domain features, LaBraM, CBraMod (3);

  • VitalDB AF detection, ECG: domain features, ECGFounder, CLEF, CSFM (4);

  • VitalDB AF detection, PPG: domain features, PulsePPG, CSFM (3).

These tasks contain bounded positive episodes with both onset and offset, which allows an event to be classified as detected or missed. MC-MED sepsis is also binary, but its detection label remains positive after onset (sepsis, normally become persistent for a long while after onset) and therefore does not define bounded episodes under our current event-level evaluation. We consequently exclude MC-MED from Sens@1FA/h.

We agree that the original caption was not sufficiently explicit. We will define the 10 settings directly in Figure 6 and add the per-dataset results to the appendix:

Table 16. Sens@1FA/h at detection by dataset (number of feature streams in parentheses).

DatasetSnapshotLSTMTCNMamba-3TXLBioLT
CHB-MIT (3)0.1110.0180.0630.1810.2510.253
VitalDB ECG (4)0.3690.1170.2880.2640.2070.281
VitalDB PPG (3)0.3540.2490.3680.2790.3650.417
Figure 6 average (10)0.2870.1270.2440.2440.2670.314

The per-dataset table will be added to the appendix.

Q4. Figure 8: BioLT seems to decrease beyond 500 windows while Transformer-XL does not, yet the text says BioLT is strongest before and after.

Thanks for pointing this out. We will correct the wording. Both Transformer-XL and BioLT do not show a significant decline beyond the 500-window boundary, and the observed change remains within the seed variation.

Reviewer_xqVgOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces BioLT, a framework for long-context biosignal modeling that captures temporal dependencies across recordings. It is especially meant as an advance over previous methods that process individual 10-second windows independently. BioLT aggregates information across windows continuously, using a hierarchical Transformer approach that aggregates information into chunks and accross chunks. The authors evaluate bioLT on four ECG, PPG, and EEG datasets, where it outperforms baselines such as LSTMs, TCNs, Mamba-3, and Transformer-XL, achieving the best detection AUROC in 15/21 settings and 20-minute forecasting AUROC in 13/21. The authors also provide provided ablations to support the benefits of gap encoding and compressed memory.

Critical strengths
  1. The motivation is compelling and the paper is well motivated and well-written. Modeling long, irregular biosignal streams is highly relevant to continuous clinical monitoring, and the paper identifies meaningful limitations of snapshot-based approaches.
  2. I appreciate that the authors evaluate their approach across multiple datasets, signal modalities (ECG, PPG, EEG), and prediction tasks, including both detection and forecasting. They also consider different feature representations, including handcrafted features and embeddings from several foundation models. This makes the empirical evaluation quite comprehensive and provides some evidence that the proposed approach is not restricted to a particular biosignal modality or feature extractor.
Critical weaknesses
  • Limited novelty: The overall framework is interesting, but I find the methodological novelty somewhat limited. The main components, such as time-gap encoding, chunk-based attention, hierarchical encoding, an memory, are already well-established ideas in sequence modeling. The contribution seems to be mostly in combining these existing techniques for biosignals and using recent Transformers.. I believe similiar ideas have been explored in the past given TCNs or LSTMs.
  • Missing hierarchical baselines: BioLT effectively models temporal dependencies at two levels: local attention within chunks and compressed representations propagated across chunks. This immediately reminded me about hierarchical LSTMs.. While the paper compares against a standard LSTM, it does not evaluate this hierarchical alternative. I think this is a clear gap and makes the comparison rather unfair. I believe an hierarchical LSTM variant could potentially achieve similiar results while also providing an upper-level memory bound as the memory usage of the LSTM is constant. I'd suggest something like a lower-level reccurent encoder that processes individual windows, which are then processed by an "upper-level" LSTM able to maintain context across chunks. Even further probably this could be modelled better by TCNs.. One could stack TCNs for storing information across chunks. Or, one could use a TCN for doing a lower-level processing of one chunk and then the LSTM to dynamically keep track across chunks. Given less architectural complexity I could imagine those methods to work equally well, if not better.
Questions for authors
  1. Please clarify why you only compare against a standard LSTM rather than a hierarchical variant? Given that BioLT essentially dependencies at two temporal levels, comparing with a hierarchical LSTM would be more natural. What evidence suggests that the proposed compressed-memory mechanism is actually necessary compared to these simpler alternatives?
  2. Please clarify what the main methodological contribution is beyond combining existing techniques. In particular, how does the proposed memory mechanism fundamentally differ from established approaches to compressed memory and hierarchical sequence modeling?
4 comments
AuthorsOpenReview ↗

Response to Reviewer xqVg (1/2)

W1 / Q2. Limited novelty: what is the contribution beyond combining existing techniques, and how does the memory differ from established compressed-memory and hierarchical models?

Thanks. We appreciate that we are not proposing an entirely new architecture/framework for every sequential modeling problem, nor a “perfect” model that can be used indefinitely. Instead, the biosignal modeling community has largely focused on “snapshot” modeling, while how to extend this toward long-term modeling, especially for streaming biosignals, remains much less clear.

Our contribution instead lies in identifying the problem, adapting appropriate and domain-informed approaches to address it, and empirically validating both the need for these design choices and their effectiveness.

We will make these clearer in our revision.

What differs. Compared with Compressive Transformer families, the key differences are domain-inspired memory supervision. We validate the resulting memory design through both component ablations, different memory strategies, and direct memory probes.

Table 5. Ablation of core components (MC-MED sepsis forecasting, H=10H{=}10, AUROC %; revised Table 4). Δ\Delta is relative to BioLT.

ConfigurationWhat the model seesECGΔ\DeltaPPGΔ\Delta
BioLTcurrent chunk + 32-slot memory + Δs\Delta s60.29±0.64–63.19±0.48–
w/o Δs\Delta sno elapsed-time channel58.02±1.71−2.2760.61±1.43−2.58
w/o mem. bankcurrent chunk only (≤30 min)56.68±0.23−3.6157.83±0.34−5.36
w/o bothcurrent chunk only, no Δs\Delta s53.82±0.17−6.4753.74±0.95−9.45

Why these choices for streaming biosignals. The design is motivated by two practical yet challenging issues of continuous biosignal monitoring. First, recordings are interrupted: exposing the elapsed time between valid windows improves AUROC by 2.3–2.6 points on MC-MED (Table 4). Second, recordings are open-ended: the recurrent memory keeps per-update computation independent of accumulated history (Table A.17), while removing the memory reduces AUROC by 3.6–5.4 points, the largest component effect in Table 4.

Beyond the architecture, we systematically evaluate this formulation across four cohorts, three modalities, multiple window-level representations, detection and forecasting, causal inference over full recordings beyond the training length, and event-level alarm evaluation.

W2 / Q1. Missing hierarchical baselines: why compare only against a standard LSTM? A lower-level encoder per window with an upper-level LSTM (or TCN) across chunks.

Our LSTM baseline already sits on a per-window encoder: each 10 s window is encoded by domain features or a foundation model, and the LSTM runs over the window sequence. That is the “per-window encoder + upper LSTM” structure, but it is not the two-level temporal hierarchy that mirrors BioLT. We therefore added the chunk-level analogue, Hier-LSTM (Table 6). It uses BioLT’s 180-window chunks: a causal 2-layer LSTM inside each chunk, each chunk summarised by its last state and masked mean, and a 2-layer upper LSTM over chunk summaries.

The prediction at window tt of chunk nn uses the in-chunk state at tt and the upper state after chunk n−1n-1: strictly causal, constant memory per update, trained from scratch on the task loss.

Due to time limitations, we report here the detection (H=0H{=}0) comparison between Hier-LSTM and BioLT on domain features (Table 6). We will report the remaining forecasting results in the camera-ready version.

Table 6. Hier-LSTM (new) vs BioLT, detection (H=0H{=}0), domain features, AUROC (%), mean±std over 3 seeds; higher mean in bold.

DatasetHier-LSTMBioLT
CHB-MIT68.1±7.582.0±7.9
MC-MED ECG79.2±1.778.8±0.6
MC-MED PPG79.4±0.781.0±1.1
VitalDB ECG94.7±0.594.9±1.6
LTAFDB87.5±7.794.7±4.7

We appreciate the reviewer’s suggestion. At detection, introducing an explicit chunk hierarchy makes the LSTM baseline substantially stronger: on MC-MED, Hier-LSTM improves over the standard LSTM (Tables 2/3 of the paper) by 18–20 AUROC points. BioLT has the higher mean on 4 of the 5 datasets. The margin is largest on CHB-MIT (82.0 vs 68.1) and LTAFDB (94.7 vs 87.5), while the two models are close on MC-MED and VitalDB (differences of 0.2 to 1.6 points). We will therefore add Hier-LSTM to the main comparisons.

Beyond the shared hierarchical structure, the main difference is how cross-chunk history is represented: Hier-LSTM compresses the past into a single recurrent state, whereas BioLT maintains multiple addressable memory slots that are recurrently updated over time. To compare the two on substantially longer recordings, Table 6 also includes the MIT-BIH Long-Term AF Database (LTAFDB; 24 h Holter, 39 patients), as also reported in our response to Reviewer 1JJn.

AuthorsOpenReview ↗

Response to Reviewer xqVg (2/2)

Q1. What evidence suggests the compressed memory is necessary compared to these simpler alternatives?

Thanks. We follow the overall recurrent-memory design of Transformer-XL, as it is the pioneering work in this direction, and we do not intend to claim novelty for the idea of “memory” itself.

Instead, we would like to highlight how we redesign the memory mechanism and the overall framework to address the particular challenges of streaming biosignals, i.e., gap-aware decoding and physiology-inspired memory modeling. Our goal is not to design an entirely new architecture for biosignals, but to provide the community with a practical tool, together with guidance on what should be considered when modeling streaming biosignals.

We evaluate this from two complementary perspectives: (i) a foreground predictive task, asking whether the “memory” enabled by our proposed framework, improves the downstream prediction, and (ii) a background history-retention task, asking whether information from earlier history remains recoverable from the learned state.

(i) Foreground predictive task. On top of existing results in the original version that demonstrates the superior or comparable performance for streaming tasks, compared to other alternatives, we find that the distinction becomes clearer when substantially longer history is available. On the added 24 h AF-onset forecasting task, BioLT outperforms the other methods across various forecasting horizons (Table 7). This suggests that the richer cross-chunk memory becomes useful when the relevant context extends much further into the past.

Table 7. LTAFDB (MIT-BIH Long-Term AF Database, 24 h), domain features: AF detection (H=0H{=}0) and AF-onset forecasting HH minutes ahead, AUROC (%), mean±std over 3 seeds; best in bold, second best in italics.

HH (min)SnapshotLSTMTCNHier-LSTMMamba-3TXLBioLT
087.9±8.489.9±2.087.3±12.087.5±7.793.2±1.093.8±4.294.7±4.7
2047.8±0.654.0±8.256.1±5.951.8±6.059.7±2.260.9±4.062.8±2.7
3047.5±0.358.4±4.155.1±5.553.3±8.659.5±2.159.6±3.460.4±3.5

(ii) Background history-retention task. Downstream performance alone does not tell us whether long-term information is actually retained. We therefore directly probe the frozen state at chunk kk, using a linear readout to reconstruct the mean features of chunk k−ak-a and to predict whether an AF event occurred in that past chunk.

The probe shows that BioLT retains more recoverable information about both past physiological features and past AF events than Transformer-XL, with the difference persisting well beyond the immediately preceding chunk. For example, past AF occurrence remains recoverable at 180 min (AUROC 0.70), whereas the Transformer-XL cache is near chance (0.48).

Together, the two analyses provide complementary evidence: the foreground experiment tests whether longer-term memory improves prediction when long history is available, while the background probe tests whether that history is actually preserved in the learned state. We therefore do not claim that compressed memory is necessary for every task; rather, the results suggest that its advantage becomes clearer when relevant physiological information extends over longer temporal contexts. We will make it clearer in the revised version.

Table 8. Memory probe, VitalDB ECG AF (H=20H{=}20), mean over 3 seeds (std ≤0.03 for R2R^2); 30–118 positive chunks per distance.

How far back30 min60 min90 min120 min180 min
R2R^2, mean features of the past chunk
BioLT memory0.980.770.660.520.30
Transformer-XL cache0.860.630.520.450.27
Snapshot (current window)0.500.440.410.370.30
AUROC, did an AF event occur in that chunk
BioLT memory0.680.680.680.660.70
Transformer-XL cache0.600.500.560.580.48
Snapshot (current window)0.520.490.480.600.48

Thank you very much for your detailed response. Several of my concerns have been addressed, and I appreciate the additional experiments and ablations. I will raise my score to 3.

Many thanks for your positive comments, much appreciated!

Reviewer_1JJnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a deep learning framework called BioLT (Biosignal Long-Context Transformer), which is specifically designed for analyzing physiological signals collected over a Long period of time and continuously, including electrocardiogram (ECG), photovolumetric pulse wave (PPG), and electroencephalogram (EEG). Its core objective is to enable AI models that could only analyze physiological signals for a few seconds to continuously monitor patients' conditions for several hours, understand long-term physiological changes, and predict disease risks in advance. The traditional model mainly answers: Did this 10-second ECG show any arrhythmia? And BioLT hopes to further answer: Based on the patient's physiological changes over the past few hours, has arrhythmia occurred now? Is there a possibility of any abnormality in the next 20 minutes? The innovation focus of this paper is not to design a better ECG feature extractor, but to endow the existing physiological signal feature extraction models with long-term memory and flow prediction capabilities.

Critical strengths

Strength 1: Transform the short-window physiological signal model into a long-time flow model. It does not require training a new ECG or EEG Foundation Model from scratch. Instead, it can add temporal modeling capabilities on top of the existing feature extractor. Strength 2: Use the signal missing time as an explicit input. BioLT does not merely attempt to fix missing data; instead, it models irregular sampling using the real time intervals between Windows. Strength 3: Supervising compressed memory with physiological statistics. By reconstructing losses through mean, standard deviation, trend and dynamic range, it helps the Memory Bank with limited capacity retain long-term historical characteristics. This is a rather distinctive design in the thesis.

Critical weaknesses

Weakness 1: Long-term experiments are still limited. The paper demonstrates that the model can handle records that exceed the length of the training window and also provides an efficiency test under a 32-hour history. However, this does not mean that the effectiveness of continuous monitoring in real hospitals for several weeks has been verified. Weakness 2: The efficiency of streaming reasoning is different from that of a complete system. The efficiency test of BioLT mainly focuses on the Model calculation for processing new chunks and cannot directly represent the entire system latency including the acquisition of original signals, preprocessing, and feature extraction of the Foundation Model.

Questions for authors

BioLT combines block-based causal attention, learnable memory queries, and FIFO memory banks. Can the author clarify the key methodological differences between BioLT and existing memory enhancement Transformers (especially Transformer-XL)? How much of the performance improvement is specifically attributed to physiologically inspired memory supervision rather than the memory architecture itself?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 1JJn (1/2)

Thank you for your valuable comments.

W1. Long-term experiments are still limited; weeks of hospital monitoring are not verified.

Thanks, our original experiments were conducted across four real-world data, of which however as you mentioned, the longest recordings are 16.4 h (VitalDB).

To verify the effectiveness of “very long” continuous monitoring, we add the MIT-BIH Long-Term AF Database (24 h Holter, 39 patients): AF detection (H=0H{=}0) and forecasting AF onset H=20H{=}20 and 3030 minutes ahead, same protocol as in the original paper, and domain features as the window-level feature.

We also include a hierarchical two-level LSTM (Hier-LSTM), following reviewer xqVg’s suggestion, as an additional long-sequence baseline. (W1/Q2).

Table 1. LTAFDB (MIT-BIH Long-Term AF Database, 24 h), domain features: AF detection (H=0H{=}0) and AF-onset forecasting HH minutes ahead, AUROC (%), mean±std over 3 seeds; best in bold, second best in italics.

HH (min)SnapshotLSTMTCNHier-LSTMMamba-3TXLBioLT
087.9±8.489.9±2.087.3±12.087.5±7.793.2±1.093.8±4.294.7±4.7
2047.8±0.654.0±8.256.1±5.951.8±6.059.7±2.260.9±4.062.8±2.7
3047.5±0.358.4±4.155.1±5.553.3±8.659.5±2.159.6±3.460.4±3.5

BioLT has the highest mean at both forecasting horizons. Snapshot is at chance there, and the three models with cross-chunk memory (Mamba-3, Transformer-XL, BioLT) lead LSTM, TCN and Hier-LSTM.

On the claims of our performance, we will restrict the accuracy claims to continuous recordings of up to about 24 hours, and restrict our claims to continuous recordings of up to approximately 24 h, rather than referring broadly to streams lasting “hours or days”.

We will also state explicitly in the Limitations that multi-day and weeks-long monitoring remains untested, where additional issues such as changes in sensor placement, patient state, and care context may become important.

Nevertheless, we view the present work as a “proof of concept” for extending established window-level biosignal representations to continuous streaming modelling. Validation over multi-day and weeks-long retrospective validation or prospective monitoring is an important next step as suitable longitudinal datasets become available.

W2. Model latency is not system latency.

Correct; thanks for this, we will separate them.

BioLT is designed as a temporal backbone on top of existing window-level representations, rather than as a raw-waveform encoder. The online pipeline therefore consists of two computational stages:

_{1:t}\rightarrow \hat{y} _t} _{\text{temporal backbone}}$$ For comparing temporal models, the relevant quantity is the second stage: all temporal baselines and BioLT receive the same pre-extracted window representations, so the window-encoding cost is shared and does not affect the comparison between temporal backbones. Table A.17 reports this incremental temporal-model latency and shows that the cost of a BioLT update remains approximately constant as recording history grows. For completeness, we additionally measure the first stage for every window representation used in the paper. Feature extraction is independent of history length because each 10 s window is encoded separately: **Table 2. Window-encoder cost per 10 s window.** |Window encoder|Device|Params|ms / window| |:---|:---:|:---:|:---:| |Domain features, ECG|CPU|–|31.1| |Domain features, PPG|CPU|–|200.9| |Domain features, EEG|CPU|–|637.9| |CSFM (ECG / PPG)|GPU|108.6M|2.4 / 2.5| |ECGFounder|GPU|30.7M|4.8| |CLEF|GPU|30.7M|4.7| |PulsePPG|GPU|28.5M|2.0| |LaBraM|GPU|5.8M|2.6| |CBraMod|GPU|4.9M|3.1| Thus, the total computational processing time of a BioLT pipeline is the cost of the chosen window encoder plus the BioLT update. Even with the slowest encoder evaluated here (EEG domain features, 0.64 s), the combined processing time remains below 0.7 s per 10 s window. We will revise the paper to distinguish *window-encoding latency*, *temporal-backbone latency*, and their sum as *computational processing latency*. We will not refer to these measurements as full system latency, since acquisition, transmission, buffering, and device-specific processing depend on the deployment environment.
AuthorsOpenReview ↗

Response to Reviewer 1JJn (2/2)

Q. Key differences from Transformer-XL? How much of the gain is due to the physiology-inspired memory supervision rather than the memory architecture?

Thanks. We follow the overall recurrent-memory design of Transformer-XL, as it is the pioneering work in this direction, and we do not intend to claim novelty for the idea of “memory” itself.

Instead, we would like to highlight how we redesign the memory mechanism and the overall framework to address the particular challenges of streaming biosignals, i.e., gap-aware decoding and physiology-inspired memory modeling. Our goal is not to design an entirely new architecture for biosignals, but to provide the community with a practical tool, together with guidance on what should be considered when modeling streaming biosignals.

As the reviewer suggests, one important and the major difference from Transformer-XL lies in how the memory is updated, particularly the physiology-inspired memory supervision. We therefore evaluate its effectiveness from two aspects: (1) downstream modeling performance, Table 3, and (2) whether the learned memory actually retains long-term physiological information, Table 4.

(i) Forward predictive downstream performance. In most settings, BioLT performs better (and most statistically better) than Transformer-XL and other memory-based sequential models (Table 3). This shows that the overall BioLT framework provides stronger predictive performance when used for downstream detection and forecasting.

(ii) Backward history retention. Downstream predictive performance does not necessarily tell us whether historical information is actually retained in memory. We therefore directly probe the frozen memory states and ask whether information from earlier chunks can still be recovered. Compared with Transformer-XL, the BioLT memory retains substantially more information about both past physiological features and past AF events over longer temporal distances (Table 4).

Table 3 is a two-sided paired Wilcoxon test over the 21 dataset–feature settings (three-seed means), as in App. Table A.16. Entry: median AUROC difference (BioLT − baseline) and pp; bold: p<0.05p<0.05.

Table 3. BioLT minus baseline, median over 21 settings (pp).

HH (min)Transformer-XLMamba-3
0+0.62 (0.001)+2.54 ($<$0.001)
1+0.27 (0.43)+0.53 (0.10)
5+0.39 (0.029)+1.15 (0.019)
10+1.52 (0.001)+0.91 (0.014)
20+1.76 ($<$0.001)+2.22 (<\0.001)
30+1.53 ($<$0.001)+1.42 (0.001)

Table 4 is a linear readout from the frozen memory at chunk kk: it predicts the mean features of chunk X-min ahead, and whether an AF event occurred in it.

Table 4. Memory probe, VitalDB ECG AF (H=20H{=}20), mean over 3 seeds (std ≤0.03 for R2R^2); 30–118 positive chunks per distance.

How far back30 min60 min90 min120 min180 min
R2R^2, mean features of the past chunk
BioLT memory0.980.770.660.520.30
Transformer-XL cache0.860.630.520.450.27
Snapshot (current window)0.500.440.410.370.30
AUROC, did an AF event occur in that chunk
BioLT memory0.680.680.680.660.70
Transformer-XL cache0.600.500.560.580.48
Snapshot (current window)0.520.490.480.600.48

Thanks for your addtional experiment results. I will raise my score to 3.

Thank you very much, we are grateful for the feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.