acceptodds
Under review as a conference paper at ICLR 2027

Learning to Access Latent Memory for ECG Restoration under Distribution Shift

Abstract

Electrocardiogram (ECG) denoising is commonly learned through direct reconstruction from corrupted inputs to clean targets, which can couple restoration to training-specific signal-corruption relationships and lead to substantial degradation when signal or corruption characteristics shift at test time. To address this challenge, we treat clean ECGs not only as reconstruction targets, but also as a source of reusable latent structure that can be learned and retained independently of the corrupted-input mapping. We introduce a two-stage framework that first curates this clean structure in a key-value memory through clean-signal reconstruction, and then uses Memory Access Alignment (MAA) to align corrupted observations with the retained memory for reconstruction. During alignment, only the encoder bottleneck and memory router are adapted, while the curated memory and decoder remain frozen. Across multiple ECG datasets and both convolutional and Transformer-based backbones, the proposed framework consistently improves reconstruction under shifts in signal and corruption characteristics. Mechanistic analyses further show that closer agreement with the memory-access patterns induced by clean signals does not necessarily correspond to better reconstruction, while decoder-sensitive readout changes account for most of the improvement or deterioration induced by adaptation. These findings support learning effective access to retained clean structure as a strategy for robust restoration under distribution shift. Our code is available at https://anonymous.4open.science/r/MAA_ECG_Denoising-A0D4/README.md.

Primary area:
learning on time series and dynamical systems
Keywords:
ECG restoration, distribution shift, out-of-distribution generalization, representation learning, memory-augmented neural networks, latent alignment, robust signal processing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Qi4AOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a two-stage framework for ECG denoising under distribution shift. First, a memory is learned from clean ECGs only, and the decoder can reconstruct the signal only through this memory. Then, with memory and decoder frozen, the encoder bottleneck and the router are adapted so that noisy inputs access the memory as the corresponding clean inputs would (Memory Access Alignment).

Critical strengths

The overall idea of the paper is interesting, in particular, the separation between what the model knows about clean ECGs (stored in a memory curated from clean signals only) and how corrupted inputs access this knowledge. In principle, this method could be applied to other physiological signals and other restoration tasks.

Critical weaknesses
  1. Clarity. The description of the memory-curation stage is imprecise, which makes the method difficult to understand. The notation is overloaded: kk denotes both the number of retrieved slots and the key dimension, TT both the signal length and the teacher, and NN both the number of slots and the noisy input. The temporal structure of the latent representation is also unclear. Several other essential details are missing, including the dimension of z_T, the router architecture, which ECG lead is used, and how the memory is initialized and trained through the top-k selection. Finally, the claims that the memory “curates” or “reflects clean ECG structure” are not supported by any analysis of the learned slots. Moreover, in the tables (which also lack a uniform style), values are highlighted in different colors without any guidance.

  2. Definition of the OOD setting. The paper never clearly specifies what “out-of-distribution” refers to: a shift in the noise, in the population, or in the acquisition setting. On QTDB, the noise comes from the same NSTDB recordings used in training (only the channel differs). The shift is therefore mainly in population and acquisition, not in corruption. On SIMEMG, population, acquisition device, and contamination change simultaneously, and the EMG contamination is similar to the muscle-artifact noise already seen during training. As a result, the claim that MAA improves robustness to “shifts in signal and corruption characteristics” cannot be verified. It remains unclear which type of shift the method actually addresses.

  3. Unclear comparison with previous approaches. The work is not clearly positioned and it is unclear where it stands in comparison to the baselines.

Questions for authors
  1. Figure 1: The synthetic noise applied to MITDB and QTDB datasets is dependent on the ECG itself since it is scaled relative to each window's ECG power. Therefore, ECGs with different amplitudes receive different absolute amounts of noise. Moreover, noise and ECGs also share frequency bands, so how separable they are depends on the ECGs themselves. Moreover, MITDB and QTDB differ in sampling rate, acquisition settings, and clinical composition. In short, Figure 1 is presented as isolating a shift in the clean ECG under a shared noise source, but it actually mixes together different sources of variation between the datasets. Which of these factors explains the degradation?

  2. Statistical significance: It is not clear whether the reported improvements are statistically significant. Several of the claimed gains fall within one standard deviation. Moreover, in Table 3, frozen access with the LUNet backbone (Stage 1 only, no adaptation) achieves an RMSE of 4.5787 and a CC of 0.9785 on SIMEMG. This outperforms LUNet + MAA and every baseline in Table 1. In other words, on the only dataset with real contamination, the proposed Stage 2 degrades performance. Nevertheless, frozen access is not included in Table 1, and Section 3.1 states that LUNet + MAA “achieves the best mean SIMEMG performance across all four metrics among the evaluated methods”. Could the authors include frozen access in Table 1, revise this claim accordingly, and explain why memory-access alignment hurts reconstruction under real contamination?

  3. Disentangling the source of the shift: The evaluation mixes several factors: population, acquisition methods, noise type, and noise level. In the current state, it is impossible to determine what the document is robust to. Moreover, although the test sets include several SNR levels and artifact types, the reported results are averaged over all of them. An ablation study should vary one factor at a time:

  • Varying the noise only, same population, but different noise between training and test (e.g., training on baseline wander and testing on electrode motion).
  • Different populations for training and test, with the same noise configuration (same artifact type and SNR).
  • Results reported separately for each SNR level and artifact type.
13 comments
AuthorsOpenReview ↗

Response to Question 1: Clarification of Figure 1, and Additional Cross-Dataset Degradation Analysis of Figure 1

We thank the reviewer for the constructive feedback.

Clarification of Figure 1

We would like to clarify that Figure 1 aims to illustrate the cross-database generalization limitations of existing ECG denoisers. We acknowledge that comparing MITDB-trained and QTDB-trained models on identical QTDB test recordings under a shared NSTDB-based corruption protocol does not isolate differences in clean ECG characteristics. Instead, it reflects the composite distribution shifts that may occur in practical deployment, where patient populations, ECG morphology, acquisition conditions, and noise characteristics may vary simultaneously. To further investigate the observed degradation, we explore cross-dataset noise characteristics and examine amplitude-normalized reconstruction errors and performance gaps across ECG amplitude, SNR, spectral overlap, and artifact types. For the additional experiments, we retain the same MITDB training, validation, and test splits as in the main experiments. The 90 QTDB recordings used for zero-shot OOD evaluation are further partitioned at the recording level into training, validation, and test subsets, with 15 recordings held out for testing. LUNet and ImputeECG are trained separately on MITDB and QTDB and evaluated on identical corrupted QTDB test inputs across three seeds (42, 43, and 44).

Noise characterization across datasets

We analyzed noise characteristics across the MITDB, QTDB, and SIMEMG test sets. MITDB and QTDB exhibit nearly identical input SNR distributions (7.50 ± 5.59 dB), high-frequency noise power fractions (15.90% vs. 15.88%), and envelope coefficients of variation (0.5907 vs. 0.5901), but differ substantially in mean noise RMS (0.2290 vs. 1.5326). This indicates that the shared synthetic-noise protocol controls relative corruption severity, while signal-dependent scaling yields different absolute noise amplitudes across datasets. In contrast, SIMEMG exhibits a distinct contamination profile, particularly a substantially higher high-frequency power fraction (60.97%).

Cross-database degradation after amplitude-normalized evaluation

We compared MITDB-trained and QTDB-trained LUNet and ImputeECG models on the same 15 held-out QTDB recordings. Both models exhibit higher reconstruction errors when trained on MITDB, with paired RMSE differences of 0.2626 (95% CI: [0.0822, 0.4953]) for LUNet and 0.2054 (95% CI: [0.0597, 0.3954]) for ImputeECG. Importantly, the performance gap remains when reconstruction errors are measured relative to clean ECG amplitude using PRD, with differences of 5.05 percentage points (95% CI: [3.28, 6.87]) and 3.38 percentage points (95% CI: [1.78, 5.04]), respectively. The PRD gap also increases across clean ECG amplitude quartiles for both architectures. Thus, the degradation is not merely due to measuring absolute errors, although amplitude differences cannot be established as its cause.

Cross-database degradation across corruption characteristics

We further examined performance gaps across ECG–noise spectral overlap, input SNR, and artifact types using the same QTDB test set and three training seeds. Both architectures exhibit positive mean RMSE gaps across all evaluated noise types and SNR levels. The gaps increase under stronger corruption, from 0.0796 at 15 dB to 0.5310 at 0 dB for LUNet, and from 0.0566 to 0.4215 for ImputeECG. Degradation also persists across all evaluated artifact categories, with the largest gaps under baseline wander for LUNet (0.3595) and electrode motion for ImputeECG (0.2362). In contrast, the performance gap does not exhibit a consistent increasing relationship with ECG–noise spectral overlap. These results indicate that cross-database degradation is not restricted to a particular noise type or severity level, while stronger corruption is associated with larger performance gaps.

These analyses show that MITDB and QTDB share similar relative noise characteristics but differ in absolute noise amplitudes, while SIMEMG exhibits a different contamination profile. Cross-database degradation persists under amplitude-relative error evaluation and across the tested noise types and SNR levels, with larger absolute RMSE gaps under stronger corruption and no consistent relationship with spectral overlap. These findings clarify that QTDB represents a composite cross-database shift under a shared synthetic-noise protocol, whereas SIMEMG additionally involves naturally acquired contamination. Although SIMEMG shares characteristics with NSTDB muscle artifact, its higher high-frequency power fraction distinguishes it from the aggregate synthetic-noise test distribution, not necessarily from muscle artifact alone. These analyses do not isolate the individual causal contributions of ECG morphology, acquisition conditions, and signal-dependent noise scaling. We have revised the Figure 1 caption and accompanying discussion to clarify this interpretation, with additional analyses provided in Appendix A.15.

AuthorsOpenReview ↗

Response to Question 2: Frozen access missing from Table 1 and MAA under real contamination and degradation analysis.

Frozen access missing from Table 1

We appreciate the reviewer's observation. We acknowledge that LUNet with frozen memory access outperforms LUNet + MAA on SIMEMG, as explicitly reported in Table 3. We would like to clarify the objectives of Table 1. Table 1 is designed to compare the complete MAA framework against the original backbones and existing ECG denoising methods, whereas Table 3 presents an ablation study of internal framework configurations to isolate the contribution of Stage 2 memory-access alignment. Accordingly, frozen memory access is reported in Table 3 rather than Table 1, reflecting the distinct evaluation objectives of these two tables. The statement in Section 3.1 that LUNet + MAA achieves the best mean SIMEMG performance across all four metrics refers specifically to the methods evaluated in Table 1, rather than all configurations considered in the ablation study. It does not imply that Stage 2 consistently outperforms frozen memory access. Nevertheless, to avoid ambiguity, we will clarify this distinction in Section 3.1.

Memory-access alignment under real contamination and degradation analysis

We agree that Stage 2 alignment does not universally improve frozen memory access under real contamination. On SIMEMG, alignment improves ImputeECG reconstruction but deteriorates LUNet performance. To investigate potential factors contributing to this difference, we examined the SIMEMG acquisition protocol and conducted a controlled experiment (Appendix A.17).

SIMEMG dataset characteristics: Unlike synthetic contamination, where noise is added to the same clean ECG waveform, SIMEMG records noisy ECGs from wrist/finger electrodes and simultaneous cleaner references from the shoulder/upper-arm region. The acquisition method assumes similar cardiac potentials along relaxed arms, with EMG contamination generated primarily in the hands (Atanasoski et al., 2023). During dataset preparation, reference recordings with excessive estimated EMG contamination were rejected; nevertheless, small residual EMG noise may remain in retained references and affect denoising-method comparisons.

Differences in electrode placement and potential residual contamination in SIMEMG references may contribute to LUNet's deterioration, since Stage 2 is trained on noisy–clean pairs sharing the same underlying ECG waveform. We therefore designed a controlled experiment to examine whether SIMEMG-derived contamination reproduces the deterioration when noisy inputs and clean targets share the same underlying waveform, using both LUNet and ImputeECG.

Controlled SIMEMG-derived contamination versus real SIMEMG: We extracted residuals from paired SIMEMG recordings, scaled them relative to clean QTDB ECGs, and added them to QTDB signals while retaining the original clean targets. We refer to this evaluation setting as QTDB with additive SIMEMG-derived residuals (controlled). For each backbone, we evaluated frozen-access and MAA models using their respective unchanged weights on both the controlled setting and real SIMEMG, without retraining or adaptation. The results are summarized below.

SettingBackboneFrozen RMSE ↓MAA RMSE ↓Frozen CC ↑MAA CC ↑
QTDB with additive SIMEMG-derived residuals (controlled)LUNet0.27170.22580.77040.7888
QTDB with additive SIMEMG-derived residuals (controlled)ImputeECG0.66460.40880.67160.6767
Real SIMEMGLUNet4.57875.28830.97850.9730
Real SIMEMGImputeECG8.29045.84330.92750.9602

Under controlled QTDB contamination, Stage 2 alignment reduces LUNet RMSE by 16.89% and improves CC by 0.0184 relative to frozen access. In contrast, on real SIMEMG, alignment increases LUNet RMSE by 15.50% and decreases CC by 0.0055. ImputeECG benefits under both settings, reducing RMSE by 38.49% and 29.52%, respectively.

These results show that LUNet's deterioration on real SIMEMG is not reproduced by transferring SIMEMG-derived residuals to clean QTDB signals. Thus, SIMEMG-derived residuals do not necessarily cause harmful alignment under the evaluated additive contamination setting. Moreover, this contrast is consistent with the possibility that differences in acquisition conditions and noisy–reference signal correspondence influence alignment effectiveness. However, because the controlled and real settings also differ in their underlying ECG signals and reference measurements, the experiment does not isolate the specific factor responsible for the contrasting outcomes.

Reference: Atanasoski, V., et al. (2023). A Database of Simultaneously Recorded ECG Signals With and Without EMG Noise. IEEE Open Journal of Engineering in Medicine and Biology, 4, 222–225. https://doi.org/10.1109/OJEMB.2023.3330295

AuthorsOpenReview ↗

Response to Weakness 2: Definition of the OOD setting

Definition of the OOD setting.

We thank the reviewer for highlighting the need to distinguish different sources of distribution shift. We agree that population, acquisition, and corruption shifts should not be treated interchangeably. We clarify that our evaluation considers two distinct OOD settings:

  1. Cross-database ECG shift (QTDB). Models are trained on MITDB and evaluated on QTDB, which differs in recording cohort and acquisition characteristics. Both datasets employ the same NSTDB-based synthetic contamination protocol, using different noise channels for training and evaluation. Thus, QTDB evaluates cross-database ECG generalization under a shared corruption-generation protocol, rather than generalization to an entirely unseen noise source. Stage 2 MAA improves over frozen memory access across all seven evaluated synthetic noise conditions for both backbones (Appendix A15). However, these results do not independently isolate the effects of population and acquisition differences.

  2. Composite real-world shift (SIMEMG). SIMEMG differs from the source setting in recording population, acquisition configuration, and naturally occurring contamination. Although its EMG contamination belongs to an artifact category represented during source training, the measured contamination characteristics differ substantially from those of the synthetic source noise. Thus, SIMEMG evaluates generalization under a composite real-world shift, rather than an entirely unseen artifact category. Under this setting, Stage 2 exhibits backbone-dependent behavior: it improves ImputeECG but degrades LUNet relative to frozen memory access. We acknowledge this limitation and investigate its reconstruction behavior in Appendix A17.

Scope of generalization. Our results demonstrate that MAA improves reconstruction on QTDB under a shared synthetic contamination protocol and across the evaluated synthetic noise conditions. They do not establish universal robustness to unseen corruption or independently isolate population, acquisition, and noise effects. Appendix A16 additionally evaluates cross-noise transfer during Stage 2 alignment. We will revise the OOD definitions and generalization claims throughout the manuscript to reflect these distinctions.

Reviewer_hALFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents ECG denoising under distribution shift. The main idea is based on a dictionary learning approach consisting of two stages: 1) memory curation, in which a dictionary of clean ECG representations is learnt in order to capture informative signal patterns, and 2) memory access alignment, in which retrieval is adapted using noisy ECGs in order to effectively access the fixed dictionary without modifying its contents. During the inference stage, a noisy ECG is encoded to query the dictionary and the retrieved entries are passed to a learned decoder to reconstruct the clean ECG. The approach is trained on MITDB with NSTDB noise and tested zero-shot on QTDB and SIMEMG (real EMG contamination) using LUNet and ImputeECG as backbones. It shows improvements compared to the baselines.

Critical strengths
  • Consistent improvement over both backbones under the tested distribution shifts, on all three ECG datasets and all four metrics (Table 1).
  • The paper presents an interesting method of separating the learning of what information to store from the learning of how to retrieve and use it for denoising.
  • The method is clear and easy to follow.
Critical weaknesses
  • The approach requires paired training data i.e., noisy and corresponding clean training samples. That is a clear limitation.
  • There is a risk of suppressing clinically meaningful features. Memory optimised for reconstructing common, clean ECG patterns may not preserve unusual but clinically relevant waveforms. Uncommon arrhythmias, for example, might be oversmoothed or reconstructed inaccurately. Neither the retrieval nor spectral matching can guarantee that these will be preserved.
  • MITDB and QTDB use synthetic corruption from NSTDB. It his thus difficult to draw practical conclusions.
  • A reconstruction could have a low overall error while distorting a diagnostically important feature. The paper tests several relevant morphological measures (morphological fidelity), where the proposed method does not uniformly improve performance. It shows mixed results.
  • Ablation demonstrates that the model depends on its memory (no retraining). The only memory-free reference is “LUNet (w/o skip control)” in Table 1. MAA improves on it, but the row is not discussed in the paper and exists for LUNet only. Removing the skip connections alone already lowers QTDB RMSE so part of the reported gain comes from this bottleneck. Two-stage training with the same alignment but without the memory is not teste. A fairer comparison for that claim would include a model trained without memory under the same data, architecture, and training budget.
  • The approach in general requires two stages, a teacher–student setup, multiple losses, and hyper-parameter tuning. The training process is complex. It might be heavily tuned to the given datasets but hard to get working in practice.
  • The paper is mostly fine written but there is some space to improve the clarity, especially in the experimental part.
Questions for authors
  • Does it generalise to unseen noise? Results on SIMEMG look promising, but it would be interesting to see if gains remain statistically reliable across independent subjects and noise conditions etc.
  • What exactly is “LUNet (w/o skip control)” in Table 1? How does the same two-stage alignment perform without the memory, for both LUNet and ImputeECG?
  • How robust is the proposed method to changes in the hyper-parameters?
  • Why is frozen access missing from Table 1. Also what does memory access alignment contribute under real contamination (given that it worsens the LUNet result on SIMEMG)?
9 comments
AuthorsOpenReview ↗

Response to Weakness 5 and Questions 2 and 4: LUNet Skip-Connection Controls, Memory-Free Two-Stage Adaptation, and Frozen-Access Evaluation.

We thank the reviewer for the feedbacks.

Clarification of LUNet and skip connections

The original LUNet employs multi-resolution encoder–decoder skip connections and a direct bottleneck connection. The LUNet (w/o skip control) baseline removes the multi-resolution skip connections while preserving the direct bottleneck connection. The proposed MAA framework is designed to replace the direct bottleneck connection with key–value memory retrieval. Therefore, LUNet (w/o skip control) serves as the appropriate architectural baseline for LUNet + MAA. We include this baseline in Table 1 to show how much improvement comes from adding MAA rather than simply removing skip connections. Since this configuration differs from the original LUNet architecture, we also report the original LUNet results for comparison with the standard backbone. Furthermore, all additional memory-free and memory-based ablations use the same skip-free architecture to ensure a fair comparison. We have clarified these architectural differences and the purpose of each baseline in the revised manuscript.

Two-stage adaptation without memory

To evaluate the contribution of the memory, we conduct additional memory-based versus memory-free adaptation experiments using LUNet and ImputeECG across MITDB (ID), QTDB (OOD), and SIMEMG (OOD). In addition to the original backbones, we evaluate six configurations: 1. Memory-free frozen access, where the clean-pretrained encoder–decoder is evaluated directly on noisy test inputs without Stage 2 adaptation; 2. Memory-free encoder adaptation; 3. Memory-free decoder tuning; 4. Memory-free joint encoder–decoder tuning; 5. Memory with frozen access, where the clean-pretrained memory-based model is evaluated without Stage 2 adaptation; and 6. The complete MAA framework with learned memory access alignment.

In Stage 1, the memory-free encoder–decoder is pretrained directly on clean ECG signals, with encoder representations passed directly to the decoder without memory retrieval. During Stage 2, memory-free encoder adaptation and MAA follow the same frozen-decoder strategy, with each model retaining its respective Stage 1 pretrained decoder. Since the memory-dependent readout alignment loss requires retrieved memory representations, it cannot be directly applied to the memory-free model. We therefore retain the applicable layerwise latent alignment objectives, with equal weights of 1 across the included layers. Memory-free encoder adaptation versus MAA serves as the primary controlled comparison, while decoder-only and joint encoder–decoder tuning provide additional controls. All experiments follow the same data splits, corruption protocols, and training epochs, with learning rates tuned separately on the validation set.

Experimental results: MAA achieves lower RMSE than the evaluated memory-free Stage 2 adaptation variants across both backbones and all three datasets. Compared with memory-free encoder adaptation, MAA reduces RMSE from 0.1240 to 0.1214 (2.1%) on MITDB, 0.5822 to 0.5631 (3.3%) on QTDB, and 5.8892 to 5.2883 (10.2%) on SIMEMG with LUNet. For ImputeECG, the corresponding reductions are from 0.1338 to 0.1225 (8.4%) on MITDB, 0.6994 to 0.5595 (20.0%) on QTDB, and 7.3312 to 5.8433 (20.3%) on SIMEMG.

The additional frozen-access comparisons show that memory-based and memory-free configurations achieve similar RMSE on MITDB and QTDB, suggesting that incorporating memory alone does not consistently improve reconstruction under these conditions. However, on SIMEMG with LUNet, memory-free frozen access (RMSE 4.5247) and memory-based frozen access (4.5787) outperform MAA (5.2883). In contrast, ImputeECG benefits from alignment, reducing RMSE from 8.2904 under memory-based frozen access to 5.8433 with MAA. These findings indicate that the benefit of access alignment depends on the backbone and corruption conditions.

Overall, the complete MAA framework outperforms the evaluated memory-free two-stage adaptation strategies across both backbones and all three datasets. Nevertheless, Stage 2 alignment is not uniformly beneficial compared with frozen access under external corruption shifts. The complete comparisons are provided in Appendix A.14 (Table A11), with results reported as mean ± standard deviation across three random seeds (42, 43, and 44).

Frozen access missing from Table 1

We acknowledge that LUNet with frozen memory access outperforms LUNet + MAA on SIMEMG, as reported in Table 3. However, Table 1 focuses on comparing the complete MAA framework with the original backbones and existing ECG denoising methods, whereas memory-based frozen access is evaluated separately in the ablation study (Table 3) to isolate the contribution of Stage 2 alignment. Therefore, the statement in Section 3.1 that LUNet + MAA achieves the best mean SIMEMG performance across all four metrics refers specifically to the methods evaluated in Table 1, rather than the internal ablation configurations.

AuthorsOpenReview ↗

Response to Question 4: Memory-Access Alignment Contribution under Real Contamination and Degradation Analysis

We agree that Stage 2 alignment does not universally improve frozen memory access under real contamination. On SIMEMG, alignment improves ImputeECG reconstruction but deteriorates LUNet performance. To investigate potential factors contributing to this difference, we examined the SIMEMG acquisition protocol and conducted a controlled experiment (Appendix A.17).

SIMEMG dataset characteristics

Unlike synthetic contamination, where noise is added to the same clean ECG waveform, SIMEMG records noisy ECGs from wrist/finger electrodes and simultaneous cleaner references from the shoulder/upper-arm region. The acquisition method assumes similar cardiac potentials along relaxed arms, with EMG contamination generated primarily in the hands (Atanasoski et al., 2023). During dataset preparation, reference recordings with excessive estimated EMG contamination were rejected; nevertheless, small residual EMG noise may remain in retained references and affect denoising-method comparisons.

Differences in electrode placement and potential residual contamination in SIMEMG references may contribute to LUNet's deterioration, since Stage 2 is trained on noisy–clean pairs sharing the same underlying ECG waveform. We therefore designed a controlled experiment to examine whether SIMEMG-derived contamination reproduces the deterioration when noisy inputs and clean targets share the same underlying waveform, using both LUNet and ImputeECG.

Controlled SIMEMG-derived contamination versus real SIMEMG

We extracted residuals from paired SIMEMG recordings, scaled them relative to clean QTDB ECGs, and added them to QTDB signals while retaining the original clean targets. We refer to this evaluation setting as QTDB with additive SIMEMG-derived residuals (controlled). For each backbone, we evaluated frozen-access and MAA models using their respective unchanged weights on both the controlled setting and real SIMEMG, without retraining or adaptation. The results are summarized below.

SettingBackboneFrozen RMSE ↓MAA RMSE ↓Frozen CC ↑MAA CC ↑
QTDB with additive SIMEMG-derived residuals (controlled)LUNet0.27170.22580.77040.7888
QTDB with additive SIMEMG-derived residuals (controlled)ImputeECG0.66460.40880.67160.6767
Real SIMEMGLUNet4.57875.28830.97850.9730
Real SIMEMGImputeECG8.29045.84330.92750.9602

Under controlled QTDB contamination, Stage 2 alignment reduces LUNet RMSE by 16.89% and improves CC by 0.0184 relative to frozen access. In contrast, on real SIMEMG, alignment increases LUNet RMSE by 15.50% and decreases CC by 0.0055. ImputeECG benefits under both settings, reducing RMSE by 38.49% and 29.52%, respectively.

These results show that LUNet's deterioration on real SIMEMG is not reproduced by transferring SIMEMG-derived residuals to clean QTDB signals. Thus, SIMEMG-derived residuals do not necessarily cause harmful alignment under the evaluated additive contamination setting. Moreover, this contrast is consistent with the possibility that differences in acquisition conditions and noisy–reference signal correspondence influence alignment effectiveness. However, because the controlled and real settings also differ in their underlying ECG signals and reference measurements, the experiment does not isolate the specific factor responsible for the contrasting outcomes.

Reference: Atanasoski, V., et al. (2023). A Database of Simultaneously Recorded ECG Signals With and Without EMG Noise. IEEE Open Journal of Engineering in Medicine and Biology, 4, 222–225. https://doi.org/10.1109/OJEMB.2023.3330295

AuthorsOpenReview ↗

Response to Question 1: Generalization Across Noise Conditions and Subject-Level Statistical Reliability

To evaluate generalization and statistical reliability, we conducted additional analyses across noise conditions, random seeds, and independent subjects (Appendices A.16–A.17).

Generalization across synthetic noise conditions

We first compared frozen memory access and MAA, trained on the full source-noise mixture, across seven individual and combined noise conditions on synthetically contaminated MITDB and QTDB, with results aggregated across available random seeds (Appendix A.16, Table A.15). Before alignment (frozen memory access), LUNet achieves its lowest RMSE under muscle artifact (MA) on both datasets, whereas ImputeECG performs similarly across noise types. After alignment (MAA), both backbones substantially improve RMSE and CC across all evaluated conditions, with baseline wander (BW) yielding the best performance. However, noise-dependent performance remains backbone-specific: on QTDB, aligned LUNet performs better under MA than electrode motion (EM) (RMSE 0.5554 vs. 0.6196), whereas ImputeECG shows the opposite trend (0.6482 vs. 0.5779). These results demonstrate that Stage 2 alignment improves reconstruction across the evaluated synthetic noise conditions on both the source MITDB dataset and the external QTDB dataset. Since these noise families are represented during training, this experiment evaluates cross-condition robustness and cross-dataset generalization rather than generalization to entirely unseen noise categories.

We further investigated the effect of training-noise composition by training Stage 2 separately on BW, EM, MA, or their full mixture (All), and evaluating performance on MITDB, QTDB, and SIMEMG (Appendix A.16, Table A.16). The results indicate that training Stage 2 on a noise type resembling the target contamination does not necessarily produce better reconstruction performance. In particular, although SIMEMG primarily contains MA-like contamination, MA-only alignment does not provide a clear advantage over BW-only alignment for LUNet. BW-only and MA-only training achieve similar RMSE (5.0746 and 5.0785), both outperforming full-mixture training (5.2883). Nevertheless, all aligned configurations remain worse than frozen access (4.5787). For ImputeECG, MA-only and full-mixture training achieve comparable RMSE (5.7965 and 5.8433, respectively), both outperforming frozen access (8.2904). Overall, ImputeECG improves over frozen access under three of the four training configurations. Although training-set sizes differ across configurations, these findings suggest that matching the broad noise category between training and testing is insufficient to guarantee effective alignment, particularly when synthetic training contamination differs from naturally acquired noise.

Subject-level consistency and statistical reliability under real contamination

To examine whether the observed effects on SIMEMG are consistent across independent subjects rather than driven by particular recordings, we conducted a paired subject-level analysis across 14 subjects and three seeds (Appendix A.17, Table A.17). Compared with frozen memory access, MAA improves ImputeECG reconstruction for 13 of 14 subjects, reducing subject-macro RMSE by 26.6%. In contrast, LUNet deteriorates for all 14 subjects, with subject-macro RMSE increasing by 20.8%.

BackboneSubject-level outcome (MAA vs. frozen)Two-sided exact sign-test p-value
ImputeECG13/14 subjects improve0.00183
LUNet14/14 subjects deteriorate0.00012

The tests use one paired outcome per subject after aggregation across seeds, assuming independent subjects and no ties. The exact sign tests provide evidence of consistent directions of change across independent subjects, showing predominantly beneficial alignment effects for ImputeECG and consistently harmful effects for LUNet. These tests assess directional consistency rather than the magnitude of reconstruction improvements.

These analyses show that Stage 2 alignment improves reconstruction across the evaluated synthetic noise conditions and maintains these improvements on the external QTDB dataset under the same noise families. On naturally contaminated SIMEMG, the subject-level results establish that alignment effects are consistent but backbone-dependent, with improvements for ImputeECG and deterioration for LUNet. Thus, our findings support robustness across evaluated source-noise conditions and cross-dataset transfer.

Reviewer_ZQYkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies ECG denoising under distribution shift and proposes a two-stage framework called Memory Access Alignment (MAA). In the first stage, a clean-input teacher is trained to reconstruct ECG signals through a learnable key–value memory. In the second stage, the memory and decoder are frozen, while the student encoder bottleneck and memory router are adapted using latent alignment, memory-readout alignment, and reconstruction losses. The method is evaluated using MITDB for in-distribution testing, QTDB for cross-dataset generalization, and SIMEMG for generalization to naturally acquired EMG contamination. The authors implement the framework with both LUNet and ImputeECG and report improvements over the corresponding original backbones. Additional experiments examine the role of memory, different adaptation configurations, and the relationship between memory-access changes and reconstruction quality. An EEG denoising experiment is also included.

Critical strengths
  1. The improvements under distribution shift are meaningful and are demonstrated with two different backbone architectures. The experimental results are the strongest part of this paper. On QTDB, MAA reduces RMSE from 0.6490 to 0.5631 for LUNet and from 0.6272 to 0.5595 for ImputeECG. Improvements are also reported on SIMEMG, where the contamination is acquired rather than synthetically generated from the training noise source. I appreciate that the authors evaluate both convolutional and Transformer-based architectures rather than relying on a single model. The exclusion of overlapping QTDB records and the use of multiple training seeds also make the comparison more convincing. These results suggest that the framework is worth investigating beyond the particular backbone used to develop it.

  2. The analysis of memory access goes beyond simply reporting reconstruction accuracy. I found Section 3.4 and the corresponding appendix experiments particularly useful. The authors show that better agreement with the clean-input teacher does not necessarily lead to better reconstruction. The decoder-sensitive subspace experiments further examine which parts of the memory-readout changes account for the observed reconstruction effects. This is an interesting distinction. It suggests that recovering the teacher's representation and obtaining a useful correction to the output are not necessarily the same objective. The authors also report cases where adaptation hurts reconstruction rather than presenting only favorable outcomes. I see this analysis as a substantive contribution to the paper.

Critical weaknesses
  1. The experiments do not sufficiently isolate the contribution of the learned memory from the broader two-stage training strategy. My main concern is whether the key–value memory itself is responsible for the reported generalization improvements, or whether a substantial part of the benefit comes from clean-signal pretraining, freezing the decoder, and aligning noisy-input representations with clean-input representations. The memory ablations in Table 2 show that shuffling, zeroing, or removing the memory after training severely damages reconstruction. However, this is not particularly surprising, since the decoder has been trained to operate on representations retrieved through that memory. These experiments establish that the trained network depends on its memory pathway, but they do not establish that a learnable memory is better than an alternative clean-pretrained latent representation. I recognize that the authors include a no-skip LUNet control and compare reconstruction-only, alignment-only, and combined training in Table A10. These are useful experiments, but they address somewhat different questions. A more convincing comparison would use a clean-pretrained encoder–decoder without the key–value memory, followed by the same freezing and teacher–student alignment procedure. Without such a comparison, I am not yet convinced that the proposed memory mechanism, rather than the training strategy surrounding it, is the essential source of improvement. This matters because retaining and accessing clean latent memory is the paper's central methodological claim.

  2. The behavior on real contamination raises an important question about when memory-access alignment is actually beneficial. The result that concerned me most appears in Table 3. On SIMEMG, the LUNet-based model with frozen access, before Stage 2 alignment, achieves an RMSE of 4.5787 and a CC of 0.9785. After joint encoder and router alignment, RMSE increases to 5.2883 and CC decreases to 0.9730. In other words, on the naturally contaminated ECG dataset, the proposed alignment procedure substantially worsens an already effective clean-memory model. The final MAA model still outperforms the original LUNet baseline, as shown in Table 1, but this is a different comparison. It does not establish that the alignment stage improves the model's ability to handle this particular shift. I appreciate that the authors report this result and examine its reconstruction mechanism. Nevertheless, it leaves an important practical question unanswered: when should the clean memory be accessed without further alignment, and when does alignment help? This distinction matters for the generalization claim. A method intended for deployment under unknown distribution shifts should ideally not require prior knowledge of the target corruption to determine whether its adaptation procedure will be beneficial.

Questions for authors
  1. Can the authors more directly establish the necessity of the key–value memory? In particular, how would MAA compare against a clean-pretrained encoder–decoder without memory, using the same frozen-decoder strategy and comparable student alignment losses?

  2. What explains the deterioration caused by alignment on SIMEMG with LUNet, and how general is this failure? Table 3 shows that frozen access performs substantially better than joint alignment on this dataset. Is the deterioration concentrated in particular recordings or subjects, or is it relatively consistent across the dataset? Can the authors identify what makes source-trained alignment harmful under this corruption shift, and whether a selection strategy using only source-domain information could avoid this deterioration?

4 comments
AuthorsOpenReview ↗

Response to Question 1 and Weakness 1: Necessity of Key–Value Memory and Comparison with Memory-Free Adaptation

We thank the reviewers for the constructive comments. To evaluate the contribution of key–value memory, we conduct additional memory-based versus memory-free adaptation experiments using LUNet and ImputeECG across MITDB (ID), QTDB (OOD), and SIMEMG (OOD). The complete results are reported in Appendix A.14 (Table A11).

Experimental settings and variants

In addition to the original backbones, we evaluate six configurations:

  1. Memory-free frozen access, where the clean-pretrained encoder–decoder is evaluated directly on noisy test inputs without Stage 2 fine-tuning;
  2. Memory-free encoder adaptation;
  3. Memory-free decoder tuning;
  4. Memory-free joint encoder–decoder tuning;
  5. Key–value memory with frozen access, where the clean-pretrained memory-based model is evaluated without Stage 2 adaptation; and
  6. The complete MAA framework with learned memory access alignment.

In Stage 1, the memory-free model is pretrained on clean ECG signals by passing encoder representations directly to the decoder, without key–value retrieval. During Stage 2, memory-free encoder adaptation and MAA follow the same frozen-decoder strategy, with each model retaining its respective Stage 1 pretrained decoder. The memory-dependent readout alignment loss is removed in the memory-free variants, while the applicable layerwise student alignment objectives are retained with equal weights of 1 across the included layers. The primary controlled comparison is between memory-free encoder adaptation and MAA, with decoder and joint encoder–decoder tuning providing additional controls. The trainable configurations follow matched initialization protocols, data splits, corruption settings, and training epochs. Learning rates are tuned separately for each configuration and seed using validation performance. All experiments are conducted with three random seeds (42, 43, and 44), and results are reported as mean ± standard deviation across seeds.

Comparison with memory-free adaptation

MAA achieves lower RMSE than all memory-free encoder, decoder, and joint encoder–decoder tuning variants across both backbones and all three datasets. Compared with memory-free encoder adaptation, MAA reduces RMSE from 0.1240 to 0.1214 (2.1%) on MITDB, 0.5822 to 0.5631 (3.3%) on QTDB, and 5.8892 to 5.2883 (10.2%) on SIMEMG with LUNet. For ImputeECG, MAA reduces RMSE from 0.1338 to 0.1225 (8.4%) on MITDB, 0.6994 to 0.5595 (20.0%) on QTDB, and 7.3312 to 5.8433 (20.3%) on SIMEMG.

Contribution of memory and access alignment

Without access alignment, memory-based and memory-free frozen-access configurations yield nearly identical RMSE on MITDB and QTDB across both backbones. Specifically, LUNet achieves RMSEs of 0.2209 versus 0.2217 on MITDB and 1.4639 versus 1.4701 on QTDB, while ImputeECG achieves 0.2283 versus 0.2285 and 1.5196 versus 1.5262, respectively (memory-free versus memory-based). These results suggest that incorporating curated memory without adapting its access pathway is insufficient to improve reconstruction under the evaluated MITDB and QTDB corruptions. In contrast, MAA consistently outperforms the tuned memory-free variants, supporting the benefit of learning how corrupted observations access the stored representations. On SIMEMG with LUNet, memory-free frozen access (RMSE 4.5247) and memory-based frozen access (4.5787) outperform MAA (5.2883), indicating that access alignment is not uniformly beneficial under this shift.

These experiments provide evidence that memory-assisted alignment improves reconstruction beyond clean pretraining and memory-free student adaptation. Because the memory-free controls exclude the memory-dependent readout alignment objective, the results support the combined contribution of key–value memory and learned access alignment rather than isolating their individual effects.

AuthorsOpenReview ↗

Response to Question 2 and Weakness 2: Alignment Effectiveness and Failure Mechanisms

We agree that Stage 2 alignment does not universally improve frozen memory access, particularly under naturally contaminated SIMEMG. We conducted additional analyses to examine its noise-dependent effectiveness, subject-level consistency, and reconstruction failure mechanisms.

Effects of Stage 2 alignment across noise conditions

Since SIMEMG primarily contains muscle-artifact (MA)-type contamination, we compared frozen access and MAA, trained on the full source-noise mixture, across seven individual and combined noise conditions on synthetically contaminated MITDB (ID) and QTDB (OOD) (Appendix A.16, Table A.15). Before alignment, LUNet achieves its lowest RMSE under MA on both datasets, whereas ImputeECG performs similarly across noise types. After alignment, both backbones substantially improve RMSE and CC across all conditions, with baseline wander (BW) yielding the best performance. However, noise-dependent performance remains backbone-specific: on QTDB, aligned LUNet performs better under MA than electrode motion (EM) (RMSE 0.5554 vs. 0.6196), whereas ImputeECG shows the opposite trend (0.6482 vs. 0.5779).

Furthermore, we investigated training-noise composition by training Stage 2 separately on BW, EM, MA, or their full mixture (All), and evaluating MITDB, QTDB, and SIMEMG (Appendix A.16, Table A.16). The results show that noise-matched training is not consistently optimal. On SIMEMG, BW-only training achieves the lowest LUNet RMSE among aligned configurations (5.0746), compared with MA-only (5.0785) and All (5.2883). However, frozen access still achieves a lower RMSE (4.5787). Conversely, ImputeECG improves under three of four training configurations. Although training-set sizes differ, these findings indicate that alignment effectiveness depends on both noise conditions and backbone architecture, rather than noise-type matching alone.

Subject-level consistency

We investigated whether alignment-induced deterioration on SIMEMG is widespread or concentrated in particular subjects. Across 14 subjects and three seeds, LUNet deteriorates for all 14 subjects, with subject-macro RMSE increasing from 4.7640 to 5.7555 (+20.8%). In contrast, ImputeECG improves for 13 of 14 subjects (8.0515 to 5.9130; −26.6%). Thus, deterioration is systematic at the subject level for LUNet but backbone-dependent (Appendix A.17, Table A.17).

Reconstruction mechanism

We further investigated alignment-induced corrections relative to frozen access (Appendix A.17).

  • Correction behavior: We define the alignment-induced correction as the difference between MAA and frozen-access predictions, and the required correction as the difference between the clean target and frozen-access prediction. We compare these corrections to determine whether Stage 2 alignment moves the reconstructed signal toward or away from the clean target. At each signal point, corrections are classified as wrong-direction (moving away from the target), overshooting (moving toward the target but increasing squared error), or beneficial (not increasing error); negligible or unnecessary corrections are considered separately. For LUNet, 43.76% are wrong-direction, 19.09% overshoot, and 36.88% are beneficial, compared with 25.51%, 13.65%, and 60.71% for ImputeECG. LUNet exhibits a lower correction cosine (0.198 vs. 0.725), despite comparable relative magnitudes (0.868 vs. 0.811). Moreover, its correction energy (+18.9911) outweighs the error-reducing interaction term (−8.6508), yielding ΔMSE=+10.3404\Delta\mathrm{MSE} = +10.3404, whereas ImputeECG achieves −41.5398. These results associate deterioration with poorly directed and excessive corrections rather than magnitude alone.

  • SNR-dependent decomposition: We estimated SNR from paired clean/noisy SIMEMG recordings before normalization and grouped windows into 5-dB bins to compare frozen-access and MAA reconstruction. LUNet improves only at −5–0 dB (ΔRMSE=−1.0778\Delta\mathrm{RMSE} = -1.0778), with deterioration beginning at 0–5 dB (+0.1903) and reaching +1.4615 at 15–20 dB. ImputeECG improves through 10–15 dB (−0.6914), but deteriorates at 15–20 dB (+0.2718), reaching +1.2049 at 20–25 dB. Thus, both backbones transition from improvement to deterioration at different SNR ranges, suggesting that alignment can over-correct relatively accurate frozen-access predictions. These are descriptive, not universal SNR thresholds; SNR is used only for retrospective analysis, not inference.

Overall, these findings demonstrate that Stage 2 alignment provides consistent benefits under the evaluated synthetic noise conditions, but its effectiveness under natural contamination is backbone- and distribution-dependent. LUNet's deterioration is systematic across subjects and associated with poorly directed or excessive corrections, particularly when frozen-access reconstruction is already relatively accurate.

AuthorsOpenReview ↗

Response to Question 2 and Weakness 2: SIMEMG Degradation and Source-Only Selection

The preceding analyses of alignment effectiveness and failure mechanisms characterize how Stage 2 alignment deteriorates reconstruction on SIMEMG but do not fully explain why LUNet performs worse than frozen access. We therefore examined the SIMEMG acquisition protocol and conducted a controlled experiment to investigate potential contributing factors (Appendix A.17).

SIMEMG dataset characteristics

Unlike synthetic contamination, where noise is added to the same clean ECG waveform, SIMEMG records noisy ECGs from wrist/finger electrodes and simultaneous cleaner references from the shoulder/upper-arm region. The acquisition method assumes similar cardiac potentials along relaxed arms, with EMG contamination generated primarily in the hands (Atanasoski et al., 2023). During dataset preparation, reference recordings with excessive estimated EMG contamination were rejected; nevertheless, small residual EMG noise may remain in retained references and affect denoising-method comparisons.

Differences in electrode placement and potential residual contamination in SIMEMG references may contribute to LUNet's deterioration, since Stage 2 is trained on noisy–clean pairs sharing the same underlying ECG waveform. We therefore designed a controlled experiment to examine whether SIMEMG-derived contamination reproduces the deterioration when noisy inputs and clean targets share the same underlying waveform, using both LUNet and ImputeECG.

Controlled SIMEMG-derived contamination versus real SIMEMG

We extracted residuals from paired SIMEMG recordings, scaled them relative to clean QTDB ECGs, and added them to QTDB signals while retaining the original clean targets. We refer to this evaluation setting as QTDB with additive SIMEMG-derived residuals (controlled). For each backbone, we evaluated the frozen-access and MAA models using their respective unchanged weights on both the controlled setting and real SIMEMG, without retraining or adaptation. The results are summarized in the table below.

SettingBackboneFrozen RMSE ↓MAA RMSE ↓Frozen CC ↑MAA CC ↑
QTDB with additive SIMEMG-derived residuals (controlled)LUNet0.27170.22580.77040.7888
QTDB with additive SIMEMG-derived residuals (controlled)ImputeECG0.66460.40880.67160.6767
Real SIMEMGLUNet4.57875.28830.97850.9730
Real SIMEMGImputeECG8.29045.84330.92750.9602

Under controlled QTDB contamination, Stage 2 MAA reduces LUNet RMSE by 16.89% and improves CC by 0.0184 relative to frozen access. In contrast, on real SIMEMG, alignment increases LUNet RMSE by 15.50% and decreases CC by 0.0055, while ImputeECG improves under both settings, reducing RMSE by 38.49% and 29.52%, respectively.

These results show that LUNet's deterioration on real SIMEMG is not reproduced by transferring SIMEMG-derived residuals to clean QTDB signals. This contrast is consistent with the possibility that differences in acquisition conditions and noisy–reference signal correspondence influence alignment effectiveness. Nevertheless, the controlled experiment does not isolate these factors from other differences, including ECG population and signal characteristics, and the transferred residuals may themselves contain cross-location differences.

Implications for source-only selection

We agree that selecting between frozen and aligned access using source-domain information is a promising direction for improving robustness under unknown shifts. However, our controlled experiment highlights an important consideration: LUNet benefits from alignment under controlled SIMEMG-derived contamination but deteriorates on real SIMEMG, despite using unchanged model weights. This suggests that selection criteria based on synthetic contamination alone may not adequately capture differences associated with real acquisition conditions and reference measurements. Although this experiment does not directly evaluate source-only selection, it highlights the importance of considering factors beyond additive noise characteristics. Developing and validating a criterion that can anticipate harmful alignment without target labels or acquisition-specific evaluation results remains a valuable direction for future work.

Reference: Atanasoski, V., et al. (2023). A Database of Simultaneously Recorded ECG Signals With and Without EMG Noise. IEEE Open Journal of Engineering in Medicine and Biology, 4, 222–225. https://doi.org/10.1109/OJEMB.2023.3330295

AuthorsOpenReview ↗

Follow-up on Additional Analyses and Reviewer Concerns

Dear Reviewer,

Thank you again for your thoughtful feedback and constructive suggestions. We particularly appreciate your comments on evaluating the contribution of key–value memory and better understanding the results on SIMEMG.

In response to your concerns, we conducted additional controlled experiments, including a clean-pretrained model without key–value memory under the same frozen-decoder strategy, and subject-level analyses of memory alignment on SIMEMG for both backbones. We have incorporated these findings into our rebuttal and revised manuscript, with clearer explanations of the contributions and limitations of the proposed approach, including cases where frozen memory access performs better than alignment.

We hope these additional analyses provide a clearer understanding of our method and help address the questions raised in your review.

As the discussion period approaches its conclusion, we would greatly appreciate any further thoughts you may have, particularly on whether our additional evidence has sufficiently addressed your main concerns. We would be happy to provide further clarification on any remaining questions.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.