acceptodds
Withdrawn from ICLR 2027

Learning Transferable Noise-Environment Representations for Cross-Domain Time-Series Restoration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Time-series restoration under domain shift is challenging because degradation patterns vary across sensing environments, while conventional models learn a single mapping from corrupted to clean observations. We study whether observable background context can provide transferable representations of degradation conditions. We formulate restoration as noise-environment conditioned learning and introduce PCD-Net, which encodes background segments into environment representations and learns a differentiable routing distribution over a prototype memory to condition waveform reconstruction through cross-attention. This separation between environment representation and restoration enables selective cross-domain adaptation, where degradation-related representations are updated together with a lightweight reconstruction adjustment module while the general restoration backbone is preserved. Experiments across seismic, mine microseismic, and industrial waveform domains show that the learned environment representations improve restoration under varying noise conditions and provide an effective basis for parameter-efficient cross-domain adaptation. These results suggest that explicitly modeling observable degradation context can enhance the transferability of time-series restoration models.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Time-Series Modeling, Domain Adaptation, environment representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_BZbwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies cross-domain time series restoration and proposes the noise-environment-conditioned restoration framework PCD-Net. It first encodes the background information into environment descriptor. Then, it learns a differentiable routing distribution over a prototype memory to inject the resulting environment information into waveform reconstruction. It further proposes a selective cross-domain adaptation strategy that freezes the waveform restoration backbone while updating the relevant module. Experiment results show improved restoration quality and substantial gains from cross-domain adaptation.

Critical strengths
  1. Explicitly modeling observable degradation context is sensible. The paper identifies the use of pre-event background segments which is ignored by conventional approaches. It is intuitive and well aligned with continuous monitoring scenarios.
Critical weaknesses
  1. The claim of transferable and parameter-efficient cross-domain adaptation is not sufficiently validated against adaptation baselines. The cross-domain experiments mainly compare four variants of the proposed framework and does not compare against existing adaptation methods. Moreover the claim of parameter efficiency is also difficult to assess because the paper does not report the number or percentage of trainable parameters and adaptation memory/training cost.

  2. Its core contribution differentiable prototype-conditioned environment representation is not new. The similar method has been well studied by previous works [1][2][3], and they are not disscussed in Related Work section. Further clarification on their difference is necessary.

[1] Timedp: Learning to generate multi-domain time series with domain prompts.

[2] Towards a Unified Generative Model for Scarce Time Series with Domain Experts.

[3] TimeXL: Explainable multi-modal time series prediction with LLM-in-the-loop.

Questions for authors
  1. Please clarify the number or fraction of target samples used and whether performance remains strong in genuinely low-data adaptation settings.

  2. Please report the absolute number and percentage of trainable parameters, as well as adaptation time or memory cost if available.

  3. How are the learned prototypes distributed and utilized? Please report the prototype frequencies and pairwise prototype similarity. The statement that non-zero softmax weights does not guarantee prototype diversity.

  4. How does selective adaptation compare with existing domain-adaptation methods? Results available with one or two representative adaptation baselines would significantly strengthen the cross-domain claim.

Reviewer_JgjSOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies waveform restoration (mainly seismic denoising) under domain shift and proposes to condition the restorer on an event-free background segment recorded before the event, treated as observable evidence of the noise environment. PCD-Net encodes the background with a context encoder, computes a temperature-scaled softmax over cosine similarities to KK learnable prototypes ("differentiable routing"), and injects the prototypes into the waveform pathway via cross-attention whose logits are biased by λrlog⁡(α+ϵ)\lambda_r \log(\alpha+\epsilon). For transfer, "selective adaptation" updates the context encoder, prototype memory and routing parameters together with a bottleneck decoder adapter while keeping the waveform encoder/decoder frozen. Validation: (i) STEAD source-domain denoising against Bandpass, Wavelet, DeepDenoiser, DPRNN, Restormer1D and a prototype-conditioned TCN (Table 1, five seeds; Fig. 4 by input-SNR bin); (ii) a component ablation (Table 2); (iii) four adaptation strategies (frozen / decoder adjustment / selective / full fine-tuning) on an industrial dataset (Table 3) and a mine microseismic dataset (Table 4); (iv) KK/τ\tau sensitivity (Fig. 6) and complexity (Table 5). The supplementary material provides code and result files for the STEAD comparison, the ablations, and the efficiency benchmark.

Critical strengths
  1. Practically motivated setting. In continuous monitoring a pre-event noise segment is available at no cost, so asking whether it can be turned into a transferable conditioning signal is a reasonable question for seismic/microseismic restoration.
  2. Transparent supplementary pipeline. The paired evaluation code, checkpoint hashes, configs and raw result tables are released, and on STEAD they confirm that the PCD-Net checkpoint outperforms the included baselines (ΔSNR 7.77 dB vs. 6.20 dB for the best baseline in global_compare.csv, consistent with Table 1/Fig. 4). This strength carries limited weight, however, because the same files indicate that the improvement is not attributable to the proposed components (W1).
Critical weaknesses

W1. Table 2 cannot be reconciled with the authors' own supplementary results, and those results contradict the paper's central claims.

Table 2 is the paper's only evidence that the background representation, routing and prototype memory matter. The supplement contains the outputs of the authors' ablation scripts (paper_experiments/ablation_v9_seed0/routing/routing_ablation_table.csv and .../context/context_ablation_table.csv, with audit JSONs recording checkpoint hashes and configs). Every row of Table 2 equals a supplementary row after relabeling and/or adding round-number offsets (ΔSNR / CC / RMSE / ST-MAE):

Table 2 row (paper)Supplementary rowPaper − supplementary
PCD-Net: 7.8554 / 0.7667 / 0.0402 / 0.0269"w/o prototype memory": 7.8554 / 0.7667 / 0.0402 / 0.02690 / 0 / 0 / 0
w/o Prototype: 7.1653 / 0.7032 / 0.0406 / 0.0272"Full PCD-Net": 7.7653 / 0.7632 / 0.0406 / 0.0272−0.6 / −0.06 / 0 / 0
w/o Routing: 6.7970 / 0.6919 / 0.0605 / 0.0471"w/o routing (uniform alpha)": 7.7970 / 0.7619 / 0.0405 / 0.0271−1.0 / −0.07 / +0.02 / +0.02
w/o Background: 5.7306 / 0.6628 / 0.0607 / 0.0474"No background" (all-zero context): 7.7306 / 0.7628 / 0.0407 / 0.0274−2.0 / −0.10 / +0.02 / +0.02
w/o Decoder Adapter: 7.1170 / 0.7626 / 0.0406 / 0.0274"weighted-sum + concat fusion": 7.7170 / 0.7626 / 0.0406 / 0.0274−0.6 / 0 / 0 / 0

Each Table 2 row is compatible with exactly one supplementary model under this relation, and the match is unique even when only the ΔSNR column is used. Results from an independent run would agree in the last three ΔSNR digits with probability of roughly 10−1110^{-11} for all five rows.

A label mix-up inside the pipeline seems excluded, for four reasons:

  • evaluate_routing_ablation_v9.py binds labels to the checkpoint arguments and validates each checkpoint's config against its label.
  • The "Full PCD-Net" row is bit-identical in ΔSNR, CC and RMSE (e.g., 7.765267381063327) to the "Correct background" row of the context ablation, which evaluates only the full checkpoint.
  • The same numbers reappear as the learned-τ point of Fig. 6's τ row.
  • They coincide with the PCD-Net numbers of the main comparison (global_compare.csv, 7.7652 dB), which match Table 1 and Fig. 4.

The inconsistency is visible even from the PDF alone. The "PCD-Net" row of Table 2 (7.8554 dB) lies 7.5 standard deviations above the five-seed PCD-Net result in Table 1 (7.7548±0.01347.7548 \pm 0.0134 dB). The actual full model in the supplement (7.7653 dB) agrees with Table 1.

Taken at face value, the authors' files show that zeroing or randomizing the background at inference changes ΔSNR by −0.035 dB, uniform routing gives +0.03 dB, removing the prototype memory gives +0.09 dB (the best variant), and weighted-sum concatenation instead of cross-attention gives −0.05 dB. All effects are below 0.1 dB, versus the 0.7–2.1 dB drops claimed in Table 2, and two of them have the opposite sign. No decoder adapter appears anywhere in the released code, so I could not see how a "w/o Decoder Adapter" variant was evaluated.

Fig. 6 is inconsistent even within itself. The selected configuration (K=16K=16, τ=0.3\tau=0.3) appears in both rows. In the top (KK) row it reaches ≈9.1 dB ΔSNR and CC ≈0.81. In the bottom (τ\tau) row the same configuration reaches 7.765 dB and CC 0.7632, which equals the supplementary full model and agrees with Table 1. In addition, K=8K=8 is reported at ≈3.2 dB although removing the prototypes altogether yields 7.86 dB. According to routing_sensitivity_v9.py, the τ\tau row is an inference-time sweep of a single checkpoint (it spans only ≈0.02 dB), and τ\tau is in fact a learnable parameter (learnable_routing=True). The KK results, by contrast, are read from an external CSV that is not included. The same script's overlay plot labels that series "K sweep (V7 Top-M)" (lines 121, 142), which suggests that the KK row comes from an earlier model variant with hard Top-M selection rather than from the PCD-Net evaluated elsewhere in the paper.

Hence the statements in Sec. 4.2 (lines 411–421), and the abstract's claim that "the learned environment representations improve restoration", are not supported by the submitted evidence. This weakness alone is decisive for my rating.

W2. The cross-domain / parameter-efficient adaptation contribution is unsubstantiated and inconsistent with the released implementation.

  • Tables 3–4 compare only PCD-Net variants. No baseline is adapted to the target domains, and no standard PEFT alternative (adapters, LoRA, BitFit, partial fine-tuning of a baseline) is compared, so it cannot be judged whether the "environment representation" makes the model more transferable than an ordinary network. Full fine-tuning is best on both target domains, and trainable-parameter counts are never reported, so "parameter-efficient" is not quantified.
  • The paper describes a bottleneck decoder adapter (Eqs. 14, 26) with the encoder and decoder frozen. The released adaptation code (transfer_v9.py) offers context_encoder, environment, decoder_only and environment_decoder; the last two unfreeze all decoder blocks (dec4–dec0, out_head), and no adapter module exists. It is unclear which configuration produced "Decoder Adjustment" and "Selective Adaptation".
  • In the released adaptation scripts (defaults of run_adaptation_ablation_v9.ps1 and the README example), target-domain events are corrupted with the STEAD noise bank (wave1.hdf5) for both adaptation and evaluation. The additive-noise distribution is then identical in source and target, so the shift is in the signal rather than in the "noise environment" that motivates the method.
  • The adaptation entry points accept only stead and mining. No file refers to the industrial dataset of Table 3, and no result files for Tables 3–4 are provided.
  • Neither target dataset is described. Missing details include the source, the size and the sampling rate (the split script admits 50 Hz and 100 Hz mine data, while inputs are "6000 samples at 100 Hz"). The paper also does not say how clean references are obtained or how many target samples are used. The ethics statement says all datasets are public, but no source is given for either target dataset.

Secondary concerns (these would remain even if W1/W2 were resolved)

  • Test vs. validation. All reported STEAD numbers are computed on v7_seed0_val.csv. This file is exactly wave2.csv.sample(frac=0.1, random_state=0): the same 20,000 traces in the same order, i.e. the seed-0 validation split of the training pipeline. The PCD-Net checkpoint config has val_frac=0.1, seed=0 and val_samples=2000, and the best checkpoint is kept by validation CC. KK and τ\tau were "selected according to validation performance", and Fig. 6's τ\tau row reproduces the reported test value (7.765 dB) at the learned τ\tau. There appears to be no untouched test split.
  • "Five random seeds." The supplement contains seed-0 runs only. Bandpass and Wavelet are deterministic yet show non-zero std in Table 1, which implies the test data change across seeds. In that case the reported stds should be close to the sampling standard error, ≈0.09–0.15 dB from the bootstrap CIs in paper_model_comparison_table.csv. Instead they are 10–50× smaller (e.g., 0.0074 dB for Wavelet, 0.0030 dB for DPRNN).
  • Baselines.
    • DPRNN and DeepDenoiser are far worse than the identity map at high input SNR (ΔSNR −6.9 and −3.4 dB in the 5–10 dB bin; Fig. 4 and per_group_compare.csv), which suggests under-training or misconfiguration.
    • The code describes DeepDenoiser as a time-domain U-Net, whereas the original estimates STFT masks.
    • The baseline implementations (v3/...) are not included.
    • PCD-Net has the most parameters (3.95M vs. 0.91–2.63M), a column that was dropped from Table 5.
    • Simple strong baselines are missing: (a) the same backbone without conditioning, or with directly concatenated/FiLM background features (the "w/o prototype" variant, which feeds the background embedding as a single attention token, is in fact the best model); (b) a Wiener/spectral-subtraction filter using the pre-event noise PSD; (c) the background-conditioned GDiffuSE-/CDiffuSE-style baselines that are implemented in compare_v9.py but not reported.
  • The background is likely part of the input. STEAD traces are exactly 6000 samples with a median P arrival at sample ≈700. The 400-sample pre-P "background" therefore seems to be a sub-window of the 60 s input xx itself, carrying no information unavailable to the baselines. This is consistent with the near-zero context ablation.
  • Undisclosed objective / reproducibility.
    • The training loss is never described; the configs combine MSE, frequency, gradient, "detect", quality-head and intermediate losses with event-region weighting.
    • The reproducibility statement points to "the full training objective, the quality-aware training target, and the adaptive inference refinement procedure" in Appendix A, but none of these is there.
    • The stated batch size (2) differs from the configs (4 for PCD-Net, 8 for the baselines).
    • Figs. 5 and 7 are illegible.
  • Related work. None of the following is discussed:
    • Conditioning a denoiser on noise-only segments is well established in speech enhancement: noise-aware training (Seltzer et al., ICASSP 2013; Xu et al., Interspeech 2014), dynamic noise embeddings with adaptation (Lee et al., APSIPA ASC 2020), and noise-model guidance fitted on short noise-only segments (GDiffuSE, Yanir et al., 2025, arXiv:2510.04157).
    • Degradation-embedding-weighted prompt/prototype components appear in all-in-one image restoration (AirNet, CVPR 2022; PromptIR, NeurIPS 2023).
    • Eq. 26 is the standard bottleneck adapter (Houlsby et al., ICML 2019).
Questions for authors

Clarifications that could change my assessment:

  1. Please explain how each row of Table 2 was produced and reconcile it with routing_ablation_table.csv and context_ablation_table.csv (mapping in W1). Which numbers are correct? Were the ablated variants retrained or evaluated at inference time? Please provide the raw logs and per-sample outputs behind Table 2.
  2. How were the KK-sweep numbers in Fig. 6 (top row) obtained, and from which model? The plotting script labels them "K sweep (V7 Top-M)". Why does the selected configuration (K=16K=16, τ=0.3\tau=0.3) reach ≈9.1 dB in the top row but 7.765 dB in the bottom row of the same figure (and 7.75 dB in Table 1)?
  3. Which code implements the lightweight decoder adapter (Eqs. 14, 26)? Which strategies correspond to "Decoder Adjustment" and "Selective Adaptation" in Tables 3–4, and how many trainable parameters does each have?
  4. For both target domains:
    • What noise corrupts the target events? The released scripts use STEAD noise.
    • What are the data sources and sizes?
    • How are clean references obtained?
    • How many target samples are used for adaptation and for testing?
  5. What varies across the five seeds of Table 1 (training, test noise/SNR draws, or both)? Please report per-seed results.
  6. Is there a STEAD test split that is disjoint from the validation set used for checkpoint selection and for choosing KK/τ\tau?
  7. Is the background segment bb a sub-window of the input xx? Do the baselines use the same training objective and training budget as PCD-Net?

Suggestions (not affecting my score):

  • Add same-backbone baselines without conditioning and with direct background conditioning.
  • Add a Wiener filter that uses the pre-event noise PSD.
  • Adapt the baselines on the target domains and compare against standard PEFT methods.
  • Evaluate with real target-domain noise.
  • Quantify whether the learned routing actually separates noise environments, e.g., with routing statistics per station or site.
Reviewer_bbZDOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PCD-Net, which encodes the noise environment from a pre-event background segment and guides waveform restoration through learnable prototypes, differentiable routing, and cross-attention. It further proposes adapting environment-related modules and lightweight decoder adapters to the target domain while keeping the restoration backbone frozen. The method is evaluated on seismic, mine microseismic, and industrial waveform data.

Critical strengths
  1. The research problem has practical relevance. Using background information available before an event to handle noise variations across sensing environments is well motivated.
  2. The approach is reasonably coherent. Background encoding, prototype-conditioned restoration, and selective adaptation form a reasonably clear framework for addressing changes in the noise environment.
  3. The paper reports promising results. The main comparison includes traditional methods, deep learning-based restoration models, and ProtoCond-TCN, with means and standard deviations over five random seeds. Adaptation is also reported to improve over the frozen model in both target domains.
Critical weaknesses
  1. Key ablation findings are inconsistent with the accompanying experimental records. All four metrics for the full PCD-Net in Table 2 exactly match those of the "w/o prototype memory" variant in the accompanying records, at the precision reported in the paper. In those records, the full model achieves a ΔSNR of 7.7653, compared with 7.8554 without prototype memory and 7.7970 without routing. These results do not show the performance drops described in the paper. The discrepancies directly affect the assessment of the prototype and routing contributions, so the correspondence between the table and the records needs clarification.
  2. The implemented scope of selective adaptation does not match the paper's description, and the claimed cost advantage lacks quantitative support. Section 3.3 and Appendix A.3 describe freezing the main decoder and training only newly added low-dimensional adapters and environment-related modules. However, decoder adaptation in the submitted implementation updates all parameters of the decoder blocks at every level and the output layer. The actual strategies used in Tables 3 and 4 therefore need clarification. The materials also lack the trainable parameter counts, proportions, and adaptation costs corresponding to these results. Since full fine-tuning achieves better restoration performance in both target domains, the performance–cost trade-off of selective adaptation needs to be quantified.
  3. The transfer contribution of the source-domain environment representations has not been isolated. Improvements from updating the environment branch during adaptation do not by themselves distinguish the contribution of reusing environment knowledge learned during source-domain pretraining from that of learning such knowledge anew on the target domain. Comparing adaptation results using pretrained versus reinitialized environment-branch weights, with the same restoration backbone, target data, and optimization budget, would more directly test the transfer value of the pretrained environment representations.
  4. Key training and data construction procedures cannot yet be fully verified. The paper and accompanying materials provide some basic settings, but parts of the implementation needed for loss computation, the underlying model, and data loading are missing. The full definition of the restoration loss, the construction of noisy inputs and clean references, the sources of the target-domain data, and the actual number of adaptation samples remain unclear. These details directly affect reproducibility and the assessment of the cross-domain adaptation results.
Questions for authors
  1. Please explain the discrepancies in values and variant assignments between Table 2 and the accompanying ablation records, and provide the corresponding experimental settings and sources of the results.
  2. The paper describes freezing the main decoder and updating only lightweight adapters within the decoder, whereas decoder adaptation in the submitted implementation updates the main decoder blocks. Please clarify which parameters were updated for Tables 3 and 4, and report the corresponding trainable parameter counts and adaptation costs to support the parameter-efficiency claim.
  3. With the same restoration backbone, target data, and optimization budget, how do adaptation results differ when the environment branch retains its source-pretrained weights versus being reinitialized?
  4. Please provide the missing key implementations and the complete restoration loss definition. Also clarify how noisy inputs and clean references are constructed, where the target-domain data come from, and how many samples are actually used for adaptation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.