- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes TEX-MEX, a method for integrating structural and temporal information into the contrastive objective. It does this by extracting semantically similar patterns from time series using the encoder's output features and combining them with temporal similarity, which draws positive and negative pairs from the temporal context of similar and dissimilar windows. The authors validate this by evaluating linear probe classification performance on physiological time series signals such as PPG, ECG, and HAR datasets. In addition, they extend their pretraining recipe to a unimodally pretrained PPG foundation model and compare it with general time-series as well as PPG-specific foundation models.
Critical strengths
The paper presentation is generally understandable. The tables and figures are legible and easy to follow to an extent. Also, attempting to combine semantic and temporal information in the contrastive objective while avoiding traditional augmentations that risk distorting time series signal is desirable in time series representation learning.
Critical weaknesses
The contrastive objective needs significantly more explanation of the learning signal and the choices for positive selection. Specifically, the initial similarity scores between the samples in the batch and the anchor are computed from representations produced by an untrained encoder. Why should these random similarity priors guide the entire learning process, and what incentive does the encoder have to learn discriminative features as training progresses? Also, the selection of random past and future windows assumes non-IID data. This is not the case for most time-series representation learning methods for classification, which assume IID data and shuffled batches, with all the information needed for learning contained within a single instance of a predefined window size (UCR and UEA). I understand that TNC does not follow this convention, but most recent methods do (TS2Vec, SoftCLT, TFC, TSTCC).
Also, the components of the loss function do not appear to be well motivated. The term, which assigns a lower weight to the dissimilar sample, should be sufficient as a soft contrastive objective; adding an component with a reverse weight that goes into the denominator is redundant at best. In addition, the statement in lines 204–205 that "P and N do not require representations to preserve features that distinguish a window from its temporal shift" is unsubstantiated, as and in Equation 1 already have anti-collapse terms in the denominator, which are usually sufficient to prevent representation collapse in contrastive learning.
Given that the loss function contains several redundant components, which would inadvertently increase training time, it would be interesting to see ablations on a more comprehensive classification benchmark suite, such as the UCR and UEA benchmarks, rather than on the more niche CGM metabolic subphenotyping task.
Overall, this paper's main contribution appears to be a substitution of the DTW similarity in SoftCLT with similarity computed using a randomly initialized encoder. For IID data, the proposed shift is not practical, and the initial similarity scores from a randomly initialized encoder backbone provide a poor prior. Therefore, a clear positive and negative selection mechanism to guide the contrastive objective is important. Also, it is important to clarify that SoftCLT is not a standalone architecture but rather is applied to other contrastive methods with a clear learning signal, such as TS2Vec.
TS-TCC - Eldele, E., et al. "Time-Series Representation Learning via Temporal and Contextual Contrasting." IJCAI, 2021.
TFC - Zhang, X., et al. "Self-Supervised Contrastive Pre-Training for Time Series via Time-Frequency Consistency." NeurIPS, 2022.
Questions for authors
Can the author clarify the following points?
-
Is the encoder backbone a pretrained backbone or randomly initialized?
-
How are the datasets processed? assuming iid with shuffle True, or keeping shuffle False as in non-iid?
-
In designing the loss components in eqn 2, 3, and 4, did you use any parallelization techniques in your code?
-
What is the main invariance your method aims to learn?
-
Can you elaborate on the main motivation for including the extra H term in the loss function in eqn 4?
Minor issue:
Lines 206 - 208 mention contrasting the "temporal shifts" of similar batch members against their corresponding unshifted windows; however, the H component doesn't factor in any temporal shifts
7 comments
Response to Reviewer U2kw (2/5)
(§1, continued)
"SoftCLT is not a standalone architecture but rather is applied to other contrastive methods with a clear learning signal, such as TS2Vec."
Our SoftCLT comparison uses TS2Vec, following the original paper's default setup. We have clarified this in the revised manuscript (Section 4.1, line 267).
2. Learning signal from a randomly initialized encoder
"Why should these random similarity priors guide the entire learning process, and what incentive does the encoder have to learn discriminative features as training progresses?"
The encoder is indeed randomly initialized rather than pretrained. Because the similarity weights are recomputed at every step from the current encoder, they become increasingly informative as training progresses (Fig. 3), and two properties make the random starting point workable.
First, a valid positive signal is guaranteed from the first training step. Each anchor's own temporal shift always contributes to the positive term P because its source-window self-similarity is , regardless of initialization. Training therefore does not depend entirely on accurately identifying similarities between different recordings before any learning has occurred. This provides a reliable starting signal, although the substantial performance drop without graded weighting (Table 3) shows that the self-shift alone is insufficient to achieve TEX-MEX's full performance.
Second, we respectfully note that random convolutional features have been shown to carry useful structure. At initialization, the encoder applies the same fixed transformation to every window, so similar input patterns tend to receive similar representations. Random convolutional architectures have been shown to preserve useful structural properties before training (Saxe et al., 2011), and in time series, ROCKET achieved strong classification performance on the UCR archive using random convolutional features (Dempster et al., 2020). In this way, the randomly initialized encoder can therefore offer a reasonable starting point for similarity estimation, which TEX-MEX then improves as training progresses (Fig. 3). Our new grader ablations are consistent with this: although the encoder's similarity starts from random initialization, replacing it with fixed DTW or REBAR distances, which are available from the first step, degrades 11 and 10 of 12 metrics, by up to 7.8 and 7.7 AUROC, respectively (revised Table 3).
The incentive to learn discriminative features comes from the contrastive objective itself. It rewards representations that distinguish the temporal contexts associated with similar source windows from those associated with dissimilar ones. As the encoder learns these distinctions, its similarity estimates become more informative, sharpening the weights used in subsequent training. Figure 3 evaluates this learned similarity directly as a distance function for identifying windows that share a class: after one epoch, it is within approximately 2 percentage points of REBAR, and by mid-training it exceeds REBAR by 4-5 points.
These observations support the viability of learning from random initialization. TEX-MEX begins with a reliable positive signal and input-dependent similarity estimates, its grading function becomes more informative during training, and the resulting representations achieve strong downstream performance. The graded-weighting ablation further demonstrates that this similarity-based supervision is important to TEX-MEX's effectiveness (Table 3). We have clarified these points in the Methods section of the paper (Section 3, lines 196–201).
(continued in part 3/5)
Response to Reviewer U2kw (3/5)
3. Data assumptions
"Also, the selection of random past and future windows assumes non-IID data… For IID data, the proposed shift is not practical."
TEX-MEX does use context beyond a single window, but this requirement is modest, and the continuous recordings it relies on are also a common type of time series data for representation learning. At each step, we randomly sample a batch of windows across the pretraining dataset, as in standard SSL, and pair each with a past or future window from its own recording, using a single temporal offset shared across the batch (Section 3). Our PULSE comparison, for example, retains the same source-window sampling procedure, with TEX-MEX additionally retrieving each shifted counterpart.
Specifically, TEX-MEX needs only recordings long enough to provide a shifted partner, typically a short distance into the past or future. As the reviewer notes, established general-purpose SSL methods such as TNC and CPC construct temporal supervision from continuous recordings, and more recent works such as REBAR and RelCon draw comparisons from windows across a recording. Large-scale models such as MOMENT and Chronos also harness extended sequences during pretraining. Further, even where only pre-segmented sequences are available, sufficiently long ones can be divided into shorter windows, as in our CGM experiments (Section 4.2).
We agree that short, isolated sequences, such as those in some UCR/UEA datasets, may be less suited to TEX-MEX pretraining when subdividing them leaves too little signal structure for meaningful cross-recording similarity, and we now note this limitation in the paper's conclusion (Section 6, lines 521–523). At the same time, long, continuous recordings are a widespread form of time series data, arising in wearable sensing, clinical monitoring, and many other domains where signals are recorded over extended periods. TEX-MEX targets continuous biosignals in general and is especially well suited to mHealth and wearable data, and we have scoped our claims accordingly (Section 6, lines 521–526). Within this setting, TEX-MEX broadly achieves state-of-the-art performance across five signal domains (Tables 1-2, 4).
Finally, this requirement applies only during pretraining: the pretrained encoder processes downstream windows independently, without their surrounding temporal context.
4. Motivation for N and H
"The term, which assigns a lower weight to the dissimilar sample, should be sufficient as a soft contrastive objective; adding an component with a reverse weight that goes into the denominator is redundant at best."
This question highlights an important distinction between TEX-MEX's loss and SoftCLT's. SoftCLT can rely on soft assignments alone because its targets are fixed and external, whereas TEX-MEX's are learned. SoftCLT trains the model's softmax similarities to match soft assignments computed from DTW distances on the raw data (Lee et al., 2024). Because these targets do not depend on the encoder, a collapsed representation cannot satisfy them.
In TEX-MEX, the similarity weights are derived from the encoder being trained, so an objective built only from them could be trivially satisfied by collapse: if all representations coincide, the weights become uniform and agree with the uniform similarities. N prevents this by explicitly placing the temporal contexts of dissimilar source windows in the denominator, so that P must attract the temporal contexts of similar windows relative to dissimilar ones.
N's similarity-based grading is also important. Uniformly treating other windows as negatives risks repelling semantically related examples, whereas N assigns lower negative weight to windows that appear similar to the anchor. This robustness to false negatives is a motivation shared by prior soft contrastive approaches, including RelCon and SoftCLT (Xu et al., 2025; Lee et al., 2024). We have added an additional ablation replacing N's graded weights with uniform negative weights. This degrades 10 of 12 metrics, by up to 6.6 AUROC (revised Table 3). The experiment supports the value of similarity-dependent negative grading rather than treating all candidate negatives equally. N therefore provides complementary negative supervision, while its grading helps reduce the effect of false negatives.
(continued in part 4/5)
Response to Reviewer U2kw (1/5)
We thank the reviewer for their detailed and thoughtful feedback. Their questions have helped us clarify several aspects of TEX-MEX's motivation, learning dynamics, and loss design. We have revised the manuscript and added targeted ablations and runtime measurements in response. In brief:
We address each concern below.
1. Relation to SoftCLT
SoftCLT models cross-recording similarity and temporal proximity through separate losses, whereas TEX-MEX combines them within the same contrastive pair, allowing each window to learn from the varied temporal evolutions of similar patterns across recordings rather than only its own observed temporal context. Specifically, SoftCLT's instance-wise loss relates different series at the same timestamp, while its temporal loss relates nearby timestamps within a single input (Lee et al., 2024). Neither constructs pairs linking related windows across both recordings and time.
TEX-MEX instead grades similarity between source windows and uses their temporally shifted counterparts as contrastive examples. A single recording provides a limited view of how a pattern evolves, whereas similar patterns across recordings may occur in a much broader range of temporal contexts. By composing these relationships, TEX-MEX can learn from this diversity rather than restricting temporal supervision to one observed trajectory. We discuss this distinction further in the General Response ("What distinguishes TEX-MEX?").
The advantage of this distinction is supported empirically. TEX-MEX outperforms SoftCLT on 18 of 20 results, by up to 14.1 AUROC (Tables 1-2). Removing the similarity-based grading that enables cross-recording temporal-context contributions also degrades 11 of 12 metrics, by up to 17.1 AUROC (Table 3; also shown in the General Response).
TEX-MEX also makes learned, adaptive similarity grading both effective and highly efficient. SoftCLT reports that applying embedding-space contrastive approaches such as NNCLR and ASCL to TS2Vec reduced performance (Lee et al., 2024, Sec. 4.6). In TEX-MEX, however, the encoder's similarity measure improves during training and ultimately surpasses DTW and REBAR at predicting shared class membership (Fig. 3). In our new grader ablations, substituting fixed DTW or REBAR similarity within TEX-MEX degrades 11 and 10 of 12 metrics, by up to 7.8 and 7.7 AUROC, respectively (revised Table 3). The encoder-based grader also reuses representations already computed in the forward pass, whereas per-batch DTW and REBAR grading increase training time per step by 5.9x and 13x. SoftCLT's approach of precomputing pairwise DTW similarities requires a quadratic number of preprocessing comparisons in the number of training inputs, an overhead TEX-MEX avoids.
(continued in part 2/5)