acceptodds
Under review as a conference paper at ICLR 2027

HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series

Abstract

Critical events in multivariate time series, from turbine failures to cardiac arrhythmias, demand accurate prediction, yet labeled data is scarce because such events are rare and costly to annotate. We introduce HEPA (Horizon-conditioned Event Predictive Architecture), built on two key principles. First, a causal Transformer encoder is pretrained via a Joint-Embedding Predictive Architecture (JEPA): a horizon-conditioned predictor learns to forecast future representations rather than future values, encouraging the encoder to capture predictable temporal dynamics from unlabeled data alone. Second, we freeze the encoder and finetune only the predictor toward the target event, producing a monotonic survival cumulative distribution function (CDF) over horizons. With fixed architecture and optimiser hyperparameters across all benchmarks, HEPA handles water contamination, cyberattack detection, volatility regimes, and eight further event types across 11 domains, matching or exceeding leading time-series architectures including PatchTST, iTransformer, MAE, and Chronos-2 on at least 10 of 14 benchmarks each, with an order of magnitude fewer tuned parameters and, on lifecycle datasets, an order of magnitude less labeled data.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
time series, self-supervised learning, joint-embedding predictive architecture, event prediction, representation learning, label efficiency, transfer learning, predictive maintenance

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_z78fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HEPA, a self-supervised framework for event prediction in multivariate time series. The framework employs a causal Transformer encoder with JEPA-based pretraining to predict future representations, followed by predictor finetuning on a frozen encoder to generate event probabilities through a discrete-time survival CDF. Experiments across 14 benchmarks from 11 domains compare HEPA with various time-series models, including ablation studies and label-efficiency evaluations. The results demonstrate competitive event prediction performance, particularly on lifecycle datasets with limited labeled data. However, the necessity of predictor finetuning remains unclear, while the theoretical assumptions and experimental evidence supporting the claimed benefits require further justification and validation.

Critical strengths

The paper addresses event prediction in multivariate time series under limited labeled data, which is a practically relevant problem. It presents a unified framework for event prediction across different domains by combining JEPA-based self-supervised representation learning, horizon-conditioned predictor finetuning, and discrete-time survival modeling. The experiments cover 14 benchmarks across 11 domains, with comparisons against various baselines, ablation studies, and statistical significance analyses. The results demonstrate competitive performance and promising label efficiency on certain lifecycle datasets, while the paper also acknowledges several limitations.

Critical weaknesses
  1. Unclear necessity of predictor finetuning. The proposed predictor finetuning is similar to training a horizon-conditioned nonlinear head on a frozen pretrained encoder. Moreover, Appendix I.2 shows that pretrained predictor weights provide little benefit over random initialization (at most 0.003 in pooled AUPRC). The necessity and unique advantages of predictor finetuning remain unclear.

  2. Insufficient justification of theoretical assumptions. Proposition 1 relies on the target sufficiency assumption (A1), but the paper does not adequately justify this assumption given that the target encoder remains fixed after initialization. Moreover, the theoretical result only addresses event-information retention and does not directly establish the benefits of predictor finetuning or label efficiency.

  3. Insufficient experimental validation. The paper lacks a comparison with a frozen randomly initialized encoder to verify the benefits of self-supervised pretraining. In addition, the 1%, 2%, and 5% label experiments only report HEPA's results on two C-MAPSS datasets, without comparisons against other baselines, making the claimed label efficiency difficult to assess.

Questions for authors
  1. Given the similar performance of pretrained and randomly initialized predictors, why is it necessary to retain and finetune the pretrained predictor rather than train a new nonlinear head?

  2. How do the authors justify the target sufficiency assumption (A1) in Proposition 1, given that the target encoder remains fixed after initialization?

  3. How does Proposition 1 theoretically support the effectiveness of predictor finetuning?

  4. How does HEPA compare with other baselines under 1%, 2%, and 5% labeled data?

  5. Why is the finetuning-mode ablation in Table 4 conducted using an earlier architecture rather than the final HEPA architecture?

5 comments
AuthorsOpenReview ↗

Predictor vs. fresh head, assumption A1, low-label baselines, final-architecture ablation

We thank the reviewer for their questions. Following their suggestion, we now compare all baselines at 1, 2 and 5% labels (Q4): the frozen pretrained HEPA encoder has the highest mean of all methods on C-MAPSS-3 at every budget.

Q1 and W1. We agree that the pretrained predictor weights matter little; the finetuned predictor's advantage comes from its architecture. With full labels it beats a fresh MLP head that takes Δt as input on MBA (+0.07 [+0.04, +0.09]) and GECCO (+0.08 [+0.02, +0.13]) and is within 0.01 of it on the other three. Its pretrained weights, however, add little (re-initialising them changes h-AUROC by −0.01 to +0.02, every interval including 0).

HEPA (pretrained predictor, finetuned) minus each variant, 100% labelsC-MAPSS-1C-MAPSS-3GECCOPSMMBA
Re-initialised predictor, finetuned−0.010.00+0.010.00+0.02
Fresh MLP head on the frozen encoder−0.010.00+0.08*−0.01+0.07*

* paired 95% interval excludes 0. With ≤5% labels, in contrast, the fresh head is the more stable choice on C-MAPSS (Q4), and the revision no longer claims that the predictor weights transfer.

Q2–Q3 and W2. We agree that A1 needed more support than the submission gave, so we now test it directly. The test shows that A1 holds at longer horizons but not at Δt = 10: adding the past to the random target's embedding of (t, t+Δt] raises event AUROC by +0.05 on GECCO and +0.13 on MBA at Δt = 10, and changes it by at most 0.02 at Δt = 50 and 100.

Linear-probe AUROC, test split, 5 seedsGECCO Δt=1050100MBA Δt=1050100
target embedding H* of (t, t+Δt]0.870.890.920.720.950.99
H* and the context embedding H_t0.920.890.900.850.950.99
H_t alone0.910.810.810.840.860.72

At Δt = 50 and 100, A1 holds within the probe's resolution. On PSM, however, every embedding carries little of the event (AUROC 0.50–0.55), so the test says little there. Accordingly, the revision states A1 this way, removes the sentence calling it "mild for well-trained target encoders" (the target is a fixed random network) and moves Proposition 1 to the appendix as a statement about information retention. Finally, as the reviewer notes, Proposition 1 says nothing about the head, predictor finetuning or label efficiency, so the case for finetuning rests on Q1 and Q4.

Q4 and W3. We added all baselines at 1, 2 and 5% labels, and the comparison supports the label-efficiency claim for the pretrained HEPA encoder. On the two subsets of Table 2, at 1–5% labels the frozen HEPA encoder with a fresh MLP head has the highest mean h-AUROC of all methods on C-MAPSS-3 and is ahead of Chronos-2 there by +0.15 [+0.05, +0.24] at 2%; on C-MAPSS-1 it is level with the best baselines (paired intervals include 0).

h-AUROCC-MAPSS-1 1%2%5%C-MAPSS-3 1%2%5%
Frozen HEPA encoder + MLP head0.790.830.840.830.930.94
Same head, random frozen encoder0.720.750.800.680.800.83
Chronos-2 (119M), same head0.720.800.820.700.780.83
PatchTST0.810.660.650.700.710.58
iTransformer0.590.590.650.640.620.62
MAE0.430.560.590.510.810.85
One-stage training0.580.520.630.620.750.86

The table reads the frozen HEPA encoder with an MLP head because, at these budgets, HEPA with its own predictor head depends strongly on which engines are labelled (sd up to 0.29 across subset draws). The second row is the frozen randomly initialised encoder the reviewer asked for: under the same head the pretrained encoder is ahead of it by +0.12 [+0.06, +0.18] at C-MAPSS-3 2%, and with 1–5% labels it has the higher mean in all 12 C-MAPSS settings. On C-MAPSS-2 and -4, too, the frozen encoder with this head has the higher mean at 2% (0.88 against Chronos-2's 0.78, and 0.89 against 0.75), with intervals that include 0. The revision therefore names the MLP head on the frozen encoder as the setting for scarce labels.

Q5. We now run it on the final architecture: a unified ablation in h-AUROC (7 arms, 5 label budgets, 5 seeds) replaces Table 4. At full labels it shows that finetuning the predictor beats a linear probe on all 5 datasets (+0.01 to +0.30) and that re-initialising it changes h-AUROC by at most 0.02.

We thank the authors for their detailed response and additional experiments. The new results provide further evidence of the effectiveness of HEPA's self-supervised pretrained encoder, particularly under limited labeled data on certain lifecycle datasets. However, this advantage is not consistent across all evaluated settings, and the pretrained predictor weights appear to contribute little to downstream performance. Therefore, we believe the paper should more clearly position its main contributions in light of these findings. Our remaining concerns have been satisfactorily addressed by the authors' response and additional experiments.

AuthorsOpenReview ↗

Thank you, we agree: the architecture is the general contribution, and pretraining pays off on lifecycle data with scarce labels

We thank the reviewer for the follow-up and are glad that the remaining concerns are addressed. We agree with the positioning the reviewer describes; the revised paper follows it, as summarised below, and we will make it explicit in the contributions list of the final version.

1. The main contribution is the horizon-conditioned architecture and its downstream recipe. One architecture with fixed hyperparameters matches or exceeds each baseline on at least 10 of 14 benchmarks across 11 domains (Section 1, Table 1). Within it, finetuning the predictor is what makes the recipe work: at full labels it beats a linear probe on the frozen predictor on all 5 datasets of the unified ablation (+0.01 to +0.30, Table 4). As the reviewer notes, however, its pretrained weights add little, and Section 3.2 says so explicitly: what transfers is the frozen encoder and the horizon-conditioning structure, not the predictor's weights.

2. The pretrained encoder is what makes scarce labels sufficient on lifecycle data, and the paper scopes the claim there. With 1–5% labels, the pretrained encoder has a higher mean than a random encoder under the same head in all 12 C-MAPSS settings (significantly in 10; Table 10). Moreover, on C-MAPSS-3 it has the highest mean of all methods at every budget, ahead of Chronos-2 by +0.15 [+0.05, +0.24] at 2% (Table 2). On GECCO, PSM and MBA, in contrast, the two encoders are level (Appendix P.2). Accordingly, Section 5.4 states that the benefit of pretraining is specific to scarce labels on lifecycle data, and that the advantage is not universal.

We are grateful for the reviewer's engagement throughout the discussion; the reviewer's questions led to the random-encoder control and the unified ablation, which we believe make the paper considerably stronger. We hope the revised paper now reflects the suggested positioning, and that this might be considered in the final assessment.

Reviewer_PBCXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HEPA, a self-supervised framework for time series event prediction. It uses JEPA to predict future representations during pretraining, and then freezes the encoder and finetunes the predictor for event prediction. The model outputs event probabilities over different horizons using a survival CDF. Experiments on 14 benchmarks show competitive performance, especially with limited labels on RUL datasets.

Critical strengths
  1. The paper studies an important problem: predicting future events from multivariate time series with limited labels. The experiments cover many different domains and show promising results, especially on C-MAPSS.

  2. The proposed framework is lightweight, and the authors provide relatively detailed experiments and ablation studies.

Critical weaknesses
  1. Limited novelty. The idea of jointly modeling anomaly detection and RUL is not entirely new. Previous multi-task learning methods have studied similar problems, and predicting event probabilities over different horizons is closely related to survival analysis. JEPA-based future representation prediction has also been explored in prior work. The authors should better clarify the main novelty beyond combining these existing ideas.

  2. The necessity of two-stage training is unclear. The paper uses self-supervised pretraining followed by predictor finetuning. However, the ablation shows that end-to-end finetuning achieves almost the same performance, while pretrained predictor weights provide little benefit over random initialization. Also, the encoder is pretrained separately on each dataset rather than using a large unlabeled corpus. It is unclear how much benefit comes from pretraining and whether this approach is better than directly training the same architecture with supervision.

Questions for authors
  1. Can the authors provide a controlled comparison between one-stage supervised training and the proposed two-stage method using the final architecture, especially under different label budgets?

  2. How does performance change with the amount of unlabeled pretraining data? Is the benefit of JEPA pretraining still clear when only a small dataset is available?

3 comments
AuthorsOpenReview ↗

Novelty, one-stage vs. two-stage training, amount of pretraining data

We thank the reviewer for the thoughtful review. The comparison they suggested, one-stage supervised training on the final architecture at several label budgets, now shows where the two-stage design pays: with 2% of the labels the frozen pretrained HEPA encoder has a higher mean than one-stage training on all four C-MAPSS subsets, by up to +0.32 h-AUROC.

Novelty. Each ingredient has precedent, but HEPA contributes both a new JEPA variant and a new way of combining the parts. Its JEPA is built for events: a causal encoder and a single predictor, conditioned on a continuous horizon Δt, that predicts the representation of the whole future interval (t, t+Δt]. The pretext task therefore asks the same question as the downstream task, so the encoder learns features of the coming interval, and the horizon-conditioned predictor serves directly as the event head, giving an event probability for every horizon on a frozen encoder. By contrast, TS-JEPA predicts masked patches inside the window for classification, and CF-JEPA's three fixed-zone predictors are discarded after pretraining (see the table). This pretraining pays when labels are scarce: at 2% labels the pretrained encoder beats a random one under the same head on all four C-MAPSS subsets, by +0.06 to +0.12 h-AUROC, every interval above 0.

IngredientPrior workIn HEPA
Future-representation predictionCPC; TS-JEPA; MTS-JEPA; CF-JEPAOne predictor conditioned on a continuous Δt; CF-JEPA uses three fixed-zone linear predictors, an EMA target and the same VICReg-style terms, and discards its predictors after pretraining
Hazards composed into a monotone CDFDRSA; DeepHit; Dynamic-DeepHitEvent surface over all horizons on frozen self-supervised features
One metric for anomaly and lifecycle datamulti-task AD + RUL modelsBoth scored by horizon-integrated AUROC (Heagerty and Zheng, 2005); an anomaly stream is read as a sequence of short lifecycles, each ending at the next onset

In addition, under a matched protocol HEPA is ahead of MTS-JEPA on 8 of 9 datasets (App. H). The revision corrects our description of CF-JEPA as in the table.

Q1 and W2. Yes; we ran this comparison on the final architecture at several label budgets. With full labels on C-MAPSS the two approaches are within 0.04 of each other. With few labels, however, the pretrained encoder wins: at 2% labels it beats a random encoder under the same head on all four subsets (by up to +0.12, every interval above 0), and the frozen HEPA encoder has a higher mean than one-stage training on all four, by +0.11 to +0.32 (significantly on 2).

Gain of HEPA over one-stage training, same budget (positive = HEPA better)HEPA, 100%HEPA, 2%Frozen HEPA encoder + MLP head, 2%
C-MAPSS-1−0.04*+0.21+0.32*
C-MAPSS-2−0.01−0.03+0.11
C-MAPSS-3+0.01*+0.02+0.18*
C-MAPSS-4+0.01+0.02+0.14

* paired 95% interval excludes 0. One stage trains the encoder and HEPA's head end to end; the last column trains only a small MLP head, which takes Δt as input, on the frozen HEPA encoder. With full labels, two-stage training therefore mainly buys a smaller trained model (198K or 264K parameters against 2.2M). We agree with the reviewer that the pretrained predictor weights add little over a fresh initialisation (re-initialising changes h-AUROC by −0.01 to +0.02 at full labels); what carries over is the pretrained encoder and the horizon-conditioned architecture.

Q2. The benefit depends on the dataset: more unlabelled data clearly helps on C-MAPSS-3, and its effect on the others is small and mixed. At 100% labels more unlabelled data from the same dataset helps on C-MAPSS-3 (+0.03 from none to all), is within noise on GECCO (+0.08 [−0.02, +0.17]) and slightly lowers C-MAPSS-1. Moreover, pretraining on the four C-MAPSS subsets pooled changes the result on C-MAPSS-1 and -3 by at most 0.01.*

Gain over no pretraining (random encoder), 100% labels25% of the data50%100%pooled C-MAPSS-1 to -4, vs 100%
C-MAPSS-1−0.01−0.01−0.02*+0.01
C-MAPSS-3+0.03*+0.03*+0.03*−0.01
GECCO+0.08+0.07+0.08–

* paired 95% interval excludes 0. No pretraining means the frozen random encoder with the same predictor finetuning. We did not test a large cross-domain corpus, and the revision states this as a limitation. At 2% labels pretraining on all the data lowers the mean on GECCO (0.57 → 0.45) but raises it on C-MAPSS-1 (0.59 → 0.73) and C-MAPSS-3 (0.73 → 0.77), the regime where Q1 shows the pretrained encoder pays.

AuthorsOpenReview ↗

Two-stage vs. one-stage: the pretrained encoder is ahead when labels are scarce

Thank you again for your review; your question about the two-stage recipe prompted a direct test, and the result supports the recipe where it matters most. At full labels, one-stage training does as well (Table 4), as one would expect when labels are plentiful. With scarce labels, however, the two-stage recipe does better: at 2% labels the frozen pretrained encoder reaches 0.93 h-AUROC on C-MAPSS-3 against 0.75 for one-stage training, and it stays ahead at 1, 2 and 5% on both C-MAPSS-1 and -3 (Table 2). This is exactly the regime HEPA is designed for, and Section 5.4 states the claim for it. Happy to answer anything else before the discussion closes tomorrow.

AuthorsOpenReview ↗

Novelty (Section 2) and one-stage vs. two-stage training (Table 2)

Thank you again for your review. In case it is useful before the discussion closes today: Sections 2 and 3 set out what HEPA adds to prior time-series JEPAs. Its single predictor is conditioned on a continuous horizon and predicts the representation of the whole future interval (t, t+Δt], the same interval the event question asks about, and this horizon-conditioned structure is kept as the event head, whereas prior JEPAs typically discard the pretraining head and probe only the encoder. Table 2 gives the one-stage against two-stage comparison you asked for, at 1, 2 and 5% labels. We would be glad to hear whether this addresses your concerns.

Reviewer_KvYLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HEPA, a multivariate time-series event prediction framework for label-scarce settings. It uses JEPA-style future representation prediction for self-supervised pretraining, followed by a frozen encoder and a fine-tuned predictor to estimate event probabilities over multiple future horizons. Its discrete-time survival formulation ensures that cumulative event probabilities are non-decreasing with the prediction horizon. Experiments across 14 benchmarks spanning 11 domains demonstrate competitive performance and label efficiency on several lifecycle prediction tasks.

Critical strengths

1.Practical and parameter-efficient. HEPA combines self-supervised time-series representation learning with lightweight downstream adaptation. Its unified event probability formulation facilitates comparisons across tasks with different prediction targets and evaluation protocols.

2.Broad empirical evaluation. The paper covers multiple domains, label budgets, and failure cases. The comparison with MAE is particularly valuable for isolating the effects of different pretraining objectives under a largely consistent downstream pipeline.

Critical weaknesses

1.The independent contribution of JEPA remains unclear. The predictor initialization ablation shows only a small difference between pretrained and randomly initialized predictors, but both use a pretrained encoder. A frozen randomly initialized encoder baseline would help quantify the contribution of JEPA pretraining.

2.Baseline comparisons involve confounding factors. HEPA and Chronos-2 use different output parameterizations, making it difficult to attribute performance differences solely to representation quality. Comparisons with specialized event prediction methods under a unified evaluation protocol are also limited.

Questions for authors

1.Can the authors further clarify the contribution of JEPA pretraining through a frozen randomly initialized encoder baseline or existing ablation results?

2.How much of HEPA's advantage over Chronos-2 comes from its monotonic output parameterization rather than representation quality?

3.How does HEPA compare with existing event prediction methods under comparable evaluation protocols? If additional experiments are not feasible, could the authors better clarify the limitations of the current comparisons?

2 comments
AuthorsOpenReview ↗

Random-encoder control, head vs. representation, comparison scope

We thank the reviewer for the constructive review. The control they suggested, a frozen randomly initialised encoder, now shows directly where JEPA pretraining helps most: with 1–5% labels the pretrained encoder is ahead in all 12 C-MAPSS settings.

Q1 and W1. We added the frozen randomly initialised encoder, and it shows where JEPA pretraining contributes: when labels are scarce. With 1–5% labels the pretrained encoder has the higher mean in all 12 C-MAPSS cells, by +0.04 to +0.15 h-AUROC (significantly in 10), and up to 10% labels it is never significantly behind. With full labels the trained head closes most of the gap. Both encoders get the same small MLP head, which takes the horizon Δt as input (264K parameters; the paper gives this head to every frozen foundation model, 395K for Chronos-2's wider features).

Pretrained minus random frozen encoder, same headC-MAPSS-1C-MAPSS-2C-MAPSS-3C-MAPSS-4
100% labels−0.02+0.02+0.03*+0.03*
10%−0.01+0.04*+0.05*+0.05*
5%+0.04+0.04*+0.11*+0.07*
2%+0.08*+0.06*+0.12*+0.12*
1%+0.08*+0.15*+0.14+0.11*

* paired 95% interval excludes 0. On the anomaly datasets GECCO, PSM and MBA the two encoders are level with this head at every budget (every interval includes 0), so the revision states the pretraining benefit for the regime where it appears: C-MAPSS with few labels.

Q2 and W2. Mostly representation. In fact, in the paper only the foundation-model comparisons mix the two: PatchTST, iTransformer and MAE use HEPA's own horizon-conditioned head and hazard output, while the frozen foundation models, Chronos-2 among them, got an unconstrained MLP head. To separate the two, we now run both encoders under both heads. With the same unconstrained head HEPA's encoder has a higher mean than Chronos-2 on 7 of 9 event datasets (significantly ahead on 5), and switching both to the hazard head moves the gap by more than 0.05 on only 2 (BATADAL 0.00 to +0.16, GECCO +0.11 to +0.02).**

HEPA − Chronos-2, 100% labelsGECCOPSMMBAETTm1BATADALSMAPWeatherBeijing-AQVIX
both encoders, unconstrained head+0.11*+0.09*+0.26*+0.09*0.00+0.04*−0.07*+0.01+0.02
both encoders, hazard→CDF head+0.02+0.12*+0.25*+0.10*+0.16*+0.06*−0.07*−0.01−0.01
HEPA as submitted vs Chronos-2 + hazard head+0.19*+0.12*+0.37*+0.10*+0.26*+0.11*−0.07*+0.02−0.03

Row 1 is the ablation without the monotone hazard output, and rows 1–2 hold the head fixed. Row 3 minus row 2 is what HEPA's finetuned predictor adds, which is largest on GECCO and MBA. All rows are fresh runs (5 seeds), so they differ from Table 5 in App. G.

Q3. We added one comparison under our protocol and, as the reviewer suggests, state the limits of the current comparisons. We now compare with a discrete-time hazard network trained end to end: the same encoder and hazard head, fitted directly to the labels with a per-horizon BCE (one-stage training). With 2% of the labels the frozen HEPA encoder of Q1 has the higher mean on all four C-MAPSS subsets, by +0.11 to +0.32; the paired interval is above 0 on two of the four and includes 0 on the other two. We did not run specialised survival models such as DeepHit or Dynamic-DeepHit under our protocol, and the revision states this as a limitation. MTS-JEPA, the closest self-supervised method for anomaly prediction, is compared under a matched protocol in App. H, where HEPA is ahead on 8 of 9 datasets.

AuthorsOpenReview ↗

Random-encoder baseline: pretrained encoder ahead in all 12 low-label C-MAPSS settings

Thank you again for your review. The random-encoder baseline you suggested is now in the paper, and it turned out to be a useful test: with the same head, the pretrained encoder has the higher mean in all 12 C-MAPSS settings with 1–5% labels, significantly in 10 (Table 10). Happy to answer anything else before the discussion closes tomorrow.

Reviewer_Rkf2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HEPA, a self-supervised framework for multi-horizon event prediction in multivariate time series. A causal Transformer learns to predict future interval representations using a fixed target encoder and variance-covariance regularization. Downstream adaptation freezes the encoder and fine-tunes a horizon-conditioned predictor and event head, producing a monotone event-probability CDF through discrete hazards. Across 14 benchmarks in 11 domains, HEPA reports the best mean h-AUROC on 9 full-label benchmarks while tuning approximately 198K parameters.

Critical strengths
  1. The paper effectively communicates the core ideas, methodologies, and results, making it accessible to readers with a background in the field.
  2. The downstream design is parameter-efficient and shows promising performance with limited labels on C-MAPSS.
  3. The paper provides extensive comparisons across 14 benchmarks and transparently reports negative results and methodological limitations.
Critical weaknesses
  1. The evaluation does not clearly distinguish early warning from predicting the continuation of ongoing anomalies, making the early-warning claims difficult to assess. The comparative baselines are also not SOTA models.
  2. The ablations use different architectures and metrics, making it difficult to isolate the contribution of predictor fine-tuning and identify the source of the performance gains.
Questions for authors
  1. Which trainable parameters receive gradients from SIGReg in Appendix I.5, given that the target encoder is frozen and its output is detached?
  2. Could the authors provide a unified ablation study using the final architecture and h-AUROC to isolate the contributions of encoder pretraining and predictor fine-tuning?
  3. Table 2 specifies the number of labeled engines, but are validation labels used for early stopping included in these budgets? Do the reported seeds also vary the labeled subset, and how sensitive are the results to this choice?
4 comments
AuthorsOpenReview ↗

SIGReg gradient path, unified ablation, label budgets, early-warning scoring

We thank the reviewer for these precise questions. The experiments they asked for show a clear win for pretraining: with only 2% of the labels, the pretrained encoder beats a random one under the same head on all four C-MAPSS subsets, by up to +0.12 h-AUROC.

Q1. We have resolved this, and the results do not depend on the choice of regulariser. In App. I.5 the target encoder was trained, so SIGReg's gradients went to it (patch embedding, two bidirectional layers, attention pool). In the main configuration the target is fixed and detached, so SIGReg reaches only the predictor and the context encoder. We reran the comparison in the main configuration, and the two regularisers perform the same: the mean difference over 13 datasets is below 0.01, with the paired interval including 0 on 11. The revision replaces I.5 with this run.

Q2 and W3. Yes, and the unified ablation separates the two contributions. At 100% labels the gain comes mostly from finetuning the predictor and much less from its pretrained weights: finetuning beats a linear probe on all 5 datasets (+0.01 to +0.30), while re-initialising the predictor changes h-AUROC by at most 0.02. Encoder pretraining, in contrast, pays with few labels: at 2% labels and under the same head, the pretrained encoder beats a random one on all four C-MAPSS subsets (+0.06 to +0.12, every interval above 0).

Gain of HEPA over each variant (positive = HEPA better)C-MAPSS-1C-MAPSS-3GECCOPSMMBA
frozen random encoder (HEPA's head), 100% labels−0.02*+0.03*+0.08+0.02+0.03
linear probe on the frozen predictor, 100%+0.01+0.03*+0.30*+0.04+0.07
re-initialised predictor, 100%−0.010.00+0.010.00+0.02
one-stage training, 100%−0.04*+0.01*+0.100.00−0.01

Each cell is the h-AUROC of HEPA as submitted minus that of the variant in the row, so +0.03 means HEPA scores 0.03 higher. * paired 95% interval excludes 0. The grid (7 arms, 5 label budgets, 5 seeds) replaces Table 4 in the revision.

2% labels, same MLP head: pretrained minus random frozen encoderC-MAPSS-1C-MAPSS-2C-MAPSS-3C-MAPSS-4
h-AUROC+0.08*+0.06*+0.12*+0.12*

Q3. Thank you for catching this; we have fixed it. Validation labels were used for early stopping but not counted in the budget, and the seeds redrew the training engines but not the validation engines. The budget now counts validation labels, and every seed redraws both. This also answers the sensitivity question: most of the low-label variance comes from which engines are labelled (sd 0.23–0.29 across subset draws, 0.00–0.11 across initialisations).

sd of h-AUROC on the two subsets of Table 2 (C-MAPSS-1, -3), 2% and 5% labelsacross 5 subset drawsacross 3 initialisations
HEPA0.23–0.290.00–0.11
one-stage training0.06–0.130.04–0.09

In the submission, by contrast, the 15 validation engines stayed the same for every seed. A 2% run now sees 2 training engines and 1 validation engine, with at least one engine in each split. The revision therefore reports every new low-label number over subset draws.

W1. We now separate early warning from the continuation of an ongoing anomaly, and scoring only true early warnings costs little. We score only the windows whose context ends before the event has started (onset-only h-AUROC). This score is at most 0.05 below the all-window score of the same runs on each of 9 event datasets, and above chance on 7 of them.

HEPA, 100% labelsGECCOMBAPSMETTm1BATADALSMAPWeatherBeijing-AQVIX
onset-only0.90*0.71*0.530.78*0.56*0.64*0.81*0.73*0.49
share of Δt=1 positives already inside an event0.990.981.000.860.971.000.910.890.00

* 95% interval above 0.5. Onset-only scoring treats an anomaly stream as a series of short lifecycles, each ending at the next onset; on run-to-failure data, in contrast, every window already ends before the failure.

W2. The comparison does include current models: Chronos-2 (2025), a 119M-parameter pretrained time-series foundation model, and MTS-JEPA (2026), a self-supervised method for anomaly prediction compared under a matched protocol in App. H. Chronos-2 is the strongest comparator for our claim because it is a pretrained encoder that can be given exactly the same labels and head as HEPA. Domain-specific methods enter only through Table 1's domain metrics, a limitation the revision states. We now also run Chronos-2 under both output heads and add one-stage training, a discrete-time hazard network trained end to end, and the trained baselines (PatchTST, iTransformer, MAE) use HEPA's own horizon-conditioned head and hazard output, so the comparison isolates the encoder and its training.

AuthorsOpenReview ↗

Unified ablation on the final architecture, in h-AUROC (Table 4)

Thank you again for your review, it made the paper better. In case it helps before the discussion closes tomorrow: the ablation you asked for now uses the final architecture and h-AUROC (Table 4), and the label-budget details are in Table 2, Appendix P.1 and Section 5.4. Happy to answer anything that is still unclear.

Thank the authors for the detailed rebuttal and experiments. The rebuttal has addressed most of my questions and provides substantially stronger evidence for the effectiveness of the proposed pretraining strategy. I would support raising the score.

AuthorsOpenReview ↗

Thank you for raising the score

Thank you for the careful follow-up. Your questions about the unified ablation and the label budgets made the paper clearer and more rigorous.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of new experiments and the revised PDF

We thank the reviewers for their thoughtful reviews. New experiments address every question.

In particular, the new runs strengthen the paper's label-efficiency result. With 2% of the labels, the frozen HEPA encoder with a small MLP head beats the same head on a frozen random encoder on all four C-MAPSS subsets, by +0.06 to +0.12 h-AUROC. Moreover, it has a higher mean than training encoder and head end to end, by +0.11 to +0.32, with the paired 95% confidence interval above 0 on 2 of the 4 subsets.

New experimentWhat it shows
Unified ablation on the final architecture in h-AUROC: 7 arms, 5 label budgetsFinetuning the predictor beats a linear probe on all 5 datasets; its pretrained weights add at most 0.02 over a fresh initialisation
Frozen randomly initialised encoderWith 1–5% labels, JEPA pretraining has the higher mean than a random encoder in all 12 C-MAPSS settings, by up to +0.15 h-AUROC
PatchTST, iTransformer, MAE, Chronos-2 and one-stage training at 1, 2 and 5% labelsOn the Table 2 subsets, the frozen HEPA encoder has the highest mean on C-MAPSS-3 at every budget (+0.15* over Chronos-2 at 2%) and matches the best baselines on C-MAPSS-1
Monotone hazard head against an unconstrained head, on the HEPA and Chronos-2 encoders, 9 event datasetsThe head moves the gap between the encoders by more than 0.05 on only 2 of the 9
Onset-only h-AUROC on 9 event datasetsPredicting an event before it starts scores at most 0.05 below the all-window score of the same runs
Pretraining on 0, 25, 50 or 100% of the dataset (0% = random encoder), and on the four C-MAPSS subsets pooled; full labels for finetuningPretraining on all vs none: C-MAPSS-3 +0.03*, GECCO +0.08 [−0.02, +0.17], C-MAPSS-1 slightly lower. Pooled C-MAPSS pretraining: within 0.01 on C-MAPSS-1 and -3
SIGReg in place of our variance–covariance regulariserThe same h-AUROC: the mean difference over 13 datasets is below 0.01

All new numbers are h-AUROC averaged over 5 seeds, and each seed also redraws the labelled subset. Differences are absolute and computed before rounding, and * marks a paired 95% confidence interval that excludes 0. The label budget now counts the validation labels used for early stopping as well as the training labels. Every number comes from fresh runs under this protocol, so some differ from the submitted tables; the revision reports these runs in the replaced Tables 2 and 4 and in a new appendix that states the protocol.

Finally, the revised PDF replaces Table 2 by the low-label comparison and Table 4 by the unified ablation, adds the random-encoder control in a new appendix, states assumption A1 per horizon with Proposition 1 moved to the appendix, and corrects our description of CF-JEPA.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.