- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework named ZEST to deal with the time series state detection task. ZEST identifies state changes and recurring states in unseen time series by reading their plots with a fine-tuned vision-language model. It builds training examples from unlabeled real signals and predicts both a segmentation and a hierarchy for grouping similar states.
Critical strengths
This paper develop a framework that combines real waveform fragments with controlled state scheduling to generate labeled multivariate sequences without manual annotation. The proposed framework can predict boundaries and recurring state identities on unseen sequences.
Critical weaknesses
-
The main concern is the paper has limited model novelty. The state detector uses an existing VLM, and a standard autoregressive loss. The main contribution lies on task related supervision construction and output formulation.
-
The motivation is not clear. State detection is closely related to time-point classification. This has been studied by many previous methods, like TNC[1] and PLanTS[2]. And the paper mainly discusses limitations of contrastive methods that construct training pairs using temporal proximity, but ignores other contrastive methods use signal similarity or address false negatives, like SoftCLT[3] and PLanTS[2].
-
For the experiments, Zest is trained on a large corpus with generated state labels, where baselines use different training setups. Comparing against other detectors trained on the same corpus or against representation learning methods adapted for time point classification may help the results more convincing
[1]Unsupervised representation learning for time series with temporal neighborhood coding.
[2]PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Series.
[3]Soft contrastive learning for time series.
Questions for authors
-
Beyond the supervision construction and output format, what is technically new in the state detector itself?
-
What is the fundamental difference between the state detection task studied in this paper and time-point classification? Does ZEST handle any setting or capability that cannot be addressed by existing time-point classification methods?
-
The motivation focuses on temporal-proximity-based contrastive learning, but how does it apply to methods such as SoftCLT and PLanTS that use signal similarity or address false negatives?
-
Since ZEST is trained on generated sequences with task-specific state labels, while the baselines use different training setups, how to determine whether its gains come from the generated supervision or from the VLM-based structured prediction approach?
4 comments
[2/3] Response to W2 and Q2–Q3: Task Formulation and Motivation
Thank you for these comments. We address three points: (1) the relationship between time series representation learning, state detection, and time-point classification (W2 and Q2 Sentence 1); (2) ZEST’s objective and novelty (Q2 Sentence 2); and (3) how false-negative handling relates to our motivation (Q3).
1. Representation learning, state detection, and time-point classification
Time series representation learning maps time points, windows, or entire instances into representations for downstream tasks. TNC [8], PLanTS [9], and SoftCLT [10] are self-supervised contrastive representation learning methods. Downstream tasks still require a predictor or decoder.
Time series state detection (TSSD) is an established task [1–4] that identifies changes in the system states reflected in time series. Its results can be expressed as pointwise state labels. Representative deep learning methods, including Time2State [1] and E2USD [2], follow a two-stage pipeline: learning window representations through contrastive learning, then clustering them to obtain state labels. Contrastive representation learning is therefore an important technical foundation for these TSSD methods. We discuss these methods in Sections 1, 3, and 6.
Time-point classification describes an output format—assigning a class label to each time point—but does not by itself define a specific task. For example, anomaly detection [6–7] and state detection [1–4] can both produce pointwise labels but differ in semantics and objectives. The main classification experiments in PLanTS and SoftCLT are instance-level; PLanTS also analyzes state transitions through representation trajectories. TNC evaluates window-based hidden-state classification and provides a representation learning foundation for TSSD. Time2State analyzes and compares TNC and develops an objective tailored to state detection.
In summary, contrastive learning underpins these two-stage TSSD methods, and state detection is a specific task whose results can be expressed as pointwise state labels.
2. ZEST’s objective and novelty
Objective: ZEST aims to enable zero-shot, end-to-end TSSD: once trained at scale, it directly predicts state boundaries and identities for unseen sequences, without target labels, parameter updates, hyperparameter tuning, or target-side clustering.
Novelty: Existing TSSD methods [1–2] follow a “representation learning + clustering” pipeline. Their original pipelines do not support the zero-shot, end-to-end detection targeted by ZEST. Section 3 highlights three issues.
- First, many false negatives persist even after filtering, impairing representation learning (Figure 2(a–b)).
- Second, even with encoders pretrained using ground-truth state relations and frozen at inference, these methods do not consistently outperform their target-trained counterparts under the tested training budget (Figure 2(c–d)).
- Finally, improving the encoder alone does not eliminate target-side clustering. State labeling still requires this separate step, so the pipeline remains two-stage rather than end-to-end.
3. How false-negative handling relates to our motivation
First, false-negative handling cannot eliminate the need for target-side fitting in two-stage TSSD, although it can improve representations. SoftCLT uses signal and temporal distances to assign soft pair weights, while PLanTS incorporates periodicity and signal similarity.
Second, our analysis already includes improvements to representation learning and false-negative handling. Specifically, Time2State analyzes limitations of TNC and introduces LSE-Loss to model intra- and inter-state relationships. E2USD further develops false negative cancellation contrastive learning to filter negatives based on similarity.
Third, Section 3 shows that false negatives persist after filtering (Figure 2(a–b)). Even pretraining encoders with ground-truth state relations from our generated corpus yields no consistent detection gains over target-specific training (Figure 2(c–d)). These results show that improving pair supervision alone does not ensure consistent detection transfer.
These findings motivate ZEST’s joint design of state supervision construction and direct state structure prediction, enabling detection without target-specific fitting.
We acknowledge the contributions of TNC, SoftCLT, and PLanTS and have added their citations to the manuscript.
[3/3] Response to W3 and Q4: Experimental Comparisons and Sources of Improvement
Thank you for these suggestions. We first discuss our contributions and experimental design (W3), then discuss the sources of performance improvement (Q4).
1. Contributions and experimental validation (W3)
[W3 Sentence 1] As discussed above, our goal is zero-shot, end-to-end TSSD without target-specific fitting. Our technical contributions are scalable supervision construction (data strategy) and adaptation for state detection (adaptation strategy). Our experiments therefore evaluate the two strategies. We adapt Qwen 3.5 2B to structured state prediction using our generated supervision. In Table 1, the resulting ZEST model outperforms both the original backbone and most baselines on unseen sequences. Together, these results validate the effectiveness of our technical contributions.
[W3 Sentence 2] Section 3 already examines your suggestion by pretraining the encoders of existing two-stage methods on our generated corpus. We construct positive and negative pairs directly from state labels, avoiding false negatives introduced by temporal-proxy pairing. Under the tested training budget, these methods do not consistently outperform their target-trained counterparts (Figure 2(c–d)).
In summary, the main experiments evaluate ZEST’s detection capability, while Section 3 examines how existing methods perform with the generated data and state supervision.
2. Sources of performance improvement (Q4)
Both the generated supervision and the structured prediction contribute to the final performance. The manuscript evaluates their roles through corresponding experiments, as detailed below.
For supervision construction, ZEST-leaf uses leaf-level segmentation without granularity selection, yet ranks first in 11 of 12 comparisons and outperforms the original Qwen 3.5 2B across all benchmarks (Section 5.2). This demonstrates the overall benefit of task adaptation with generated supervision. Section 5.4 further compares different data construction methods. Our prototype mining and expansion strategy outperforms random cropping, validating the supervision construction strategy itself.
For structured generation, ZEST-oracle’s gains over ZEST-leaf show that the generated merge trees contain better segmentations than the default leaf-level output (Section 5.2). Oracle selects granularity using ground-truth labels, showing the best performance attainable among these candidates. Section 5.3 further shows that selecting granularity with a few target labels improves detection at time points not used for selection, demonstrating the practical value of hierarchical outputs. Furthermore, Appendix E.5.1 compares different input representations. Image input achieves better detection performance or efficiency than the numerical encoder and text tokens, supporting the value of visual input in our task adaptation strategy.
Together, these experiments show that both supervision construction and task adaptation contribute to ZEST’s detection performance.
[1] Chengyu Wang, Kui Wu, Tongqing Zhou, and Zhiping Cai. 2023. Time2State: An Unsupervised Framework for Inferring the Latent States in Time Series Data. Proceedings of the ACM on Management of Data, 1(1), Article 17, 1–18.
[2] Zhichen Lai, Huan Li, Dalin Zhang, Yan Zhao, Weizhu Qian, and Christian S. Jensen. 2024. E2Usd: Efficient-yet-effective Unsupervised State Detection for Multivariate Time Series. Proceedings of the ACM Web Conference 2024, 3010–3021.
[3] Yasuko Matsubara, Yasushi Sakurai, and Christos Faloutsos. 2014. AutoPlait: Automatic Mining of Co-evolving Time Sequences. Proceedings of the ACM SIGMOD International Conference on Management of Data, 193–204.
[4] Arik Ermshaus, Patrick Schäfer, and Ulf Leser. 2025. CLaP – State Detection from Time Series. Proceedings of the VLDB Endowment, 19(1), 70–83.
[5] Arik Ermshaus, Patrick Schäfer, and Ulf Leser. 2023. ClaSP: Parameter-free Time Series Segmentation. Data Mining and Knowledge Discovery, 37, 1262–1300.
[6] Shreshth Tuli, Giuliano Casale, and Nicholas R. Jennings. 2022. TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data. Proceedings of the VLDB Endowment, 15(6), 1201–1214.
[7] Zahra Zamanzadeh Darban, Geoffrey I. Webb, Shirui Pan, Charu C. Aggarwal, and Mahsa Salehi. 2024. Deep Learning for Time Series Anomaly Detection: A Survey. ACM Computing Surveys, 57(1), Article 15, 1–42.
[8] Sana Tonekaboni, Danny Eytan, and Anna Goldenberg. 2021. Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding. International Conference on Learning Representations (ICLR).
[9] Jia Wang, Xiao Wang, and Chi Zhang. 2025. PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Series. arXiv preprint arXiv:2509.05478.
[10] Seunghan Lee, Taeyoung Park, and Kibok Lee. 2024. Soft Contrastive Learning for Time Series. International Conference on Learning Representations (ICLR).
Follow-up on Our Responses
We sincerely appreciate the time and effort you have devoted to reviewing our work. Your questions have helped us clarify the task setting and present our contributions and experimental evidence more clearly.
As the discussion period draws to a close, we would be grateful to hear whether our responses and revisions address your concerns, and would be happy to provide any further clarification.
Thank you again for your time and consideration!
[1/3] Response to W1 and Q1: Novelty and Technical Contributions
We thank the reviewer for the comments and questions on the novelty, motivation, and experimental evaluation of our work. Below, we clarify ZEST’s task setting and technical contributions, explain the rationale behind its design, and discuss the experimental evidence supporting our approach.
For Weakness 1 and Question 1, we recognize that the original manuscript may not have sufficiently explained our contributions, potentially giving the impression that ZEST simply fine-tunes an existing VLM.
In the following, we would like to clarify our novelty and technical contributions.
1. Novelty: data construction and task adaptation for TSSD
Our contributions are twofold: scalable supervision construction from unlabeled real-world time series and hierarchical structured generation for direct detection on unseen sequences. Using a VLM backbone and an autoregressive loss is not our main contribution.
We focus on constructing high-quality TSSD supervision at scale and adapting pretrained VLMs for state detection. This goal has a clear motivation. On the one hand, VLMs’ visual priors show promise for end-to-end state detection, but performance is limited without task adaptation (Table 1). On the other hand, the two-stage methods in Section 3 show limited transfer gains after pretraining and still require target-specific clustering. We therefore investigate how data construction and task adaptation can jointly enable zero-shot, end-to-end detection.
2. Technical contributions: addressing supervision acquisition and state structure modeling
The first challenge is to combine real-world dynamics with precise state supervision. Real time series often lack dense state labels, while purely synthetic signals may miss real-world diversity. Thus, we mine and expand reusable waveform prototypes from unlabeled real signals using prediction consistency (Section 4.1.1), then schedule multivariate states to control their differences, durations, and recurrence (Section 4.1.2). This preserves real-world patterns while determining state identities and boundaries, enabling TSSD supervision at scale (Section 4.1).
The second challenge is to adapt general visual capabilities for state structure prediction. TSSD requires accurate state identification while accommodating different state granularities. Thus, we develop hierarchical structured generation with joint supervision of segment boundaries, state identities, and a state merge tree. One prediction yields a default leaf-level segmentation and consistent coarser alternatives, without target-specific training or clustering (Section 4.2).
Together, these designs enable zero-shot, end-to-end TSSD with a deep learning model. Data construction ablations validate the supervision strategy, while comparisons using the same backbone and results on six benchmarks support the full adaptation framework (Sections 5.2 and 5.4). We have clarified our contributions and motivations in the Abstract, Introduction and Section 3.3, and explained in Sections 4.1–4.2 how our designs address these challenges.