- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates the use of frozen foundation model representations for change point detection (CPD) in multivariate time series and generic event boundary detection (GEBD) in videos. The authors aim to perform these tasks without additional task-specific training. The authors propose a framework called GAS-FM. It extracts representations from pretrained frozen foundation models. These high dimensional representations are then converted into univariate signals using Generalized Autosimilarity Statistics (GAS). The resulting signals are processed using existing statistical change-point detection methods to identify changes in the input sequences.
The paper also introduces Change Encoding Maps and Change Encoding Rate (CER). These are used to evaluate whether the extracted representations preserve information about changes in the input sequences.
The proposed framework is evaluated on multiple benchmarks for multivariate time series CPD and video based GEBD. Different pretrained foundation models are considered in the experiments. The results are compared with existing state-of-the-art (SOTA) methods
Critical strengths
- The paper explores the use of pretrained foundation models for change-point detection without requiring additional task-specific training. This reduces the need to train separate models for change detection. The method is evaluated on both time-series and video datasets, showing its applicability across different types of sequential data.
- The experiments include 13 time series foundation models and four vision foundation models. This allows the authors to compare different model architectures and sizes. The results show that larger models do not always perform better. This suggests that the model architecture and the quality of learned representations are important for change-point detection.
- The proposed Change Encoding Rate (CER) provides a way to measure whether foundation-model representations capture changes in the original data. It also helps distinguish limitations in the extracted representations from those in the detection process. The authors observe that a high CER does not always result in a high F1 score.
- The framework uses frozen foundation models and avoids additional representation training. This reduces the training cost compared with methods such as TS-CP2. The paper also includes ablation experiments on denoising, similarity measures, detection methods, and noise robustness. These experiments help explain the contribution of different components of the framework
Critical weaknesses
- The theoretical analysis does not fully establish whether GAS preserves all relevant changes in the data. Theorem 2 assumes that the scalarization function is change-preserving. It then shows that the resulting signal satisfies the conditions required by SCAN. However, this does not prove that GAS satisfies this assumption in all cases. Proposition 1 considers certain changes in the mean, covariance, and lag-one dependence, but only under specific conditions. The authors should clarify which types of changes GAS can preserve and which changes may remain undetected.
- Some mathematical expressions in the paper appear to contain errors. In Section 3.3, the normalization used for cosine similarity under the Mahalanobis geometry appears to be missing a square root. Similarly, Proposition 1 in Appendix A.4 appears to contain an extra factor of (1/d) in the second-moment expression. The authors should verify these equations and clarify whether these errors affect the theoretical results.
- The novelty of GAS compared with existing similarity-based methods is not sufficiently clear. GAS includes commonly used measures such as dot product, cosine similarity, and covariance-adjusted similarity. The framework also relies on SCAN, an existing change-point detection method. Comparisons with simpler approaches, such as distances between consecutive embeddings and PCA-based projections, would help establish the specific contribution of GAS.
- The proposed method does not consistently outperform existing methods across all datasets. On Kinetics-GEBD, it achieves an average F1-score of 0.849, compared with 0.845 for FlowGEBD. This improvement is relatively small. On TAPOS, the proposed method achieves 0.569, compared with 0.623 for FlowGEBD. Similarly, on HASC, GAS-Moirai-Base achieves 0.502, compared with 0.530 for RuLSIF. The authors should provide a more detailed analysis of the lower performance on HASC and TAPOS.
- The runtime comparison mainly highlights the reduction in training cost rather than inference speed. Table 7 shows that the proposed method requires 1.72 seconds on Yahoo A4 using a GPU, while TS-CP2 requires only 0.65 seconds for inference after training. Although the proposed method avoids additional training, it does not achieve faster inference in this comparison. The authors should provide a detailed runtime breakdown for feature extraction, similarity calculation, and change-point detection.
- The runtime comparison does not clearly distinguish training time from inference time. The comparison with TS-CP2 includes 100 epochs of training, while the proposed method does not require additional training. This demonstrates a reduction in training cost but does not necessarily establish faster inference. Table 7 reports a runtime of 1.72 seconds for the proposed method on Yahoo A4 using a GPU. In comparison, TS-CP2 requires 0.65 seconds for inference after training. The authors should report separate runtimes for feature extraction, similarity calculation, and change-point detection to make the comparison more informative
Questions for authors
Q1 Theorem 2 assumes that the scalarization function is change-preserving. The authors should specify the conditions under which GAS satisfies this assumption and provide examples of changes that GAS may fail to detect. Q2 The equations in Section 3.3 and Proposition 1 appear to contain mathematical inconsistencies. The authors should verify and correct these expressions and explain whether the corrections affect the theoretical results. Q3 The proposed GAS formulation includes several existing similarity measures. The authors should add comparisons with consecutive embedding distances and PCA-based projections to demonstrate the specific advantage of GAS. Q4 The proposed method performs worse than some existing methods on HASC and TAPOS. The authors should analyze these results and identify whether the lower performance is caused by the foundation model representations, GAS transformation, or the detection method. Q5 CER relies on known change-point labels for evaluation. The authors should clarify how CER can be used without ground-truth annotations and verify the false change-encoding rate using separate no-change samples. Q6 Table 7 shows higher inference time for GAS-FM than TS-CP2 on Yahoo A4. The authors should provide a breakdown of feature extraction, GAS computation, and change-point detection times to clarify the computational advantages of the proposed framework. Q7 Figure 4(a) shows a large improvement in F1-score when total-variation denoising is used. How much of the final improvement comes from denoising rather than GAS itself? Can the authors compare GAS with simpler embedding-based methods using the same denoising settings?
Acknowledging authors response
I read the authors response. I am glad that the comments helped to further investigate this important topic.