acceptodds
Under review as a conference paper at ICLR 2027

Change-point detection using foundation models

Abstract

Foundation models have achieved remarkable success across a broad range of tasks, including time-series forecasting and video analysis, yet their potential for change detection remains largely unexplored. We investigate whether pretrained foundation-model representations encode change-relevant information and how such representations can be used for change detection across different contexts without task-specific training. This question is studied in two settings: change-point detection (CPD) in multivariate time series and generic event boundary detection (GEBD) through temporal video segmentation. Our model-agnostic framework extracts frozen foundation-model representations and projects them onto a change-sensitive univariate signal using a data-driven transformation, enabling the direct application of established univariate CPD methods. Across multiple foundation models and benchmarks, the resulting training-free approach is competitive with specialized CPD and GEBD methods, achieving strong performance on CPD benchmarks and matching or exceeding several GEBD baselines at moderate boundary tolerances. It also eliminates the model-training cost required by many self-supervised approaches. Our results suggest that pretrained foundation-model representations can provide a fast and general-purpose basis for change detection across both time-series and video data.

Primary area:
learning on time series and dynamical systems
Keywords:
Foundation Models, Change-Point Detection, Generic Event Boundary Detection, Multivariate Time Series, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qfyJOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates the use of frozen foundation model representations for change point detection (CPD) in multivariate time series and generic event boundary detection (GEBD) in videos. The authors aim to perform these tasks without additional task-specific training. The authors propose a framework called GAS-FM. It extracts representations from pretrained frozen foundation models. These high dimensional representations are then converted into univariate signals using Generalized Autosimilarity Statistics (GAS). The resulting signals are processed using existing statistical change-point detection methods to identify changes in the input sequences.

The paper also introduces Change Encoding Maps and Change Encoding Rate (CER). These are used to evaluate whether the extracted representations preserve information about changes in the input sequences.

The proposed framework is evaluated on multiple benchmarks for multivariate time series CPD and video based GEBD. Different pretrained foundation models are considered in the experiments. The results are compared with existing state-of-the-art (SOTA) methods

Critical strengths
  1. The paper explores the use of pretrained foundation models for change-point detection without requiring additional task-specific training. This reduces the need to train separate models for change detection. The method is evaluated on both time-series and video datasets, showing its applicability across different types of sequential data.
  2. The experiments include 13 time series foundation models and four vision foundation models. This allows the authors to compare different model architectures and sizes. The results show that larger models do not always perform better. This suggests that the model architecture and the quality of learned representations are important for change-point detection.
  3. The proposed Change Encoding Rate (CER) provides a way to measure whether foundation-model representations capture changes in the original data. It also helps distinguish limitations in the extracted representations from those in the detection process. The authors observe that a high CER does not always result in a high F1 score.
  4. The framework uses frozen foundation models and avoids additional representation training. This reduces the training cost compared with methods such as TS-CP2. The paper also includes ablation experiments on denoising, similarity measures, detection methods, and noise robustness. These experiments help explain the contribution of different components of the framework
Critical weaknesses
  1. The theoretical analysis does not fully establish whether GAS preserves all relevant changes in the data. Theorem 2 assumes that the scalarization function is change-preserving. It then shows that the resulting signal satisfies the conditions required by SCAN. However, this does not prove that GAS satisfies this assumption in all cases. Proposition 1 considers certain changes in the mean, covariance, and lag-one dependence, but only under specific conditions. The authors should clarify which types of changes GAS can preserve and which changes may remain undetected.
  2. Some mathematical expressions in the paper appear to contain errors. In Section 3.3, the normalization used for cosine similarity under the Mahalanobis geometry appears to be missing a square root. Similarly, Proposition 1 in Appendix A.4 appears to contain an extra factor of (1/d) in the second-moment expression. The authors should verify these equations and clarify whether these errors affect the theoretical results.
  3. The novelty of GAS compared with existing similarity-based methods is not sufficiently clear. GAS includes commonly used measures such as dot product, cosine similarity, and covariance-adjusted similarity. The framework also relies on SCAN, an existing change-point detection method. Comparisons with simpler approaches, such as distances between consecutive embeddings and PCA-based projections, would help establish the specific contribution of GAS.
  4. The proposed method does not consistently outperform existing methods across all datasets. On Kinetics-GEBD, it achieves an average F1-score of 0.849, compared with 0.845 for FlowGEBD. This improvement is relatively small. On TAPOS, the proposed method achieves 0.569, compared with 0.623 for FlowGEBD. Similarly, on HASC, GAS-Moirai-Base achieves 0.502, compared with 0.530 for RuLSIF. The authors should provide a more detailed analysis of the lower performance on HASC and TAPOS.
  5. The runtime comparison mainly highlights the reduction in training cost rather than inference speed. Table 7 shows that the proposed method requires 1.72 seconds on Yahoo A4 using a GPU, while TS-CP2 requires only 0.65 seconds for inference after training. Although the proposed method avoids additional training, it does not achieve faster inference in this comparison. The authors should provide a detailed runtime breakdown for feature extraction, similarity calculation, and change-point detection.
  6. The runtime comparison does not clearly distinguish training time from inference time. The comparison with TS-CP2 includes 100 epochs of training, while the proposed method does not require additional training. This demonstrates a reduction in training cost but does not necessarily establish faster inference. Table 7 reports a runtime of 1.72 seconds for the proposed method on Yahoo A4 using a GPU. In comparison, TS-CP2 requires 0.65 seconds for inference after training. The authors should report separate runtimes for feature extraction, similarity calculation, and change-point detection to make the comparison more informative
Questions for authors

Q1 Theorem 2 assumes that the scalarization function is change-preserving. The authors should specify the conditions under which GAS satisfies this assumption and provide examples of changes that GAS may fail to detect. Q2 The equations in Section 3.3 and Proposition 1 appear to contain mathematical inconsistencies. The authors should verify and correct these expressions and explain whether the corrections affect the theoretical results. Q3 The proposed GAS formulation includes several existing similarity measures. The authors should add comparisons with consecutive embedding distances and PCA-based projections to demonstrate the specific advantage of GAS. Q4 The proposed method performs worse than some existing methods on HASC and TAPOS. The authors should analyze these results and identify whether the lower performance is caused by the foundation model representations, GAS transformation, or the detection method. Q5 CER relies on known change-point labels for evaluation. The authors should clarify how CER can be used without ground-truth annotations and verify the false change-encoding rate using separate no-change samples. Q6 Table 7 shows higher inference time for GAS-FM than TS-CP2 on Yahoo A4. The authors should provide a breakdown of feature extraction, GAS computation, and change-point detection times to clarify the computational advantages of the proposed framework. Q7 Figure 4(a) shows a large improvement in F1-score when total-variation denoising is used. How much of the final improvement comes from denoising rather than GAS itself? Can the authors compare GAS with simpler embedding-based methods using the same denoising settings?

Reviewer_3JyaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether frozen pretrained foundation-model representations can be reused for change-point detection (CPD) and generic event-boundary detection (GEBD) without task-specific training. The proposed framework extracts latent representations using frozen foundation models, evaluates their change-awareness through a Change-Encoding Map, and applies Change-Preserving Scalarization to transform the multivariate representations into a univariate signal for change-point detection.

Critical strengths
  1. The paper presents a clear and well-organized framework for leveraging frozen foundation models for CPD and GEBD without task-specific representation training, and evaluates the approach across multiple model families and modalities.

  2. The proposed Change-Encoding Map provides an intuitive way to evaluate the change-awareness of pretrained representations. The paper also provides theoretical conditions under which a change-preserving scalarization produces a scalar process compatible with SCAN.

Critical weaknesses
  1. The experimental evidence does not yet establish broad effectiveness. Performance on HASC is relatively weak. In addition, the three time-series benchmarks are relatively old. Evaluation on newer and more diverse datasets, together with stronger recent baselines under a matched protocol, is needed to demonstrate effectiveness. The substantially higher HASC results summarized in a recent survey [1] further motivate such a comparison, although these numbers should not be compared directly without confirming identical protocols.

  2. The methodological novelty appears limited. The framework primarily combines frozen representations with existing elements such as adjacent-embedding similarity, covariance adjustment, denoising, and SCAN. The theoretical results provide conditional justification but do not generally establish that the chosen foundation model and GAS scalarization preserve all relevant changes. GAS itself mainly generalizes familiar dot-product and cosine similarity measures.

[1] Gupta, M., Wadhvani, R., and Rasool, A. “Comprehensive analysis of change-point dynamics detection in time series data: A review.” Expert Systems with Applications 248 (2024): 123342.

Questions for authors
  1. Provide comparisons on newer datasets and against stronger recent methods using matched splits, preprocessing, metrics, and detection tolerances.

2.Explain the primary methodological novelty beyond combining frozen foundation-model representations, similarity-based scalarization, denoising, and SCAN. In particular, explain which properties of GAS are new relative to existing adjacent-embedding similarity measures.

  1. Discuss how sensitive is the method to the model- and dataset-specific hyperparameter search, and how well does a single fixed configuration generalize across foundation models and datasets.
Reviewer_Q9waOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether pretrained foundation model representations encode change-relevant information usable for change-point detection (CPD) in time series and generic event boundary detection (GEBD) without task-specific training. The presented GAS-FM framework extracts frozen representations and projects them onto change-sensitive univariate signals using Generalized Autosimilarity Statistics (GAS). The authors introduce so-called change-encoding maps. They employ an empirical-null maximum mean discrepancy (MMD) framework to quantify whether distributional changes contained in the input data, remain distinguishable in representation space. Conceptually, they establish a framework to assess whether pretrained foundation models act as effective change encoders. The main claimed technical contribution seems to be in the quoted text: "characterizing the information to retain through representation and scalarization", and nonparametric change-point detection along with "theoretical guarantees". However, in what ways the state-of-the art is advanced by this work (beyond introduced change-preserving scalarizations and motivation to choose scaled dot product) is not easy to distill from the paper. It would be helpful if conceptual and theoretical analysis be more structured, e.g. important questions could be (1) Which changes in the original data generation process remain identifiable in the embeddings? (2) Which multivariate changes survive the reduction to one dimension, and which can become invisible? (3) False-alarm control, detection power, and localization accuracy.

In my understanding the main new results are empirical insights from an extensive study of 17 (encoder-only, encoder-decoder, and decoder-only architectures x model size) pretrained models - 13 for time-series and 4 for video. However, it is also not obvious how conclusive or deep these insights are.

Critical strengths

The problem of change detection is challenging and practically relevant. Many approaches have been proposed including those considered by the authors. The main potential strength of the paper is in facilitating "computationally efficient change-point detection using established statistical methods". The studied framework requires no task-specific training. This can be considered consequential as training-free methods dramatically reduce barriers to deployment and enable rapid iteration. If foundation models already encode change information, this represents a fundamental efficiency gain over learning task-specific representations, with immediate practical applicability. The novel element of the framework is the CER (Change-Encoding Rate) metric that allows assessing whether pretrained representations preserve change information on new datasets.

The experimental study is extensive and the results are interesting overall. Inclusion of both time-series and video is compelling.

Critical weaknesses

It is non-trivial to distill the key contributions of the paper with respect to the body of existing work. The author do state their contribution. They do have the related work section.

Yet, I still find the positioning convoluted. While I appreciate Figures 1 and 2 - they provide good intuition, they do not help to position novel elements of the framework as its individual ingredients have been explored in related work. What are the most distinct contributions? theoretical guarantees? empirical insights? better overall performance thanks to the choice of scaled dot product or another design choice?

See also question to authors for additional questions.

Questions for authors

The paper emphasizes "training-free" and "unsupervised", but apparently requires some validation-set tuning. Section D.3 states context window, scalarization statistic, and denoising coefficient are selected on validation sets from a grid (Table 4). In real deployment scenarios without labeled change points, can practitioners select suitable hyperparameters. If they cannot, how likely would this significantly degrade performance? In other words, how sensitive is performance to hyperparameter selection without validation labels? Reporting results with default/heuristic hyperparameters that don't require tuning would be useful.

CER varies considerably from one dataset to the other. Figures 5-6 show high CER on Yahoo A4, USC-HAD, and Kinetics-GEBD but substantially lower CER on HASC and TAPOS-GEBD. The paper acknowledges this is "dataset-dependent" but provides no analysis of what makes some datasets have weak change encoding. Can practitioners predict when the method will work. Should low CER datasets require task-specific training anyway or would it not help either?

What is the out-of-distribution performance, i.e. how do models trained on different distributions perform on new CPD tasks?

1 comment
Reviewer_Q9waOpenReview ↗

Acknowledging authors response

I read the authors response. I am glad that the comments helped to further investigate this important topic.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.