acceptodds
Withdrawn from ICLR 2027

FROM ATTENTION WEIGHTS TO ATTENTION ZONES: LOCALIZING FAULT-RELEVANT SIGNALS AND TIME INTERVALS IN MULTIVARIATE TIME-SERIES

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Root cause identification from multivariate time-series data in automotive systems is challenging because labels are typically available only at the vehicle level, not for individual signals or time steps, leading to a weakly supervised learning problem. This paper presents an attention-zone identification framework based on a hybrid convolutional–transformer architecture that jointly models local channel-wise patterns and global temporal structure to localize fault-relevant signals and time windows. Channel-wise CNN encoders preserve signal-level interpretability, while a transformer encoder enable attribution of the classification decision to specific channels (“attention channels”) and time intervals (“attention zones”) via attention weights, without post-hoc explanation methods. The method is evaluated on (i) a benchmark Racket Sports dataset augmented with synthetic localized faults and (ii) a large-scale vehicles dataset representing a complex real-world data analysis task. The results demonstrate that the proposed attention-zone framework provides robust, intrinsically interpretable root-cause insights for weakly supervised multivariate time-series diagnostics in complex automotive systems.

Primary area:
learning on time series and dynamical systems
Keywords:
Root-cause identification, Multivariate time-series diagnostics, Weakly supervised learning, Attention-zone localization, Interpretable CNN–Transformer architecture

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_znxGOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The general task the authors solve in the paper is the following: Given class labels for multivariate time series samples that broadly classify the samples into faults and normal operation, they want to identify channels and time steps that distinguish the faulty samples from the normal ones, aiding root cause analysis.

Understanding which channels and time steps discriminate the classes from each other can be a valuable hint to solve this, therefore the authors propose a hybrid convolutional-transformer architecture for multivariate time series classification with builtin channel level and temporal interpretability.

They first embed each channel with a dedicated 1D-CNN. The resulting per-channel embeddings are used as tokens and a cls token is appended to it. They then expand the resulting tensor to exist S (num channels) times and append the original multivariate time series to it. They project the resulting tokens with a shared linear layer into a d-dimensional embedding space and add a learnable positional encoding before passing it to a transformer encoder layer. the final cls token is passed to the classifier. The idea is, that with this setting, the cls token can allocate attention weights to each univariate encoding individually, and to the raw time series tokens across channels, which enables interpretation of the cls attention weights as feature importance indicators. They argue that attention from cls to cnn tokens reveals channel importance and attention from cls to raw time step tokens reveals temporal importance. They train the model with input dropout to simulate missing or corrupted channels, by either dropping the raw input or the cnn embeddings of a randomly selected subset of samples.

They test their architecture on one dataset from the UEA Time Series Classification archive and augment it with synthesized faulty classes (synthetic stuck value in channel 1 for two different time ranges, adding a class label for both). They show that the channel with the fault is the one with the highest importance (attention) when classifying the fault class correctly and that the attention on the raw data tokens peaks where the fault is inserted. The second dataset is from actual field vehicles and contains a real fault due to component issues. They show that the method correctly points at the pressure 2 channel.

The problem that the authors investigate is an important one and solving it with one architecture that is inherently interpretable is interesting. Despite that, the work is not comparing against a single baseline and the proposed architecture misses proper ablations of the individual components and algorithmic decisions (individual embeddings vs shared ones, CNN vs MLPs, input dropout vs no input dropout vs the proposed structured input dropout, ...), which makes it difficult to assess the actual benefit. Additionally, I think this line of research overlaps with research in the area of time series anomaly detection and localization. (1) Contrasting against sota TSAD approaches (e.g. alora-T) and (2) evaluation on localization benchmarks like SMD or SWaT would strengthen the paper significantly. Lastly, I think a comparison against common interpretability methods like SHAP would improve the manuscript further. The code and dataset are not accessible. That said, I recommend a reject and think that another round of polishing is necessary to finalize the manuscript.

Critical strengths
  • Given fault labels the method can be trained end-to-end and comes with inherent interpretability.
Critical weaknesses
  • It's hard to assess the actual benefit of the proposed method because evaluations on standard benchmarks and against proper baselines are missing.
  • The architecture and algorithmic decisions are not properly ablated.
Questions for authors
  • In the methods section you describe a reshaping to M = 1 + S.F + L. Is each sequence element in M a token and embedded individually to d dimensions?
  • Do you plan to release the second dataset?
  • You state that the input level dropout during training is essential. Do you have any experiments that back this, that maybe not have made it into the manuscript?
Reviewer_oFDiOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a hybrid CNN-Transformer framework that identifies root causes in automotive multivariate time-series data under weak vehicle-level supervision. By leveraging channel-wise CNNs and transformer attention mechanisms, the model directly pinpoints fault-relevant signals and time windows with intrinsic interpretability, eliminating the need for post-hoc explanation methods.

Critical strengths

This paper addresses a practical industrial challenge, diagnosing faults when only vehicle-level failure labels are available, by proposing a simple and deployment-friendly framework that combines channel-wise CNNs with a single-layer Transformer to directly inspect attention weights without relying on post-hoc explanation tools. Validated on a real-world vehicle fleet dataset, the model identified "Pressure 2 during low-pressure periods," matching the root cause independently confirmed by engineers.

Critical weaknesses
  1. Insufficient Experiment:

    a. There is no comparison with other baseline. Such as TimeMIL[1], TAIL-MIL[2] et al. and interpretability methods (SHAP[3]).

    b. Absence of Ablation Studies. Although the paper highlights the importance of Dropout, it provides no ablation experiment to show the impact of its removal.

    c. Overly Simple Synthetic Faults. Simply inserting high-amplitude segments into a single channel, which almost any method can detect. Furthermore, Table 1 only reports "Faulty-1."

    d. Lack of Quantitative Metrics: such as IoU, pointing game, and AUPRC, as well as faithfulness evaluations.

  2. Severely Inadequate Literature Review: The manuscript cites only eight references in total, which is highly insufficient for a research article.

  3. Limited Novelty: Relying on CLS attention for interpretability is a well-established technique commonly used in ViT and attention rollout line of work. The hybrid architecture—pairing channel-wise encoders with concatenated tokens—is a routine engineering assembly rather than a foundational methodological contribution. The proposed strategy of Dropout also commonly used in other work.

Reference

[1] Chen, Xiwen, et al. "TimeMIL: Advancing multivariate time series classification via a time-aware multiple instance learning." arXiv preprint arXiv:2405.03140 (2024).

[2] Jang, J., & Kwon, H.-Y. (2025). TAIL-MIL: Time-Aware and Instance-Learnable Multiple Instance Learning for Multivariate Time Series Anomaly Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 39(17), 17582–17589.

[3] Lundberg S M, Lee S I. A unified approach to interpreting model predictions[J]. Advances in neural information processing systems, 2017, 30.

Questions for authors

None.

Reviewer_uVmkOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a hybrid CNN-transformer for multivariate time-series fault diagnosis using sequence-level labels. Separate CNNs extract channel features. These features and raw time-series tokens are combined with a CLS token in a single transformer layer. The CLS representation is used for classification, while attention from the CLS token is used to estimate channel and time-point importance. The paper considers multiclass classification on augmented RacketSports and healthy/faulty binary classification on real-world fleet vehicle data. Evaluation mainly consists of channel-selection statistics and example attention maps.

Critical strengths

The paper addresses a practical fault-diagnosis problem using only vehicle-level labels. Identifying relevant signals and time intervals could help engineers focus their investigations.

Critical weaknesses
  1. Section 3.1 does not establish that higher CLS attention means a greater contribution to the predicted class. Attention weights alone do not account for the value vectors or the classifier.

  2. Equation (2) defines the attention output, which includes multiplication by V. The following text instead describes it as attention weights, making the explanation inconsistent.

  3. The method does not clearly define an attention zone. It describes time-point scores without explaining what constitutes a zone.

  4. The model uses multiple attention heads. How their weights are combined into the temporal scores in Figures 5 and 8 is not specified.

  5. The paper does not report overall classification performance or baseline comparisons. Temporal localization is illustrated through examples. Table 1 reports channel selection only for correctly classified Faulty-1 samples. These results are insufficient to assess performance across the evaluation set.

Questions for authors
  1. Why should higher CLS attention indicate a greater contribution to the predicted class, given the role of value vectors and the classifier?

  2. Are importance scores computed from the attention weights or the attention output in Equation (2)? Please clarify the notation.

  3. What constitutes an attention zone, and how is it identified from the time-point scores?

  4. How are the attention heads combined to produce the temporal scores in Figures 5 and 8?

  5. Can you report classification performance against relevant baselines on held-out data? For synthetic faults, can you also report localization results across all held-out fault samples, including misclassified ones?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.