acceptodds
Under review as a conference paper at ICLR 2027

TimeSWARM: Soft Weighted Assignment-Based Relational Modeling of Channel-Patches for Multivariate Time Series Classification

Abstract

Multivariate time series classification (MTSC) requires jointly modeling discriminative temporal patterns and dependencies across variables or channels. Existing methods typically process channels independently, represent interactions only at the channel level, or route an entire multivariate patch as a single unit, thereby overlooking heterogeneous local dynamics across channels and temporal regions. We address this limitation by coupling local representation specialization with cross-channel relational learning at the channel-patch level. To this end, we propose TimeSWARM, a mixture-of-experts framework for dynamic channel-patch specialization and relational modeling. Each channel is partitioned into local patches, and an input-conditioned router combines temporal and spectral descriptors to produce expert assignments for each channel-patch. Sparsified routing weights are used to aggregate multi-scale temporal experts, enabling specialization across channels, temporal regions, and samples. The resulting dense routing distributions are reused as soft latent-group assignments for cross-channel interaction, so that the same routing mechanism determines both how each local pattern is represented and which channels it communicates with. Extensive experiments on the UEA multivariate time series archive show that TimeSWARM achieves the best average accuracy and average rank among the compared methods, together with the highest number of dataset-level top-1 results.

Primary area:
learning on time series and dynamical systems
Keywords:
time series, multivariate time series classification, deep learning, mixture of experts, channel correlation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gofdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TimeSWARM, a mixture-of-experts framework for multivariate time series classification. Each channel is split into local patches, and a temporal-spectral router assigns each channel-patch to multi-scale experts; the same dense routing distribution is reused as a soft latent-group assignment for cross-channel interaction. This couples local representation specialization with relational modeling without a separate graph or clustering module, and yields the best average accuracy and rank on the UEA archive.

Critical strengths
  • Reusing MoE routing as the relational structure is clean: it couples local specialization with cross-channel interaction without a separate graph or clustering module, and yields a provably low-rank, symmetric, row-stochastic channel relation (Section 3.3).

  • Evaluation is broad and strong: full UEA-30 plus UEA-10/UEA-25 subsets and UCI-HAR, with the best average accuracy, average rank, and most top-1 results, backed by pairwise Wilcoxon tests.

  • The ablation cleanly isolates the three components; removing any one degrades accuracy on both UEA-30 and UEA-10.

  • The appendix analysis goes beyond headline accuracy with routing visualization, expert-utilization diversity, and dense-routing versus induced-relational structure.

Critical weaknesses
  • No comparison against the MoE-based MTSC methods it cites (AnchorMoE, TVSRNet); the Table 1 baseline set is largely inherited from SGN.

  • The efficiency motivation is undercut: all experts are evaluated for every channel-patch, sparsification affects only aggregation, and no parameter counts, runtime, or memory are reported.

  • Per-dataset grid search (patch period, kernels, MoE layers, overlap, differencing) with best test performance from a single seed makes the modest margins hard to assess; no variance across seeds is reported on UEA.

  • The relational step is low-rank soft group averaging, conceptually close to existing channel-grouping and factorized message passing, so the core novelty is incremental.

Questions for authors
  1. Please add AnchorMoE and TVSRNet (the cited MoE-based MTSC methods) as baselines, or justify their omission.

  2. Please report parameter counts, training/inference time, and peak memory on representative datasets.

  3. Please report mean and standard deviation over multiple seeds on UEA and clarify whether the test split is used for hyperparameter or model selection.

  4. Please add an ablation replacing the routing-induced relation with a learned or fixed channel graph to show the routing coupling specifically helps.

3 comments
AuthorsOpenReview ↗

Review Reminder

Dear Reviewer gofd,

We sincerely thank you again for your constructive feedback and for recognizing the potential strengths of our proposed shared expert-induced relational modeling mechanism.

Following your valuable suggestions, we have conducted several additional experiments, including comparisons with recent MoE-based methods, computational efficiency analyses, multi-seed evaluations, and controlled architectural ablations to further validate our core methodological contribution.

As the discussion period ends in less than 24 hours, we hope you will have an opportunity to review our additional results and the corresponding revisions. We greatly appreciate your encouraging assessment of our work and hope that the new experimental evidence further strengthens your confidence in our proposed method.

Thank you once again for your time, consideration, and valuable support.

Best regards,

The Authors

Thank you for the detailed response and clarifications. Most of my concerns have been adequately addressed. I will maintain my original rating.

Thank you very much for your response and consideration! We sincerely appreciate your constructive feedback and are glad that your concerns have been addressed.

Reviewer_KAFkOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes TimeSWARM, a mixture-of-experts framework for multivariate time series classification. The method performs dynamic expert assignment at the channel-patch level and reuses the resulting routing probabilities as soft latent-group assignments for cross-channel interaction. It further incorporates multi-scale convolutional experts and inter-patch contextualization to capture temporal features. The authors evaluate TimeSWARM on the UEA multivariate time series classification archive and present comparisons with existing methods, component ablations, and routing visualizations. The paper reports improvements in average classification accuracy and ranking over the compared baselines.

Critical strengths

S1. A potentially useful modeling approach. The paper combines channel-patch-level expert specialization with cross-channel relational modeling through shared routing distributions, offering an interesting architectural design for multivariate time series classification.

S2. Broad benchmark coverage. The evaluation covers all 30 datasets in the UEA archive, including datasets with diverse characteristics. This provides a broad setting for examining the proposed approach, although the reliability of the resulting comparisons depends on the evaluation protocol.

S3. Component-level analysis. The paper includes ablation studies and routing visualizations that examine the roles of expert specialization, cross-channel interaction, and inter-patch contextualization.

Critical weaknesses

W1. The test-performance reporting procedure raises serious concerns about experimental validity. Appendix A.2 explicitly states that the authors report the best test performance across evaluated hyperparameter configurations. Although the paper clarifies that the test set is not used for gradient-based optimization, selecting or reporting the best configuration according to test performance can still introduce selection bias and overestimate generalization performance. The authors state that this follows prior UEA evaluation practices, but consistency with previous reporting conventions does not resolve the concern about test-set-based selection. This issue directly affects the reliability of the paper's main performance claims.

W2. The fairness and consistency of baseline comparisons are not sufficiently established. The reported baseline results are primarily taken from previous publications, while TimeSWARM is evaluated through a separate hyperparameter grid search. Although the authors attempt to follow established benchmark protocols, the paper does not sufficiently demonstrate that the compared results were obtained using consistent hyperparameter selection and test-reporting procedures. Therefore, the observed performance differences may be affected by evaluation protocol discrepancies, making it difficult to confidently attribute the reported improvements to the proposed method.

W3. The independent contribution of routing-coupled relational modeling is insufficiently demonstrated. A central contribution of TimeSWARM is reusing expert-routing probabilities to construct cross-channel relationships. Although the ablation study shows that removing the cross-channel interaction module reduces classification accuracy, this only establishes the usefulness of cross-channel interaction within the proposed architecture. It does not demonstrate that reusing the expert-routing distributions is more effective than independently learned channel relationships or alternative soft-grouping mechanisms. More direct controlled comparisons are needed to validate this key design choice.

Questions for authors

Q1. Appendix A.2 states that the best test performance across evaluated configurations is reported. Were the final hyperparameter configurations selected based on test-set accuracy? If so, how do the authors address the resulting selection bias? Could the authors provide results obtained by selecting hyperparameters exclusively on validation data and evaluating the selected configurations once on the test set?

Q2. Since the main baseline results are drawn from previous publications, could the authors clarify whether these results follow the same hyperparameter selection and test-reporting procedures as TimeSWARM? What evidence supports the comparability and fairness of the reported performance comparisons?

Q3. Could the authors provide controlled comparisons between routing-coupled relational modeling and alternative approaches, such as independently learned channel relationships or separate soft-grouping mechanisms, to establish the independent contribution of the shared-routing design?

1 comment
AuthorsOpenReview ↗

Review Reminder

Dear Reviewer KAFk,

We sincerely thank you again for your detailed review and valuable suggestions.

Following your feedback, we have conducted substantial additional experiments in this short period of time for the rebuttal stage, particularly validation-based multi-seed evaluations against the strongest recent baselines to address concerns regarding test-set selection bias and evaluation fairness, along with controlled ablations validating our core methodological contribution.

As the discussion period ends in less than 24 hours, we would greatly appreciate it if you could kindly review our rebuttal and additional results. We remain happy to address any further questions promptly. If our additional evidence has satisfactorily addressed your concerns, we would be grateful if you could kindly reconsider your evaluation.

Thank you again for your time and consideration.

Best regards,

The Authors

Reviewer_pqRTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TimeSWARM, a framework for multivariate time series classification that combines channel-patch-level conditional expert specialization, routing-induced cross-channel relational modeling, and inter-patch contextualization. The central idea is to reuse mixture-of-experts (MoE) routing distributions as soft latent-group assignments to facilitate cross-channel information exchange.

Critical strengths
  1. The paper addresses local temporal pattern modeling and cross-channel interactions in multivariate time series classification. Its observation that channel relevance may vary across temporal regions provides a reasonable motivation for the study.
  2. The paper formally describes the computational procedures for routing distributions, cross-channel interactions, and inter-patch contextualization. These descriptions help readers understand how the model is implemented.
Critical weaknesses
  1. The current novelty argument is insufficient to establish that the proposed shared-routing relational modeling mechanism provides a clearly differentiated and substantive methodological contribution.
  2. There is a gap between the reported component-removal experiments and the claimed methodological contribution. The current ablations do not sufficiently isolate the source of the proposed method's advantages.
  3. This unvalidated random design directly affects the core expert aggregation process but lacks the necessary ablation and stability analysis, weakening the methodological justification for a key implementation choice.
  4. The current statistical reporting does not adequately support the claim of statistically significant improvements over most competing methods. Repeated runs under a clearly defined evaluation protocol, together with complete test statistics and appropriate treatment of multiple comparisons, are needed.
  5. There is an evidential gap between the visualizations and the interpretation of the resulting latent groups. The claims should be limited to qualitative observations unless further quantitative validation is provided.
Questions for authors

Please refer to the critical weaknesses.

1 comment
AuthorsOpenReview ↗

Review Reminder

Dear Reviewer pqRT,

We sincerely thank you again for your thoughtful review. As the discussion period ends in less than 24 hours, we would greatly appreciate it if you could kindly review our detailed rebuttal and the additional experimental evidence provided, particularly the controlled architectural ablations and multi-seed evaluations addressing your concerns.

We would be happy to provide any further clarification or information as soon as possible. If you find that our responses have satisfactorily addressed your concerns, we would be grateful if you could kindly reconsider your evaluation. Thank you again for your time and consideration.

Best regards,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their constructive feedback and for recognizing the strengths of our work, particularly its well-motivated channel-patch-level modeling, promising architectural design, and strong empirical performance.

We would like to emphasize that the primary novelty of TimeSWARM lies in jointly modeling local expert specialization and cross-channel relationships through the same input-conditioned channel-patch routing distributions, without requiring a separate graph-learning or grouping mechanism. Our original submission already demonstrated state-of-the-art performance across all 30 UEA datasets, achieving the highest average accuracy (80.8%), best average rank (2.2), and most top-1 results (16) among the compared methods, supported by component-level ablations, routing analyses, and multi-seed experiments on UCI-HAR.

We have carefully addressed all reviewers' comments, questions, and concerns through detailed point-by-point responses, clear methodological clarifications, and additional supporting experiments. We believe our comprehensive responses and additional evidence reinforce the novelty, methodological soundness, and strong empirical findings already established in our original submission, while directly addressing the reviewers' concerns.

We sincerely appreciate the reviewers' valuable suggestions and hope our clarifications and additional analysis will support a favourable reconsideration of our work.

Thank you again for your time and consideration.

Best regards,

The Authors

AuthorsOpenReview ↗

Urgent Review Reminder

Dear Reviewers pqRT, KAFk, and gofd,

With only a couple of hours remaining before the discussion deadline, we would be grateful if you could take a moment to review our detailed responses and additional experimental results, which we have worked extensively to prepare over the past weeks.

As we have not yet received any follow-up feedback, we sincerely expect to hear your thoughts before the discussion closes. We hope our responses adequately address your concerns and would greatly appreciate your reconsideration of your overall assessments.

Thank you very much for your time and consideration!

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.