acceptodds
Under review as a conference paper at ICLR 2027

LISTS: Local Instance Selection for Time Series Classification

10 views

Abstract

Deep time series classification models commonly aggregate temporal representations into a single global representation, which may obscure class-discriminative information contained in local regions. To address this limitation, we propose LISTS, a Local Instance Selection framework for Time Series classification. LISTS sequentially selects a small number of local temporal representations that can complement the global context and explicitly retains them alongside the global representation for classification. We formulate this selection process as a sequential decision-making problem and optimize the selector using reinforcement learning with the classification objective as its learning signal. We further provide a theoretical analysis that motivates the proposed global-local formulation. Experiments on all 30 UEA benchmark datasets demonstrate strong classification performance, with LISTS achieving 74.4% average accuracy and the best average rank among ten methods, while a controlled synthetic study verifies that LISTS identifies class-informative local regions.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Classification, Local Instance Selection, Reinforcement Learning, Sparse Selection

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_VMCSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper looks at the idea of using local information in neural network encodings. The idea is that deep learning encodings are often pooled in a global fashion before being used in the final classification layer, which throws away information. This paper instead proposes that local instances can augment the global information and improve performance. They propose to do the instance selection via reinforcement learning. The authors compared their method against several standard benchmarks on the UEA multivariate time series datasets, and provided some synthetic experiments. Performance seems competitive with standard benchmarks such as MINIROCKET in accuracy, though I could not find any timing information.

Critical strengths
  • The basic idea is interesting: to exploit the local information in encodings that gets thrown away after global pooling. The basic algorithm is actually quite simple, which should be read as a positive here, and as such should be adaptable to other models.

  • The results look competitive with other benchmarks, though as usual, how much tweaking and tuning was done to achieve them is difficult to assess. The synthetic data example is nice and Figure 3 is great, and really helps sell the idea to the reader.

Critical weaknesses
  • Unfortunately, I found this paper very difficult to read and parse, especially as the underlying idea is not immediately obvious. It took me some time to realise that the local features are being selected per instance, by the RL mechanism, and then given the same coefficients regardless of which local feature (i.e., what tt) they are. This is because these ideas are buried in the text, when it should be presented up-front. For this idea to make sense it relies on the local features somehow having the same semantic meaning regardless of which local instance they are. That might be perfectly valid for the architecture they have chosen but is not discussed at all, which is strange as this seems to be the key idea underlying the paper. It is not immediately obvious to me why different features from the encoder should work well with the same logistic regression coefficients. Some discussion of this topic is crucial.

  • There are also issues with notation that make what is ultimately a reasonably simple idea, at least mathematically, initially quite difficult to understand. Symbols are introduced without sizes (e.g., WjW_j, WgW_g -- these are matrices), and poor notation obscures what is actually going on. To wit, eq (8) defines qϕc(y∣gi,r…)q_{\phi c}(y | gi, r … ) as the classifier, but then the text before eq (10) says the classifier and encoder and trained using (10). It was not immediately obvious to me whether only the weights WjW_j are updated, or whether the encoder parameters are updated as well as qϕcq_{\phi c} doesn't show any dependence on ϕe\phi_e.

  • A presentational issue is the presence of what seems to me to be irrelevant Propositions. I do not really see what use they serve the paper or exposition? Proposition 1 seems to link mutual information to Bayes log-risk, and Proposition 2 seems to be a restatement of the data processing inequality. These would appear to be standard results, but even beyond that, neither of these results appear to be invoked anywhere in the paper other than as high level justifications of the general idea of using local lists; there doesn't really appear to be much use specifically for the authors architecture. They seem to add clutter to the paper without seeming to provide any payoff.

  • Finally, the paper also introduces a very elaborate mechanism for training via reinforcement learning. Eq (7) appears to be a sparse linear model -- albeit one in which the features are adaptively chosen per instance. It is not clear to me that reinforcement learning is necessary to achieve this. There is a small ablation against an attention alternative which attains poorer results, but overall it seems possible that the level of complexity around using RL is not necessary and the paper does not provide any compelling arguments, beyond a single empirical ablation study (presented without any confidence intervals), to justify its use.

  • Finally -- it seems to me that what the authors are proposing is really just a sparse, adaptive form of global pooling. This is because the selected ritr_i^t s enter the model in the order of selection, not in any temporal order. Model (7) could be thought of as giving additional weight only to the selected representations, beyond their contribution to the global average, and then pooling them by a weighted linear average. In fact, from (1), if t=(t1,…,tk){\bf t} = (t_1,\ldots,t_k) denote the indices of the kk selected instances, then you can rewrite (7) as

logitsi=b+∑t[WgT+∑jI(t=tj)Wj]rit{\rm logits}_i = b + \sum_t \left[ \frac{W_g}{T} + \sum_j I(t = t_j) W_j \right] r_i^t

So this appears to be an adaptive (matrix weighted) pooling mechanism rather than augmenting global with local features, per se.

Questions for authors
  1. What value do the Propositions bring to the paper? Where and how are they used to justify your particular choices beyond high level justifications.

  2. To follow up -- why do we need Proposition 1 to justify why we need a Proposition to know that local information may exist? It seems that it would be trivial to propose time series classification problems where the local position of a simple temporal feature would be the prime (only) discriminator between two classes.

  3. What is "sg"?

  4. Where are the timing results? There are some ms / instance results reported in Table 3 but I think that timing results for training and at prediction time, vis a vis the other benchmarks, are necessary.

  5. Before Eq (10) it is said you update the encodings and classifier, but (10) shows no dependency on the encoding. Is this a typo or are you also updating the encoding?

  6. Why is it that different local instances can use the same weights effectively in the classifier? If these were just standard features to a classifier, then this would obviously not make any sense; I couldn't use height for one individual and weight for another as my first feature with the same coefficient, which, if I understood, is essentially what you are doing. So what is the fundamental aspect of the model that makes it work for these encodings? This is important to understanding the idea.

  7. Please comment on the connection to adaptive global pooling.

3 comments

Thank you again for your suggestions! We have updated the manuscript with additional experiments, computational efficiency analyses, and improvements to the presentation of our method. All revisions are highlighted in blue in the updated PDF. We have also summarized the main changes in our general comment. We hope these revisions address your concerns.

The authors have addressed a number of my questions. I acknowledge that the presentation and experimental evaluation has been substantially improved. However, some of my fundamental concerns remain. To wit:

  1. Regardless of how the authors wish to interpret equation (7) it remains a form of (adaptive) weighted global pooling over the features. In fact, concatenation followed by a linear classifier is algebraically equivalent to weighted additive aggregation over that concatenated representation. I do not believe that the distinction that the authors draw between their approach, and pooling, is particularly compelling.

  2. I maintain that Propositions 1 and 2 do not add anything substantive to the paper. Specifically, Proposition 1 appears to be a standard relationship between conditional mutual information and Bayes log-risk. Proposition 2 appears to be the data-processing inequality. Beyond being well known results, neither provide any specific insight into the proposed algorithm; in fact, they are sufficiently general that they could be invoked to justify an enormous range of methods. I do not see that they contribute meaningfully to understand the specific idiosyncracies of LIST.

  3. Finally, I remain unconvinced that reinforcement learning is necessary, or even particularly well justified for this problem. In fact, in the authors expanded experimental comparisons (which are appreciated), the results indicate that LISTS does not significantly outperform either Attention-3 or Top-3 under the authors own statistical testing procedure. Both are simpler alternatives with lower computational costs and are more readily understood. As such, the results reinforce my original concern than the addition complexity (computationally, and methodologically) of reinforcement learning is inadequately justified.

As a final note, the paper combines together a number of established machine learning techniques, without, in my view, providing any compelling theoretical or methodological argument for why this specific combination is preferable to some other combination, or a simple method. The provided theory is too general to answer these questions. I acknowledge the empirical results are promising, but without strong methodogical justification it is difficult to know whether these will generalize beyond the repositories on which the method was developed.

I appreciate the authors' efforts in revising the paper -- and do not dispute that the method achieves competitive performance on the evaluated data sets. However, I believe that the paper is not of the standard required for ICLR and am happy to retain my original score.

Thank you for your follow-up response. We appreciate the time you have taken to review our revised manuscript. However, we believe that several of your remaining concerns shift the discussion away from the actual contributions of LISTS. We respectfully disagree that these concerns undermine the methodological contribution or novelty of our work. Below, we address each point directly.


1. Algebraic Equivalence Does Not Challenge Our Core Contribution

We agree that Eq. (7) can be algebraically rewritten as a form of adaptive weighted pooling. However, this algebraic equivalence does not undermine any of the methodological contribution or novelty of LISTS.

Our claimed novelty has never been the invention of a new algebraic aggregation operator. Rather, it lies in the problem formulation and the mechanism for selecting complementary local information beyond an existing global representation.

Eq. (7) only describes how the selected representations contribute to the final prediction. It does not characterize how these representations are selected or how the selection policy is optimized. Therefore, we believe that repeatedly focusing on the algebraic form of Eq. (7) overlooks the actual contribution claimed in our paper.

2. Propositions 1 and 2 Directly Motivate Our Key Design Choices

We have already clearly explained how Propositions 1 and 2 directly motivate our key design choices in both the methodology section of our paper and our rebuttal:

Proposition 1 formally quantifies the complementary class information provided by local instances beyond the global context. This directly motivates our objective of selecting local instances that provide complementary information.

Proposition 2 establishes that compressing global and local representations may discard class-relevant information. This directly motivates our design choice to preserve the selected local evidence alongside the global representation through concatenation.

Furthermore, regarding your concerns that these propositions are "too general" or "could be invoked to justify an enormous range of methods," we would like to clarify that the generality of a theoretical result does not diminish its relevance to a specific methodological design. On the contrary, many foundational results in machine learning are broadly applicable, and this generality is precisely what makes them valuable, as they can provide theoretical principles and motivation for a wide range of methods.

3. Reinforcement Learning Is a Justified Design Choice, Not a Claim of Necessity

We have never claimed that reinforcement learning itself is our novelty, main contribution or the only possible optimization method. REINFORCE is a design choice for optimizing our sequential discrete selection policy.

Every machine learning framework necessarily involves design choices, and a design choice does not need to be the only viable option, or significantly outperform every alternative, to be justified. The relevant question is whether the choice is well motivated and supported by empirical evidence.

In our case, REINFORCE provides a practical mechanism for optimizing sequential discrete selection, with relatively modest computational overhead. Regarding performance, LISTS-3 achieves the highest average accuracy among the compared selection strategies on the 30 UEA datasets. Although not all of these differences are statistically significant after Holm correction, this does not imply that REINFORCE is unjustified. Furthermore, on the synthetic Sleep benchmark, LISTS-3 achieves substantially stronger localization performance, with All-Hit@3 reaching 86.00%, compared with only 38.00% for Attention-3.

These results provide concrete empirical support for our choice of REINFORCE, demonstrating favorable performance in both classification and localization with relatively modest computational overhead.

4. Response to the Reviewer's Final Note

We respectfully disagree that using established machine learning techniques undermines methodological justification. Most deep learning frameworks build upon existing components, and methodological justification does not require inventing every component from scratch.

In LISTS, our theoretical analysis provides a clear rationale for the proposed framework. Our experiments, especially the ablation studies, also provide empirical justification for our key design choices.


Finally, we hope that our clarifications address your remaining concerns. Thank you again for your time and consideration of our work.

Reviewer_bkzjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LISTS, which formulates local instance selection for time series classification as a sequential decision-making problem. The encoder produces local representations at each temporal position, and GAP yields a global context. Conditioned on the global context and the selection history, the selector samples during training and greedily selects k=3 local representations at inference. The global context and the selected local representations are then concatenated and passed to a linear classifier. The selector is trained with REINFORCE using the classification log-likelihood as the reward. The paper also provides an information-theoretic analysis showing that local evidence may contain complementary class information beyond the global context, and that preserving the concatenation is preferable to further compression. Experiments are conducted on 30 UEA datasets and a synthetic Sleep dataset.

Critical strengths
  1. The motivation is clear, and the method is easy to understand.
  2. The synthetic experiments demonstrate the claims of the method.
  3. The method achieves the highest average accuracy on 30 UEA datasets and covers the main design ablations, indicating practical competitiveness.
Critical weaknesses
  1. LISTS is essentially a different instance-combination mechanism rather than a newly designed network or module. It is recommended to add experiments showing the improvements achieved when LISTS is combined with other networks, so as to demonstrate its value.
  2. As the core of the method, RL-based instance selection is not a novel mechanism. It lacks comparative analysis with other instance-selection strategies, such as non-sequential Top-k attention, contiguous segment selection, and saliency/gradient-based selection. The current ablation based only on random selection and attention-based selection is insufficient to demonstrate the advantage of the proposed instance-selection mechanism.
Questions for authors
  1. How much computational overhead does adding instance selection introduce? The paper lacks the most critical complexity analysis, as well as efficiency comparisons with other related methods. Table 3 only reports the growth of the method itself with respect to the number of selected instances.
  2. Using instance information to enhance discriminative ability is not first proposed by LISTS. Therefore, it is recommended to discuss more in the first figure and related work how the instance-utilization mechanism differs from that of other methods, so as to better highlight the novelty of this work.
  3. The theory states that "if complementary information is selected, there is a benefit", but there is no sufficient theory to prove that complementary information is actually selected.
  4. The reporting of parameter sensitivity is limited. Table 3 only analyzes three instance-selection budgets; the feature of instances is affected by the effective receptive field of InceptionTime, but this parameter is not analyzed.
  5. A code repository that can be reproduced is missing.
  6. It is recommended to supplement the significance test in the ablation part to prove the advantages of the mechanism.
8 comments

Thanks for the response. The rebuttal is largely a set of planned revisions rather than concrete, evaluable results; given the absence of specific experimental analyses, it does not resolve my concerns about novelty, the necessity of RL, and empirical support. I will therefore keep my score for now and would reconsider only if the rebuttal provides the promised concrete results and analyses.

Thank you for your feedback! We have updated the manuscript with additional experiments and analyses. All revisions are highlighted in blue in the updated PDF. We have also summarized the main changes in our general comment. We hope these revisions address your concerns.

AuthorsOpenReview ↗

Additional Experimental Results and Clarifications (Part 1/2: Critical Weaknesses)

We thank the reviewer again for the constructive feedback. We have completed the additional experiments and revised the manuscript accordingly. Below, we revisit each of your concerns and provide the updated experimental results and analyses. All revisions are highlighted in blue in the updated PDF.

Weaknesses:

1. Backbone Generalizability

We have extended LISTS to two additional backbones, FCN and ResNet, alongside InceptionTime. The results are reported in Section 4.5 (Table 4). LISTS achieves statistically significant improvements over the corresponding Global Only baseline across all three backbones, demonstrating the generalizability of our framework.

BackboneGlobal Only Acc.LISTS-3 Acc.95% CI (pp)Wilcoxon p
FCN0.7050.724[0.45, 3.33]0.030985
ResNet0.7110.735[0.53, 4.29]0.026229
InceptionTime0.7160.744[1.30, 4.66]0.010760

2. Selection Strategy Comparison

Following the reviewer's suggestion, we have added two non-sequential selection strategies, Top-k attention (Top-3) and contiguous segment selection (Segment-3), to provide a more comprehensive comparison with alternative selection mechanisms. The results are reported in Section 4.4 (Table 3).

MethodAcc. ↑95% Bootstrap CI (pp)Holm pParams (M) ↓FLOPs (G) ↓Latency (ms) ↓Train/epoch (s) ↓
Global Only0.716[1.3035, 4.6555]0.0496580.5110.5318.938.20
Random-30.720[0.6941, 4.1431]0.0496580.5140.5309.008.24
Attention-30.731[-0.1292, 2.7357]0.1419810.5310.53611.139.37
Top-30.729[0.3294, 2.9022]0.1419810.5220.5399.108.38
Segment-30.725[0.5198, 3.6399]0.0496580.5220.5399.679.26
LISTS-30.744——0.5310.58311.8310.53
Reviewer_CLQWOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes framework LISTS that sequentially selects local temporal instances alongside the global representation for classification. It extends the existing literature of global-local modelling for TSC with a new a global-local representation. Theoretical analysis of this proposed global-local formulation is also provided.

Critical strengths

A new framework which selects local temporal instances alongside the global representation for classification. A theoretical analysis of a new global-local formulation.

Critical weaknesses

It would be useful to justify why reinforcement learning is chosen among other alternative techniques to optimise the classification objective of the the selector.

Questions for authors

See the above comments.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank the reviewers for their helpful suggestions. We have revised the manuscript with additional experiments, analyses, and improvements to the presentation. All revisions are highlighted in blue in the updated manuscript. The main changes are summarized below.

1. Selection Strategy Comparison We have added comparisons with two non-sequential selection strategies, Top-3 and Segment-3, in Section 4.4 (Table 3). LISTS-3 outperforms these two methods on the 30 UEA datasets.

2. Backbone Generalizability We have added experiments with two additional backbones FCN and ResNet , alongside InceptionTime in Section 4.5 (Table 4). LISTS-3 achieves statistically significant improvements over the corresponding Global Only baseline across all three backbones.

3. Receptive Field Sensitivity We have added a receptive field sensitivity analysis in the Appendix D.3, comparing smaller and larger receptive fields with the default configuration of InceptionTime. We evaluate the effects of receptive field size on performance and computational efficiency.

4. Computational Efficiency We have added computational efficiency comparisons across different selection strategies in Section 4.4 (Table 3) and comparisons with existing TSC baselines in Appendix D.2 (Table 8). Both comparisons report parameter counts, FLOPs, inference latency, and training time per epoch. The evaluation protocol is in Appendix B.3.

5. Statistical Significance We have added confidence intervals and Wilcoxon signed-rank tests in Sections 4.3, 4.4 and 4.5 (Tables 2, 3 and 4). The results show statistically significant improvements over several component ablations and alternative selection strategies. The evaluation protocol is in Appendix B.3.

6. Presentation Improvements We have expanded Section 2 (Related Work) to more clearly distinguish LISTS from relevant methods. We have also revised Section 1 (Introduction) and Section 3 (Methodology) to more clearly explain the selection framework and implementation of our method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.