From Which to When: Multiple Instance Learning for Temporal Localization in Time Series

Multiple instance learning (MIL) enables time series classification from annotations that list the classes in a sequence without their temporal locations. We extend MIL for time series from sequence classification to temporal localization, which requires locating each class at the timesteps where it occurs. Knowing which classes occur does not determine when they occur, and we therefore identify two design principles for recovering the latter. Specifically, evidence should be selected sparsely and separately for each class, and instances outside the selected evidence should receive supervision constrained by the bag label. We propose StepMIL, which combines class-wise sparse temporal pooling with Local Soft-Target Propagation (LSTP). The pooling bases each class prediction on a few timesteps selected for that class. LSTP propagates bag-constrained soft targets between neighboring timesteps, weighted by source reliability and edge affinity, and is used only during training. We evaluate StepMIL on six datasets from human activity recognition, sleep staging and electrocardiography, with bags of up to 90,000 instances, against five MIL methods for bag classification. StepMIL improves temporal localization and sequence classification together, attaining the highest instance macro-F1 and Cohen's $\kappa$ on every dataset and the highest bag mAP on five of the six datasets.