Explanation-Guided Active Learning by Exploring Diverse Decision Rationales

Skeleton-based action recognition relies on large-scale annotations, motivating active learning to identify informative samples under limited labeling budgets. Existing acquisition strategies typically quantify informativeness using predictive uncertainty, feature-space coverage, or gradient-based sensitivity, without considering the evidence underlying model decisions. Samples supported by similar spatio-temporal evidence may provide redundant supervision even when they are distant in prediction, feature, or gradient space. Conversely, samples with similar representations may exhibit distinct decision patterns and thus provide complementary supervision. We propose an explanation-guided active learning framework that selects samples based on their decision evidence. Using Layer-wise Relevance Propagation, we identify prediction-preserving spatio-temporal evidence and encode it into fixed-dimensional explanation representations. Batch acquisition then improves coverage in this explanation space, promoting diversity in decision evidence beyond predictions, features, or gradients. We further bound the generalization risk by the explanation-space covering radius, optimization error, and explanation reconstruction error, providing theoretical support for our criterion. Experiments on two skeleton-based action recognition datasets consistently outperform existing active learning baselines.