- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TRAILS, a post-convergence approach for unified action quality assessment (AQA). It has two steps: Trajectory Interpolation (TI), which selects an intermediate model between the final raw checkpoint and its EMA counterpart based on training loss, and Statistics Recalibration (SR), which updates sport-specific BatchNorm statistics using unlabeled test data. TRAILS is evaluated on RG, FisV, and LOGO with MoAKE as the base model. It improves the average SRCC from 0.760 to 0.777 and reduces R-ℓ2 from 2.074 to 2.008.
Critical strengths
The paper offers useful analyses beyond the main results. Specifically, the authors examine several aspects of the converged model, including gradient conflicts across sports, dominant Hessian curvature, layer-wise parameter displacement, and sensitivity to different weight constructions. These analyses make the paper more informative than a straightforward combination of two post-processing techniques.
Critical weaknesses
- The proposed approach combines raw-to-EMA weight interpolation with sport-specific BatchNorm recalibration, both of which build closely on existing techniques. However, the paper falls short of clearly articulating its distinct methodological novelty beyond a straightforward combination of these existing components. The authors should better highlight what fundamental algorithmic insights or theoretical advantages differentiate TRAILS from prior works.
- The evaluation lacks comparisons against several prominent Action Quality Assessment (AQA) architectures and state-of-the-art baselines. Specifically, to robustly demonstrate TRAILS's effectiveness, the evaluation should include representative methods such as MUSDL (CVPR '20), CoRe (ICCV '21), TPT (ECCV '22), and NAE (CVPR '24).
- While the paper evaluates various test-set-based recalibration strategies, it omits evaluating sport-specific BatchNorm recalibration using only training data. This limitation makes it difficult to decouple the performance gains specifically attributable to sport-specific normalization from those driven by transductive test-time adaptation. An ablation study isolating these two factors is strongly recommended to clarify the source of improvement.
Questions for authors
- What is the key methodological novelty of TRAILS compared to existing weight interpolation and BatchNorm adaptation techniques? Furthermore, could the authors clarify how the optimization-geometry analysis directly informs or guides the proposed method's architectural design?
- Have the proposed components (TI and SR) been evaluated on other representative AQA architectures beyond MoAKE? Providing results on alternative backbones would significantly strengthen the claim that the proposed improvements are generalizable rather than architecture-specific.
- How does sport-specific BN recalibration using training data compare with recalibration using unlabeled test data? Including this comparison would help isolate and clarify whether the observed performance gains stem primarily from domain- or sport-specific normalization or from transductive test-time adaptation.
2 comments
Q3: Sport-Specific Recalibration versus Test-Time Adaptation
A3: The primary objective of SR is to accommodate sport-specific feature distributions within a unified AQA model, rather than to perform global test-distribution adaptation. Although different sports share the same model weights, their feature statistics can differ substantially. SR addresses this heterogeneity through sport-specific BatchNorm recalibration without modifying the shared model weights. Importantly, this mechanism does not inherently require test-set statistics.
To further clarify this distinction, we conduct additional experiments comparing global and sport-specific recalibration using training data with the original test-set recalibration protocol. All configurations use identical TI-selected weights and differ only in their BatchNorm statistics.
Table: Effects of different BatchNorm recalibration protocols (SRCC ↑).
| Method | Test-set BN | RG | FisV | LOGO | Avg. |
|---|---|---|---|---|---|
| TI (Original BN) | No | 0.7654 | 0.6729 | 0.8336 | 0.7651 |
| TI + Global-SR (Train) | No | 0.7721 | 0.6646 | 0.8370 | 0.7668 |
| TI + Sport-SR (Train) | No | 0.7749 | 0.6729 | 0.8482 | 0.7750 |
| TI + Sport-SR (Test) | Yes | 0.7771 | 0.6729 | 0.8501 | 0.7767 |
As shown in the table, Sport-SR (Train) improves average SRCC from 0.7651 to 0.7750, outperforming Global-SR (Train) (0.7668) and approaching Sport-SR (Test) (0.7767). These results demonstrate that SR's improvements primarily arise from modeling sport-specific feature heterogeneity, rather than requiring access to the complete test distribution. Moreover, sport-specific BatchNorm statistics can be precomputed from training data and fixed during deployment, enabling independent online inference without accessing other test samples or future observations.
We thank the reviewer for the constructive feedback and positive assessment of our post-convergence analyses. We provide point-by-point responses to the concerns raised below.
Q1: Methodological Novelty and Optimization-Guided Design
A1: The central contribution of TRAILS lies in task-driven post-convergence model-state decoding for unified AQA. While existing AQA research has primarily focused on architectural design and training-time optimization, TRAILS explores a complementary post-convergence perspective to address heterogeneous optimization behavior and sport-dependent feature distributions within a shared model.
Our gradient alignment and Hessian curvature analyses reveal distinct optimization characteristics across sports, motivating the exploration of alternative model states beyond the converged checkpoint. Inspired by mode connectivity, which suggests that favorable model configurations may exist along paths connecting different parameter states, we explore the training-derived Raw-to-EMA trajectory. TI then selects a favorable shared weight configuration by minimizing the sample-weighted training score loss, without additional gradient-based optimization. As demonstrated in the supplementary Complete Trajectory Coefficient Search, the selected state reduces the training score loss by 4.57% relative to Raw, providing empirical support for this optimization-guided selection strategy.
SR complements TI by recalibrating sport-specific BatchNorm statistics while preserving the selected weights, thereby accommodating heterogeneous feature distributions across sports. Their complementary contributions are supported by the ablation results: TI and SR individually improve average SRCC from 0.7600 to 0.7651 and 0.7683, respectively, while their combination further increases it to 0.7768. Together, TI and SR enable task-driven post-convergence model-state decoding in both weight and statistics spaces, improving unified AQA performance without additional gradient-based training or architectural modifications.
Q2: Baseline Comparisons and Cross-Architecture Generalization
A2: Existing AQA methods, including MUSDL, CoRe, and TPT, are primarily designed to train and evaluate separate models for individual sports, whereas unified AQA aims to assess heterogeneous sports using a single model with shared parameters. NAE addresses narrative action evaluation rather than numerical quality-score regression. We have further clarified these differences in the revised Related Work section.
To further address this concern, we conduct cross-architecture experiments under the unified multi-sport setting. Specifically, we evaluate a MoAKE variant with a shared temporal encoder and adapt TPT (ECCV 2022), an independent AQA architecture, to the same unified evaluation protocol. The results are reported in the revised Generalization Across Backbones section.
Table: Cross-architecture evaluation in terms of average SRCC (↑).
On the MoAKE variant, TI and SR independently improve average SRCC from 0.7427 to 0.7512 and 0.7478, respectively, while their combination further increases it to 0.7574, demonstrating their complementary effectiveness under an alternative unified AQA configuration. On the independent TPT backbone, TI improves average SRCC from 0.5698 to 0.5778, also exceeding the EMA-best result of 0.5767. Since TPT does not contain BatchNorm layers, SR is not applicable without architectural modifications. These results provide further evidence that score-guided post-convergence model-state decoding is effective beyond the original MoAKE architecture.