- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a new method for zero-shot skeleton action recognition. Specifically, the overall framework is based on Flora. First, it incorporates low-rank covariance estimation for cross-modal alignment. Furthermore, it designs a test-time adaptation technique to calibrate unseen action-class centers. Experimental results show that the proposed method achieves competitive performance.
Critical strengths
- The introduction of low-rank covariance estimation for cross-modal alignment is appreciated, as it models the intra-class variance of skeleton features at a fine-grained level, which is beneficial for more precise alignment.
- The paper is well organized. Meanwhile, the experiments are relatively thorough and comprehensive, and the proposed method achieves competitive performance.
Critical weaknesses
- The motivation for incorporating LRCA is not well explained, and there is insufficient evidence or justification to demonstrate its effectiveness beyond recognition accuracy. Meanwhile, the performance gains brought by this design are limited. A more thorough analysis at the feature level would help justify the necessity of this module.
- The test-time adaptation requires at least a batch of unseen samples to estimate class means and variances. However, during inference, each unseen action sample should ideally be recognized independently rather than relying on a batch of test data. Thus, I believe the assumption underlying this inference setting is too strong and may not be suitable for standard inductive zero-shot learning.
- The proposed framework, including both training and ZSL/GZSL inference, is largely based on Flora, while the newly introduced designs are relatively incremental, raising concerns about the technical contributions and novelty of this work.
Questions for authors
- Why are flow-matching errors computed over a grid? Flora has demonstrated that single-step inference is sufficient. However, the motivation and benefits of this grid-based design are not clearly explained.
- It is unclear why cross-reconstruction errors are incorporated into the flow-matching errors. Meanwhile, the two types of errors may have different scales, raising concerns about the appropriateness of directly adding them together.
- The GZSL inference framework is derived from Flora, so an appropriate citation should be provided for Eq (12).
Response to Reviewer WQRm
We thank the reviewer for the constructive feedback. W and Q denote the numbered weaknesses and questions in the review.
1. Why is LRCA necessary, and what does it learn? (W1). Flora's diagonal Gaussian conditionals cannot represent dependencies between latent dimensions. LRCA introduces the low-rank covariance component LL⊤ to model these correlations and align them across skeleton and semantic representations.
The effectiveness is supported by both recognition and structural evidence.
These results show that LRCA contributes both improved recognition and better alignment of the learned cross-dimensional covariance structure, beyond an additional projection head.
2. Does TSC impose an unrealistic batch requirement? (W2). No requirement is imposed on the core ZSL model. LRCA and DDI classify individual queries. TSC is an optional enhancement for deployments where an unlabeled query collection is available, not a prerequisite for zero-shot recognition. Its value is measurable when that information exists: holding the full pipeline fixed, Table 5 shows TSC raises 48/12 ZSL from 62.45% to 66.06% and GZSL-H from 58.36% to 59.68%. The TSC-off LRCA-only model independently outperforms reproduced Flora (64.7% versus 63.7%, Table 3). A separate matched-access three-seed control also favors low-rank over diagonal without TSC (61.11% versus 59.42%). This directly answers the deployment concern: batch access is a condition for using the optional TSC enhancement, not a requirement to run LoAD. The same trained model remains usable for independent, single-query ZSL inference.
3. What advances beyond Flora? (W3). LoAD uses the Flora flow predictor but changes what the model aligns (correlated class-conditionals) and how it makes decisions (class-conditioned reconstruction for ZSL; flow evidence for GZSL). This is supported by the LRCA-only gain and structural controls above, rather than only a final-model score. Reconstruction fusion also helps under one fixed λ=0.1, not just split-specific weights (see Q2). TSC contributes a separate optional inference improvement.
4. Is multi-time flow scoring necessary? (Q1). No. Our reported benchmark results use single-time scoring at t=0.5, consistent with Flora's single-step inference. The time-grid formulation is a generalization included for analysis, not a requirement of LoAD.
We additionally evaluated multi-time scoring independently, using the same fixed checkpoint without TSC. It changes ZSL accuracy from 58.96% to 59.02% on 48/12. Thus, LoAD's reported improvements do not depend on multi-time scoring or additional ODE integration.
5. Why combine errors of different scales? (Q2). FM tests velocity consistency; deterministic reconstruction tests whether the class semantics explain the observed skeleton feature. They are combined as Efm+λEvae, where λ calibrates their influence. Rather than requiring split-wise tuning, one fixed λ=0.1 yields:
On 48/12, fusion corrects 33 FM-only errors versus 13 newly incorrect predictions. The consistent net benefit supports the scores' complementary evidence, despite their different raw scales.
6. GZSL attribution (Q3). The GZSL seen/unseen ratio gate in Eq. (12) is inherited from Flora; we will cite the source alongside the equation. Reconstruction is used for ZSL class ranking; the GZSL gate and class scores remain FM-only.