- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes a new task named UADR, which aims to produce quantitative performance scores and qualitative explanations jointly. To address this task, it constructs a new dataset covering three types of actions and corresponding LLM-generated action difference descriptions. Furthermore, it designs a framework with a kinematic routing module to capture fine-grained action cues from videos. Experimental results demonstrate competitive performance across various datasets.
Critical strengths
- The paper introduces a new action difference reasoning dataset that includes paired expert comments and LLM-generated action difference descriptions.
- The proposed framework not only performs well on its own benchmark but also achieves competitive performance on existing action assessment and interpretation datasets.
Critical weaknesses
- The motivation for unifying quantitative assessment and qualitative interpretation into a new single task is unclear. It appears more like an architectural choice, with limited insights into the potential benefits of such unification. Additionally, the dataset covers only three types of actions, raising concerns about the practical value of this new task setting, given that numerous studies have already explored fine-grained action assessment and interpretation separately.
- The paper is not well written, and several technical designs lack sufficient justification. Additionally, the ablation studies do not evaluate the subcomponents of each module, raising concerns about the effectiveness of the proposed complex framework.
- The proposed method lacks sufficient novelty and provides limited methodological insights. The framework combines numerous existing techniques, including multi-head attention, kinematic information, GNNs, and KL divergence, without adequately explaining the motivations or benefits of their integration.
Questions for authors
- How is the optimal kinematic path determined for a given video? Specifically, when 256 candidate paths per video are used to compute for KL divergence during training, how are their scores obtained, and how is the optimal path selected?
- Why does the cross-modal semantic grounding module align the selected kinematic routes with both language and visual information? The underlying motivation and potential benefits are not sufficiently explained.
- What is the motivation for computing a bidirectional residual graph embedding? Additionally, what are the advantages of jointly performing quantitative assessment and qualitative interpretation rather than handling them separately?
4 comments
Continued response
(continue from the previous response)
-
How to determine the optimal kinematic path: Every candidate path is scored by the same network, and the highest-scoring one is selected. The full procedure is below (Sec. 3.2). We add the algorithm box below for better understanding:
Input: video V and its 3D keypoints f_V = E_V(V) # pretrained video encoder, N keyframes for each of the 256 paths l = (j1, j2, j3, j4), with every j in {elbow, shoulder, hip, knee}: f_M(l) = angle, velocity and 3D position of j1..j4 over the keyframes Q(l) = sigmoid( F_R( MHCA( F_M(f_M(l)), F_V(f_V) ) ) ) # Eq. (1) l* = argmax over l of Q(l) # the selected route p_r = Q / (sum over l of Q(l)) # used only by L_KL, Eq. (2) -
Cross-modal semantic grounding module align the selected kinematic routes with both language and visual information: Because the two branches play different roles. Language is the teacher, available only in training. Vision is the student, used at inference. Sec. 3.3 gives this motivation (L243–249 and L259–261), and we now state it more explicitly:
- is the commentary-conditioned view of the selected route. It exists only in training.
- is the video-conditioned view of the same route. It is the only one used downstream, to enhance the keypoint and mesh features, and the only one available at inference.
- The semantic loss (Eq. 3) pulls f_VM toward , so that the video branch learns to carry what the commentary says matters.
- Neither branch alone would do. Aligning with language alone would require commentary at test time. Using vision alone would leave the route with no signal about what experts consider important. The benefit is measured in the submission: without the loss, falls from 0.76 to 0.63, rises from 3.37 to 6.06, and METEOR falls from 33.0 to 28.9 (Tables 1 and 3).
-
Motivation for computing a bidirectional residual graph embedding: A description must credit each performer with their own strengths. The instruction in Fig. 2 asks what player 1 does better than player 2 and what player 2 does better than player 1. The two directions of the residual serve these two halves.
- H₁ − H₂ and H₂ − H₁ carry the same information linearly, but not after the non-linear projector F_H. Each direction is projected into its own tokens, and each set emphasizes the dimensions on which one performer exceeds the other. This is the same reason that keeping both signs of a rectified feature preserves information that a single rectifier discards.
- Swapping the two videos swaps the two halves. The decoder therefore always sees the advantage of performer 1 first and the advantage of performer 2 second, which helps it attribute each strength to the right performer.
-
Advantages of jointly performing quantitative assessment and qualitative interpretation: We list three points below, all of them are supported in the paper:
- Consistency. Both outputs are read from the same graph embeddings, so the description is decoded from the evidence that produced the scores. In the data, the two annotations agree on who is better in 94.7–96.4% of pairs (Appendix C), and a pipeline of separate models has no guarantee of preserving this.
- Shared supervision. Language supervision improves scoring ( 0.63 without the semantic loss, 0.76 with it), and all four ablations shared by Tables 1 and 3 degrade both outputs.
- Coverage. No specialist baseline handles both outputs. MAGR produces no text, and fine-tuned Qwen3-VL reaches only 0.46.
We hope these answers clarify the motivation, the design and the evidence, and we are glad to discuss any point further.
Thanks to the authors for their efforts. All my concerns have been addressed, and I have raised my rating from 2 to 3. Nevertheless, I still believe that the presentation quality could be further improved. I hope the authors can further clarify the motivation and improve the figures in the final version.
We greatly appreciate your time and constructive feedback.
We greatly appreciate your time and constructive feedback.
Author response to review comments
We thank the reviewer for the comments and address each question below.
Motivation for the unified task, and its scope: The unification is motivated by the use case, not by the architecture, and the submission shows its benefits in the data, in the model and in the results.
Use case. A trainee needs to know how good each execution is and what differs between the two. A score alone is not actionable. A description alone has no magnitude and cannot be tracked over time. Fig. 1 shows the intended output: scores of 7.5 and 8.4, together with what each player does better.
Data. In ADR the two annotations express one judgment. Three raters who read only the reference description identified the performer with the higher expert score in 94.7–96.4% of 900 pairs (Appendix C). A system whose description contradicts its scores is visibly wrong, and separately trained systems carry no guarantee against that.
Model. Both outputs are read from the same kinematic graph embedding: the score from H, and the description from H₁, H₂ and their residual (Sec. 3.4). The description is decoded from the evidence that produced the scores.
Practical value. The two outputs have so far been studied separately is the motivation for UADR. Table below lists the closest methods by input and output. None takes two arbitrary performers and returns a score for each together with a description of their differences. The pairwise setting is also deliberate (Appendix C, “Dataset design choice”). Comparing each performer with a fixed expert prototype would not support feedback between two non-expert performers, or on the strengths of both sides.
Writing, justification of the design, and ablations: Reviewer NCYB lists “Extremely clear writing” as the first strength of the paper and finds the notation “very formal and easy to understand”. We will nevertheless improve the presentation further on presentation.
Justification on technical designs: Each design is motivated where it is introduced in Sec. 3, and each is supported by an ablation. The table below gives the location of the motivation and the measured effect on ADR (Tables 1 and 3).
Ablation studies: In the paper, we evaluate the sub-components of every module. Several of these ablations are in the appendix (Tables 6, 7, 9 and 13), which may be why they were missed. We organize ablation studies in the table below for better understanding.
(to be continued)