- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a Semantic-Adaptive Relation Fusion (SARF) method, which couples semantic-guided visual relation learning with semantic response recalibration and instance-adaptive semantic–visual supervision. The framework consists of three modules: TFA, SLR, and SVF. Extensive experiments demonstrate the effectiveness of the proposed method, and detailed experimental settings ensure its reproducibility.
Critical strengths
-
Comprehensive experiments. The paper evaluates on multiple benchmarks (SSv2, HMDB51, Kinetics-400) under both standard and generalized few-shot settings, with thorough ablation studies (Table 2) that isolate the contribution of each proposed module (TFA, SLR, SVF). This provides convincing evidence that the performance gains are not due to a single component but stem from the synergistic design.
-
Clear and coherent writing. The paper is well-organized and easy to follow. The motivation is clearly stated, the method description flows logically from high-level intuition to technical details, and the figures/tables effectively illustrate the key ideas. This makes the work accessible even to readers not deeply familiar with the FSAR literature.
Critical weaknesses
-
The technical contribution appears incremental relative to TS-FSAR and existing semantic adaptation techniques. SARF retains TS-FSAR’s core feature extraction and episodic matching framework, together with its base training objectives. Its main additions combine attention-weighted text conditioning for relation features, residual logit adaptation, and instance-conditioned semantic–visual fusion for auxiliary supervision. Although these choices are reasonable, the paper does not clearly establish what technical insight their integration provides beyond adapting familiar mechanisms to this framework. Table 2 includes useful comparisons with alternative designs, but the ablations are restricted to SSv2-Small 5-way 1-shot, and the additional gain from SVF is 0.44 percentage points over TFA+SLR. Results across multiple training seeds and additional settings, together with a quantification of the added training cost, would help establish the robustness and practical value of the proposed combination.
-
The motivation and positioning relative to prior semantic-aware FSAR methods need to be sharpened. The broad rationale of using pretrained vision–language semantics to compensate for limited support examples is already well established, as exemplified by CLIP-FSAR and TS-FSAR. The paper acknowledges these advances and instead emphasizes the limited direct involvement of textual semantics in relation construction. However, it does not sufficiently establish why existing forms of semantic conditioning are inadequate for this purpose, or what specific limitation TFA resolves beyond the TS-FSAR framework on which it is built. A more precise comparison of where and how semantics enter these methods, together with a clearer explanation of the resulting limitations, would strengthen the claimed research gap.
Questions for authors
-
The paper positions itself against CLIP-FSAR, MAFSAR, and TS-FSAR, but the technical distinctions remain somewhat vague. Could the authors provide a more concrete comparison of algorithms to clarify what is genuinely new?
-
Table 4 in the Supplementary Materials ablates the effect of different text descriptions. Could the authors report a variant where the detailed descriptions are replaced by simpler ones (e.g., "a video of [action]" or just "[action]")? This would help quantify how much the carefully crafted prompts contribute to the final performance in this paper.
2 comments
Rebuttal follow-up
Dear Reviewer,
I hope this message finds you well.
As the discussion period is drawing to a close, we would like to respectfully check whether our rebuttal has helped address the initial concerns you raised. Please let us know if you have any further questions or if there is anything else we can clarify. We would be more than happy to discuss further. Your perspective is crucial for us to improve the paper.
Thank you again for your time and valuable review!
Best regards!
Response to Reviewer usS6
Thank you for the careful assessment. We have sharpened the contribution statement, added robustness and cost measurements, and completed the requested prompt control.
The individual operations are intentionally lightweight. The contribution is their coordinated use to address three limitations of existing semantic-aware FSAR: semantic information does not directly participate in local relation construction, transferred semantic responses are not adapted to the video task, and semantic-visual auxiliary supervision lacks a query-dependent balance. TFA addresses the first limitation by selecting category descriptions using video-conditioned attention and injecting them before relation construction. The variants without semantic attention, with direct concatenation, and with additive fusion obtain 59.85%, 59.74%, and 59.81%, respectively, compared with 60.89% for TFA. The learned global scalar reaches 61.32%, whereas query-dependent SVF reaches 61.99%, further supporting instance-dependent supervision.
To evaluate robustness across independent training runs, we repeated both TS-FSAR and SARF using training seeds 0, 1, and 2 on SSv2-Small and HMDB-51. For each dataset, all checkpoints were evaluated on the same 10,000 test episodes:
SARF improves over TS-FSAR in every paired run. The average gains are 1.56 ± 0.41 percentage points on SSv2-Small and 0.75 ± 0.42 percentage points on HMDB-51, demonstrating that the improvement is not confined to a particular training seed or dataset.
We additionally report both training and inference costs under identical hardware and settings:
SARF adds 1.3% inference latency. Training time and peak training memory increase by 13.6% and 9.4%, respectively. These measurements are now included in the supplementary material.
To directly examine prompt sensitivity, we evaluated SARF using the two simplified prompt forms requested by the reviewer. The raw action name
[action]and the templatea video of [action]achieve 60.75% and 61.35%, respectively, compared with 61.99% using the inherited detailed descriptions. Although richer descriptions provide additional semantic information, simplifying them results in relatively small reductions of 1.24 and 0.64 percentage points. This indicates that SARF remains effective without relying exclusively on detailed descriptions. Importantly, TS-FSAR and SARF use exactly the same inherited descriptions, frozen text encoder, and aggregation procedure in the main comparison. Therefore, the improvement of SARF cannot be attributed to access to different or richer textual information.[action]a video of [action]