- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
SignAlign achieves improved sign language retrieval by addressing the problem of preserving the dissimilarity among different text embeddings through the TSSA module and the problem of the model paying higher attention to few common video embeddings through the CTSA module. The motivation of this work is clear and the experimental results verify the ability of the proposed approach to achieve improved results. Furthermore, an extensive ablation study justifies the selection of the values of the hyperparameters used in the work.
Critical strengths
One of the strengths of this work is their clear motivation and contributions. They propose two modules, each serving a specific purpose to address specific limitations of previous methodologies. Thus, the reader can clearly follow the reasoning behind the proposed work. A second strength of this work is the experimental results and ablation study. By testing the proposed work in 3 well-known large sign language datasets, the authors show the improvements achieved by their method. In addition, an extensive ablation study justifies the value assigned to each hyperparameter employed by the proposed method.
Critical weaknesses
None
Questions for authors
The following should be addressed by the authors:
- The RGB encoder and the vision transformer should be clarified. Which models are used?
- The acronym EOT (line 205) should be defined.
- Tables 5,6 & 7 should clearly present the V2T and T2V columns for better understanding.
- Some minor English errors “retrieved for the other” appears twice in line 20 and “assigns” in line 116.
3 comments
Thank you very much! My comments have been addressed. I'll keep my score
We sincerely thank the reviewer for the positive score and encouraging assessment, as well as the valuable suggestions for improving our paper.
Response to Reviewer Xg5q
We sincerely thank the reviewer for the support and encouraging assessment of our work!
Q1: Which RGB encoder and vision Transformer are used?
A1: The RGB encoder is I3D; we use the publicly released, fixed 1024-dimensional I3D segment features from CiCo (Cheng et al., 2023). The vision Transformer is a trainable temporal Transformer with 12 layers, hidden width 768, and 12 attention heads. Its compatible Transformer blocks, class token, layer normalizations, and output projection are initialized from the visual tower of CLIP ViT-B/32 (Radford et al., 2021). We have clarified these details in Appendix A.1 (Lines 767–775).
Q2: What does EOT mean?
A2: EOT stands for “end-of-text.” TSSA uses the EOT-token hidden state followed by CLIP’s pretrained text projection as the sentence representation. We have defined EOT at its first occurrence in line 205.
Q3: Unclear labeling of the T2V and V2T columns in Tables 5–7.
A3: We have explicitly labeled the two R@1 columns in Tables 5–7 as “T2V R@1 ↑” and “V2T R@1 ↑,” respectively, from left to right.
Q4: Minor English errors.
A4: We have removed the duplicated “retrieved for the other,” corrected “assigning weights” to “assigns alignment weights,” and proofread the manuscript.