- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes ZipSign, a training-free compression framework for sign language translation. ZipSign combines hand- and face-preserving spatial token merging with recognition-confidence-guided temporal pruning. The experiments on TwoStream and Sign2GPT across PHOENIX14T and CSL-Daily show inference speedups with limited translation-quality degradation.
Critical strengths
A key strength is the training-free design, which accelerates existing SLT models without retraining. Across two backbones and two datasets, the reported 1.7–2.0× speedups at moderate compression, with limited translation-quality degradation.
Critical weaknesses
- Spatial merging combines region protection, norm-directed fusion, weighted pooling, and resolution-adaptive budgets. A weighted-pooling-only baseline and separate ablations of these components are needed to identify which mechanisms produce the gains.
- On CSL-Daily, ZipSign reduces Sign2GPT latency by 41.4% while estimated GFLOPs remain nearly unchanged. Although FLOPs and latency need not scale proportionally, please provide component-wise profiling and clarify whether FLOP estimates cover the complete autoregressive decoding process, with matched decoding settings and reported output lengths.
- The manuscript would benefit from careful proofreading. Examples include “we splits … and performs” → “we split … and perform” (L222), “temporal redundant” → “temporally redundant” (L254), and “a segments” → “segments” (L271).
Questions for authors
The related work would benefit from discussing “Sign Language Translation with Iterative Prototype" (IP-SLT, ICCV 2023).
2 comments
Response to Weaknesses 2: Thanks.
(1) Component-wise Profiling.
We conducted component-wise profiling on Sign2GPT with CSL-Daily using an RTX A6000 GPU. As shown in Table 9, ZipSign reduces total latency by 41.4% (378.58 → 221.78 ms/sample), primarily through a 47.7% reduction in autoregressive decoding time (339.59 → 177.74 ms/sample). Spatial pooling incurs an additional 5.06 ms, while feature fusion saves 1.98 ms and temporal selection introduces 2.15 ms overhead. The visual backbone remains nearly unchanged. Overall, the 161.85 ms reduction in decoder latency substantially outweighs the additional processing overhead.
This acceleration is achieved by temporal pruning, which shortens the visual sequence supplied to the pretrained language model (XGLM), reducing the visual context processed during autoregressive decoding. Thus, decoder acceleration dominates the overall inference speedup despite relatively small changes in estimated GFLOPs.
Table 9: Component-level inference latency breakdown on Sign2GPT with CSL-Daily test set (ms/sample, mean ± std).
| Component | Baseline | ZipSign (50%) |
|---|---|---|
| Visual Backbone | 22.64 ± 3.50 | 22.46 ± 5.94 |
| Spatial Pooling | 5.25 ± 2.44 | 10.31 ± 5.05 |
| Feature Fusion | 11.10 ± 2.61 | 9.12 ± 2.78 |
| Temporal Token Selection | — | 2.15 ± 5.66 |
| Translation Decoder | 339.59 ± 4.10 | 177.74 ± 6.03 |
| Total Latency | 378.58 ± 9.68 | 221.78 ± 7.52 |
(2) GFLOPs Accounting, Decoding Settings, and Output Lengths.
-
GFLOPs Accounting. Our FLOP estimates cover the complete autoregressive decoding process. Total GFLOPs change little because the computationally expensive visual backbone remains unchanged after token compression. In contrast, XGLM decoding is sequential and memory-bound, so it accounts for little of the FLOPs but much of the latency. Fewer visual tokens shorten each decoding step, which reduces latency without notably changing FLOPs.
-
Decoding settings. Both methods use the same frozen checkpoint and identical XGLM generation settings (decoding strategy, beam size, maximum length, stopping criteria). Latency covers full generation, excluding keypoint extraction.
-
Output lengths. Output lengths are nearly identical on PHOENIX14T (19.16 vs. 19.35) and CSL-Daily (12.24 vs. 12.22) (Table 10), so the speedup comes from lower per-step cost rather than shorter outputs.
Table 10: Average generated-output lengths using the Sign2GPT backbone on PHOENIX14T and CSL-Daily test sets.
| Dataset | Baseline | ZipSign (50%) |
|---|---|---|
| PHOENIX14T | 19.16 | 19.35 |
| CSL-Daily | 12.24 | 12.22 |
Response to Weaknesses 3 Grammatical Errors and Proofreading Issues: We thank the reviewer for pointing out these grammatical errors. We have corrected all the identified issues and carefully proofread the manuscript to improve its grammatical accuracy and overall clarity. All corrections are highlighted in green in the revised manuscript.
Response to Question 1: We thank the reviewer for this suggestion. We have added a discussion and citation of IP-SLT (ICCV 2023) in the Related Work section to provide a more comprehensive overview of existing SLT methods.
Additional Component Ablations and Inference Efficiency Analysis of ZipSign.
Response to Weaknesses 1 Insufficient Component Ablations of Spatial Merging: We thank the reviewer for this valuable comment.
To isolate the contributions of individual spatial components, we conducted incremental ablations on TwoStream with the PHOENIX14T test set. Starting from weighted-pooling-only baseline, we progressively introduce region protection, resolution-adaptive budgets, and norm-directed fusion. All experiments use the same frozen checkpoint, with temporal pruning disabled.
As shown in Table 8, introducing region-protected merging improves BLEU-4 by 0.50. Resolution-adaptive budgets further improve BLEU-4 by 0.28. Adding norm-directed fusion yields an additional 1.14 BLEU-4 improvement. Overall, the complete spatial module improves BLEU-4 by 1.92 and ROUGE-L by 0.89 over weighted pooling alone, demonstrating the complementary contributions of these components.
Table 8: Ablation of spatial merging components on TwoStream with the PHOENIX14T test set (temporal pruning disabled).