- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes RefineSLT, a framework for improving semantic fidelity in pretrained sign language translation models while keeping their encoders and decoders frozen. It consists of three components: CueMiner extracts semantic cue pseudo-labels from training translations using masking-based embedding changes and an IDF prior; CueGrounder predicts cue representations and their temporal visual support from video features and initial decoding states; and CueRefiner uses cue confidence and estimated translation coverage to control residual updates to visual features. The original decoder then generates a refined translation in a second pass. Experiments on PHOENIX-2014T, How2Sign, and Auslan-Daily report consistent BLEU and ROUGE improvements across multiple backbones, evaluated on test subsets excluding reference translations of three words or fewer.
Critical strengths
(1) RefineSLT enhances pretrained translators while keeping their encoders and decoders frozen and reusing the original decoder. This makes the framework potentially useful for improving existing systems without retraining the entire backbone. (2) The paper evaluates multiple backbones across three sign language datasets and reports consistent translation improvements. Ablations with a parameter-matched adapter, random cues, and IDF-based cues provide useful controls for assessing the proposed components.
Critical weaknesses
(1) Missing comparisons with recent competitive methods. The experiments omit recent methods[1-6] and the related-work discussion overlooks [1-6], which address contextual representation learning, efficient multi-stream modeling, and feature enhancement for SLT. Discussing these approaches would better position the proposed refinement framework.
(2) The paper does not directly establish that CueMiner identifies meaning-critical content, that hidden-state similarity reliably measures translation coverage, or that cue-conditioned attention localizes the corresponding visual evidence. The ablations demonstrate component utility but provide limited support for these interpretations. Direct cue-quality, coverage-detection, and localization evaluations are needed to substantiate the central claims.
(3) BLEU, ROUGE, and reference-cue exact matching do not adequately measure successful semantic correction or harmful changes to initially correct content. The reported 92% success rate also lacks a sufficiently precise definition of attempted corrections and their coverage. Furthermore, excluding references of three words or fewer limits comparability with standard benchmarks.
(4) Important details concerning cue extraction, loss formulations, training stages, and backbone configurations are missing. \textbf{ The “without gloss supervision” description and shared I3D implementation also require clarification given the original backbones’ supervision and architectures. } Finally, the lightweight claim lacks parameter and runtime measurements despite requiring two decoding passes, while limited ablation scope and missing uncertainty estimates leave experimental reliability insufficiently assessed.
(5) The writing could be improved, and Figure 2 is suspected to have been generated by AI.
[1]Learning effective sign features without text for gloss-free sign language translation,
[2] Mixsigngraph: A sign sequence is worth mixed graphs of nodes
[3] ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation
[4] Lost in Translation, Found in Context: Sign Language Translation with Contextual Cues
[5] Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction
[6] SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale
[7] Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation
Questions for authors
See the weaknesses.