- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes SEMAR, a method for allocating a limited refinement budget in sign language video generation. It learns to predict the semantic gain from additional refinement steps for local body-part regions, using labels obtained by comparing checkpoints along shared-noise refinement trajectories. A greedy allocator combines these predictions with measured hardware costs to select regions and refinement depths under a latency budget. Experiments on PHOENIX14T and How2Sign compare routing strategies and report quality–latency trade-offs across evaluators, refiner architectures, and GPUs.
Critical strengths
The paper addresses a practical question: how to allocate limited refinement compute across regions of a generated sign language video, accounting for whether additional refinement is likely to improve the output.
The matched-backbone experiments keep the generator, refiner, and action space fixed across routing policies, helping isolate the effect of the allocation strategy.
Critical weaknesses
-
The citation formatting needs correction. Parenthetical references should use \citep, while references serving as part of the sentence should use \citet.
-
It lists studies without sufficiently distinguishing pose-based sign language production from direct video generation and video refinement. The coverage and discussion of relevant work are insufficient to establish how SEMAR differs from existing regional refinement and adaptive-computation methods.
-
Equation (1) introduces a text-conditioned coarse video generator, but its architecture, initialization, training objective, and training procedure are not adequately described. The subsequent 12-block DiT description concerns the residual refiner rather than the coarse generator. Given the difficulty of sign language video generation, this backbone is a substantive component. The authors should identify the implementation, establish its generation quality, and demonstrate which errors local refinement can correct.
-
The presentation is difficult to follow. Components are introduced before their roles and connections are explained. For example, Section 3.1 abruptly moves from the coarse generator to “the default residual model” without first explaining the residual formulation. Dense terminology and frequent references to scattered appendix sections make the training and inference pipelines unnecessarily difficult to reconstruct. I have reason to suspect that the manuscript might be AI-generated.
-
The semantic supervision requires stronger validation. The routing labels depend on an in-house evaluator with sentence, gloss, handshape, orientation, contact, and non-manual components. However, the supervision and training of its local prediction heads, and their individual reliability, are insufficiently documented. Independent machine evaluators provide useful supporting evidence, but without signer evaluation, it remains unclear whether the reported gains correspond to improved linguistic fidelity an
Questions for authors
see above