SEMAR: Semantic Marginal-Value Routing for Budget-Adaptive Sign Language Video Refinement

An important sign-language region may already be correct, while a visible error may not improve with further refinement. We study compute allocation through the Semantic Value of Compute (SVoC): the expected semantic-utility gain of an executable refinement increment per unit of measured hardware cost. SEMAR obtains paired marginal-gain labels from shared-prefix trajectories, predicts gains for articulator–time tubes and refinement depths, and selects increments under a packed-plan latency budget. In matched-backbone evaluation at 50% residual budget, SEMAR achieves 20.1 WER at 682 ms P95, compared with 27.8 for uniform routing, 21.4 for predicted error reduction, and 21.0 for predicted cumulative gain (direct) at comparable latency. Relative to unconditional full refinement, it reduces P95 latency by 37% with a 2.8-point WER gap. Shared-prefix supervision reduces marginal-label variance by 53%. Experiments across DGS and ASL, signer-disjoint data, refiner families, and three GPUs support semantic marginal-value routing as a practical quality–latency trade-off.