Vibrics: Unified Evaluation of Video Generation Pipelines with Request-Specific Rubrics

Video generation pipelines rewrite open-ended user requests into model-ready prompts before rendering videos. Existing evaluators focus on final outputs, providing little evidence about rewriting decisions and often missing fine-grained, request-specific requirements. We introduce Vibrics, a unified evaluation framework that freezes a request-specific rubric derived from the original request and reference media before observing any candidate. Its criteria evaluate the rewritten prompt, any available process trace, and the delivered video under one shared standard. Video criterion decisions yield Fulfillment (request satisfaction), Craft (visual quality), and calibrated Utility, while the full criterion record provides evidence-grounded diagnoses of unmet requirements across the pipeline. To benchmark complete pipelines at scale, we build Vibrics-Bench from 240 human-reviewed, open-ended production requests that reflect what users would type or paste, span eight input types and professional scenarios, and require substantive rewriting. Crossing five rewriting methods with five generators yields 25 pipelines per request and 6,000 videos, all evaluated under one frozen rubric per request. Vibrics agrees with expert consensus on 85.3% of comparable criterion decisions, while its Fulfillment, Craft, and Utility rankings reach 81%/81%/78% of human-human agreement. Experts also find that 89.6% of 1,000 diagnostic reports identify the delivered video's most important problem. On the 40-request annotated subset, Craft varies primarily with the generator, whereas Fulfillment varies with both the generator and the rewriting method. Together, Vibrics provides comparable scores and diagnoses for complete pipelines, while Vibrics-Bench provides a shared benchmark for evaluating rewriters, generators, and their combinations.