Learning Executable Scoring Criteria for Action Quality Assessment

Action quality assessment (AQA) relies on scoring criteria that determine which evidence matters and how it contributes to a quality judgment. Existing methods typically encode these criteria implicitly in learned mappings or specify them explicitly through hand-designed evaluation structures. The latter fix how evidence is organized and combined, preventing score supervision from revising these structural choices. Yet translating domain knowledge into an effective scoring structure requires accounting for the reliability and complementarity of model-derived evidence, which must be established from data. We therefore formulate scoring criteria themselves as learnable structures: domain knowledge guides the construction of candidate criteria, while score supervision selects among these alternatives and calibrates their numerical realization. To this end, we propose Learning Executable Scoring Criteria for Action Quality Assessment (CriScore), a neuro-symbolic framework that represents learned scoring criteria as executable task-level programs. Domain knowledge guides the induction of evaluation concepts, evidence bindings, and scoring relations within a predefined evidence and operation space. Overall-score supervision trains the evidence predictors and supports program selection and numerical calibration, adapting the induced criteria to observed assessment performance. The resulting program is reused across new observations through neuro-symbolic quality reasoning, producing quality scores together with execution traces. Experiments across four AQA datasets demonstrate the effectiveness of CriScore, supporting the value of learnable scoring criteria for action quality assessment. The code and data are available at https://anonymous.4open.science/r/CriScore/.