Transfer One to All: Anchor Query-Specific Rubric Set Learning with Coarse Preferences

Recent work generates query-specific rubric sets to provide fine-grained criteria for evaluating responses in LLM post-training. When scores assigned under rubric sets are used as rewards, the policy learns to favor higher-scoring responses. Therefore, the contribution of rubric sets to policy improvement should be an explicit criterion for rewarding rubric-set generation. Existing methods learn rubric sets based on metrics such as diversity, non-redundancy, and response discrimination, but these measures do not directly assess whether the induced policy updates encourage the desired overall behavior. In this paper, we are the first to study how a coarse preference can serve as an anchor for generating fine-grained, query-specific rubric sets. Although coarse preferences and generated rubric sets evaluate responses using different criteria, they both provide rewards for training the same policy. Therefore, we derive a reference policy-update direction from the coarse preference and augment the rubric generator's existing training reward with an alignment term measuring how closely each rubric set's induced update matches this reference. For each query with a positively aligned candidate, the best-aligned rubric set supplies response scores for updating the policy. Extensive experiments on multiple benchmarks demonstrate improvements in both response discrimination using rubric sets and downstream policy performance. Our code is available on anonymous GitHub.