acceptodds
Under review as a conference paper at ICLR 2027

EvoJudge: Dual Evolution of Reusable Judging Skills and Rubrics for LLM-as-a-Judge

Abstract

LLM-as-a-judge plays an increasingly important role in model development, supporting fine-grained evaluation and providing reward signals for training. However, obtaining reliable judgments often requires highly capable frontier models, whose costs limit their use in large-scale evaluation and supervision. Inspired by recent advances in self-evolving agents, % which extract recurring patterns and convert them into reusable experience, in this paper, we introduce EvoJudge, a dual evolution framework that improves LLM judges along two complementary dimensions: how to judge, represented by procedure skills that guide evaluation strategies, and what to judge, represented by criterion rubrics that specify evaluation standards for each type. Starting from a few labeled samples, EvoJudge reflects on original judgment and synthesizes the reusable experience into skill and rubric libraries. In addition, periodic validation detects performance regressions, triggers selective rollback and provides corrective guidance for subsequent evolution. Experiments across model families demonstrate obvious judging improvements, with Qwen3-30B-A3B gains , , and on VerifyBench-hard, RewardBench2, and RMBench-hard, respectively, and enables Qwen3.5-35B-A3B to match frontier LLMs like GPT-5.6 Sol and Kimi-k3. Our analyses further demonstrate the complementary benefits of skills and rubrics, the effectiveness of periodic validation, and the transferability across various models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM-as-a-Judge, Evolving agent, Skill, Rubric

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Qnj1OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes EvoJudge, a framework that improves a frozen LLM judge by learning two libraries of reusable guidance from labeled examples: procedural skills describing how to evaluate responses and task-specific rubrics specifying evaluation criteria. The framework generates and consolidates this guidance, periodically checks validation performance, and uses selective rollback and corrective feedback when performance degrades. At inference time, retrieved skills and a task-type rubric guide the judge without updating model parameters. Experiments on VerifyBench-Hard, RewardBench2, and RM-Bench-Hard report accuracy improvements across multiple judge backbones. Additional analyses examine component contributions, periodic validation, cross-model transfer, and generated output length.

Critical strengths
  1. Practical improvement without model parameter updates EvoJudge learns reusable evaluation guidelines that can be applied to frozen judge models. It offers a practical paradigm for evaluator tuning in scenarios where fine-tuning is unavailable or undesirable, extending the potential applicability of this method beyond the specific models tested in the work.
  2. Empirical gains across multiple evaluation settings The proposed method improves the performance of two Qwen-based judges on all three benchmarks. For Qwen3.5-35B-A3B, it also outperforms the evaluated baseline and skill-based alternatives on each benchmark. These results demonstrate that the framework can enhance judgment accuracy.
Critical weaknesses

The independent contribution of the dual-evolution design is insufficiently established. The paper lacks budget-matched comparisons with directly optimizing the judge prompt, such as GEPA, or evolving a unified experience library. These comparisons are important because the gains could arise from additional guidance and supervised feedback rather than the proposed separation into skills and rubrics. Moreover, Section 4.4 describes inference-time component ablations, but the library counts in Table 6 leave unclear whether these settings use components of the same evolved libraries or independently evolved ones. This ambiguity limits the ablations’ ability to establish the value of the central design.

The coverage and robustness of the learned experience under heterogeneous task distributions remain unclear. The experiments use 200 induction examples and 100 validation examples per benchmark, without sufficiently examining sensitivity to sample selection, supervision size, or task composition. In a heterogeneous distribution, this limited sample may underrepresent rare evaluation requirements, and guidance useful for some categories may interfere with others. Aggregate improvements and cross-model transfer do not establish robustness to these conditions. This limits the evidence that the learned experience can reliably support diverse judging tasks beyond the particular sampled settings.

Questions for authors

Under the same backbone, labeled-data access, and optimization budget, how does EvoJudge compare with optimizing only the judge prompt, for example using GEPA, or evolving a unified experience library containing both procedures and criteria?

How sensitive are the results to the number and selection of induction examples and to the composition of mixed-task data? Per-category results across different sampling seeds and budgets would help assess coverage of underrepresented tasks. Please also clarify whether induction, validation, and test sets are separated by original question ID, keeping related response variants within the same split.

What is the end-to-end cost when including experience generation and consolidation, repeated validation, retrieval, and additional input tokens? Relative to the relevant baselines, how many subsequent judgments are required to amortize the upfront evolution cost?

Reviewer_eP9HOpenReview ↗
Rating
1
Confidence
4
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes EvoJudge, a training-free framework for improving LLM's judging performance. This framework accumulates judging knowledge by analyzing previous judging experiences. In this framework, the judging knowledge is categorized into: skills that guides how to judge and rubrics that tells what to judge. Both skills and rubrics are managed like a memory system with periodic validation to prevent performance degradation. During the inference, the LLM judge will retrieve the skill and rubrics according to the input sample. Experiments shows that EvoJudge can improve the LLM’s judging performance on several LLM-as-a-Judge benchmarks with different backbone LLM, and outperforms other training-free methods.

Critical strengths

The proposed framework is intuitive and sounds reasonable, it can be applied to any modern LLMs and provides a full cycle for deriving and maintaining the judging skills and rubrics. The framework can also be implemented using existing concepts like skills and retrieval frameworks without introducing additional engineering complexity.

Critical weaknesses

Generalizability is missing: (1) All the experiments require labelled data from the same distribution as the benchmark for evaluation. It cannot tells us the generalizability of this method. The labelled data is also hard to obtain in real world settings.; (2) The size of the judging knowledge library is small and not shared among different benchmarks. Based on my understanding, the motivation of having a library is being able to manage large number of skills and rubrics, however, the largest library in the experiments contains only 54 entries (rubrics and skills). It’s not clear how is the method’s performance affected by more rubrics and skills, especially when there are rubrics and skills generated for edge cases.

Questions for authors
  1. How does the method compare to SFT method?
  2. Can you use a shared library for all testing benchmarks?
  3. Can you use a dataset with a different distribution as the testing benchmarks to extract skills and rubrics, and test how well the models use these out-of-distribution skills and rubrics?
Reviewer_9umVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EvoJudge improves frozen LLM judges by learning two reusable knowledge libraries: procedural skills describing how to judge, and task-type rubrics specifying what to judge. Using 200 induction examples and 100 validation examples per benchmark, it reflects on judgments, consolidates experience, and uses periodic rollback and corrective guidance to regulate updates. Experiments cover three benchmarks and multiple model backbones.

Critical strengths

Practical approach without parameter updates. The framework improves judges through reusable external guidance, making it applicable to models whose weights cannot be modified.

Encouraging empirical improvements. Average accuracy increases by 8.99 and 5.66 percentage points for the two principal Qwen backbones.

Explicit regulation of accumulated knowledge. Periodic validation and rollback address the possibility that newly generated guidance can degrade performance.

Useful supporting analyses. Component ablations, cross-model transfer experiments, and qualitative examples provide insight into the framework’s behavior, although several conclusions require stronger controls.

Critical weaknesses
  1. The incremental contribution of dual evolution is insufficiently isolated. Eval-Skill already combines workflows and principles, while Auto-Rubric learns reusable evaluation criteria. The existing comparisons do not establish whether separate libraries outperform a unified representation under matched supervision, optimization, and context budgets.

  2. Evaluation and ablation protocols are unclear. Validation examples actively inform future guidance, but comparable supervision access for baselines is not documented. Benchmark aggregation requires clarification. Furthermore, the description of inference-time component ablations is difficult to reconcile with differing library counts in Table 6.

  3. Evidence for corrective guidance is limited. On Qwen3, guidance adds only 0.38 points over rollback alone using the displayed averages, while reducing RM-Bench accuracy from 71.55 to 71.03. Missing uncertainty estimates prevent a firm conclusion about its incremental benefit. The proposed explanation involving overly restrictive guidance remains untested.

  4. The efficiency analysis is incomplete. Output-token reductions omit additional input context, retrieval overhead, and offline evolution costs. Consequently, the practical cost advantage is not established.

  5. Generalization and reproducibility remain underexplored. Transfer is evaluated within one benchmark, and vanilla baselines for two transfer targets are missing. Behavior without task-type metadata is untested. Retrieval settings, operational prompts, decoding parameters, exact splits, and the sources of variability across repeated runs also need documentation.

Questions for authors
  1. Are the skill-only and rubric-only results independently evolved systems or inference-time removals from jointly evolved libraries? How are the library counts reconciled, and would a unified-library baseline perform similarly under matched budgets?

  2. What labeled examples, task-type metadata, and optimization budgets are available to each baseline? What exact aggregation rules and category coverage produce the reported benchmark scores, and why are rubric baselines considered inapplicable to VerifyBench-Hard?

  3. Is the guidance-versus-rollback difference reproducible across independent evolution runs? How sensitive is regulation to validation size, threshold, and frequency, and how is reference accuracy maintained after partial rollback?

  4. What are the total input/output tokens, model calls, latency, and amortized cost, including evolution? At what workload size does EvoJudge become more economical than competing approaches?

  5. What are the vanilla accuracies for Qwen3.5-9B and Qwen3.5-27B? How are missing task labels or absent rubrics handled, and do frozen libraries generalize to unseen benchmarks or task types?

  6. Can the authors provide complete prompts, retrieval and decoding settings, split identifiers, and uncertainty estimates that distinguish inference variability from variability across complete evolution runs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.