acceptodds
Under review as a conference paper at ICLR 2027

LexReward: A Taxonomy-Driven Reward Framework for Legal Language Models

Abstract

Legal language models require reward signals that capture not only answer correctness but also the multidimensional quality of legal responses. Existing reward methods, however, often rely on coarse-grained holistic judgments, providing limited domain specificity and interpretability. We introduce LexReward, a taxonomy-driven framework for legal reward modeling. LexReward characterizes legal response quality along three complementary dimensions: Style, covering lexical and syntactic quality; Element, assessing legal subjects, facts, statutes, and decisions; and Chain, evaluating the order, completeness, correctness, and non-redundancy of legal reasoning. For each dimension, we develop rubrics that specify evaluation criteria and quality levels. The resulting rewards are used to construct pairwise preference data for Direct Preference Optimization (DPO) and reward-model training. Experiments show that the rubric-based rewards reliably distinguish legal responses of different quality and that DPO training on the preference data improves performance across all three dimensions. The learned reward models, LexRM, also support effective downstream optimization: each dimension-specific reward model improves policy performance in its corresponding dimension through reinforcement learning, without requiring reference answers at reward time. Dimension-wise analyses further support the effectiveness of the proposed taxonomy and reward construction. All code and data will be made available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Reward Modeling, Rubric-Based Evaluation, Legal Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KqiUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  • Proposes LexReward, a taxonomy of legal response quality with three dimensions: Style (lexical/syntactic), Element (subjects, facts, statutes, decisions), and Chain (order, completeness, correctness, non-redundancy).
  • Taxonomy reportedly built with legal experts (top-down specification + bottom-up error analysis). Rubric rewards:
  • Style is rule-based: distance from a corpus of Chinese court judgments (KL over legal/sentiment terms, conjunction rate, sentence-length statistics, 3–6-gram coverage).
  • Element uses an LLM judge without a reference answer.
  • Chain is rule-based or LLM-judged, depending on the task.
  • Rubric scores over a five-model candidate pool produce preference pairs, which are used for DPO and to train LexRM (Bradley–Terry, Qwen3-8B). LexRM is then used for Best-of-5 selection and as the GRPO reward.
  • Validation: selection accuracy on CLASE, Legal∆, and LexChain; DPO and GRPO (LexRM vs. outcome/ROUGE rewards); sub-criterion ablations.
Critical strengths
  • Well-motivated problem: outcome-only rewards ignore whether a legal conclusion is properly grounded. A structured, interpretable reward is a useful direction for legal RL.
  • GRPO with LexRM improves all five Legal∆ metrics over SFT (+6.04 average; SLP +9.9, CAC +8.0) without reference answers at reward time. This is the strongest evidence that dimensional rewards add something beyond outcome rewards.
Critical weaknesses

Taxonomy and rubrics are unvalidated, and the expert process is undocumented.

  • Missing: the experts’ number and roles, the error-analysis protocol, and agreement statistics.
  • Unclear whether experts set the rubric levels and weights.
  • No comparison of rubric/LexRM scores with expert ratings, and the Element judge is never checked against gold labels.

The Style metric is likely length-biased

  • Forward KL over the full vocabulary mostly measures vocabulary coverage, so longer responses score better; n-gram coverage behaves the same way.
  • The single pooled reference penalizes case-specific vocabulary.
  • No expert input or validation for this is reported.
  • CLASE tests authentic vs. generated text, which is what the metric was calibrated on, so the Style gains may reflect optimizing toward the metric.

Hallucination is not measured, and the reward may encourage it?

  • No metric for fabricated statutes or wrong entities.
  • The statute rubric gives the top score to law name + article number + content, but the judge has no reference or retrieval to verify them, so specific but fabricated citations can score well.
  • The Style reward rewards legal terms whether or not they are correct.

Overstated claims and no significance testing.

  • Many deltas are under 1 point with no seeds or confidence intervals: DPO on Element is +0.35, and LexRM-Chain beats Skywork-Qwen by 0.27.
Questions for authors
  1. Hallucination: Do you measure fabricated statutes or wrong entities before and after DPO/GRPO? Without a reference or retrieval, how does the judge avoid rewarding specific but fabricated citations?
  2. Experts: Please document the expert roles, the error-analysis protocol, and agreement statistics. Which rubric choices came from experts and which were tuned on data?
  3. Style length bias: How does the Style reward correlate with response length, and how do held-out authentic judgments of similar length score?
  4. Training: Is the 1e-7 learning rate (about 140 steps) correct for the RMs and LoRA DPO? This looks very low! Please also give the hyperparameters for GRPO, DPO, and the exact rule-reward definitions.
  5. Variance: Please report seeds and confidence intervals for Tables 1–3.

Suggestions:

  • Add Hallucination as part of the metric. In fact please check possibilities of adding other safety metrics w.r.t legal domain.
  • Add a small expert-rated validation set and a robustness check with a second LLM judge or perform a perturbation of the rubric.
4 comments
AuthorsOpenReview ↗

Response to Reviewer KqiU

We sincerely thank the reviewer for the thoughtful and constructive feedback. We particularly appreciate the concerns regarding expert validation, the robustness of Style rewards, legal hallucinations, and statistical reliability. These suggestions are valuable for clarifying both the contributions and limitations of LexReward. In response, we provide further details of our expert-guided taxonomy construction, report additional independent expert evaluations and robustness experiments, and clarify our training and evaluation settings. We address each concern below.

W1 & Q2: Taxonomy Construction, Expert Validation, and Rubric Reliability

We appreciate the reviewer's emphasis on the importance of expert involvement and independent validation. We would like to clarify that LexReward is fundamentally an expert-guided taxonomy grounded in established legal principles, rather than a classification derived solely from empirical error frequencies.

Expert panel and construction procedure.

Our taxonomy was developed by an expert panel comprising one law professor and two PhD-level legal researchers, following an iterative process that combines bottom-up error induction with top-down expert refinement.

For bottom-up induction, we analyzed five legal benchmarks (CLASE, MSLR, LexChain, LexEval, and JuDGE), covering 27 legal task types. For each task, we selected up to 100 instances and employed Qwen3-27B to identify potential deficiencies by comparing candidate responses with preferred responses or reference answers. Among 2,456 successfully analyzed instances, 862 were identified as containing at least one potential error.

One PhD-level legal expert subsequently reviewed and consolidated these preliminary error patterns, identifying their underlying legal quality requirements to produce an initial taxonomy draft.

During top-down refinement, the full expert panel iteratively examined the proposed criteria according to established principles of legal writing, judicial decision-making, and legal argumentation, together with Chinese judicial writing standards. The experts evaluated the legal relevance, conceptual coherence, completeness, and boundaries of the proposed criteria, consolidated overlapping concepts, and identified normative requirements that could not be fully captured by empirical error patterns.

All three experts ultimately reached consensus on the final taxonomy, including its three dimensions (Style, Element, and Chain) and their fine-grained criteria.

We acknowledge that conventional inter-annotator agreement statistics were not calculated during this consensus-based construction process. The taxonomy was developed through iterative expert deliberation rather than independent annotation of a fixed set of categories. We will clarify this distinction and document the experts' roles and construction procedure in the revised manuscript.

Independent expert validation of rubric-derived preferences.

To further evaluate whether our rubric rewards align with professional legal judgments, we conducted additional human evaluations across all three dimensions.

We recruited three additional legal experts (two with master's degrees in law and one with a PhD in law), each independently responsible for evaluating response pairs in one assigned dimension. This arrangement was intended to minimize inconsistencies arising from different experts applying evaluation criteria across dimensions.

An expert from the original taxonomy construction panel further reviewed disputed cases to ensure annotation quality and consistency with the original taxonomy. This review identified 16, 6, and 20 ambiguous pairs in Style, Element, and Chain, respectively, which were excluded from the adjusted analysis.

The adjusted expert agreement rates are summarized below:

DimensionExpert Agreement Rate
Style79.76%
Element90.43%
Chain83.08%

These results provide additional human-based evidence that our rubric-derived preference signals are broadly aligned with professional legal assessments.

For Chain, we sampled 150 response pairs equally across three preference-pair construction categories. After excluding 20 disputed pairs, the remaining 130 pairs achieved 83.08% expert agreement. Notably, Category C achieved 100% explicit expert agreement (50/50), providing encouraging evidence that the Chain rubric captures legally meaningful reasoning distinctions.

We note that these results measure the alignment between rubric-derived preferences and expert judgments, rather than conventional inter-annotator agreement among multiple independent annotators.

Taxonomy and rubric operationalization.

We would also like to distinguish between the expert-defined taxonomy and its practical implementation as reward functions.

The taxonomy establishes the legal quality criteria that should be evaluated, while the rubric rewards operationalize these criteria into measurable signals. Context-independent criteria, particularly those related to professional legal expression, are approximated using linguistic statistics, whereas context-dependent criteria concerning legal elements and reasoning are assessed using structured LLM-based rubrics.

We acknowledge that the choices involved in implementing these criteria, including scoring levels and numerical weighting, require clearer documentation. We will distinguish the expert-defined legal requirements from the implementation-specific scoring choices in the revised manuscript.

W2 & Q3: Style Reward Robustness and Potential Length Bias

We sincerely thank the reviewer for this insightful comment. We agree that corpus-level linguistic statistics may be sensitive to response length and vocabulary distribution, and that such statistics cannot independently guarantee textual coherence or substantive legal quality.

Additional Analysis of Potential Length Bias.

Motivated by this concern, we conducted an additional analysis of our Style preference data to investigate whether the reward systematically favors longer responses.

Specifically, we compared the lengths of chosen and rejected responses in our Style preference pairs and found that 41.2% of the chosen responses were shorter than their corresponding rejected responses. This finding suggests that our Style reward does not simply favor longer responses, and that response length alone does not determine the preference direction.

While individual statistical features may still exhibit some length sensitivity, these results indicate that the Style reward captures characteristics of professional legal expression beyond response length, rather than merely rewarding longer text.

Supplementary perplexity-based filtering.

To partially address this limitation, we introduced a supplementary perplexity (PPL) filtering mechanism using a frozen Qwen3-4B model.

Specifically, we calibrated a two-sided PPL acceptance interval using 7,913 authentic judicial reasoning passages, with another 1,979 passages held out for validation. The resulting acceptance interval is [2.603, 10.949].

Texts outside this interval, including highly unpredictable legal terminology combinations and abnormally predictable repetitions, are rejected before Style evaluation. Texts passing the filter retain their original five-component Style scores.

Importantly, PPL serves as a preliminary textual quality filter rather than an additional dimension of our taxonomy.

The additional adversarial evaluation produced the following results:

Adversarial CategorySamplesRejection Rate
Synthetic legal terminology passages60100.00%
Shuffled word sequences10100.00%
Shuffled character sequences10100.00%
Repeated authentic judicial passages6088.30%
Short-text adversarial attacks15075.30%

These results indicate that the supplementary filter can effectively reject several types of abnormal inputs that were previously problematic for the statistical Style reward.

W3 & Q1: Hallucination and Legal Factual Reliability

We appreciate the reviewer's concern regarding fabricated statutes, incorrect entities, and the potential for reward models to favor superficially convincing but legally incorrect responses.

We agree that reliable legal reward modeling must distinguish professional legal presentation from substantive legal correctness.

Hallucination-related criteria in the taxonomy.

Although hallucination is not introduced as a separate top-level dimension, several of its manifestations are explicitly covered by our expert-defined criteria.

In particular, the Element dimension evaluates the correctness and completeness of legally relevant information, including Subjects, Facts, Statutes, and Decisions. The Chain dimension further evaluates the correctness of legal reasoning and the connections among these elements.

Our bottom-up error analysis also identified factual misidentification and fabrication, inaccurate or inapplicable legal grounds, and incorrect application of legal rules as recurring deficiencies. These patterns informed the construction of the corresponding taxonomy criteria.

Therefore, the taxonomy explicitly recognizes hallucination-related errors as important aspects of legal response quality, rather than treating the presence of legal terminology or detailed statutory citations as sufficient evidence of correctness.

Limitations of reference-free reward implementation.

We agree that the current reference-free implementation cannot independently guarantee the authenticity or applicability of every statutory citation.

For example, although the Statutes rubric considers the specificity and correctness of legal references, an LLM-based evaluator without access to authoritative legal sources may fail to distinguish a genuine citation from a plausible but fabricated one.

Similarly, the Style reward primarily measures characteristics of professional legal expression and does not independently verify whether the legal terms used in a response are factually appropriate.

We therefore do not interpret a high Style score or a detailed statutory citation as sufficient evidence of legal correctness.

Scope and future improvements.

Our present contribution is to establish a structured, expert-grounded framework for evaluating legal response quality and to investigate its operationalization into reward signals.

We agree that stronger legal factual verification would benefit from authoritative legal retrieval, explicit statutory citation checking, and dedicated hallucination evaluations. These mechanisms could complement the existing Element and Chain criteria without necessarily requiring hallucination to become an additional independent top-level dimension.

We appreciate this suggestion and regard hallucination-specific evaluation and robustness analysis as important directions for improving the reliability of the current reward implementation.

W4 & Q5: Statistical Reliability and Interpretation of Performance Improvements

We thank the reviewer for emphasizing the importance of statistical reliability and careful interpretation of relatively small performance differences.

We agree that improvements of less than one percentage point should not be interpreted as conclusive evidence of superiority without appropriate uncertainty estimates.

Random seeds and reproducibility. To clarify our experimental settings, we used the following five random seeds:

1, 2, 3, 4, and 42.

We will explicitly document these seeds and the corresponding experimental configurations in the revised manuscript.

Interpretation of small improvements. We acknowledge that some reported improvements, such as the 0.35-point DPO improvement on Element and the 0.27-point difference between LexRM-Chain and Skywork-Qwen, are relatively small.

We agree that these differences should be interpreted cautiously and should not, by themselves, be taken as evidence of statistically significant superiority.

Our primary objective is to investigate whether expert-defined legal quality criteria can be translated into useful reward signals across multiple dimensions, rather than to establish the superiority of every individual method on every benchmark.

Accordingly, we regard the broader performance patterns across Style, Element, and Chain, together with our additional expert evaluations, as more informative than isolated marginal improvements.

Statistical reporting. We appreciate the request for confidence intervals and significance testing. We will clarify the seed configurations and supplement the reported comparisons with uncertainty estimates where the corresponding repeated-run results are available.

We will also revise statements about small performance differences to avoid implying statistical significance that has not been established.

Q4: Training Configurations, Learning Rate, and Rule-Based Rewards

We appreciate the reviewer's careful examination of our training configurations and the concern regarding the learning rate and number of optimization steps. We would like to clarify the differences between reward model training and policy optimization, as well as the definitions of our rule-based rewards.

The Style, Element, and Chain reward models are all trained using full-parameter fine-tuning (full_param=true), without LoRA, whereas SFT, DPO, and GRPO use LoRA for policy optimization.

As reported in Appendix B.2, the reward models are trained using the Bradley–Terry objective for two epochs, with a learning rate of 1 × 10⁻⁷, a batch size of 32, and DeepSpeed ZeRO-3.

Regarding the reviewer's concern about approximately 140 optimization steps, we would like to clarify that the training datasets differ across dimensions, containing 3,800 preference pairs for Style, 2,842 for Element, and 3,800 for Chain. Therefore, the same number of optimization steps should not be assumed for all three reward models. The actual optimizer step counts depend on the corresponding training configurations, including effective batch size and gradient accumulation.

We agree that both the learning rate and the actual optimizer step counts should be reported clearly for reproducibility. However, a relatively small learning rate alone does not establish that the training configuration is ineffective; its suitability should be considered together with the optimization setup and observed training behavior.

For policy optimization, the LoRA configurations are summarized below:

HyperparameterSFTDPOGRPO
LoRA rank (r)323232
LoRA alpha646464
LoRA dropout0.0500
Target modulesSame modules*Same modules*Same modules*

All three methods target q_proj__, k_proj__, v_proj__, o_proj__, gate_proj__, up_proj__, and down_proj__.

Additional DPO configurations, including beta, learning rate, batch size, and sequence length, are provided in Appendix B.2. For GRPO, we use TRL 0.26.2, with mask_truncated_completions=True and scale_rewards="group". For each dimension, we separately train the SFT → GRPO(RM) and GRPO(rule) variants, as detailed in the configuration table in Appendix B.2.

The detailed definitions and implementations of the rule-based rewards are provided in Appendix B.2.

Reviewer_LeotOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce LexReward, a taxonomy-driven framework for legal reward modelling that characterizes legal response quality along three complementary dimensions: Style, Element, and Chain. For each dimension, they develop rubrics that specify evaluation criteria and quality levels. The resulting rewards are used to construct pairwise preference data for DPO and reward-model training. Experiments show that the rubric-based rewards reliably distinguish legal responses of different quality and that DPO training on the preference data improves performance across all three dimensions. The learned reward models, LexRM, also support effective downstream optimization: each dimension-specific reward model improves policy performance in its corresponding dimension through reinforcement learning, without requiring reference answers at reward time.

Critical strengths

Introduction of a structured taxonomy of legal response quality that decomposes the requirements of a high-quality legal response into three complementary dimensions (Style, Element, and Chain) and fine-grained criteria, providing a principled foundation for legal reward modelling

Development of interpretable, rubric-based rewards for each fine-grained criterion in the taxonomy using rule-based and LLM-based evaluation, and demonstrating that these rewards effectively distinguish response quality in Best-of-5 selection tasks

Training LexRM, the first family of Chinese legal reward models, on rubric-derived preference data and demonstrating its effectiveness in Test-Time Scaling (TTS)

Critical weaknesses

The authors have classified the taxonomy into 3 classes with the help of legal experts. However, there is no inter-annotator agreement or correlation with the experts' opinions.

The biggest concern with this work is that at no point is human correlation conducted to validate the effectiveness of the answers produced by adopting LexRM. The entire work is driven by comparing models against one another using an LLM-as-a-judge evaluator. This is an essential comparison. However, without human correlation, the effectiveness of answers produced for the legal domain remains unvalidated.

Questions for authors

How do you confirm that your taxonomy is good enough to capture all the nuances of the legal domain? You have mentioned that you worked with legal experts, but you did not discuss their role or how they agreed on this taxonomy, such as through inter-expert agreement or a similar paradigm.

How good is the LLM-as-a-judge evaluator for legal tasks?

Line 376: "To generate candidate responses". How do you determine that the candidate responses are relevant to the task?

LoRA hyperparameters are not provided for reproducibility.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Leot

We sincerely thank the reviewer for the thoughtful and constructive feedback. We particularly appreciate the emphasis on expert agreement, human validation, and experimental reproducibility, which are important considerations for reliable legal reward modeling. In response, we provide further details of our expert-guided taxonomy construction, report additional evaluations involving independent legal experts, and clarify our evaluation protocols and implementation settings. We address each concern below.

W1: Expert-Guided Taxonomy Construction and Validation

We thank the reviewer for highlighting the importance of expert involvement and agreement in establishing a reliable legal quality taxonomy. We would like to clarify that LexReward is fundamentally an expert-guided taxonomy grounded in established legal principles, rather than a classification derived solely from LLM-generated error patterns.

Expert-guided taxonomy construction.

Our taxonomy was developed by an expert panel comprising one law professor and two PhD-level legal researchers, following an iterative process combining bottom-up error induction with top-down expert refinement.

For bottom-up induction, we analyzed five legal benchmarks (CLASE, MSLR, LexChain, LexEval, and JuDGE), covering 27 legal task types. For each task, we selected up to 100 instances and employed Qwen3-27B to identify potential deficiencies by comparing candidate responses with preferred responses or reference answers. Among 2,456 successfully analyzed instances, 862 were identified as containing at least one potential error. One PhD-level legal expert subsequently reviewed and consolidated these error patterns, identifying their underlying legal quality requirements and producing an initial taxonomy draft.

During top-down refinement, all three experts iteratively examined the proposed criteria according to established principles of legal writing, judicial decision-making, and legal argumentation, together with Chinese judicial writing standards. They evaluated the legal relevance, conceptual coherence, completeness, and boundaries of the criteria, consolidated overlapping concepts, and incorporated normative requirements not fully captured by empirical error patterns.

All three experts ultimately reached consensus on the final taxonomy, including its three dimensions (Style, Element, and Chain) and their fine-grained criteria. Their involvement therefore extended throughout the construction process, from identifying legal quality requirements to refining evaluation criteria and establishing the final taxonomy, rather than merely assigning labels to predefined categories.

We acknowledge that conventional inter-annotator agreement statistics were not calculated during this consensus-based construction process, as the taxonomy was developed through iterative expert deliberation rather than independent annotation of a fixed set of categories.

Additional independent expert validation.

To further evaluate whether our rubric-derived preferences align with professional legal judgments, we recruited three additional legal experts (two with master's degrees in law and one with a PhD in law), each independently responsible for evaluating response pairs in one assigned dimension (Style, Element, or Chain). This dimension-specific assignment was intended to minimize potential inconsistencies arising from different experts applying evaluation criteria across dimensions.

An additional expert from the original taxonomy construction panel reviewed disputed cases to ensure annotation quality and consistency with the taxonomy. This review identified 16, 6, and 20 ambiguous pairs in Style, Element, and Chain, respectively, which were excluded from the adjusted analysis.

The resulting expert agreement rates are summarized below:

DimensionExpert Agreement Rate
Style79.76%
Element90.43%
Chain83.08%

These results provide additional human-based evidence that our rubric-derived preferences are broadly consistent with professional legal assessments across the three dimensions. We note that these measures reflect the alignment between rubric-derived preferences and expert judgments, rather than conventional inter-annotator agreement among multiple independent annotators evaluating the same dimension.

We will clarify the expert panel's composition, construction procedure, consensus process, and additional validation results in the revised manuscript.

W2: Human Validation and Downstream Evaluation of LexRM

We appreciate the reviewer's emphasis on human validation of LexRM-optimized responses. We agree that professional legal judgments are particularly valuable for assessing the reliability of reward signals and the substantive quality of generated legal responses.

We would like to clarify that our empirical evaluation combines established legal benchmark protocols with additional independent expert assessments, providing complementary evidence of LexRM's effectiveness.

Evaluation using established legal benchmarks.

Our downstream evaluation follows the standard protocols of established legal benchmarks, covering both candidate response selection and policy optimization. Specifically, CLASE, LegalΔ, and LexChain assess professional legal expression, substantive legal task performance, and legal reasoning-related capabilities, respectively.

These benchmarks provide standardized evaluation settings for comparing baseline models with models optimized using LexRM. Importantly, our evaluation does not rely exclusively on holistic LLM-as-a-judge assessments. For instance, LegalΔ includes F1-based evaluation for statute and charge prediction and accuracy-based evaluation for sentencing prediction, case analysis, and legal calculation. Other evaluations follow their respective benchmark-specific protocols.

We believe that performance improvements under these established evaluation protocols provide meaningful evidence of LexRM's effectiveness on the legal capabilities covered by these benchmarks.

Complementary independent expert evaluation.

Beyond benchmark-based evaluation, we conducted additional human assessments to examine whether our rubric-derived preference signals reflect professional legal judgments.

As detailed in W1, three additional legal experts independently evaluated response pairs across Style, Element, and Chain, with adjusted expert agreement rates of 79.76%, 90.43%, and 83.08%, respectively.

For Chain, we sampled 150 response pairs equally across three preference-pair construction categories (50 per category). After excluding 20 disputed pairs identified during expert review, the remaining 130 pairs achieved 83.08% expert agreement. Notably, Category C achieved 100% explicit expert agreement (50/50), providing encouraging evidence that our Chain rubric captures legally meaningful reasoning distinctions.

These expert assessments support the quality of the rubric-derived supervision signals, while the downstream benchmark results evaluate their practical utility in response selection and policy optimization.

Scope of the current validation.

We recognize the reviewer's distinction between validating preference supervision and directly evaluating optimized generations with human experts. Our additional expert evaluations primarily address the former, and we do not present them as a substitute for a comprehensive human assessment of all downstream generations.

Nevertheless, we believe that expert validation of the supervision signals and performance improvements on established legal benchmarks provide complementary evidence for the effectiveness of LexRM. Direct expert assessments of optimized responses, particularly under more challenging legal reasoning scenarios, would further strengthen this evaluation. We appreciate this suggestion and will consider incorporating such evaluations in future work.

Q1: Taxonomy Coverage and Expert Consensus

We thank the reviewer for this important question. Our objective is to systematically organize important and evaluable criteria of legal response quality, rather than claim exhaustive coverage of every nuance across all legal domains, tasks, and application scenarios.

The three dimensions of LexReward address complementary aspects of legal response quality: Style concerns professional legal expression, Element concerns the completeness and correctness of legally relevant information, and Chain concerns the logical connections and reasoning processes among these elements.

As explained in W1, the taxonomy was constructed through empirical error analysis covering five legal benchmarks and 27 legal task types, followed by iterative refinement by an expert panel comprising one law professor and two PhD-level legal researchers. All three experts reached consensus on the final taxonomy and its fine-grained criteria.

We additionally analyzed rubric scores from 4,615 candidate responses to examine the empirical relationships among dimensions. After controlling for generator identity, Style exhibited nearly zero correlation with Element (Spearman ρ = 0.01) and Chain (ρ = −0.03), while Element and Chain showed a moderate positive correlation (ρ = 0.31). These findings provide further evidence that our dimensions capture distinguishable, though not entirely independent, aspects of legal response quality.

We acknowledge that the current taxonomy has been developed and evaluated primarily within the scope of the Chinese legal tasks considered in this study. We will clarify this scope and the expert consensus procedure in the revised manuscript.

Q2: Reliability of LLM-as-a-Judge Evaluation for Legal Tasks

We appreciate the reviewer's concern regarding the reliability of LLM-based evaluation in the legal domain. We agree that legal correctness and reasoning quality require careful assessment, particularly in cases involving statutory applicability, conflicting authorities, or complex legal reasoning.

We would first like to clarify that our evaluation does not rely exclusively on LLM-as-a-judge. We adopt established legal benchmarks and their corresponding evaluation protocols rather than relying on a single holistic LLM evaluator.

For example, LegalΔ reports F1 scores for statute and charge prediction, as well as accuracy-based metrics for sentencing prediction, case analysis, and legal calculation. CLASE and LexChain are also evaluated according to their respective benchmark-specific protocols. These evaluations provide standardized evidence of performance on the legal capabilities covered by each benchmark.

In addition, our newly conducted expert evaluations provide complementary evidence for the reliability of our rubric-derived preference signals. As reported in W1, the adjusted expert agreement rates reach 79.76% for Style, 90.43% for Element, and 83.08% for Chain, suggesting substantial alignment between rubric-derived preferences and professional legal judgments.

We nevertheless recognize that these findings do not establish the universal reliability of LLM-based evaluation across all legal tasks. In particular, reference-free evaluation cannot independently guarantee the correctness of statutory citations, resolve conflicting legal authorities, or identify every instance of legally invalid reasoning.

Our current work therefore adopts established benchmark evaluation protocols and supplements rubric-based supervision with independent expert validation, rather than claiming that LLM-based evaluation can replace professional legal judgment. We will further clarify the evaluation methods and their respective limitations in the revised manuscript.

Reviewer_51weOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

LexReward proposes a structured framework for evaluating and optimizing Chinese legal-language responses. Its taxonomy distinguishes Style, Element, and Chain: linguistic expression; coverage and treatment of subjects, facts, statutes, and decisions; and the order, completeness, correctness, and nonredundancy of reasoning. The criteria are operationalized through a mixture of deterministic rules and LLM-based assessment. Style rewards compare lexical and syntactic statistics with authentic judicial writing. Element uses a rubric-guided judge without reference answers. Chain uses task-specific heuristics or an LLM-assisted procedure, depending on the available reasoning structure.

Critical strengths

The taxonomy captures a meaningful limitation of outcome-only rewards. A correct legal conclusion can coexist with an unsupported statutory citation, an omitted decisive fact, or a defective inferential step. Distinguishing content coverage from the connections among content gives the framework more diagnostic value than a single correctness or fluency score. It also makes the intended optimization target inspectable: readers can examine which legal-quality criteria are rewarded instead of inferring them from opaque holistic preferences.

Critical weaknesses
  1. The taxonomy's expert grounding is insufficiently documented. The paper describes expert input and error analysis but does not report the number or qualifications of participants, the annotation procedure, or agreement on the proposed dimensions and rubrics. This matters because Style partly measures proximity to corpus-level judicial language statistics. Matching sentence lengths, terminology distributions, or conjunction frequencies does not necessarily indicate appropriate legal communication for a particular question and may favor formulaic or unnecessarily elaborate responses. CLASE's objective component also uses related corpus-comparison features, creating potential construct overlap between training reward and evaluation.
  2. The evidence for substantive reasoning improvement remains narrower than the terminology suggests. Chain evaluation relies on an LLM benchmark protocol, and no independent expert assessment tests whether higher scores correspond to legally valid inference rather than better presentation. The Chain GRPO gain over SFT is only 0.41 overall, without repeated-run uncertainty. Moreover, the three policy blocks concern separately trained models, so the results do not show one policy simultaneously improving expression, legal accuracy, and reasoning. Task-vector merging substantially reduces Style and Chain reward performance despite small parameter-update cosine similarities.
  3. Baseline interpretation also requires care. DPO is initialized from the base model and underperforms SFT across the aggregate dimensions; its gains over the base alone provide limited evidence of advantage over simpler training. Lawformer and Legal-BERT require clearly specified reward adaptation to be meaningful comparators. Finally, reference-free evaluation cannot by itself verify statutes or resolve ambiguous legal facts; the framework's reliability under hallucinated citations, conflicting authorities, and adversarially polished but incorrect reasoning is not established.
Questions for authors
  1. How many legal experts contributed to the taxonomy, what were their relevant qualifications, and how were disagreements resolved? Could you provide an independently annotated sample with agreement statistics and examples where Element coverage and Chain correctness lead to different judgments, demonstrating that the dimensions are distinguishable in practice?
  2. Can you test whether Style rewards prefer concise, legally correct answers over verbose responses padded with authentic judicial phrases? Length-controlled comparisons, paraphrases, and mismatched legal terminology would help establish that the reward measures appropriate expression rather than proximity to the aggregate vocabulary and sentence statistics of the reference corpus.
  3. What happens when optimizing one dimension to the others? Please evaluate each dimension-specific policy on all three benchmarks and compare with a jointly optimized policy. The current separate-model results cannot establish that better style preserves legal accuracy or that higher Chain scores improve substantive legal reliability without compensating regressions elsewhere.
  4. Could you report repeated-run uncertainty for the 0.41-point Chain improvement over SFT and compare SFT followed by DPO with the current base-initialized DPO? Please also specify how encoder baselines obtain scalar rewards and include a matched Qwen3-8B holistic legal-judge baseline to isolate the contribution of the taxonomy itself.
  5. How do the rubrics and learned reward models respond to fabricated citations, subtle statutory misapplication, persuasive but invalid reasoning, and prompt injection inside candidate answers? An independent expert audit and evaluation on unseen legal tasks would clarify whether the rewards generalize beyond benchmark-specific structure and remain dependable during policy optimization.
6 comments
AuthorsOpenReview ↗

Response to Reviewer 51we

We sincerely thank the reviewer for the thoughtful and constructive feedback, and for recognizing the value of our taxonomy in addressing the limitations of outcome-only rewards. We appreciate the acknowledgment of its diagnostic value, particularly in distinguishing legal content coverage from reasoning quality and providing transparent, interpretable optimization objectives. We address the reviewer's concerns in detail below.

W1: The taxonomy's expert grounding is insufficiently documented

We thank the reviewer for highlighting the importance of expert grounding and the validity of Style evaluation. We would like to clarify that LexReward is fundamentally an expert-guided taxonomy grounded in established legal principles, rather than a taxonomy derived solely from empirical error frequencies. Our motivation is to systematically formalize the normative requirements of high-quality legal responses, while subsequent experiments investigate whether these expert-defined criteria can be effectively operationalized as reward signals.

Expert Panel and Construction Procedure.

Our taxonomy was developed by an expert panel comprising one law professor and two PhD-level legal researchers. We adopted an iterative construction process that integrates bottom-up error induction with top-down legal knowledge, with the final taxonomy established through consensus among all three experts.

Bottom-up Error Induction.

We first conducted an empirical error analysis across five legal benchmarks (CLASE, MSLR, LexChain, LexEval, and JuDGE), covering 27 legal task types. For each task, we selected up to 100 instances and employed Qwen3-27B to identify potential deficiencies by comparing candidate responses with preferred responses or reference answers. Among 2,456 successfully analyzed instances, 862 were flagged as containing at least one potential error.

One PhD-level legal expert subsequently reviewed and consolidated the preliminary error patterns, grouping recurring deficiencies and identifying their underlying legal quality requirements to produce an initial taxonomy draft. Representative error patterns and their corresponding taxonomy criteria are summarized below.

DimensionRepresentative Error PatternObserved Instances
StyleLegal Expression Deficiencies18
Element — Subjects / FactsFactual Misidentification and Fabrication193
Element — StatutesInaccurate or Inapplicable Legal Grounds208
Element — DecisionsIncorrect Legal Outcomes and Calculations515
Element / Chain — CompletenessOmission of Essential Legal Information211
Chain — CorrectnessIncorrect Application of Legal Rules138
Chain — CorrectnessLogical Inconsistency and Invalid Inference51
Task ComplianceInstruction and Document-Format Violations183

Note: These are taxonomy-aligned summaries of the original Qwen3-27B error categories, rather than independently verified error counts for individual taxonomy leaves. Multiple error types may occur in the same instance, and the categories are not mutually exclusive.

Top-down Expert Refinement and Consensus.

Building upon the initial draft, the full expert panel iteratively evaluated and refined the taxonomy according to established principles of legal writing, judicial decision-making, and legal argumentation (Goodrich, 1990; Osbeck, 2012; Maley, 2014), together with established Chinese judicial writing standards. Crucially, the experts did not simply adopt the observed error categories. Instead, they examined the legal relevance, conceptual coherence, completeness, and boundaries of the proposed criteria, consolidating overlapping concepts and identifying normative requirements that could not be fully captured by empirical error frequencies.

Through iterative refinement, all three experts reached consensus on the final taxonomy and its fine-grained criteria (Figure 2). We emphasize that the taxonomy is primarily grounded in established legal principles and expert consensus, with empirical error analysis informing its refinement. The subsequent experiments aim to assess the practical utility of operationalizing this expert-guided taxonomy into reward signals, rather than serving as its sole validation.

Style Validity and CLASE.

We acknowledge that corpus-level linguistic statistics cannot fully capture context-dependent writing quality. However, these features operationalize expert-defined legal writing criteria, as detailed in Section 4.1. Moreover, CLASE is itself a legal-style benchmark validated against human judgments. Thus, the similarity between our Style criteria and CLASE is expected, as both reflect established standards of professional legal writing. While this introduces some measurement overlap, the CLASE results still demonstrate the practical effectiveness of our Style reward under an established domain-specific evaluation protocol.

Overall, we emphasize that LexReward's primary contributions include an expert-grounded, structured taxonomy of legal response quality, rubric-based evaluation methods, and downstream optimization experiments demonstrating how the taxonomy can be operationalized as practical reward signals. We will clarify the expert composition, iterative construction procedure, expert consensus, and the scope of Style evaluation in the revised manuscript.

W2: The evidence for substantive reasoning improvement remains narrower than the terminology suggests

We thank the reviewer for these valuable observations. We would like to reiterate that LexReward is fundamentally an expert-guided taxonomy grounded in established legal principles and expert consensus. Our primary contribution is to formalize the multidimensional requirements of high-quality legal responses. The current experiments explore a promising approach to operationalizing these expert-defined criteria into reward signals, rather than claiming an optimal or exhaustive taxonomy-to-reward implementation. We address the specific concerns below.

Independent Legal Expert Validation of Rubric Rewards.

To further assess whether our rubric rewards align with professional legal judgments, we conducted additional human evaluations across all three dimensions. For each evaluation, we constructed response pairs consisting of a higher-scoring positive response and a lower-scoring negative response selected by the corresponding rubric. Legal experts were asked to independently choose the better response in each pair, allowing us to measure agreement between expert preferences and rubric-based rankings. To minimize potential inconsistencies arising from different experts applying evaluation criteria across dimensions, we recruited three additional legal experts (two with master's degrees in law and one with a PhD in law), each independently responsible for annotating response pairs in one assigned dimension (Style, Element, or Chain). To ensure annotation quality and consistency with our original taxonomy, an additional expert from the original taxonomy construction panel reviewed disputed cases and identified 16, 6, and 20 ambiguous pairs in Style, Element, and Chain, respectively, which were excluded from the adjusted analysis.

DimensionExpert Agreement Rate
Style79.76%
Element90.43%
Chain83.08%

For Chain, we sampled 150 response pairs equally across three preference-pair construction categories (50 per category). After excluding 20 disputed pairs identified during expert review, the rubric-derived preferences agreed with expert judgments on 83.08% of the remaining 130 pairs. Notably, Category C achieved 100% explicit expert agreement (50/50), indicating particularly strong alignment for this category of preference comparisons. These results provide additional expert-based evidence that the Chain rubric reflects legally meaningful reasoning quality rather than merely superficial presentation characteristics.

Dimension-Specific Optimization and Cross-Dimensional Behavior.

We agree that our experiments do not establish simultaneous improvements across all three dimensions within a single policy. However, our objective is to examine whether each expert-defined dimension can provide a distinct and meaningful optimization signal, rather than requiring every dimension-specific reward model to improve all aspects of legal response quality simultaneously.

To investigate this distinction, we additionally evaluated the dimension-specific reward models across all three benchmarks using the same TTS setting:

Reward ModelCLASE (Style) ↑LegalΔ (Element) ↑LexChain (Chain) ↑
LexRM-Style8735.640.34
LexRM-Element44.562.5548.67
LexRM-Chain24.7563.4550.57
Random Baseline5046.1145.41

The results reveal distinct performance patterns across dimensions. LexRM-Style achieves the strongest performance on CLASE, whereas LexRM-Element and LexRM-Chain perform better on substantive legal tasks. In particular, LexRM-Chain achieves the highest LexChain score, while also demonstrating strong performance on Element.

Importantly, these cross-dimensional differences are consistent with expert-validated assessments of legal quality.

As demonstrated by our additional expert agreement evaluation in Section 1 and the expert-informed evaluation methodology of CLASE, professional legal expression and substantive legal competence are distinct aspects of response quality. For example, Llama and Gemma can exhibit stronger professional legal writing characteristics while performing less well on legal content and reasoning tasks. Thus, a Style-oriented reward model exhibiting weaker Element or Chain performance does not necessarily indicate a failure of the taxonomy; rather, it may reflect the different quality criteria being measured.

We further analyzed the rubric scores of 4,615 candidate responses before reward-model training. After controlling for generator identity, Style exhibits nearly zero correlation with Element (Spearman ρ = 0.01) and Chain (ρ = −0.03), whereas Element and Chain show a moderate positive correlation (ρ = 0.31). These observations further support the distinction between professional legal expression and substantive legal quality, while reflecting the natural dependence of legal reasoning on correctly identified legal information.

Task-Vector Merging.

We acknowledge that the performance degradation of LexRM-Merge indicates the limitations of naive task-vector merging, as low parameter-update cosine similarity does not necessarily guarantee functional compatibility. However, our primary contribution is the expert-guided taxonomy grounded in established legal principles, while task-vector merging represents only one exploratory approach to integrating its dimension-specific reward models. Its limitations therefore highlight opportunities for improving reward integration, rather than undermining the validity of the taxonomy itself.

W3: Baseline interpretation also requires care

We thank the reviewer for these thoughtful comments. We agree that the baseline results and the scope of our evaluation should be interpreted carefully. As clarified in our previous responses, LexReward is primarily an expert-guided taxonomy grounded in established legal principles and expert consensus. Our experiments aim to explore the feasibility of translating these expert-defined quality criteria into practical reward signals, rather than to establish the superiority of any particular optimization method. We include both established general-purpose reward models and legal-domain pretrained encoders to examine different approaches to legal response quality assessment.

DPO, SFT, and Preference Data Quality.

We agree that DPO's improvements over the base model do not establish its superiority over SFT. Our motivation for including DPO was to investigate whether rubric-derived preference pairs provide useful supervision for legal response generation, rather than to compare optimization algorithms.

To further examine the effect of preference supervision under different initialization settings, we additionally evaluated DPO initialized from an SFT checkpoint. The updated Chain results are summarized below:

MethodCLASE OverallElement OverallLexChain Overall
SFT + DPO8.7560.2157.24

These experiments examine the potential utility of rubric-derived preference data under different training configurations. We do not interpret them as demonstrating that DPO necessarily outperforms SFT or vice versa. In addition, the independent legal expert evaluations reported in our response to W2 provide complementary evidence that our preference pairs reflect meaningful distinctions in legal response quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.