- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
- Proposes LexReward, a taxonomy of legal response quality with three dimensions: Style (lexical/syntactic), Element (subjects, facts, statutes, decisions), and Chain (order, completeness, correctness, non-redundancy).
- Taxonomy reportedly built with legal experts (top-down specification + bottom-up error analysis). Rubric rewards:
- Style is rule-based: distance from a corpus of Chinese court judgments (KL over legal/sentiment terms, conjunction rate, sentence-length statistics, 3–6-gram coverage).
- Element uses an LLM judge without a reference answer.
- Chain is rule-based or LLM-judged, depending on the task.
- Rubric scores over a five-model candidate pool produce preference pairs, which are used for DPO and to train LexRM (Bradley–Terry, Qwen3-8B). LexRM is then used for Best-of-5 selection and as the GRPO reward.
- Validation: selection accuracy on CLASE, Legal∆, and LexChain; DPO and GRPO (LexRM vs. outcome/ROUGE rewards); sub-criterion ablations.
Critical strengths
- Well-motivated problem: outcome-only rewards ignore whether a legal conclusion is properly grounded. A structured, interpretable reward is a useful direction for legal RL.
- GRPO with LexRM improves all five Legal∆ metrics over SFT (+6.04 average; SLP +9.9, CAC +8.0) without reference answers at reward time. This is the strongest evidence that dimensional rewards add something beyond outcome rewards.
Critical weaknesses
Taxonomy and rubrics are unvalidated, and the expert process is undocumented.
- Missing: the experts’ number and roles, the error-analysis protocol, and agreement statistics.
- Unclear whether experts set the rubric levels and weights.
- No comparison of rubric/LexRM scores with expert ratings, and the Element judge is never checked against gold labels.
The Style metric is likely length-biased
- Forward KL over the full vocabulary mostly measures vocabulary coverage, so longer responses score better; n-gram coverage behaves the same way.
- The single pooled reference penalizes case-specific vocabulary.
- No expert input or validation for this is reported.
- CLASE tests authentic vs. generated text, which is what the metric was calibrated on, so the Style gains may reflect optimizing toward the metric.
Hallucination is not measured, and the reward may encourage it?
- No metric for fabricated statutes or wrong entities.
- The statute rubric gives the top score to law name + article number + content, but the judge has no reference or retrieval to verify them, so specific but fabricated citations can score well.
- The Style reward rewards legal terms whether or not they are correct.
Overstated claims and no significance testing.
- Many deltas are under 1 point with no seeds or confidence intervals: DPO on Element is +0.35, and LexRM-Chain beats Skywork-Qwen by 0.27.
Questions for authors
- Hallucination: Do you measure fabricated statutes or wrong entities before and after DPO/GRPO? Without a reference or retrieval, how does the judge avoid rewarding specific but fabricated citations?
- Experts: Please document the expert roles, the error-analysis protocol, and agreement statistics. Which rubric choices came from experts and which were tuned on data?
- Style length bias: How does the Style reward correlate with response length, and how do held-out authentic judgments of similar length score?
- Training: Is the 1e-7 learning rate (about 140 steps) correct for the RMs and LoRA DPO? This looks very low! Please also give the hyperparameters for GRPO, DPO, and the exact rule-reward definitions.
- Variance: Please report seeds and confidence intervals for Tables 1–3.
Suggestions:
- Add Hallucination as part of the metric. In fact please check possibilities of adding other safety metrics w.r.t legal domain.
- Add a small expert-rated validation set and a robustness check with a second LLM judge or perform a perturbation of the rubric.
4 comments
Taxonomy and rubric operationalization.
We would also like to distinguish between the expert-defined taxonomy and its practical implementation as reward functions.
The taxonomy establishes the legal quality criteria that should be evaluated, while the rubric rewards operationalize these criteria into measurable signals. Context-independent criteria, particularly those related to professional legal expression, are approximated using linguistic statistics, whereas context-dependent criteria concerning legal elements and reasoning are assessed using structured LLM-based rubrics.
We acknowledge that the choices involved in implementing these criteria, including scoring levels and numerical weighting, require clearer documentation. We will distinguish the expert-defined legal requirements from the implementation-specific scoring choices in the revised manuscript.
W2 & Q3: Style Reward Robustness and Potential Length Bias
We sincerely thank the reviewer for this insightful comment. We agree that corpus-level linguistic statistics may be sensitive to response length and vocabulary distribution, and that such statistics cannot independently guarantee textual coherence or substantive legal quality.
Additional Analysis of Potential Length Bias.
Motivated by this concern, we conducted an additional analysis of our Style preference data to investigate whether the reward systematically favors longer responses.
Specifically, we compared the lengths of chosen and rejected responses in our Style preference pairs and found that 41.2% of the chosen responses were shorter than their corresponding rejected responses. This finding suggests that our Style reward does not simply favor longer responses, and that response length alone does not determine the preference direction.
While individual statistical features may still exhibit some length sensitivity, these results indicate that the Style reward captures characteristics of professional legal expression beyond response length, rather than merely rewarding longer text.
Supplementary perplexity-based filtering.
To partially address this limitation, we introduced a supplementary perplexity (PPL) filtering mechanism using a frozen Qwen3-4B model.
Specifically, we calibrated a two-sided PPL acceptance interval using 7,913 authentic judicial reasoning passages, with another 1,979 passages held out for validation. The resulting acceptance interval is [2.603, 10.949].
Texts outside this interval, including highly unpredictable legal terminology combinations and abnormally predictable repetitions, are rejected before Style evaluation. Texts passing the filter retain their original five-component Style scores.
Importantly, PPL serves as a preliminary textual quality filter rather than an additional dimension of our taxonomy.
The additional adversarial evaluation produced the following results:
| Adversarial Category | Samples | Rejection Rate |
| Synthetic legal terminology passages | 60 | 100.00% |
| Shuffled word sequences | 10 | 100.00% |
| Shuffled character sequences | 10 | 100.00% |
| Repeated authentic judicial passages | 60 | 88.30% |
| Short-text adversarial attacks | 150 | 75.30% |
These results indicate that the supplementary filter can effectively reject several types of abnormal inputs that were previously problematic for the statistical Style reward.
W3 & Q1: Hallucination and Legal Factual Reliability
We appreciate the reviewer's concern regarding fabricated statutes, incorrect entities, and the potential for reward models to favor superficially convincing but legally incorrect responses.
We agree that reliable legal reward modeling must distinguish professional legal presentation from substantive legal correctness.
Hallucination-related criteria in the taxonomy.
Although hallucination is not introduced as a separate top-level dimension, several of its manifestations are explicitly covered by our expert-defined criteria.
In particular, the Element dimension evaluates the correctness and completeness of legally relevant information, including Subjects, Facts, Statutes, and Decisions. The Chain dimension further evaluates the correctness of legal reasoning and the connections among these elements.
Our bottom-up error analysis also identified factual misidentification and fabrication, inaccurate or inapplicable legal grounds, and incorrect application of legal rules as recurring deficiencies. These patterns informed the construction of the corresponding taxonomy criteria.
Therefore, the taxonomy explicitly recognizes hallucination-related errors as important aspects of legal response quality, rather than treating the presence of legal terminology or detailed statutory citations as sufficient evidence of correctness.
Limitations of reference-free reward implementation.
We agree that the current reference-free implementation cannot independently guarantee the authenticity or applicability of every statutory citation.
For example, although the Statutes rubric considers the specificity and correctness of legal references, an LLM-based evaluator without access to authoritative legal sources may fail to distinguish a genuine citation from a plausible but fabricated one.
Similarly, the Style reward primarily measures characteristics of professional legal expression and does not independently verify whether the legal terms used in a response are factually appropriate.
We therefore do not interpret a high Style score or a detailed statutory citation as sufficient evidence of legal correctness.
Scope and future improvements.
Our present contribution is to establish a structured, expert-grounded framework for evaluating legal response quality and to investigate its operationalization into reward signals.
We agree that stronger legal factual verification would benefit from authoritative legal retrieval, explicit statutory citation checking, and dedicated hallucination evaluations. These mechanisms could complement the existing Element and Chain criteria without necessarily requiring hallucination to become an additional independent top-level dimension.
We appreciate this suggestion and regard hallucination-specific evaluation and robustness analysis as important directions for improving the reliability of the current reward implementation.
W4 & Q5: Statistical Reliability and Interpretation of Performance Improvements
We thank the reviewer for emphasizing the importance of statistical reliability and careful interpretation of relatively small performance differences.
We agree that improvements of less than one percentage point should not be interpreted as conclusive evidence of superiority without appropriate uncertainty estimates.
Random seeds and reproducibility. To clarify our experimental settings, we used the following five random seeds:
1, 2, 3, 4, and 42.
We will explicitly document these seeds and the corresponding experimental configurations in the revised manuscript.
Interpretation of small improvements. We acknowledge that some reported improvements, such as the 0.35-point DPO improvement on Element and the 0.27-point difference between LexRM-Chain and Skywork-Qwen, are relatively small.
We agree that these differences should be interpreted cautiously and should not, by themselves, be taken as evidence of statistically significant superiority.
Our primary objective is to investigate whether expert-defined legal quality criteria can be translated into useful reward signals across multiple dimensions, rather than to establish the superiority of every individual method on every benchmark.
Accordingly, we regard the broader performance patterns across Style, Element, and Chain, together with our additional expert evaluations, as more informative than isolated marginal improvements.
Statistical reporting. We appreciate the request for confidence intervals and significance testing. We will clarify the seed configurations and supplement the reported comparisons with uncertainty estimates where the corresponding repeated-run results are available.
We will also revise statements about small performance differences to avoid implying statistical significance that has not been established.
Q4: Training Configurations, Learning Rate, and Rule-Based Rewards
We appreciate the reviewer's careful examination of our training configurations and the concern regarding the learning rate and number of optimization steps. We would like to clarify the differences between reward model training and policy optimization, as well as the definitions of our rule-based rewards.
The Style, Element, and Chain reward models are all trained using full-parameter fine-tuning (full_param=true), without LoRA, whereas SFT, DPO, and GRPO use LoRA for policy optimization.
As reported in Appendix B.2, the reward models are trained using the Bradley–Terry objective for two epochs, with a learning rate of 1 × 10⁻⁷, a batch size of 32, and DeepSpeed ZeRO-3.
Regarding the reviewer's concern about approximately 140 optimization steps, we would like to clarify that the training datasets differ across dimensions, containing 3,800 preference pairs for Style, 2,842 for Element, and 3,800 for Chain. Therefore, the same number of optimization steps should not be assumed for all three reward models. The actual optimizer step counts depend on the corresponding training configurations, including effective batch size and gradient accumulation.
We agree that both the learning rate and the actual optimizer step counts should be reported clearly for reproducibility. However, a relatively small learning rate alone does not establish that the training configuration is ineffective; its suitability should be considered together with the optimization setup and observed training behavior.
For policy optimization, the LoRA configurations are summarized below:
| Hyperparameter | SFT | DPO | GRPO |
| LoRA rank (r) | 32 | 32 | 32 |
| LoRA alpha | 64 | 64 | 64 |
| LoRA dropout | 0.05 | 0 | 0 |
| Target modules | Same modules* | Same modules* | Same modules* |
All three methods target q_proj__, k_proj__, v_proj__, o_proj__, gate_proj__, up_proj__, and down_proj__.
Additional DPO configurations, including beta, learning rate, batch size, and sequence length, are provided in Appendix B.2. For GRPO, we use TRL 0.26.2, with mask_truncated_completions=True and scale_rewards="group". For each dimension, we separately train the SFT → GRPO(RM) and GRPO(rule) variants, as detailed in the configuration table in Appendix B.2.
The detailed definitions and implementations of the rule-based rewards are provided in Appendix B.2.
Response to Reviewer KqiU
We sincerely thank the reviewer for the thoughtful and constructive feedback. We particularly appreciate the concerns regarding expert validation, the robustness of Style rewards, legal hallucinations, and statistical reliability. These suggestions are valuable for clarifying both the contributions and limitations of LexReward. In response, we provide further details of our expert-guided taxonomy construction, report additional independent expert evaluations and robustness experiments, and clarify our training and evaluation settings. We address each concern below.
W1 & Q2: Taxonomy Construction, Expert Validation, and Rubric Reliability
We appreciate the reviewer's emphasis on the importance of expert involvement and independent validation. We would like to clarify that LexReward is fundamentally an expert-guided taxonomy grounded in established legal principles, rather than a classification derived solely from empirical error frequencies.
Expert panel and construction procedure.
Our taxonomy was developed by an expert panel comprising one law professor and two PhD-level legal researchers, following an iterative process that combines bottom-up error induction with top-down expert refinement.
For bottom-up induction, we analyzed five legal benchmarks (CLASE, MSLR, LexChain, LexEval, and JuDGE), covering 27 legal task types. For each task, we selected up to 100 instances and employed Qwen3-27B to identify potential deficiencies by comparing candidate responses with preferred responses or reference answers. Among 2,456 successfully analyzed instances, 862 were identified as containing at least one potential error.
One PhD-level legal expert subsequently reviewed and consolidated these preliminary error patterns, identifying their underlying legal quality requirements to produce an initial taxonomy draft.
During top-down refinement, the full expert panel iteratively examined the proposed criteria according to established principles of legal writing, judicial decision-making, and legal argumentation, together with Chinese judicial writing standards. The experts evaluated the legal relevance, conceptual coherence, completeness, and boundaries of the proposed criteria, consolidated overlapping concepts, and identified normative requirements that could not be fully captured by empirical error patterns.
All three experts ultimately reached consensus on the final taxonomy, including its three dimensions (Style, Element, and Chain) and their fine-grained criteria.
We acknowledge that conventional inter-annotator agreement statistics were not calculated during this consensus-based construction process. The taxonomy was developed through iterative expert deliberation rather than independent annotation of a fixed set of categories. We will clarify this distinction and document the experts' roles and construction procedure in the revised manuscript.
Independent expert validation of rubric-derived preferences.
To further evaluate whether our rubric rewards align with professional legal judgments, we conducted additional human evaluations across all three dimensions.
We recruited three additional legal experts (two with master's degrees in law and one with a PhD in law), each independently responsible for evaluating response pairs in one assigned dimension. This arrangement was intended to minimize inconsistencies arising from different experts applying evaluation criteria across dimensions.
An expert from the original taxonomy construction panel further reviewed disputed cases to ensure annotation quality and consistency with the original taxonomy. This review identified 16, 6, and 20 ambiguous pairs in Style, Element, and Chain, respectively, which were excluded from the adjusted analysis.
The adjusted expert agreement rates are summarized below:
These results provide additional human-based evidence that our rubric-derived preference signals are broadly aligned with professional legal assessments.
For Chain, we sampled 150 response pairs equally across three preference-pair construction categories. After excluding 20 disputed pairs, the remaining 130 pairs achieved 83.08% expert agreement. Notably, Category C achieved 100% explicit expert agreement (50/50), providing encouraging evidence that the Chain rubric captures legally meaningful reasoning distinctions.
We note that these results measure the alignment between rubric-derived preferences and expert judgments, rather than conventional inter-annotator agreement among multiple independent annotators.