acceptodds
Under review as a conference paper at ICLR 2027

LegalEvolver: Agentic Legal Judgment Prediction via Rubric-Guided Self-Evolution

Abstract

As LLM-based agents develop, they have been increasingly applied in the legal domain. Legal Judgment Prediction (LJP) requires models to conduct multi-step reasoning and external legal evidence retrieval. Existing legal search agents' performance is often constrained by the quality of retrieval queries. They often formulate queries directly from the initial user request, overlooking latent legal issues and legally salient details. To address this issue, we propose LegalEvolver, a query optimization framework specifically designed to generate legally informative queries for LJP-oriented retrieval. Sampling high-quality optimized queries and their corresponding search trajectories is challenging, as relying solely on unguided high-temperature rollouts is both inefficient and constrained by the model's initial capabilities. We therefore heuristically optimize queries as seeds, improving the efficiency of trajectory generation. Furthermore, we summarize rollout experience into rubrics and use them to guide multi-round iterative self-evolution. Iteratively performing rubric optimization and rubric-guided RL enables the model to surpass its initial performance ceiling. End-to-end performance on extensive benchmarks demonstrates the effectiveness of query optimization on LJP. Comparisons with direct SFT followed by RL demonstrate the effectiveness of rubric guidance. Evolving rubrics further outperform fixed rubrics, confirming the benefits of self-evolution.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Legal Judgment Prediction, LLM Agents, Query Optimization, Rubric-Guided Reinforcement Learning, Self-Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_b2FeOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper investigates the problem of Legal Judgment Prediction (LJP). It proposes LegalEvolver, a legal judgment prediction framework that combines legal agent, high-temperature exploration, query rewriting, and evolving rubrics with policy to generate legally informative queries for LJP-oriented retrieval. Experiments on CAIL2018 and CMDL show that LegalEvolver outperforms traditional LJP classifiers, direct LLM reasoning, legal/search agents, and post-training baselines.

Critical strengths

The paper is generally well written with a clear pipeline.

The experiments are extensive, with multiple types of compared methods and ablations justifying the effectiveness.

Critical weaknesses

My major concern about this submission is novelty. The core components of LegalEvolver including legal agent, high-temperature exploration, query rewriting, and evolving rubrics with policy are all existing methods or techniques. The only novelty of this paper seems to be the preference design under the legal judgement scenario, which is insufficient for a general machine learning methodology.

Repeated experiments are lacking. In Table 1, LegalEvolver (61.5) shows higher overall performance than SmartSearch (60.4) by 1.1. Standard deviation is necessary to see if improvement is significant.

Questions for authors

How are evaluation cases and their near-duplicates excluded from the precedent retrieval corpus? Since retrieved precedents preserve law articles, charges, and prison terms, overlap between the evaluation cases and the retrieval database could directly leak target labels.

3 comments

W1. Novelty beyond existing agent, query-rewriting, and rubric-evolution techniques

Thank you for this comment.

Legal agents, high-temperature exploration, query rewriting, and reinforcement learning have precedents. We focus on the additional challenges that general-purpose agentic retrieval faces in realistic legal retrieval. We also focus on designing training signals that address these challenges.

In legal judgment prediction, the agent must identify facts that could change the judgment. It must then translate them into retrieval queries. Distinguishing similar charges requires evidence about offense elements. Predicting sentences also requires guidance on mitigating circumstances such as voluntary surrender and restitution. A single case therefore creates multiple evidence needs. The agent must choose among statutes, judicial interpretations, sentencing guidance, and precedents. Retrieved evidence may also change the current hypothesis. Subsequent queries must adapt accordingly.

Our query rewriting preserves facts relevant to the current legal issue. It also specifies the questions that remain unresolved. We compare successful and failed trajectories for the same case. We then turn retrieval differences affecting legal qualification and sentencing into rubrics. These rubrics are specific to retrieval tools and reasoning objectives. They provide feedback for intermediate queries. As the policy improves, we identify remaining retrieval errors in new trajectories. We then update the rubrics accordingly.

In the original CAIL2018 and CMDL evaluation, LegalEvolver achieves an Overall score of 61.5. In comparison, SmartSearch achieves 60.4, which is a difference of 1.1 percentage points. After an additional training round, LegalEvolver reaches 64.6. This score is approximately 4.4 percentage points above SmartSearch after the same additional round. This result further supports the effectiveness of continually updating rubrics to address specific legal retrieval errors and jointly optimizing them with the model policy.


W2. Repeated experiments, standard deviations, and the reliability of the gain

Thank you for this comment.

Due to time constraints, we previously completed only three training rounds. We have now conducted one additional round. We provide the full results and analysis in our response to Reviewer EmAR, Question 1. The key finding is that LegalEvolver's Overall score increases from 61.5 to 64.6, which is a gain of 3.1 percentage points. After the same additional round, it exceeds SmartSearch by 4.4 percentage points. Meanwhile, both Standard GRPO and SmartSearch show slight decreases in Overall after an additional training round. This indicates that longer training alone does not necessarily yield gains. It further supports the effectiveness of rubric–policy co-evolution for continued improvement.

We also conducted cross-backbone experiments to further assess the effectiveness of our method. We provide full results in our response to Reviewer URUY, Question 2. The key finding is that LegalEvolver improves over Agentic RAG and SmartSearch by 14.0 and 4.9 percentage points, respectively, on Qwen3-4B. It improves over them by 8.7 and 4.3 percentage points, respectively, on Qwen3.5-4B. These results show that LegalEvolver outperforms both retrieval-augmented and query-optimization baselines across different base models. They further support the effectiveness and cross-model applicability of rubric–policy co-evolution.


Q1. Excluding evaluation cases and near-duplicates from the precedent corpus

Thank you for this comment.

When constructing all precedent retrieval corpora, we screened them against the CAIL2018 and CMDL test sets. We did this by computing BM25 similarity between the factual descriptions (Fact) of test and corpus cases. Under our similarity computation scheme, we removed every corpus case whose similarity to any test case reached 60% or above. We compare factual descriptions because different cases can share the same charge, law article, or sentence. Facts are more useful for identifying overlapping cases and near-duplicate documents. We designed this filter to exclude test cases and highly similar documents so the agent cannot directly retrieve target judgment information. We will specify the similarity computation, threshold, and exclusion procedure in the revised manuscript.

Dear Reviewer b2Fe,

Thank you once again for your valuable comments! As the discussion phase draws to a close, we would like to ask whether we have adequately addressed all your concerns. We look forward to hearing from you!

Thank you for the detailed rebuttal and additional experiments. The empirical evidence strengthened.

However, my main concerns remain:

W1: The proposed framework mainly combines existing techniques, and the methodological novelty beyond prior query-optimization and rubric-evolution methods remains limited.

W2: Additional training rounds and cross-backbone experiments do not replace repeated runs with different random seeds and standard deviations.

Q1: The BM25-based filtering is helpful, but the definition of the 60% similarity threshold and its effectiveness in removing near-duplicates remain unclear.

I maintain my original rating of 1.

Reviewer_URUYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LegalEvolver, a training framework for legal search agents in Legal Judgment Prediction (LJP), where models predict law articles, charges, and prison terms by retrieving statutes, judicial interpretations, sentencing guidelines, and precedents. The authors find that current legal search agents often fail because they write poor search queries (e.g., copying raw case text or sticking to an early wrong charge), while standard SFT + outcome-reward RL provides no step-level feedback for intermediate searches. LegalEvolver addresses this in three stages: (1) generating diverse trajectories via high-temperature sampling and heuristic query rewriting for rejection-sampled SFT; (2) building an initial rubric bank by contrasting queries from successful and failed trajectories on the same case across four tools and three reasoning focuses; and (3) iteratively co-evolving the policy (via GRPO with rubric + outcome rewards) and the rubrics. Experiments on CAIL2018 and CMDL show consistent gains over baselines.

Critical strengths
  1. Well-motivated step-level supervision for legal search: Instead of relying only on final outcome rewards or generic query clarity rules, the paper uses the natural structure of LJP (qualification vs. sentencing errors) to contrast successful and failed trajectories on the same case.

  2. Effective rubric–policy co-evolution with clean ablations: Updating the rubrics as the agent improves prevents the process reward from becoming outdated. The step-matched comparison against Standard GRPO and Fixed-Rubric GRPO clearly shows that the performance gains come from evolving the rubrics rather than simply training for more steps.

Critical weaknesses
  1. High pipeline complexity and unquantified LLM-judge overhead: The framework relies heavily on a strong external LLM (e.g., Qwen2.5-72B) for query rewriting, rubric induction/validation, and scoring every tool call during online GRPO. The paper does not report training wall-clock time or token costs, nor does it analyze sensitivity to LLM-judge noise or potential reward hacking.

  2. Modest gains over strong baselines and single-backbone evaluation: In Table 1, the improvements over the one-stage query optimization baseline (SmartSearch) are relatively small (e.g., 61.5 vs. 60.4 overall, and 25.6% vs. 25.5% on CMDL prison term accuracy). Moreover, all post-training experiments use only a single base model (Qwen3-8B) under Chinese criminal law, leaving generalizability across model scales and other legal systems (e.g., common law) untested.

Questions for authors

See weaknesses above

4 comments
AuthorsOpenReview ↗

Response 1/2

W1. Pipeline complexity, LLM-judge overhead, scoring noise, and reward hacking

We appreciate the request to quantify the external-LLM overhead.

During initialization, Qwen2.5-72B rewrites queries for a small set of trajectories and consumes approximately 0.93 million tokens. Initial rubric induction consumes approximately 1 million tokens. The next three rounds consume approximately 0.1 million tokens per round, or 0.3 million tokens in total. Four rounds of rubric induction consume approximately 1.3 million tokens. Query rewriting and rubric induction together consume approximately 2.23 million tokens.

During GRPO, Qwen2.5-72B provides one rubric-reward evaluation per tool call. Each trajectory averages 7.79 tool calls, with approximately 80 tokens per reward evaluation. With 32 rollout groups per step and 8 trajectories per group, 400 steps generate approximately 102,400 trajectories and 797,696 reward evaluations. The estimate of 32 × 8 × 400 × 7.79 × 80 yields approximately 63.82 million tokens for online scoring. This calculation uses an estimate of 80 tokens per evaluation. We will verify complete input/output token counts and actual runtime against logs.

To assess query-quality scoring agreement, we sampled a separate set of 120 instances. We obtained human and LLM scores using the same 1–5 criteria and achieved a Kappa of 0.819. We plan to examine how different judges, repeated scoring, and score perturbations affect training or trajectory selection. We will report the downstream performance effects.

To address reward hacking, we sampled 120 trajectories from rollouts generated by the final training checkpoint. We compared human and LLM ratings of the online rubric reward using levels 0, 1, 2, and 3. This yielded a Kappa of 0.8374. This result indicates high agreement when humans and the LLM apply the same rubrics. We plan to independently inspect high-reward trajectories for the relevance of retrieved evidence and its support for the judgment.

AuthorsOpenReview ↗

Response 2/2

W2. Modest gains, single-backbone evaluation, and generalization to other legal systems

Thank you for these suggestions.

1. An additional training round widens the margin over the strong baseline.

We previously completed three training rounds due to time constraints. We have now conducted an additional round and observed further improvement. Full results and analysis are in our response to Reviewer EmAR, Question 1. LegalEvolver's Overall score increased from 61.5 to 64.6, a gain of 3.1 percentage points. It exceeds the SmartSearch score of 60.2 by 4.4 percentage points after the same number of rounds. Standard GRPO and SmartSearch both show slight decreases in Overall scores after an additional round. This indicates that extending training alone does not guarantee gains. These results support the effectiveness of continued rubric–policy co-evolution.

2. Additional backbones demonstrate gains across model configurations.

We added Qwen3-4B and Qwen3.5-4B to assess effectiveness across model configurations. LegalEvolver achieved Overall scores of 59.9 and 62.6, respectively. It exceeds the strongest baseline, SmartSearch, by 4.9 and 4.3 percentage points, respectively. These results show that the gains extend beyond Qwen3-8B to other base models. Detailed results follow.

Qwen3-4B

Method  CAIL2018CMDLOverall  
LawChargePrisonLawChargePrison
Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  
Direct Reasoning53.8 / 34.963.0 / 52.93.9 / 3.169.0 / 76.671.9 / 75.96.7 / 5.643.1
Agentic RAG63.2 / 46.865.4 / 56.04.8 / 4.169.3 / 77.875.8 / 78.15.5 / 4.745.9
Vanilla SFT73.1 / 61.069.4 / 58.419.4 / 17.375.4 / 78.173.5 / 74.521.2 / 17.553.2
Standard GRPO73.7 / 59.570.7 / 59.620.1 / 17.878.8 / 80.176.1 / 77.721.5 / 18.154.5
SmartSearch73.8 / 58.570.6 / 59.419.7 / 17.480.5 / 82.576.0 / 77.723.3 / 20.255.0
LegalEvolver79.7 / 64.777.2 / 67.423.4 / 19.584.9 / 86.983.5 / 84.224.9 / 22.959.9

Qwen3.5-4B

Method  CAIL2018CMDLOverall  
LawChargePrisonLawChargePrison
Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  
Direct Reasoning72.5 / 60.169.1 / 61.617.9 / 17.611.6 / 38.064.6 / 65.015.2 / 17.642.6
Agentic RAG81.6 / 66.780.2 / 72.321.6 / 20.540.0 / 62.179.7 / 80.321.4 / 21.254.0
Vanilla SFT77.6 / 70.074.6 / 67.618.9 / 16.475.8 / 78.474.3 / 74.621.6 / 17.455.6
Standard GRPO79.4 / 68.077.2 / 68.919.6 / 16.580.6 / 81.276.4 / 79.222.3 / 19.157.4
SmartSearch80.1 / 68.677.8 / 69.520.1 / 17.181.8 / 83.177.3 / 79.823.8 / 20.458.3
LegalEvolver85.1 / 73.383.2 / 76.523.0 / 18.985.8 / 87.484.5 / 85.125.4 / 23.362.6

3. Adapting the framework to other legal systems requires local knowledge and retrieval tools.

Jurisdictions differ in their statutes, judicial interpretations, rules for precedents, and prediction tasks. Applying this framework elsewhere requires a jurisdiction-specific legal knowledge base. It also requires adapted retrieval tools, query rubrics, and task outputs. The underlying mechanism induces rubrics from successful and failed trajectories while jointly improving the rubrics and policy. This mechanism does not depend on the statutes of a specific country. It offers a general framework for legal adaptation.

Thank your for the explanation, especially the cost.

I was wondering does the author compare the computational costs with the baselines. I am concerned that although the proposed method outperforms baselines, it leads to substantially higher cost.

Thank you for raising this important point about computational cost.

We further compared the model-call overhead of LegalEvolver and SmartSearch during training. SmartSearch incurs overhead in two stages: (1) during rollout, it scores queries in the initial trajectories, refines low-quality queries based on the feedback, and regenerates subsequent trajectory steps; (2) during reward computation, it evaluates query quality to provide process rewards for GRPO optimization. We estimate SmartSearch’s online scoring output at approximately 61.44 million tokens, comparable in scale to LegalEvolver’s 63.82 million tokens. The additional overhead specific to LegalEvolver primarily comes from iterative rubric induction, which consumes approximately 0.1 million tokens per round, a relatively small amount compared with online scoring. LegalEvolver ultimately achieves an Overall score of 64.6, outperforming SmartSearch’s 60.2 by 4.4 percentage points. These results suggest that rubric evolution delivers substantial performance gains at an affordable additional supervision cost.

Reviewer_EmAROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes LegalEvolver, a training framework for legal search agents that predicts law articles, charges, and prison terms. The framework has three stages: high-temperature exploration with heuristic query rewriting and rejection-sampling SFT; rubric construction by contrasting successful and failed trajectories for the same case across four retrieval tools and three reasoning focuses; and rubric-guided GRPO with iterative rubric refinement. The method is evaluated on charge-balanced CAIL2018 (1,000 cases) and CMDL (2,000 cases) test sets against classification, direct reasoning, agentic retrieval, and post-training baselines, with ablations, training curves, query failure analysis, and LLM-judged query quality.

Critical strengths
  • The paper identifies query formulation as a key bottleneck in legal judgment prediction and addresses it with legally informed query rewriting, incorporating case-specific facts and unresolved legal issues rather than relying on generic search queries.

  • The rubric construction method uses contrasts between successful and failed trajectories from the same case to derive fine-grained supervision for intermediate retrieval actions. This provides more informative feedback than relying only on final judgment outcomes.

  • The experiments on CAIL2018 and CMDL show consistent improvements over existing agentic retrieval and post-training baselines. The analysis of query quality and rubric evolution also helps explain how the proposed method improves legal search behavior.

Critical weaknesses
  • The improvements on SmartSearch (+1.1 Overall) and fixed-rubric GRPO (+1.15) are relatively small, thus it is difficult to determine how robust the gains are. This is especially relevant for macro-F1, which is computed over many charge categories with few examples each. The paper would benefit from stronger evidence that improvements are reliable.

  • The contribution of rubric guidance is not fully isolated from trajectory generation and SFT. Vanilla SFT already reaches 58.5 Overall, compared with 45.2 for Agentic RAG. The fixed-rubric variant also improves Overall by only 0.45 over the no-rubric-reward variant, with no gain in CAIL accuracy or Ma-F1. The results therefore support rubric evolution more clearly than rubric rewards in general.

  • The paper refers to legal experts and their LJP trajectories but does not explain how these trajectories were collected or how many experts were involved. It is also unclear whether the rewriting strategies come from actual expert data or heuristics based on the authors' legal knowledge. In addition, the LLM-based query-quality evaluation closely follows the rubric's own criteria and may not independently establish that the queries retrieve better evidence. More description should be provided.

  • The paper does not clarify whether the precedent corpus was deduplicated against the CAIL2018 and CMDL test sets. Since retrieved precedents include charges and law articles, overlapping cases could cause label leakage. It is unclear whether the classification baselines were evaluated on the same balanced test split, making the fairness of these comparisons difficult to assess.

Questions for authors

See weakness.

4 comments
AuthorsOpenReview ↗

Response 1/2

W1. Modest gains over SmartSearch and fixed-rubric GRPO: how reliable are the improvements?

Thank you for your feedback.

Due to time constraints, we initially completed only three training rounds. This limited our improvement over SmartSearch. We have now conducted one additional round. LegalEvolver's Overall score increased from 61.5 to 64.6 (a gain of 3.1 percentage points). SmartSearch scored 60.2 after the same additional round, 4.4 percentage points below LegalEvolver. The detailed results are shown below.

Method  CAIL2018CMDLOverall  
LawChargePrisonLawChargePrison
Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  Acc. / Ma‑F1  
Direct Reasoning60.9 / 42.856.9 / 34.68.7 / 6.967.5 / 77.173.6 / 78.39.0 / 8.243.7
Agentic RAG62.9 / 44.858.2 / 35.310.2 / 8.272.2 / 80.175.9 / 79.77.9 / 7.045.2
Vanilla SFT79.9 / 70.076.6 / 68.520.0 / 20.580.6 / 82.779.6 / 80.223.6 / 19.758.5
Standard GRPO80.5 / 68.577.9 / 69.220.8 / 20.683.9 / 84.881.9 / 83.523.6 / 20.459.6
SmartSearch80.9 / 67.278.2 / 69.320.5 / 20.385.4 / 87.481.9 / 83.725.3 / 22.460.2
LegalEvolver86.1 / 73.083.9 / 76.723.7 / 22.189.2 / 91.089.0 / 89.526.6 / 24.664.6

The additional round shows there was still room for improvement after the initial three rounds. As the policy improves, subsequent training can use higher-quality retrieval trajectories. This better connects legally relevant facts with retrieval objectives and improves article and charge prediction.

Improvements in prison-term prediction are more limited. Sentencing involves judicial discretion and requires weighing case-specific factors like the offense's circumstances, the harm caused, the defendant's culpability, and expressions of remorse. Optimizing retrieval alone yields limited gains on this task.

Both Standard GRPO and SmartSearch showed slight decreases in Overall scores after an additional training round. This shows that simply extending training does not always improve performance. It also supports the need to adapt query supervision to address the current policy's weaknesses.


W2. Contribution attribution: rubric guidance versus trajectory generation and SFT

Thank you for raising this attribution question.

SFT establishes effective tool-use behavior. However, imitating successful trajectories alone has limitations. It teaches the model to retrieve statutes, judicial interpretations, sentencing guidance, and precedents during reasoning. The model then uses this evidence to make predictions. SFT increases the Overall score from 45.2 (Agentic RAG) to 58.5. This highlights the importance of learning effective tool use and evidence utilization. However, a fixed set of demonstrations provides static supervision. It does not adapt to the model's changing errors. Rubric–policy co-evolution addresses this. It explores weaknesses in the current retrieval policy and derives and refines rubrics from new trajectories. This lets query-level feedback adapt to the model's capabilities and error patterns. After the additional round, LegalEvolver reached 64.6. This exceeds SFT by 6.1 percentage points and shows further gains from continued exploration and adaptive supervision.

We agree that the ablations support rubric evolution more directly than static rubric rewards alone. In the four-round comparison, fixed rubrics increase the Overall score from 59.2 (without rubric rewards) to 60.2, a gain of 1.0 percentage point. Evolving rubrics further increase the Overall score to 64.6. This outperforms fixed rubrics by 4.4 percentage points and the setting without rubric rewards by 5.5 percentage points. Simply adding static rubrics provides limited gains. We emphasize the mechanism of continually discovering and refining effective rubrics as the policy improves. The four-round ablations are reported below. Accuracy and Macro-F1 scores are averaged over article, charge, and prison-term prediction within each dataset. Parentheses indicate percentage-point changes relative to LegalEvolver.

Method     CAIL2018    CMDL Overall
Acc. / Ma‑F1  Acc. / Ma‑F1  
LegalEvolver (Ours)  64.6 / 57.3  68.3 / 68.4    64.6
w/o Rubric Evolution  60.5 (-4.1) / 52.3 (-5.0)  63.6 (-4.7) / 64.4 (-4.0)    60.2 (-4.4)
w/o Rubric Rewards  59.5 (-5.1) / 51.3 (-6.0)  62.6 (-5.7) / 62.9 (-5.5)    59.2 (-5.5)
AuthorsOpenReview ↗

Response 2/2

W3. Expert trajectory sources, query-rewriting strategies, and query-quality evaluation

Thank you for requesting clarification.

We hired one master's-level intern in law at USD 30 per hour to annotate 120 trajectories. We provided the intern with automatically generated retrieval and reasoning trajectories. They used their legal knowledge to rewrite retrieval queries and correct the trajectories. The query-rewriting strategies used in this work were derived from these 120 manually annotated trajectories.

To assess agreement between LLM-based and human query-quality scores, we sampled a separate set of 120 instances. We compared their ratings on a 1–5 scale and obtained a Kappa of 0.819. We will include the detailed scoring criteria for the four evaluation dimensions in the appendix.


W4. Precedent-corpus deduplication, label leakage, and baseline evaluation fairness

Thank you for highlighting data leakage and evaluation fairness.

When constructing all precedent retrieval corpora, we screened them against the CAIL2018 and CMDL test sets. Specifically, we computed the BM25 similarity between the factual descriptions (Fact) of test cases and corpus cases. We identified and removed cases appearing in the test sets to prevent the retrieval of judgment information. All baselines and LegalEvolver were evaluated on the same test cases. We used identical sample sizes and distributions of charges and law articles to ensure a consistent basis for comparison. We will clarify this Fact-based BM25 filtering procedure and the shared evaluation settings in the revised manuscript.

Dear Reviewer EmAR,

Thank you once again for your valuable comments! As the discussion phase is coming to an end, we would like to kindly confirm whether we have sufficiently addressed all your concerns. We are looking forward to your response!

Thank you for the rebuttal. W1 is partially addressed, as the performance gains are larger, but their statistical reliability remains unclear. W2 is largely addressed by the new ablation results, which better demonstrate the contribution of rubric evolution. W3 is partially addressed, as the annotation process is clarified, but independent validation of retrieval quality is still lacking. W4 is largely addressed by the clarifications on data deduplication and evaluation fairness. Overall, I appreciate the authors' efforts, but the remaining concerns lead me to maintain my original rating.

Comments on the paper

1 comment

General Response of LegalEvolver

We sincerely thank all reviewers for their time and constructive feedback. We have added experiments and clarified our motivation in response to your concerns.

Legal judgment prediction requires identifying decisive facts, distinguishing related charges, selecting applicable provisions, and weighing sentencing factors. In practice, these tasks rely on labor-intensive work by legal experts. Enabling agents to address them requires legal-specific reasoning within a general-purpose framework.

Our rubric evolution aims to develop legal retrieval experience tailored to the model's evolving capabilities. By comparing successful and failed trajectories, LegalEvolver identifies weaknesses in the current policy and updates its query-level supervision. The evolving rubrics guide the model to formulate legally meaningful questions and retrieve evidence from a more informed legal perspective.

Our additional evidence includes:

FocusAdditional Evidence
Stronger performanceAfter four rounds, LegalEvolver achieves 64.6 Overall, versus 60.2 for SmartSearch and 59.6 for Standard GRPO.
Rubric evolutionFour-round ablations achieve 60.2 without evolution and 59.2 without rubric rewards, versus 64.6 for the full method.
Additional backbonesOn Qwen3-4B and Qwen3.5-4B, LegalEvolver exceeds SmartSearch by 4.9 and 4.3 percentage points, respectively.
Methodological clarityWe clarify annotation sources, scoring agreement, computational overhead, corpus filtering, and shared evaluation settings.

These results support the value of continually adapting legal query supervision as the policy improves. We will incorporate the additional evidence into the revised manuscript and respectfully ask the reviewers to consider it in their reassessment.

Thank you again to all reviewers and the Area Chair.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.