acceptodds
Under review as a conference paper at ICLR 2027

Mental-R1: Aligning LLM Reasoning for Mental Health Assessment

Abstract

Mental health problems such as anxiety, depression, and suicide remain urgent global challenges, where timely and accurate assessment is critical for effective intervention. Recently, large language models have been explored for mental health assessment. However, existing general-purpose post-training methods do not align with the cognitive processes of human assessment, which may lead to unreliable reasoning outcomes. To bridge this gap, we propose Cognitive Relative Policy Optimization (CRPO), a reinforcement learning framework tailored for the mental health domain. CRPO extends group relative policy optimization by integrating stage-dependent uncertainty modeling into the policy optimization process. Specifically, we introduce a stage-wise entropy regularization mechanism that encourages broad exploration in early reasoning phases and progressively enforces confident decision-making in later stages, mimicking the human cognitive shift from uncertainty to certainty. In addition, inspired by cognitive appraisal theory, we formalize cognitive reasoning stages, thereby guiding theory-grounded interpretable inference. Experiments on 8 mental health datasets show that CRPO achieves an average improvement of 10.4 percentage points in weighted F1-score over the best reinforcement learning baseline. Furthermore, the CRPO-trained model Mental-R1 demonstrates clear advantages compared with existing large language models on reasoning-intensive cases, suggesting that CRPO enhances reasoning capabilities for mental health assessment.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM, Mental Health, Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_smnbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper observes that generic post-training (SFT or standard RL) does not reflect how mental-health professionals actually assess — which has two salient properties the authors want to build into LLM reasoning. Experiments use Qwen3-8B as the base for all methods, jointly trained on the combined train+validation splits of 8 public datasets. This is competent, well-organised incremental work: GRPO plus a stage-dependent entropy schedule plus a staged output format, applied carefully to eight public mental-health datasets with consistently positive results. The breadth of the comparison set is a genuine strength, and the appendix-level documentation of the eight datasets and their splits is useful.

Critical strengths
  1. This is an interesting idea. Mental-health assessment is a plausible setting in which how a model reasons determines whether subtle internal cues survive against salient external functioning. 2.More analysis than is typical for this kind of submission. The schedule study is a well-built control. Combined with the M × τ grid, this is a real attempt to explain the mechanism rather than just report scores. 3.All datasets are public; split protocols are documented including the sensible decision to align DepSeverity partitions with Dreaddit because they share a corpus, and the de-duplication of RSD to one post per individual. This addresses a real problem in this subfield.
Critical weaknesses

1.The paper's novelty is that its staged structure is justified by cognitive appraisal theory ("theory-grounded", "cognition-aligned", "inspired by and formalised from appraisal theory"). But nothing distinguishes "appraisal-theory stages" from any five-stage reasoning scaffold. 2.The reasoning-demand subset is selected by GPT-5, and then Mental-R1 is compared against GPT-5 on exactly those samples, yielding the paper's second headline number. 3."Reasoning demand" has no human or clinical validation — it is an unvalidated LLM judgement used to construct an evaluation set that favours reporting large gains. 4.The abstract promises "theory-grounded interpretable inference," and Section 1 lists interpretable inference as an outcome. There is no evaluation of whether the generated appraisals are clinically coherent, whether they are faithful, or whether a practitioner would find them useful. 5.Only weighted F1 is reported. Several datasets are imbalanced and several are ordinal; since the paper follows prior work in using weighted F1 it is defensible, but weighted F1 can mask minority-class collapse. 6.Maximum completion length 512 tokens with ≤4096 thinking budget elsewhere; at 5 stages plus the answer this may constrain some datasets. Worth reporting truncation rates.

Questions for authors

1.Please add a structured-CoT prompting-only baseline and an SFT-on-structured-traces baseline, to separate the staged output scaffold from the RL objective. 2.Was the M/τ grid evaluated on the same test splits used for Table 2 and Table 3? If so, please re-select on validation and report both. 3.Even a small annotation study comparing CRPO traces with GRPO traces for coherence and usefulness would substantiate the interpretability claim. If none is possible, please drop the claim.

3 comments
AuthorsOpenReview ↗

Response to Reviewer smnb

Thank you for the detailed assessment and constructive suggestions on the experimental controls, evaluation protocol, and analysis. We have uploaded a revised manuscript incorporating the additional experiments and clarifications described below.

W1. Appraisal-theory stages versus a generic five-stage scaffold

To examine the contribution of the appraisal-inspired stage definitions, we trained a CRPO variant using five generic reasoning stages: observation, analysis, interpretation, evaluation, and conclusion. This variant used the same SER schedule and parameters (M = 0.06 and τ = 3.5) as CRPO.

MethodDATDRSDDepSeverityLT-EDISDCNLDreadditFIGLIDAvg.
CRPO with generic stages59.19±2.2246.34±2.1077.08±1.4255.37±2.1959.90±1.2477.40±1.8388.61±1.5333.51±1.6262.18
CRPO63.62±1.0350.14±4.7278.86±1.1657.14±0.9761.75±1.4080.56±0.5791.57±0.6138.39±1.6665.25

CRPO achieves 65.25 average weighted F1, compared with 62.18 for the generic-stage variant, a gain of 3.08 points, with higher scores on all eight datasets. These results support the usefulness of the appraisal-inspired stage definitions under the same entropy schedule. This comparison is included in Table 2 as the w/ GS ablation and discussed in Section 4.2; the generic-stage definitions and configuration are described in Appendix C.1.

W2 / W3. Subset selection and the validity of reasoning demand

We repeated the subset selection using Gemini 4, which is outside our baseline set. The top-20% subsets selected by GPT-5 and Gemini 4 have an overlap of 78.4%. Mental-R1’s gain in average weighted F1 over the strongest baseline is 15.3 points on the Gemini-selected subsets, compared with 15.6 points on the GPT-5-selected subsets. These results show that the reported advantage persists across the two selectors. The second-selector analysis has been added to Section 4.4, and the scoring prompt is provided in Appendix H.

W4 / Q3. Evaluation of reasoning coherence and usefulness

We conducted a human evaluation comparing CRPO and GRPO on 80 test inputs, with 10 randomly sampled from each dataset's test split. Two mental health professionals independently selected the better output or a tie on each criterion, yielding 160 judgments per criterion. The outputs were presented without model or method identifiers.

DimensionCRPO preferredTieGRPO preferred
Coherence38.8%48.1%13.1%
Usefulness47.5%38.1%14.4%
Closeness to human professionals’ reasoning51.9%39.4%8.8%

Raw inter-annotator agreement ranged from 78.8% to 81.3%, with Cohen’s κ = 0.63–0.69. CRPO received more preference votes than GRPO on all three criteria: 62 versus 21 for coherence, 76 versus 23 for usefulness, and 83 versus 14 for closeness to human professionals’ reasoning. These results are reported in Section 4.6 and Figure 5, with the evaluation criteria and agreement statistics in Appendix I.1.

W5. Evaluation beyond weighted F1

We added macro-F1, which assigns equal weight to each class. CRPO achieves an average macro-F1 of 56.99, compared with 45.37 for GRPO, a gain of 11.62 points, with higher scores on all eight datasets. The gains on RSD, DepSeverity, and LT-EDI are 15.07, 13.11, and 14.22 points, respectively. The per-dataset macro-F1 results are included in Table 6 of the revised manuscript.

MethodDATDRSDDepSeverityLT-EDISDCNLDreadditFIGLIDAvg.
GRPO54.35±2.7415.86±3.1538.25±0.4527.42±1.3951.40±0.4072.29±0.8085.82±0.3017.55±0.8745.37
CRPO62.58±2.1930.93±1.9451.36±0.8241.64±1.1161.38±1.5479.27±1.6190.89±1.4537.86±2.2256.99

W6. Completion length and truncation

The maximum completion length was set to 512 tokens as a practical trade-off given the model size and available GPU memory. Across the CRPO test outputs, 2.7% of completions reached this limit. These details are reported in Appendix C.

AuthorsOpenReview ↗

Response to Reviewer smnb

Q1. Structured-CoT prompting and structured-trace SFT baselines

We added both requested baselines. Since the original training datasets do not provide reasoning traces, we constructed synthetic structured traces using the original, frozen Qwen3-8B model. For each training prompt, we sampled four candidate completions with the five-stage prompt and retained at most one trace, randomly selected from candidates that satisfied the format requirements and whose final answer matched the training label. The selected traces were then used for supervised fine-tuning.

MethodDATDRSDDepSeverityLT-EDISDCNLDreadditFIGLIDAvg.
Frozen Qwen3-8B, five-stage prompt53.62±2.4530.75±1.9159.66±1.4535.77±1.8349.42±2.2970.74±0.7278.62±0.4221.08±1.5749.96
SFT on structured traces54.93±2.3925.39±1.7866.50±1.0031.45±1.7153.15±1.4673.22±1.8681.27±1.9417.62±3.3650.44
CRPO63.62±1.0350.14±4.7278.86±1.1657.14±0.9761.75±1.4080.56±0.5791.57±0.6138.39±1.6665.25

CRPO achieves 65.25 average weighted F1, compared with 49.96 for five-stage prompting and 50.44 for structured-trace SFT. It outperforms both baselines on all eight datasets, with a 14.81-point average gain over structured-trace SFT. These results support the contribution of the CRPO training procedure beyond the tested prompting-only and supervised-learning alternatives. The baseline construction and results are provided in Appendix D and Table 5.

Q2. Was the M/τ grid evaluated on the test splits?

The original Figure 5 presented the parameter-sensitivity results on the test sets. We used the separate training, validation, and test partitions described in Appendix C, and selected parameters using the validation set. We have added the corresponding validation-set plot in Appendix E (Figure 6) and report the validation results below.

Mτ = 2.5τ = 3.5τ = 4.5
0.0461.4264.7862.61
0.0662.9665.8462.19
0.0861.7363.5262.25

The validation results show consistent overall trends with the test-set results: τ = 3.5 performs best for each tested value of M, and M = 0.06, τ = 3.5 is the best configuration on both splits. This validation-selected configuration achieves 65.84 on validation and 65.25 on test.

Dear Reviewer,

We appreciate the time and effort you have taken to review our submission and provide valuable feedback. We have carefully addressed the comments raised during the first round and submitted our responses and revision accordingly. As the discussion period is ending soon, we would greatly appreciate any further feedback you might have. Please let us know if there are any remaining concerns or questions, so we can address them in time.

Thank you again for reviewing our paper.

Best regards,

Authors

Reviewer_J8pCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Cognitive Relative Policy Optimization (CRPO), which combines stage-wise entropy regularization with five structured reasoning stages inspired by cognitive appraisal theory. The resulting model, Mental-R1, improves weighted F1 across eight mental health assessment datasets. Ablations examine the contributions of the reasoning structure and entropy schedule.

Critical strengths
  • A concrete optimization contribution. CRPO introduces an explicit stage-dependent entropy schedule, supported by comparisons against constant and reversed schedules.
  • This method improves performance across eight datasets, with component ablations supporting the usefulness of both entropy regularization and structured reasoning.
Critical weaknesses
  • Claims exceed the demonstrated contribution. Improved classification and a declining token-entropy profile do not establish alignment with human cognition or reliable clinical reasoning. The authors should narrow these claims and clarify how cognitive appraisal theory motivates the optimization design beyond defining the reasoning stages.

  • Intermediate reasoning remains insufficiently validated. The rewards check output structure and final-label correctness, but do not directly evaluate whether intermediate claims are supported by the input. In Figure 3, interpretations such as “limited emotional integration” are not clearly grounded in the statement. This raises the possibility of correct predictions accompanied by unsupported explanations. Targeted evaluation or expert-led error analysis is needed to assess this risk.

Questions for authors

Does the model improve because of the five psychological reasoning stages, or would a simpler template such as “observe, analyze, conclude” work equally well with the same entropy schedule? Please clarify what the experiments actually show and avoid claiming human-like reasoning without sufficient evidence.

In Figure 3, the model mentions “limited emotional integration,” although the input does not clearly support this interpretation. How often does the model predict the correct label while making unsupported claims along the way? Please have domain experts review a sample of outputs and report these errors.

Please acknowledge that a correct label does not guarantee a valid explanation. State clearly that Mental-R1 is intended for research and should not be used for real-world psychological or psychiatric assessment without further validation. Future work could reward evidence-supported reasoning, rather than only correct labels and output format.

3 comments
AuthorsOpenReview ↗

Response to Reviewer J8pC

Thank you for recognizing the optimization contribution and for the constructive suggestions on clarifying the scope of our claims and assessing the validity of the generated reasoning. We have uploaded a revised manuscript incorporating the additional experiments and clarifications described below.

W1. Scope of the claims and the role of cognitive appraisal theory

We have further narrowed the claims in the revised manuscript. We retain alignment with human cognition as a motivation in the Introduction and have removed wording that presents it as an established conclusion. For example, we have replaced “align” with “aims to align” and “cognition-aligned” with “cognition-inspired.” This motivation informs our design: SER encourages exploration in early reasoning stages and progressive consolidation in later stages, while the appraisal-inspired stages organize this progression and define the units for stage-wise entropy regularization.

We also added a human evaluation in which two mental health professionals independently compared outputs on 80 held-out inputs, with 10 per dataset and 160 judgments per criterion for each model comparison. For closeness to human professionals’ reasoning, CRPO received more preference votes than w/o SER (36.3% versus 15.0%, with 48.8% ties) and GRPO (51.9% versus 8.8%, with 39.4% ties). These results support the more specific conclusion that CRPO’s generated reasoning was perceived as closer to human professionals’ reasoning by the mental health professionals in this evaluation. The human evaluation is reported in Section 4.6 and Figure 5, with the criterion definitions and agreement statistics in Appendix I.1.

W2 / Q2. The Figure 3 example and unsupported explanations despite correct labels

Figure 3 shows the output: “may suggest limited emotional integration.” The word “may” explicitly marks this as a tentative inference rather than a definitive conclusion. This interpretation draws on the reported unexplained emptiness (“I do not know why”) and occasional sleep difficulty in the context of preserved work and social functioning. We specifically consulted domain experts about this inference, and they considered it reasonable.

We also conducted a systematic human evaluation of 80 CRPO outputs, with 10 randomly sampled from each dataset's test split. Two mental health professionals independently checked whether factual cues were supported by the input, whether inferences were reasonable and appropriately qualified, and whether the final conclusion was supported by the evidence and reasonable inferences.

CheckAll outputs (80)Correct label (55)Incorrect label (25)
Evidence grounding problem8.1%2.7%20.0%
Unreasonable or unsupported inference13.1%5.5%30.0%
Insufficient support for the conclusion10.6%3.6%26.0%

Rates are averaged across the two annotators, with 160, 110, and 50 judgments in the respective columns and no adjudication. Raw agreement ranged from 91.3% to 93.8%, with Cohen’s κ = 0.54–0.62. Problem flags were less frequent among correctly classified outputs on all three checks. Among the 55 correctly classified outputs, 4 (7.3%) were flagged by at least one annotator for an evidence grounding or inference problem. This rate counts outputs flagged by either annotator and is distinct from the mean annotator rates in the table. The output audit is summarized in Section 4.6, with the criteria and full results provided in Appendix I.2 and Table 7.

AuthorsOpenReview ↗

Response to Reviewer J8pC

Q1. Appraisal stages versus a generic five-stage template

We trained a CRPO variant with five generic reasoning stages (observation, analysis, interpretation, evaluation, and conclusion) using the same SER schedule and parameters, M = 0.06 and τ = 3.5. Using five generic stages keeps the stage count and entropy schedule fixed, allowing us to examine the contribution of the stage definitions.

MethodDATDRSDDepSeverityLT-EDISDCNLDreadditFIGLIDAvg.
CRPO with generic stages59.19±2.2246.34±2.1077.08±1.4255.37±2.1959.90±1.2477.40±1.8388.61±1.5333.51±1.6262.18
CRPO63.62±1.0350.14±4.7278.86±1.1657.14±0.9761.75±1.4080.56±0.5791.57±0.6138.39±1.6665.25

CRPO achieves 65.25 average weighted F1, compared with 62.18 for the generic-stage variant, a gain of 3.08 points, with higher scores on all eight datasets. These results support the usefulness of the appraisal-inspired stage definitions under the same entropy schedule. This comparison is included in Table 2 as the w/ GS ablation and discussed in Section 4.2; the generic-stage definitions and configuration are described in Appendix C.1.

Q3. Validity of explanations and intended use

We agree that a correct final answer does not necessarily establish the validity of the intermediate reasoning. We have added a discussion in Section 4.7 clarifying that Mental-R1 is intended as a copilot to support human professionals and requires further validation before use in real-world assessment.

Dear Reviewer,

We appreciate the time and effort you have taken to review our submission and provide valuable feedback. We have carefully addressed the comments raised during the first round and submitted our responses and revision accordingly. As the discussion period is ending soon, we would greatly appreciate any further feedback you might have. Please let us know if there are any remaining concerns or questions, so we can address them in time.

Thank you again for reviewing our paper.

Best regards,

Authors

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to Reviewer bbxy

Thank you for the helpful suggestions on directly evaluating intermediate reasoning and strengthening the experimental controls. We have uploaded a revised manuscript incorporating the additional experiments and clarifications described below.

W1 / Q1. Grounding and internal consistency of intermediate reasoning

We conducted a human evaluation comparing CRPO and w/o SER on 80 test inputs, with 10 randomly sampled from each dataset's test split. Two mental health professionals independently assessed each pair of outputs and selected the better output or a tie for each criterion. The outputs were presented without model or method identifiers. Both annotators’ original judgments were retained, yielding 160 judgments per criterion.

DimensionCRPO preferredTiew/o SER preferred
Grounding28.8%57.5%13.8%
Internal consistency26.9%61.3%11.9%
Closeness to human professionals’ reasoning36.3%48.8%15.0%

Raw inter-annotator agreement ranged from 73.8% to 83.8%, with Cohen’s κ = 0.57–0.70. Most judgments on grounding and internal consistency were ties. When a preference was expressed, CRPO received more votes on all three criteria: 46 versus 22 for grounding, 43 versus 19 for internal consistency, and 58 versus 24 for closeness to human professionals’ reasoning. These results suggest that CRPO’s generated reasoning was perceived as closer to human professionals’ reasoning than that of w/o SER in this evaluation. The human evaluation is reported in Section 4.6 and Figure 5, with annotation criteria and agreement statistics in Appendix I.1.

W2 / Q2(a). Frozen Qwen3-8B under direct-answer, free-form CoT, and five-stage prompts

We evaluated frozen Qwen3-8B under the three requested prompting settings.

SettingDATDRSDDepSeverityLT-EDISDCNLDreadditFIGLIDAvg.
Direct answer35.00±0.7727.97±1.2341.91±0.3221.53±1.0935.59±0.3656.74±0.2562.95±1.4512.38±0.8436.76
Free-form CoT52.57±3.0628.02±2.8950.40±1.2424.74±3.2946.18±0.8067.21±1.2276.42±1.2316.59±2.3745.27
Five appraisal stages53.62±2.4530.75±1.9159.66±1.4535.77±1.8349.42±2.2970.74±0.7278.62±0.4221.08±1.5749.96
CRPO63.62±1.0350.14±4.7278.86±1.1657.14±0.9761.75±1.4080.56±0.5791.57±0.6138.39±1.6665.25

The five-stage prompt improves average weighted F1 from 45.27 to 49.96 compared with free-form CoT, showing the benefit of the five-stage reasoning structure. CRPO reaches 65.25, approximately 15.3 points higher, and exceeds frozen five-stage prompting on all eight datasets. Structured prompting alone therefore does not account for the observed performance. These results are reported in Table 5, with the evaluation settings described in Appendix D.

W2 / Q2(b). Comparison with a representative entropy-guided method

We added an entropy-control baseline based on KL-Cov [1], which was proposed to mitigate policy entropy collapse during RL training. We used the same five appraisal stages as CRPO.

MethodDATDRSDDepSeverityLT-EDISDCNLDreadditFIGLIDAvg.
KL-Cov57.27±2.6543.01±1.7675.21±0.2952.64±0.8957.88±1.8274.84±2.5187.70±0.3930.86±0.7859.93
CRPO63.62±1.0350.14±4.7278.86±1.1657.14±0.9761.75±1.4080.56±0.5791.57±0.6138.39±1.6665.25

CRPO achieves 65.25 average weighted F1, compared with 59.93 for the KL-Cov entropy-control baseline, with higher scores on all eight datasets. These results support the advantage of SER over the KL-Cov entropy-control baseline on mental health assessment tasks. This comparison is included in Table 2 as the w/ KL-Cov ablation and discussed in Section 4.2.

[1] Ganqu Cui et al. (2025). The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models. arXiv:2505.22617. https://arxiv.org/abs/2505.22617

Dear Reviewer,

We appreciate the time and effort you have taken to review our submission and provide valuable feedback. We have carefully addressed the comments raised during the first round and submitted our responses and revision accordingly. As the discussion period is ending soon, we would greatly appreciate any further feedback you might have. Please let us know if there are any remaining concerns or questions, so we can address them in time.

Thank you again for reviewing our paper.

Best regards,

Authors

Thank you for the responses and clarifications. The rebuttal has largely addressed my concerns. I also consider the concerns raised by the other reviewers important, so I will maintain my current score. I would also understand if the final decision were to accept the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.