acceptodds
Under review as a conference paper at ICLR 2027

Evolving Support Priorities in Empathetic Reinforcement Learning

Abstract

We identify a fundamental mismatch in empathetic reinforcement learning: support priorities evolve with the dialogue state, yet existing methods typically optimize predefined reward specifications that remain fixed across turns. To model these *evolving support priorities*, we organize empathetic support along cognitive, affective, and proactive empathy, and propose **Context-Adaptive Rubric Evolution (CARE)**. At each turn, CARE generates a context-adaptive rubric by adjusting both the weights of these three empathy dimensions and their fine-grained evaluation criteria. The rubric generator is trained with turn-level rubric supervision and human preference data through supervised fine-tuning followed by preference-based reinforcement learning, and then serves as an adaptive reward interface for online empathetic RL. Integrated with both RLVER and MICA, CARE achieves state-of-the-art performance across SentientBench, EQBench3, and EMPA under three independent LLM judges. Notably, on EMPA, CARE improves EPM-Idx over the strongest baseline by at least **13 points** under all three judges, including an increase from **28.11 to 83.54** under Gemini-2.5-Pro. Further analyses show that learned rubric priorities systematically vary across dialogue stages and user emotions, demonstrating that CARE adapts what is rewarded as support needs evolve. Code is provided in the Supplementary Materials.

Primary area:
reinforcement learning
Keywords:
Large language models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_x1RUOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers the evolving of support priorities in empathetic conversation, and focuses on the issue form rubric evolution. For this purpose, this paper proposes CARE, a context-adaptive rubric evolution benchmark. CARE trains a context-conditioned rubric generator by adjusting both the weights of these three empathy dimensions (cognitive, affective, and proactive empathy) and their fine-grained evaluation criteria. Moreover, CARE serves as an adaptive reward interface for online empathetic support conversation based on reinforcement learning. This paper proves the advantages of CARE integrated with both RLVER (CARE-R) and MICA (CARE-M) across three LLMs.

Critical strengths
  1. The CARE benchmark proposed in this paper encompasses the construction of the Principle of Evolving Support Priorities dataset, as well as the training and application of the context-adaptive rubric generator. It took into account the three dimensions of cognition, affectivity, and proactivity, as well as the application of reinforcement learning technology.
  2. This paper conducted numerous experiments, not only confirming the advantages of CARE, but also conducting various analyses and presenting meaningful results.
Critical weaknesses
  1. Although this paper has innovative techniques in solving problems, overall, considering the dynamic changes of emotional support strategies is not a completely novel research topic.
  2. This paper can be further improved in terms of its organizational structure and content presentation (See Questions For Authors).
Questions for authors
  1. Line 096, what are CARE(R) and CARE(M)? Looking at the content of this article, before this line, only line 094 had "RLVER-based CARE(R)". So what about CARE(M)? In particular, the CARE(R) and CARE(M) are used to providing explanations for Figures 1(b), it is even more perplexing. After reading the entire text, it was discovered that this explanation was not provided until lines 311-312. Obviously, such a structure is not conducive to reading.
  2. For Formula 1, it is necessary to provide a further explanation on how Rt corresponds to the three-dimensional evaluation criterion, especially why only its weight is changed. Although there are example prompts in the appendix, it is still necessary to clarify the principle here. This includes the evaluation methods for each dimension, the selection of priorities, etc., which will also facilitate the understanding of the subsequent Formula 3, Formula 4, and Section 3.3 content.
  3. In the experimental results presented in Table 1, the results of RLVER seem to be inferior to those of MICA. And CARE(R) is based on RLVER, while CARE(M) is based on MICA. Then why is CARE((R)) superior to CARE(M)?
  4. The content from line 261 to 302 is intended by the authors to highlight the main contributions of this paper. However, in fact, some of the content here is repetitive with that in the subsequent pages, and it is unnecessary. Moreover, generally speaking, the experimental part should start by introducing the experimental setup (i.e. Section 4.1 in this paper).
1 comment
AuthorsOpenReview ↗

Rebuttal by Authors — Reviewer x1RU

We appreciate the reviewer's positive assessment of CARE's evolving support-priority formulation and its strong experimental validation. We clarify the methodology and address your suggestions below.

[Q1] Although this paper has innovative techniques in solving problems, overall, considering the dynamic changes of emotional support strategies is not a completely novel research topic.

[A1] Existing methods on dynamic support strategies adapt the strategy of the policy, while the reward criteria used to optimize them remain predefined:

  • STRIDE-ED (Ji et al., ACL 2026) reasons dynamically about support strategies, but is optimized with predefined Format, Emotion, and Strategy reward criteria.
  • ESCA (Li et al., AAAI 2026) selects strategies according to the state of the user, but evaluates strategy execution with predefined quality rules.
  • DyCoP (Liu et al., AAAI 2026) dynamically plans support behavior, but does not employ a correspondingly turn-adaptive rubric-based RL reward.

In contrast, CARE generates both the weights over the three empathy dimensions and the fine-grained evaluation criteria at each turn, so that the reward specification itself evolves with the dialogue state. New controls test this distinction: we either fix a learned, context-dependent rubric for the whole dialogue, or keep per-turn regeneration but permute the dimension weights across contexts (SentientBench, DeepSeek-V4-Pro judge; §4.3, Table 3):

Reward / rubric variantScore ↑Succ. (%) ↑Fail (%) ↓
Fixed from persona11.750.0074.00
Fixed from first-turn query10.560.0080.00
Shuffled weights50.204.0032.00
CARE(R), per-turn (full)91.4172.332.00

Most of the improvement is lost once the reward specification stops following the dialogue state.

Revisions: The fixed-rubric and shuffled-weight controls are added to Table 3 (L343–353) and discussed in Section 4.3 (L406–411).


[Q2] CARE(R) and CARE(M) are not clearly defined at their first mention (Lines 096–097), making their use in Figure 1(b) confusing.

[A2] Thank you for pointing this out. CARE(R) denotes the dynamic reward of CARE integrated into the RLVER training framework, and CARE(M) denotes the same reward integrated into the MICA training framework.

Revisions: CARE(R) and CARE(M) are defined in Section 1 (L91–94), and the caption of Figure 1 clarifies that CARE (Ours) denotes CARE(R) (L75–76).


[Q3] How is RtR_t derived from the three empathy dimensions in Eq. (1)? Clarify how each dimension is evaluated, how priorities are determined, and why only the dimension weights are adjusted.

[A3] We clarify the principle here:

  • What changes. Besides the weights of the three empathy dimensions, CARE also generates fine-grained, context-specific criteria within each dimension; only the three dimensions themselves are fixed.
  • Evaluation of each dimension. The three dimensions assess the understanding of the user's situation (cognitive), the recognition and validation of emotions (affective), and actionable support (proactive). The criteria within each dimension are scored and averaged, and the dimension scores are then aggregated with the corresponding weights (Eq. (5)).
  • Selection of priorities. The priorities are produced by the rubric generator, which is trained with SFT followed by preference-based reinforcement learning. Eq. (3) encourages the rubric to discriminate human-preferred responses, while the priority-alignment term in Eq. (4) constrains the weight ordering to be consistent with the support strategies in the preferred responses.

Revisions: The clarification is added after Eq. (1) in Section 3.1 (L191–195).


[Q4] In the experimental results presented in Table 1, the results of RLVER seem to be inferior to those of MICA. And CARE(R) is based on RLVER, while CARE(M) is based on MICA. Then why is CARE(R) superior to CARE(M)?

[A4] RLVER and MICA differ in both reward design and training data. CARE(R) and CARE(M) keep the original training data but replace the reward entirely with that of CARE, which removes the difference in reward design. The remaining difference is the data: RLVER simulates users whose hidden support needs surface gradually and whose emotions respond to every turn, which is exactly where a turn-adaptive reward helps most. This explains why CARE(R) outperforms CARE(M), although RLVER trails MICA under its original reward.


[Q5] Lines 261–302 contain repetitive content.

[A5] Thank you for the suggestion. We have removed this duplicated overview, and Section 4 now begins directly with the Experimental Setup (§4.1).

Revisions: Section 4 now begins directly with the Experimental Setup (Section 4.1, L263–265).

Reviewer_jHsFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper propose CARE, a framework that dynamically generates evaluation rubrics conditioned on dialogue history. These rubrics contain three empathy dimensions: cognitive, affective and proactive empathy with context-dependent weights and fine-grained criteria. The paper then evaluates CARE on SentientBench, EQBench3 and EMPA using three LLM judges and reports good improvements over existing approaches. It also provides ablations, human preference evaluation, qualitative analyses and general-capability retention tests. The idea is well motivated and the results like I said are promising. However, the experimental evidence needs stronger controls to distinguish the contribution of turn-dependent reward adaptation from improvements due to richer rubrics, additional preference training and differences in reward-model capacity.

Critical strengths
  1. Like I mentioned before, the paper tackles a well-motivated research problem. The distinction between adapting the dialogue policy and adapting the reward specification is meaningful. The paper highlights that a fixed reward formulation may be inadequate when a user's needs change during a multi-turn interaction. This is a good research direction, especially for non-verifiable, socially situated tasks.

  2. The framework is technically reasonable and clearly presented. The formulation of the reward rubric is straightforward, interpretable and compatible with existing policy-optimization algorithms.

  3. The ablation experiments are useful and the reported improvements are good, especially on SentientBench and EMPA. The qualitative examples and analyses of rubric weights across dialogue turns also make the method easier to interpret.

Critical weaknesses
  1. The paper acknowledges existing work on query-specific rubrics, personalized rubrics and DynamicRubric. The proposed contribution is the application of context-dependent rubric learning to changing states within empathetic dialogue. This is a reasonable contribution, but the underlying methodology combines established components: LLM-generated rubrics, supervised fine-tuning, preference optimization and rubric-conditioned rewards. The novelty is then primarily in the problem formulation and adaptation mechanism rather than in a fundamentally new RL algorithm. The paper would benefit from a stronger empirical comparison against dynamic rubric-learning methods adapted to the same dialogue setting, rather than relying predominantly on empathetic-dialogue baselines with different reward formulations.

  2. The main novelty is not only generating context-specific rubrics, it is also establishing that the changing reward priorities across dialogue turns improves empathetic RL. But the experiments do not directly compare CARE against a strong, learned, context-specific but temporally fixed rubric baseline. Some of the experiments which is worth considering is (i) full CARE with a newly generated rubric at every turn, (ii) a rubric generated at the first turn and reused throughout the dialogue, (iii) a rubric generated from the complete initial context or an initial support-needs estimate and held fixed, (iv) CARE with turn-wise rubric weights shuffled or permuted across contexts. Without these comparisons, the paper shows that CARE is useful, but not that evolving priorities are the decisive factor.

  3. The improvement in EMPA under Gemini is unusually large. Similarly, SentientBench success rates increase significantly, while some failure rates approach zero. While these results are plausible but my concerns are as follow: (a) first, the paper relies heavily on LLM-based evaluators. Although three judge families are used, agreement across LLM judges does not guarantee agreement with human assessments of supportive quality, (b) the benchmarks include simulated users and model-based assessments. It is possible that the optimized policy becomes effective at satisfying the evaluation environment without necessarily producing equivalently better real-world support, (c) there is insufficient analysis of whether the gains arise from genuinely improved empathy, changes in verbosity or response style or optimization toward the preferences of the judging models. So, I would request per-instance performance distributions, paired confidence intervals, qualitative failure cases and human validation of a subset of complete multi-turn conversations.

  4. Your method derives coarse support-priority supervision from support strategies identified in preferred responses. The priority-alignment reward then encourages rubric weights to match the dimension counts of those preferred responses. This introduces a conceptual issue, i.e., the support strategy expressed by a preferred response is not necessarily the same as the support strategy most needed by the user. Consequently, the training procedure may learn patterns of preferred model responses rather than independently identifying user support needs. I would like to see an evaluation in which independent human annotators label the support priorities of dialogue contexts without seeing candidate responses. These labels could then be compared against CARE's predicted dimension weights.

I am open to hearing what the authors have to say and I am willing to increase my score accordingly.

Questions for authors
  1. Can the authors provide a controlled comparison in which the same learned rubric is frozen at the first turn and reused throughout the conversation? How much improvement remains attributable specifically to turn-wise adaptation?

  2. How do the authors ensure that the rubric generator learns actual user support needs rather than reproducing support strategies already expressed in preferred model responses?

  3. Why are the EMPA improvements so much larger than those on EQBench3? Could this be due to differences in benchmark sensitivity, simulator behavior or evaluator alignment?

  4. Can the authors report human evaluations of complete dialogue trajectories including inter-annotator agreement and uncertainty estimates rather than only final-turn response preferences?

  5. How does CARE compare with existing dynamic rubric-learning approaches adapted to the same empathetic-dialogue environment?

  6. What is the additional computational cost of generating and scoring adaptive rubrics and how does the improvement compare with simply allocating more training compute to the baseline?

  7. Can the authors provide examples where CARE assigns inappropriate support priorities and investigate how these errors affect downstream policy behavior?

5 comments
AuthorsOpenReview ↗

(1/4). Rebuttal by Authors — Reviewer jHsF

We appreciate the reviewer’s recognition that CARE addresses an important problem, provides a technically sound framework, and achieves promising results. We address your questions and provide additional experiments below.

[Q1] (Weakness 2) Can the authors provide a controlled comparison in which the same learned rubric is frozen at the first turn and reused throughout the conversation? How much improvement remains attributable specifically to turn-wise adaptation?

[A1] We compare four variants using the same training protocol, differing only in the rubric generation strategy:

  • Fixed from persona: the rubric is generated once from the complete persona of the simulated user and kept fixed for the whole dialogue.
  • Fixed from first-turn query: the rubric is generated once from the first-turn query of the user and reused for the whole dialogue.
  • Shuffled weights: the rubric is regenerated at every turn, but its dimension weights are permuted across contexts.
  • Per-turn regeneration (CARE(R), full): the rubric is regenerated at every turn from the current dialogue state.

The results on SentientBench with DeepSeek-V4-Pro as the judge are presented below; they are added to Table 3 and discussed in §4.3:

Reward / rubric variantScore ↑Succ. (%) ↑Fail (%) ↓
Qwen2.5-7B-Instruct9.143.3387.67
Fixed from persona11.750.0074.00
Fixed from first-turn query10.560.0080.00
Shuffled weights50.204.0032.00
CARE(R), per-turn (full)91.4172.332.00
  • Turn-wise adaptation accounts for most of the gain. Both fixed variants achieve no successful dialogue, although they use the same learned generator and are conditioned on information about the user. Even the complete persona cannot substitute for updating the rubric as the dialogue unfolds.
  • The weights must match the context. Permuting the weights reduces the score from 91.41 to 50.20, which is below the Uniform dimension weights ablation of Table 3 (64.20).
  • A stale rubric is worse than a neutral one. Both fixed variants are below the Uniform dimension weights (64.20), Uniform rubric criteria (41.62), and scalar reward model (22.92) ablations of Table 3, and close to the untrained backbone (9.14; Table 1). Since the shuffled variant, trained with the identical pipeline, reaches 50.20, we attribute this to the reward rather than to a training failure: a rubric tailored to the opening state keeps rewarding priorities that are no longer appropriate (e.g., continued emotional validation when the user has moved on to understanding the situation), which is consistent with the shifts shown in Figure 3.

Overall, these results demonstrate that turn-wise adaptation accounts for most of CARE's performance gains.

Revisions: The fixed-rubric and shuffled-weight controls are added to Table 3 (L343–353) and discussed in Section 4.3 (L406–411).

AuthorsOpenReview ↗

(2/4). Rebuttal by Authors — Reviewer jHsF

[Q2] (Weakness 4) How do the authors ensure that the rubric generator learns actual user support needs rather than reproducing support strategies already expressed in preferred model responses?

[A2] We conducted the evaluation suggested in Weakness 4 (Appendix H, "Response-blind labels of support needs"). For 80 held-out dialogues, six annotators independently labeled which empathy dimension (cognitive, affective, or proactive) the seeker most needs at the current turn, without seeing any candidate response. The inter-annotator agreement is moderate (Fleiss' κ = 0.53, 95% CI 0.43–0.62), so we evaluate on the 57 dialogues where at least five of the six annotators agree, excluding the remaining 23 due to insufficient inter-annotator agreement. The prediction of the generator is its highest-weighted dimension (a tie counts as predicting both).

Agreement with human consensus
Uniform chance33.3%
Majority-class baseline42.1%
CARE rubric generator71.9% (41/57; 95% CI 59.2–81.9%)

The lower bound of the interval is well above both baselines. On the 28 dialogues with unanimous labels, the agreement is 60.7% (17/28). These results demonstrate that CARE's learned priorities align with independently assessed user needs.

Revisions: The response-blind need-tracking study is provided in Appendix H (L974–980).


[Q3] Why are the EMPA improvements so much larger than those on EQBench3? Could this be due to differences in benchmark sensitivity, simulator behavior or evaluator alignment?

[A3] The difference mainly reflects benchmark characteristics. EMPA evaluates dynamic multi-turn interactions with simulated users through Outcome, Efficiency, and Stability, closely matching CARE's target setting. EQBench3 uses scripted role-play and dialogue analysis to assess broader EQ abilities. While the improvement margins vary across judges, CARE consistently ranks first on every metric under all three judges, and CARE(R) outperforms all non-CARE methods in pairwise comparisons. The consistent gains on EQBench3 further demonstrate that CARE's benefits extend beyond interactive emotional support.

AuthorsOpenReview ↗

(3/4). Rebuttal by Authors — Reviewer jHsF

[Q4] (Weakness 3) Can the authors report human evaluations of complete dialogue trajectories including inter-annotator agreement and uncertainty estimates rather than only final-turn response preferences?

[A4] We conducted a blind human evaluation over complete dialogue trajectories (Appendix H, "Human evaluation of complete trajectories"). Six annotators independently judged 30 dialogue scripts; for each script, they were shown the full trajectories produced by CARE(R), RLVER, and MICA in randomized, anonymized order, and selected the best one (180 judgments in total).

MethodPreference
MICA4.4%
RLVER10.6%
CARE(R)85.0% (153/180)
  • Uncertainty. The 95% CI of the CARE(R) preference is 80.0–90.0% under a script-clustered bootstrap, and 73.3–95.0% when resampling both scripts and annotators (chance = 33.3%).
  • Script level. Under majority vote, CARE(R) wins 29 of the 30 scripts (Wilson 95% CI 83.3–99.4%), with one tie. It receives more votes than RLVER on 29 scripts and fewer on none (two-sided sign test, p < 10⁻⁸).
  • Agreement. The raw pairwise agreement is 73.6%, and Gwet's AC1 is 0.70 (95% CI 0.58–0.80).

Revisions: The human evaluation of complete trajectories is provided in Appendix H (L982–986).


[Q4, cont.] (Weakness 3) So, I would request per-instance performance distributions, paired confidence intervals, qualitative failure cases and human validation of a subset of complete multi-turn conversations.

[A4, cont.] The human validation is reported above and the failure cases in [A7]. For each benchmark instance, we average three runs per method and compute the difference in the per-instance score between CARE and its base method (positive values favor CARE). The 95% CIs are computed by paired bootstrap over all benchmark instances (Appendix H, Table 9). EQBench3 differences are on the per-instance 0–20 scale, i.e., one fifth of the 0–100 Overall scale of Table 1.

Mean difference [95% CI]:

JudgeComparisonSentientBenchEMPAEQBench3
DeepSeek-V4-ProCARE(R) − RLVER44.88 [38.93, 50.79]69.71 [65.02, 73.70]2.32 [1.78, 2.83]
DeepSeek-V4-ProCARE(M) − MICA8.42 [3.46, 13.59]42.74 [34.81, 50.74]1.25 [0.71, 1.79]
Gemini-2.5-ProCARE(R) − RLVER45.73 [39.83, 51.60]68.89 [63.51, 74.16]3.12 [2.20, 4.03]
Gemini-2.5-ProCARE(M) − MICA14.22 [8.61, 20.18]53.09 [45.18, 60.58]1.38 [0.71, 2.07]
GPT-5.5CARE(R) − RLVER31.53 [28.28, 34.88]19.25 [13.26, 25.62]1.13 [0.81, 1.43]
GPT-5.5CARE(M) − MICA12.33 [9.68, 15.08]11.28 [6.41, 16.33]0.50 [0.24, 0.74]

Median difference [Q1, Q3]:

JudgeComparisonSentientBenchEMPAEQBench3
DeepSeek-V4-ProCARE(R) − RLVER42.67 [19.33, 70.00]73.95 [65.61, 78.17]2.54 [1.13, 3.68]
DeepSeek-V4-ProCARE(M) − MICA0.00 [−0.50, 20.00]40.74 [23.03, 61.35]1.39 [−0.06, 2.28]
Gemini-2.5-ProCARE(R) − RLVER44.17 [17.83, 72.83]71.34 [57.94, 81.60]3.22 [0.67, 5.33]
Gemini-2.5-ProCARE(M) − MICA0.00 [0.00, 23.17]58.22 [37.28, 68.51]1.22 [0.06, 2.44]
GPT-5.5CARE(R) − RLVER32.00 [17.75, 42.33]13.89 [5.93, 32.03]1.22 [0.50, 2.00]
GPT-5.5CARE(M) − MICA10.17 [1.67, 23.17]8.28 [2.18, 21.48]0.50 [0.06, 1.11]
  • All 18 mean-difference CIs exclude zero, for both CARE variants on every benchmark and under every judge.
  • The gain of CARE(R) is broad-based: its first quartile is positive in all nine settings, i.e., at least three quarters of the instances improve.
  • The gain of CARE(M) on SentientBench is concentrated: under DeepSeek-V4-Pro and Gemini-2.5-Pro the median difference is 0.00, i.e., CARE(M) and MICA obtain the same final score on at least half of the instances. On EMPA, its gain is broad-based (first quartile above zero under all judges).

Revisions: Per-instance differences and paired CIs are provided in Appendix H (L988–991; Table 9, L1013–1025).

Reviewer_K4oWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

CARE trains a context-conditioned rubric generator via SFT and GRPO preference RL, then uses it as a frozen adaptive reward in RLVER and MICA. The paper reports state-of-the-art results on SentientBench, EQBench3, and EMPA under three judges.

Critical strengths
  1. CARE(R)/CARE(M) rank first and second on every aggregate metric under all three judges (Table 1), e.g. EMPA EPM-Idx 95.77 vs 52.10 and 83.54 vs 28.11 over MICA (rep.), corroborated by a blinded human study (51.5% vs 23.9%/24.6%, Table 8) and component ablations (Table 3).
  2. A reusable, fully documented recipe — distilled turn-level rubric supervision, a priority-alignment reward, and a frozen generator that plugs into existing pipelines "without changing their rollout or optimization procedures" (§1, p.2) — with gains persisting from Qwen3-8B to 32B (Table 2) and no degradation on code/math/instruction (Table 5).
Critical weaknesses
  1. Judge DeepSeek-V4-Pro (§4.1, p.6) is the same model that generated the SFT rubric supervision ("via DeepSeek-v4-pro", AI Use Statement, p.10; Appendix I, p.19), and the reward evaluator DeepSeek-V4-Flash is same-family, yet no judge-exclusion analysis is reported. The paper's own gains are smallest under GPT-5.5 (2.55/13.02 vs 6.06–7.67/43.67–55.43 elsewhere, §F, pp.16–17).
  2. The central claim "CARE adapts what is rewarded as support needs evolve" (Abstract, p.1) is not directly measured: the sole held-out check tests preference recovery (59.4%, Table 4), not need-tracking, and the SFT-only generator stays "almost uniformly cognitive-dominant" (Appendix D, p.15). Rubrics varying across turns/emotions (Figs. 3–4) does not establish that rewards track evolving support needs.
Questions for authors

See Critical Weaknesses

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors — Reviewer K4oW

We thank the reviewer for highlighting CARE's consistent top performance, as well as its effective rubric-learning framework. We address your questions and provide additional results below.

[Q1] Judge DeepSeek-V4-Pro is the same model that generated the SFT rubric supervision, and the reward evaluator DeepSeek-V4-Flash is same-family.

[A1] We took this into account when designing the evaluation. Besides DeepSeek-V4-Pro, the original submission already reports results under two other state-of-the-art judges from different model families, Gemini-2.5-Pro and GPT-5.5 (Table 1, §F), and CARE ranks first under every judge:

Judge (non-DeepSeek)EQBench3: CARE(R) − MICAEMPA EPM-Idx: CARE(R) − MICA
Gemini-2.5-Pro+7.67+55.43
GPT-5.5+2.55+13.02

Human annotators agree with this conclusion. In the blinded human study of Table 8 (15 annotators with a psychology background), CARE(R) received 51.5% of the preference votes, versus 23.9% for MICA. In a new blind human evaluation over complete dialogue trajectories, CARE(R) was preferred in 85.0% of the judgments (Appendix H). The gains are therefore not an artifact of same-family judging.

Revisions: The human evaluation of complete trajectories is provided in Appendix H (L982–986).


[Q2] The paper's own gains are smallest under GPT-5.5.

[A2] CARE ranks first on every metric of all three benchmarks under every judge. Since each judge scores on a different scale, absolute margins are not comparable across judges. Human evaluators agree with the ranking: CARE(R) is the most preferred method both for final-turn responses in Table 8 and for complete dialogue trajectories in Appendix H, with preference rates of 51.5% and 85.0%, respectively. The improvement of CARE is therefore consistent across judges, benchmarks, and human evaluation.

Revisions: The human evaluation of complete trajectories is provided in Appendix H (L982–986).


[Q3] The paper does not directly demonstrate that CARE's reward priorities track users' evolving support needs.

[A3] We measure need-tracking directly with a new human study (Appendix H, "Response-blind labels of support needs"). For 80 held-out dialogues, six annotators independently labeled which empathy dimension (cognitive, affective, or proactive) the seeker most needs at the current turn, without seeing any candidate response. The inter-annotator agreement is moderate (Fleiss' κ = 0.53, 95% CI 0.43–0.62), so we evaluate on the 57 dialogues where at least five of the six annotators agree, excluding the remaining 23 due to insufficient inter-annotator agreement. The prediction of the generator is its highest-weighted dimension (a tie counts as predicting both).

Agreement with human consensus
Uniform chance33.3%
Majority-class baseline42.1%
CARE rubric generator71.9% (41/57; 95% CI 59.2–81.9%)

The lower bound of the interval is well above both baselines. On the 28 dialogues with unanimous labels, the agreement is 60.7% (17/28). These results show that CARE's learned priorities are highly aligned with independently judged user needs.

We further evaluate the importance of context-dependent adaptation on SentientBench (§4.3, Table 3). Shuffling turn-wise weights across contexts reduces the score from 91.41 to 50.20, while fixing the rubric throughout a dialogue reduces it to 11.75 / 10.56. These results demonstrate that CARE's gains rely on aligning reward priorities with the current dialogue state.

Revisions: The response-blind need-tracking study is provided in Appendix H (L974–980). The fixed-rubric and shuffled-weight controls are added to Table 3 (L343–353) and discussed in Section 4.3 (L406–411).

I appreciate the authors’ efforts to address my concerns, and I find the additional evidence helpful in resolving them. I am inclined to maintain my current positive rating.

We sincerely thank the reviewer for the constructive comments and for recognizing our efforts. We are glad that the additional evidence helped resolve the concerns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Rebuttal by Authors

Dear Program Chairs, Senior Area Chairs, Area Chairs, and Reviewers,

We thank all reviewers for their thoughtful comments and constructive feedback.

In this paper, we propose Context-Adaptive Rubric Evolution (CARE), which enables the reward specification in empathetic reinforcement learning to evolve with the dialogue state. The contributions of this paper are as follows:

  1. A new problem formulation: We identify the mismatch between evolving support priorities and fixed reward specifications in existing empathetic RL methods, and formulate evolving support priorities along three dimensions: cognitive, affective, and proactive empathy.
  2. An adaptive reward framework: CARE trains a rubric generator that dynamically adjusts both the weights of the three empathy dimensions and their fine-grained evaluation criteria at each turn. The frozen generator can be integrated into existing pipelines (RLVER and MICA) without modifying their rollout or optimization procedures.
  3. Consistent improvements: The two CARE variants rank first and second across all aggregate metrics on SentientBench, EQBench3, and EMPA under three LLM judges. For example, CARE improves EMPA EPM-Idx from 28.11 to 83.54 under Gemini-2.5-Pro.

Following the reviewers' suggestions, we conducted the following additional experiments during the rebuttal period:

  1. Fixed-rubric and shuffled-weight controls: The SentientBench score drops from 91.41 to 11.75 / 10.56 with fixed rubrics and 50.20 with shuffled weights (§4.3, Table 3).
  2. Response-blind human labeling of support needs: CARE achieves 71.9% agreement with human consensus, compared with 42.1% for the majority-class baseline (Appendix H).
  3. Blind human evaluation of complete trajectories: CARE(R) is preferred in 85.0% of 180 judgments (Appendix H).
  4. Dynamic rubric-learning baseline: EvoRubrics achieves 26.35 on SentientBench, compared with 91.41 for CARE(R) (Appendix H).
  5. Paired confidence intervals: All 18 paired 95% confidence intervals for CARE–baseline mean differences exclude zero (Appendix H, Table 9).
  6. Compute-cost and failure-case analyses (Appendix H).

We have incorporated these additional results and clarifications into the revised paper, primarily in Appendix H. We hope these additions address the reviewers' concerns.

Best regards,

Authors of Paper #491

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.