acceptodds
Withdrawn from ICLR 2027

OCTO-CARES: Towards Human-like Emotional Support Conversation with Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As large language models (LLMs) become increasingly integrated into everyday life, they are also being used for daily emotional support. While LLMs provide reassurance and guidance through empathetic language and structured support strategies, some users may perceive these responses as formulaic and impersonal. These users may seek the sense of connection that comes from shared understanding and perspectives shaped by lived experience and interpersonal relationships, qualities that current LLMs can struggle to convey. Therefore, in this work, we aim to improve the naturalness and interpersonal resonance of LLM-generated emotional support by matching users’ support needs with relevant human peer-support exemplars. Specifically, we propose Octo-Classifier Attention-Weighted Retrieval for Empathetic Support (Octo-CARES), which combines a classifier that identifies users' support needs across eight help-seeking patterns with an attention-weighted retrieval mechanism that selects relevant human peer-support exchanges. We construct Octo-Pairs, a dataset of 2,653 naturally occurring Reddit post–comment pairs annotated with these patterns. Each pair links a concrete help-seeking situation to a highly upvoted peer response. The retrieved exchanges are incorporated into the LLM's context as exemplars of understanding and experiential insight to guide response generation. Experiments on both in-domain and out-of-domain datasets demonstrate that Octo-CARES helps LLMs generate more supportive and human-like responses. Our code is currently available via an anonymous link./r/OCTO-CARES-Towards-Human-like-Emotional-Support-Conversation-with-Large-Language-Models-417C/

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Language Model, Emotional Support, Retrieval-Augmented Generation, Human Touch

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_SUi8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how to make emotional support replies from large language models (LLMs) feel more human and less like fixed templates. The authors introduce Octo-Pair, a dataset of 2,653 Reddit post-comment pairs with eight labels. They also propose Octo-CARES, which uses attention from eight classifiers to find similar posts and their paired human replies. The LLM then uses these examples to generate emotional support replies. The results show better retrieval scores on several measures and higher win rates on most response measures under LLM-based evaluation.

Critical strengths
  • The paper studies an important problem. A reply can sound kind but still fail to make a user feel understood.
  • Using real human replies as examples is a useful idea. It gives the model both a similar situation and an example of how a person responds.
Critical weaknesses
  • The main figure is hard to follow. Figure 1 has many arrows. It is hard to see how classifier training, attention weights, retrieval, and response generation connect. The authors should redraw the figure and show the training and test steps in separate parts.
  • The training labels come from one LLM, while the test labels come from a vote among three LLMs. The paper does not report human checks or agreement with human labels. The retrieval scores may show agreement with LLM labels rather than a true match with users' support needs.
Questions for authors
  • Could human annotators label a small set of posts? How well do their labels match the labels from the three LLMs?

  • What does attention weighting add beyond normal semantic similarity? Could the authors show a few examples where it finds a better support example?

  • Could the authors compare matched comments with random or wrong comments while keeping the prompt length similar? This would help show whether the gains come from finding the right human reply.

  • Could the authors test the method on the original ESCoT dialogues without turning them into long Reddit-style posts? Could human raters also judge the quality of the replies?

Reviewer_rpBHOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper aims to address the challenge of LLMs providing emotional support. To support this aim, the authors introduce Octo-Pair, which is a dataset of 2,653 Reddit post-comment pairs with LLM generated labels for 8 different dimensions that characterize emotional support-seeking and disclosure behaviors. The paper then develops Octo-CARES, which uses information from classifiers for the 8 dimensions to retrieve relevant examples of human emotional support to aid in LLM response generation. The authors evaluate the retrieval pipeline by comparing to LLM generated labels, and they evaluate the resulting generations of Octo-CARES through pairwise comparisons via unvalidated LLM judges.

Critical strengths

The core idea of improving emotional support generations by retrieving examples of human exchanges that request similar support is very compelling, and I would like to see work that investigates this more rigorously. Additionally, the 8 dimensions proposed for understanding emotional support-seeking could be quite useful.

Critical weaknesses
  1. There is a lack of methodological detail, especially with respect to the dataset. The authors claim the Octo-Pair dataset as a contribution, but do not provide key details such as how were the reddit posts collected, which subreddits were they selected from, what were the inclusion / exclusion criteria, what post processing is done on the reddit posts (e.g., were they anonymized), and is the dataset being publicly released?

  2. There is no human validation for any of the labels or evaluations. All labels and evaluations are generated via unvalidated LLM judges. This is an important weakness for a few reasons:

2.1. The concepts that this work generates labels are not exceptionally clear / factual judgments. They are judgments about the intentions of posters with respect to emotional support. It is not obvious that LLMs (or even human annotators) would be able to accurately measure these concepts. To address this, this work needs to collect multiple human judgments for a reasonable subset of the dataset and demonstrate that there is substantial agreement among humans and between humans and LLMs.

2.2. All evaluations are done via LLM judges without human validation. Again, the concepts that are being measured (e.g., human-likeness, diversity, informativeness) are quite subjective and it is not obvious that the LLM judge setup used would be able to measure these constructs. Again, to evaluate this, the authors should include measures of agreement among humans and between humans and LLMs for all evaluated concepts.

2.3. Because of the lack of validation, the key insights that this paper provides are incredibly limited. It cannot make clear claims about the efficacy of its method, and its dataset’s utility is mostly just a collection of reddit posts.

Questions for authors

What is the justification for the seven dimensions evaluated? Where did they come from, and what reasoning is used to justify that more human-like or more diverse responses are better?

Reviewer_g7cmOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the formulaic, impersonal nature of LLM-generated emotional support responses that often fail to foster interpersonal connection. It presents Octo-CARES, a retrieval-augmented framework leveraging human peer-support exemplars: the authors build Octo-Pair, a dataset of 2,653 annotated Reddit post-comment pairs with eight labels covering self-disclosure and support-seeking patterns, and design Octo-Vector to match both semantics and support intent. Experiments show Octo-Vector outperforms existing Emotional RAG baselines on retrieval metrics, and the full framework achieves higher win rates than post-only, persona-based and multi-agent baselines across seven quality dimensions.

Critical strengths
  1. The paper is well motivated. The problem of formulaic and impersonal response generation is indeed a core limitation of current LLM-based emotional support systems, lending clear and solid relevance to this work.

  2. The Octo-Pair dataset constructed in this work holds practical value for the research community focused on emotional support conversations.

Critical weaknesses
  1. While the authors motivate their work by incorporating experiential perspectives from human peer responses, the paper provides no discussion of whether this design rationale is supported by relevant psychological theories. There is a clear lack of theoretical grounding from psychology literature to justify the core idea of leveraging peer-support exemplars for emotional support generation.

  2. The proposed Octo-CARES framework is largely a straightforward application of retrieval-augmented generation (RAG) to the emotional support conversation (ESC) task, with fairly limited methodological novelty.

  3. The experimental evaluation suffers from several notable limitations. First, the majority of comparisons are conducted against ablation variants of the authors’ own method, and both the RAG and ESC baselines adopted in the study are relatively dated. Second, critical implementation details remain undisclosed throughout the paper. These include training hyperparameters, complete evaluation protocols, and most importantly the prompt specification used for the human-likeness metric. This lack of detail significantly undermines the reproducibility of the reported experimental results.

  4. For ESC tasks, adaptive support strategies are essential in sustained multi-turn conversations. However, the proposed method relies exclusively on single-turn Octo-Pair retrieval and lacks any consideration of strategy adjustment or selection over long dialogues. While existing LLMs are often criticized for rigid, formulaic support patterns, the complete absence of explicit strategy modeling in this framework is also a meaningful shortcoming.

  5. No qualitative case studies or illustrative examples are included to demonstrate the framework’s behavior in concrete scenarios.

  6. The presentation of the paper could be improved. In particular, Figure 1 is never explicitly referenced in the main text, and the work would benefit from a dedicated section giving a high-level overview of the proposed approach.

Questions for authors

Please refer to the Critical Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.