acceptodds
Under review as a conference paper at ICLR 2027

PSYEVO: A PERSONALIZED COUNSELING AGENT THAT SELF-EVOLVES AT TEST TIME

Abstract

Mental health disorders affect a substantial proportion of the global population, yet a persistent shortage of trained practitioners leaves the majority without adequate care. Large language model (LLM)-based counselors present a promising direction for delivering scalable conversational psychological support. Offline model training alone leaves limited room to adapt to individual clients or to learn from ongoing therapeutic interaction at test time. We introduce PsyEvo, an LLM-based counseling framework that enables both client-specific personalization and response-policy improvement at test time through three components: Hierarchical Bayesian Skill Policy (HBSP) personalizes what intervention to apply by maintaining a per-client skill posterior updated from session feedback; Inter-session Listwise Preference Optimization (LiPO) improves how the selected skill is expressed by updating a shared response adapter from cross-client preference evidence; and State-conditioned Ordinal Credit Assignment (SOCA) supplies candidate preferences and trajectory credit to the two components through consistency-checked comparisons and ordinal projection. In simulated-client evaluation with shared online cohort adaptation, PsyEvo obtains 7.684 Overall on PsychEval and exceeds every component variant in each of three matched runs. Removing individual components lowers mean overall score by 0.138–0.171 under the shared configuration, supporting conditional contributions within the complete scaffold.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
LLM Counseling, Test-Time Adaptation, Client Personalization, Ordinal Credit Assignment, Preference Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7uAaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PsyEvo adapts a counseling agent between sessions through two learning paths: a private skill posterior for each client and a response adapter shared across clients. SOCA supplies candidate preferences and trajectory credit to these updates. On a simulated 100-client PsychEval cohort, the full system scores 7.684 and exceeds each component variant in three matched runs. The paper also reports a small expert-rating study and a separate summary-ranking experiment. The main evaluation measures adaptation within the interacting cohort; it does not test clinical benefit or transfer to a new cohort.

Critical strengths
  1. The separation between private and shared learning is clearly designed. Client memories and skill posteriors remain separate, while preference evidence updates a shared adapter. Fixing the adapter within a session makes the update process easier to track. The paper also distinguishes internal learning feedback from the external evaluation scores (Appendix G).
  2. The matched ablations provide useful evidence. Removing a component or replacing the grouped objective with pairwise DPO lowers the overall score by .138–.171, and the full system wins each comparison in all three runs (Table 3; Appendix E.1). The authors also explain that the client-bootstrap intervals are conditional on those runs. This is more convincing than relying on Table 1's mix of published and newly evaluated baselines.
Critical weaknesses
  1. The experiments do not yet show that matching adaptation to a particular client is what helps. Removing HBSP removes residual learning altogether. A pooled learner or shuffled-client posterior would be a better control for personalization. Different clients receiving different skill rankings is not enough to establish that they receive the right rankings. There is also no fully frozen control that retains memory, candidate generation, and response selection while disabling both learning paths (Appendix C, lines 777–788). Because the adapter changes across the cohort, performance may depend on client order and update timing; three runs give only a limited view of that variation.
  2. The evidence for SOCA's credit assignment is incomplete. Its counseling ablation changes both candidate preferences and trajectory credit, so it does not isolate ordinal expansion. Expanding small differences in noisy scores could help or hurt, and the paper needs a direct test. SummEval evaluates a different fusion rule, not the counseling graph selector or trajectory credit (Appendix K.2). The human study is encouraging, but agreement is low on several dimensions and the strongest general-purpose baseline is absent. The archived adapter trace and MusPsy results also have unresolved links to the main configuration (Appendices F.2 and K.1), so they cannot resolve these questions.
Questions for authors
  1. How does the full system compare with a control that freezes both HBSP updates and adapter updates while retaining the complete memory, candidate-generation, and response-selection scaffold? Please include comparable inference cost.
  2. Does client-specific adaptation outperform a pooled residual learner and a client-shuffled posterior control under the same update budget? This would directly test whether personalization, rather than additional learning alone, improves outcomes.
  3. Can you separate the effects of candidate preference construction, ordinal projection, and discounted trajectory propagation? How sensitive are the results to the ordinal spread parameter, particularly when raw feedback scores are close or noisy?
  4. Can you provide adapter-version/update logs linked to each main run, plus robustness to different client orders or update schedules? For the expert pilot, paired case-level differences and per-rater system rankings would help establish whether the preference persists despite low absolute agreement.
Reviewer_8cz4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

i apologize in advance to the authors for my lack of experience and expertise in this particular topic of machine learning for mental health. this lack of domain expertise makes it challenging for me to make a holistic assessment of this submission.

in this paper, the authors propose an online adaptation algorithm for a multi-session psychiatric therapy, in which the system adapts its memory, skill set and understanding of the client. this algorithm builds an adapter for a well-trained base model (e.g. Qwen3-32B in this particular work).

compared to earlier studies on the similar topic, the proposed approach works significantly better. though, this improvement looks largely due to the use of a better base model (Qwen3-32B), as the base model already outperforms most of the existing psychiatry-specific baselines. This may be an artifact of LLM-based automated judgement, but unfortunately the authors did not compare these general-purpose models (including Qwen3-32B) in their human evaluation.

considering that there are quite a lot of hyperparameters that must be carefully chosen and that the proposed approach (PsyEvo) does realy on parameter tuning (adapters), I would appreciate much more how a simpler adapter tuned using DPO on top of Qwen3-32B as well as other general-purpose open-weight models did. after all, we would prefer having an approach that is generic and simple.

i do apologize again for this short review, and i suggest reviewers do the following two items, after which i will reconsider my recommendation. (1) please run human eval on Qwen3-32B as well as other general-purpose open-weight models that are quite close to the proposed approach according to the automated scoring method. (2) please consider a much simpler variant of adapter training (without skill slots, etc.) as comparison (there are some ablation studies but i believe it can be stripped down further.)

Critical strengths

i believe this area is where AI can have a positive impact when done well, and thereby i appreciate the authors’ efforts.

Critical weaknesses

see my answer at the top.

Questions for authors

see my answer at the top.

Reviewer_do34OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PsyEvo combines client-specific skill learning with a shared response adapter over a frozen backbone. HBSP selects skills, LiPO updates the adapter, and SOCA constructs candidate preferences and trajectory credit. The main evaluation uses 100 simulated PsychEval clients sharing online adapter updates, with three matched runs and component ablations. The paper also reports a small human-rating pilot and a separate SOCA fusion variant on SummEval.

Critical strengths
  1. Client-specific skill learning, shared response updates, and external scoring are kept separate. This makes the adaptation process easier to inspect.
  2. Full PsyEvo outperforms every component variant in all three matched runs, with Overall gains of 0.138–0.171. The repeated gains support the components' contribution in this setting.
Critical weaknesses
  1. SOCA can amplify small score differences and alter HBSP's reward scale. Appendix C uses only the latest value from each projected session-reward history (p. 14): raw rewards [0.5, 0.5001, 0.5002] then give update rewards [0.5, 1, 1]. Eq. 8 also spreads near-tied turn scores across a large credit range. Each projected vector preserves its mean, but the rewards used online need not. The paper does not examine whether the gains are robust to these changes in scale and ranking.
Questions for authors
  1. When the reward history is reprojected, are previous HBSP updates recomputed? If not, how is the resulting change in the online reward scale handled?
  2. How stable are turn and session rankings across repeated scoring? Do the gains persist with near-tie handling and a matched online reward scale?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.