PSYCHO-R1: Advancing Proactive Guidance for Long-Horizon Psychological Counseling

LLM-based counselors can generate empathetic and professionally grounded responses, yet they struggle to proactively guide therapeutic progress across multi-turn interactions—a capability essential to effective counseling. This limitation stems in part from insufficient modeling of dynamic decision-making under information asymmetry, which requires state inference of clients to guide proactive inquiry and intervention. It is further compounded by trajectory-level advantages during optimization, which provide coarse supervision and fail to distinguish the contributions of individual counselor turns to subsequent conversational progress. To this end, we introduce PSYCHO-R1, a proactive counselor agent trained via a GRPO variant in an agentic interactive framework with three components: 1) an information-asymmetric multi-agent environment with a client simulator conditioned on a private profile and an evaluator providing turn-level rewards; 2) hierarchical decision-making linking state inference to proactive guidance; and 3) critic-free credit assignment using local lookahead returns to compute turn-level group-relative advantages. Experimental results demonstrate that PSYCHO-R1 achieves the best performance across all 11 dimensions spanning counselor-level, client-level, and therapeutic alliance metrics, outperforming both specialized mental-health models and substantially larger general-purpose LLMs.