MentaBench: Benchmarking Mental Health Signal Detection from Text with Small Language Models

Text-based mental health signals can enable just-in-time adaptive interventions (JITAIs), but practical deployment requires models that are accurate, efficient, low-latency, and privacy-preserving. While large language models (LLMs) perform well on mental health prediction tasks, their computational cost and reliance on centralized inference limit their applicability in sensitive or resource-constrained settings. Small language models (SLMs) offer a promising alternative; however, their reliability for mental health signal detection remains underexplored, with existing evaluations fragmented across datasets, tasks, and adaptation settings. We introduce **MentaBench**, a unified benchmark for evaluating SLMs on text-based mental health signal detection. We curated a large-scale mental health corpus which comprises 21 public datasets spanning 40 tasks and diverse text sources, and evaluates 11 representative SLMs under zero-shot, few-shot, and fine-tuning regimes. We benchmarking results show that fine-tuned SLMs on mental health datasets can outperform traditional machine learning and encoder-based baselines as well as LLMs. Evaluating on both predictive performance and deployment-related metrics, we further find that Gemma4-E4B achieves the strongest predictive performance, while LLaMA3.2-1B and LLaMA3.2-3B reach a more balanced trade-off for real-world deployment scenarios. **MentaBench** provides a standardized foundation for assessing the generalization, adaptability, and deployability of efficient language models for mental health monitoring and future JITAI systems. Code is available at: **[MentaBench Code](https://anonymous.4open.science/r/MentaBench_Code-65F9)**.