- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces a framework that combines reasoning over signals from wearables and social texts [sic!] in the context of mental wellbeing assessments, specifically for stress evaluations. The authors combine modality specific representations (extracted using state of the art foundation models) and use these to construct a structured reasoning apprach that follows an observation-reasoning-evaluation direction. Essentially, this reasoning backend is a graph-based method that not only provides numerical assessments (of mental stress in this case) but also, as claimed by the authors, provides an “evidence-based” auditing trail. The authors claim that such an interpretable approach is more valuable for practical applications of computational psychiatry, which is the field this paper targets. An experimental evaluation on a range of five existing datasets is used to demonstrate the effectiveness of the presented approach, albeit seemingly limited to quantitative results mainly. In their discussion the authors outline next steps in terms of plans to design a mobile app that would be used for actual deployments rather than evaluations on existing datasets.
Critical strengths
This paper operates in an interesting and relevant area. Mental stress is a growing concern worldwide and the analysis of a combination of both signals from wearables, which would be used to capture measurable physiological indicators that might be correlated with stress, and social signals that directly link to mental wellbeing and stress is an interesting and promising idea. I applaud the authors for their system-level approach that combines established methods for uni-modal representation extraction and multi-modal integration into a reasoning framework thatm, as per the description provided by the authors, not only outputs individual stress scores but also aims for their explanations through auditable traces, which would support practitioners and patients alike.
Critical weaknesses
While I appreciate the system focus of this paper (the individual components for both processing the signals of various modalities as well as their integrations are fairly standard) because it has great potential for substantial impact, the paper does not fully deliver what it promises. I fully agree with the authors that automated stress assessments should not be reduced to some seemingly arbitrary numbers but rather should provide explanations for these — given the significance of such assessments and their outcomes. Yet, the paper does not actually dive into these critical insights. Table 3 seems to be the only indication for results beyond quantitative scores, yet no actual details are given.
I am also concerned about the impact this paper could have in its current form. Again, the work targets system level, which is great. Yet, the authors only evaluate it (again, mainly at the quantitative level that they wanted to go beyond) on existing datasets. The authors hint at an app and an actual deployment but this is left for future work. I would argue that such an app and an actual deployment is mandatory for this paper such that a reader can learn about the actual practical impact this work could have.
The presentation of the quantitative results has some issues: Why are the data points in figure 4 connected? This is just plain wrong because it suggests some sort of connection that simply is not there. Also: The baselines seem to weak and thus result in an unfair comparison. Why would one expect those models out of the box function well? I would argue that they would need to be finetuned etc. to make this a meaningful comparison. Without it, the gap between SOTA and the proposed framework seems inflated.
Questions for authors
- Why are there no results beyond mere quantitative assessments?
- Why are the data points in figure 4 connected?
- Why did you use SOTA baselines as is, rather than adapt them to the domain?