acceptodds
Under review as a conference paper at ICLR 2027

Latte: Towards Agentic Emotional Support Systems

Abstract

Emotional-support agents are increasingly capable of understanding users and generating helpful responses, yet some support needs arise from practical circumstances that language alone cannot address. We study Agentic Emotional Support (AES), which extends emotional support beyond response generation by enabling agents to understand users' circumstances and take grounded, appropriate actions when helpful. We introduce LATTE, a benchmark of 90 open-ended scenarios in reproducible multi-application Android environments. Each scenario begins with an affective, operationally underspecified utterance and specifies no oracle action or unique terminal state. We evaluate 11 models under Emotional Support Response, Task-oriented Action, and AES using trajectory-level evaluation of Overall Support and Action Quality, together with Grounding and Consent diagnostics. Results show that AES generally improves support over both response-only and task-oriented baselines. More importantly, its benefits depend far more on decision and execution quality than on action frequency: acting more is not inherently better. A bounded Decision-Correction procedure further improves AES for some model families, but not uniformly across backbones. Together, these findings advance emotional-support agents from supportive response toward appropriate action, showing that the key is not merely to act, but to act well.

Primary area:
datasets and benchmarks
Keywords:
Agentic Emotional Support, GUI Agents, Mobile Agents, Agent Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_ghuxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LATTE, a benchmark for Agentic Emotional Support (AES), which extends traditional emotional-support dialogue toward grounded actions in interactive mobile environments.

Unlike conventional task-oriented benchmarks, LATTE begins with emotionally expressed but operationally underspecified user needs. Agents must determine whether to act, ask for clarification, provide suggestions, or refrain from acting. The benchmark contains 90 scenarios constructed from 50 environment recipes across seven Android applications, without prescribing a unique correct action sequence or terminal state.

The authors evaluate 11 models under three settings: Emotional Support Response (ESR), Task-oriented Action (TA), and Agentic Emotional Support (AES). Evaluation covers Overall Support (Empathy, Helpfulness, and Agency), Action Quality (Decision and Success), and process diagnostics for Grounding and Consent.

The results suggest that AES generally improves support compared with response-only and task-oriented approaches. The authors further report that improvements are more strongly associated with successful actions than with action-attempt frequency. Finally, they introduce a bounded Decision–Correction procedure that improves performance for several models, although its effectiveness varies across model families.

Critical strengths
  1. An important and meaningful research problem with a useful benchmark formulation.

The paper identifies an important limitation of existing emotional-support agents: providing empathetic language is not necessarily sufficient when a user's distress arises from practical circumstances.

The distinction between "What should the agent say?" and "What, if anything, should the agent do?" is meaningful.

I particularly appreciate that LATTE does not prescribe a unique correct action or terminal state. The agent must infer what kind of assistance is appropriate, and asking, acting, or deliberately refraining may all be valid choices.

This formulation is closer to realistic assistance than conventional benchmarks with explicit operational goals.

  1. A carefully constructed interactive evaluation framework with meaningful human validation.

The benchmark integrates reproducible Android environments, hidden user circumstances, a user simulator, consent monitoring, grounding audits, and trajectory-level evaluation.

The authors also validate the evaluation components using independent human annotations and report substantial agreement between human assessments and automated judgments.

These efforts strengthen the credibility and reproducibility of the benchmark and represent a meaningful contribution.

Critical weaknesses
  1. The experimental comparisons do not sufficiently isolate the effects of action availability, emotional-support objectives, and actual intervention.

One of the paper's main findings is that AES improves support over response-only emotional support. However, the AES–ESR comparison changes multiple factors simultaneously.

AES has access to the mobile environment and can inspect private evidence and execute actions, whereas ESR has no GUI access. Their system objectives also differ.

This distinction is particularly important because the Overall Support metric includes Agency and Helpfulness, which reward obtaining relevant evidence, reducing user workload, and making practical progress. ESR is structurally unable to perform some of the behaviors that receive higher scores under these rubrics.

Consequently, part of the observed AES–ESR improvement may follow directly from the experimental capabilities and evaluation criteria rather than establishing the independent benefit of agentic emotional support.

The AES–TA comparison is more controlled because it holds the GUI interface fixed and varies the interaction objective. I consider this a useful experiment. However, it still does not establish the isolated contribution of actual intervention or the interaction between action availability and emotional-support objectives.

A more convincing design would independently vary GUI access and the emotional-support objective, ideally through a factorial experiment. This would help distinguish the benefits of environmental information, practical action, and emotion-aware decision-making.

This concern is important because the paper's central contribution is not merely demonstrating that an agent with additional capabilities can provide more assistance, but establishing when and why acting improves emotional support.

  1. The central conclusion that action quality matters more than action quantity is supported primarily by observational associations rather than controlled interventions.

The paper argues that better support depends more on acting appropriately than on acting frequently.

This is a reasonable and potentially important hypothesis. However, the evidence is not yet sufficiently strong to establish it as a general scientific finding.

The model-level correlation between action-attempt rate and support improvement is approximately 0.06, whereas the correlation with successful-action rate is approximately 0.75. Yet these comparisons involve only 11 models, and action-attempt rates are concentrated within a relatively narrow range, approximately 88–96%.

A near-zero correlation within such a restricted range does not demonstrate that increasing or decreasing action frequency has no effect on support quality.

Furthermore, action quality and support improvement are both derived from judgments of the same trajectories. Although the authors use separate evaluation calls, the underlying rubrics are semantically related: successful, appropriate interventions are naturally expected to receive higher Helpfulness and Agency scores.

Thus, the association between judged action quality and judged support improvement may partly reflect the structure of the evaluation criteria rather than an independently established behavioral mechanism.

A stronger experiment would manipulate action propensity or intervention thresholds while holding the model, scenario, and evaluation protocol fixed. This could directly test whether acting more improves support, whether excessive action causes harm, and under what conditions abstention is preferable.

Finally, the main evaluation uses only one sampled trajectory per model–scenario–setting combination. This limits the ability to distinguish systematic differences from sampling variability. The Decision–Correction comparisons also use independent runs and show mixed outcomes across backbones, making repeated-trial uncertainty particularly important.

Questions for authors
  1. Can the authors isolate the causal contribution of action availability and emotional-support objectives?

Could the authors conduct a controlled factorial comparison that independently varies GUI access and emotional-support objectives?

The current AES–TA comparison isolates the objective reasonably well, but AES–ESR simultaneously changes environmental access and policy objectives.

Such an experiment would help establish whether the improvement comes from better information access, actual execution, emotion-aware action selection, or interactions among these factors.

A convincing result would materially strengthen my assessment.

  1. Can the authors experimentally test the effect of action frequency rather than relying on model-level correlations?

The paper's central conclusion is that acting more is not inherently better.

Could the authors manipulate action propensity, intervention thresholds, or action budgets for the same backbone and scenarios while keeping other factors fixed?

This would help establish whether there is an optimal intervention regime and when additional action becomes unnecessary, ineffective, or harmful.

  1. How robust are the reported gains to repeated sampling and independent evaluation criteria?

The main experiments use one trajectory per model–scenario–setting combination, and Decision–Correction is compared against independently sampled baseline trajectories.

Could the authors report repeated-trial estimates and uncertainty intervals for the main AES–ESR, AES–TA, and Decision–Correction contrasts?

Additionally, can the authors evaluate the main conclusions using an independent human preference comparison or a separately designed support-quality rubric, rather than relying primarily on related dimensions of the same evaluation framework?

Reviewer_tt3bOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Agentic Emotional Support (AES), where agents go beyond empathetic responses to understand users’ situations and take appropriate actions when needed. The authors introduce LATTE, a benchmark of open-ended emotional support scenarios in interactive Android environments, evaluating both support quality and the appropriateness of agents’ actions. Experiments show that AES generally outperforms response-only and task-oriented baselines, with benefits depending more on action quality than action frequency. The paper also proposes a Decision–Correction mechanism to review actions before execution, which improves performance for some models but not consistently across all model families. The evaluation is further supported by human annotation and validation of the benchmark components.

Critical strengths
  • LATTE brings open-ended emotional support into an interactive GUI setting, where agents need to understand the user's situation and decide whether to act and what actions to take. Unlike traditional task-oriented benchmarks, it does not assume a single correct action sequence and allows agents to ask for clarification or choose not to act. This makes the setting more realistic and provides a useful way to evaluate how well agents make decisions in emotional support scenarios.

  • The evaluation goes beyond emotional support quality to consider whether actions are appropriate, how well they are executed, and whether they respect user consent and available evidence. The authors validate their automated judgments against independent human ratings, including comparisons across different experimental settings. They also conduct human evaluations of the user simulator and auditing components. These checks add credibility to the reported results.

Critical weaknesses
  • ESR cannot access the GUI, and the user simulator is instructed to generally avoid providing detailed private information. The evaluation rubric also limits the scores of responses that lack relevant GUI evidence. These settings may amplify the performance gap between AES and ESR.
  • The AES–ESR comparison changes both environment access and the support policy. A read-only GUI baseline that can retrieve information but cannot modify application state would help clarify how much of the improvement comes from accessing information versus taking actions.
  • The simulated users are instructed not to inspect or summarize private information on behalf of the assistant. Real users, however, may be more willing to share relevant details. It remains unclear whether the AES–ESR performance gap would hold under different user information-sharing behaviors.
  • Decision–Correction combines action checking, visual localization, and final arbitration, but the paper does not evaluate these components separately. As a result, it is unclear whether the improvements mainly come from better action decisions or more accurate GUI localization and execution.
  • It would be useful to compare Decision–Correction against simpler approaches, such as self-review or action verification, under comparable inference budgets. Since the method produces inconsistent results across models (table 3) and even hurts performance in some cases, such comparisons are needed to establish whether the additional complexity is justified.
Questions for authors
  • Among the AES trajectories, what proportion involve only read-only retrieval or inspection, and what proportion involve actions that modify application state, send messages, or make external commitments?

  • If AES agents were allowed to access the same GUI information but not modify application state, how much of the improvement in Overall Support would remain? Also, would the AES–ESR performance gap still hold if simulated users were allowed to share some relevant private information naturally, while still accounting for the effort required from users?

Reviewer_gptvOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LATTE, which is a benchmark for evaluating Agentic Emotional Support. Existing benchmarks evaluate either supportive responses or task completion, but not whether an agent's chosen action is appropriate support. To bridge this gap, LATTE contains 90 open-ended scenarios in reproducible multi-application Android environments. Comprehensive experiments on 11 models under response-only, task-oriented, and AES settings show that AES generally improves support.

Critical strengths

This paper has novel problem framing. It moves emotional support from supportive responses to Agentic Emotional Support. The paper provides solid analysis and insights which shows that quality matters more than quantity.

Critical weaknesses

Model distribution is not diverse because 6 of 11 models are from Qwen family, and other models like Claude, GUI-specialized agents are missing. The comparison favors AES in design choice, since TA is explicitly told not to do emotion support, and ESR cannot see the phone while the simulated user withholds private details. So it will cap ESR’s helpfulness. The claim that acting more does not help is weak, because 11 models’ action attempts are in a narrow 88–96% window, so the near-zero correlation (ρ = 0.06) may reflect this narrow range rather than a true absence of effect.

Questions for authors

All benchmark roles use GPT-5.4-mini, and it is also one of the evaluated models. Can the author re-judge a subset from a different model family?

Reviewer_Ehs5OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a benchmark for assessing the emotional support capabilities of agents, particularly in multi-application Android environments. Building on previous work on emotional support evaluation, which mainly focuses on response quality, this paper also considers whether agents take appropriate actions. The benchmark, named Latte, includes 90 scenarios spanning 50 environment recipes and 7 apps. The authors evaluate 11 LLMs and provide scores for overall support and action quality. They also report automatic grounding and consent violation rates.

Critical strengths
  1. The paper is well motivated.
  2. The authors provide the code and data as supplementary material, which aids reproducibility and transparency.
  3. The experiments are mostly thorough.
Critical weaknesses
  1. GPT-5.4-mini plays almost all roles in the benchmark → this LLM is used as the scenario generator, the user simulator, the auditor, the judge, and a baseline in the benchmark. Unsurprisingly, its family (GPT-5.4, GPT-5.4-mini, GPT-5.4-nano) achieved the top results (Table 2). This is not addressed in the limitations.
  2. Lack of scenario diversity → after inspecting the provided samples (scenarios.jsonl), I feel like the initial user utterances are repetitive, and mainly belong to 2-3 categories: 1) user being overwhelmed (e.g., "I'm being buried"); 2) user being afraid of their tone (e.g., "seeming weird/awkward"); 3) user second-guessing themselves.
  3. Unclear link to emotional support → Adding to the previous point, it is unclear to me how most of these samples relate to emotional support (e.g., "I’ve got too many numbers in front of me and I’m getting tense. This page is packed."). I can acknowledge that in such cases, an agent could benefit from displaying empathy while providing assistance (e.g., similar to how ChatGPT or Claude operate), but I do not see how emotional support is needed.
  4. Issues with evaluation
    • The Task-oriented prompt (Prompt 7) asks the LLM to "keep emotional acknowledgment minimal". However, this variant scores higher on empathy (compared to ESR) on 3 LLMs, and the authors do not provide an explanation for this.
    • Based on Prompt 14, the judge caps helpfulness and agency at 3 when the need depends on phone content that the assistant has not obtained. Yet, the user simulator is required not to share private information with an assistant that cannot inspect the phone (i.e., ESR), and the ESR has no access to such information. Despite this, ESR still outperforms AES on helpfulness in 5 LLMs.
    • Each scenario is run once per model, which makes it unclear whether the reported contrasts would survive multiple runs (given the variance in judge outputs).
    • Adding to the above point, the judges are required to give an ordinal rating (1-5) based on the defined rubrics. However, the reported results scale these ratings to 0-100. The scaling may lead to very small differences being presented as meaningful improvements.
Questions for authors
  1. Did you add any additional steps to control for the diversity of initial user utterances?
  2. How were the annotators selected? what were their expertise?
  3. How were the prompts designed?
  4. Do you think the rankings would change if you used a different LLM as the user simulator or the judge?
  5. How confident are you that the reported contrasts survive multiple runs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.