- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a study of how the expression style of user instructions affects the task performance of LLM agents, and frames the resulting performance gaps as an expression style-induced bias.
The authors then propose the Politeness-Arousal Style Framework (PA-Style), which defines two style axes (polite vs. direct, calm vs. emotional) motivated by the Big Five model, and build a semantic-preserving stylization pipeline (LLM-based rewriting plus an LLM Semantic Comparator and Style Checker) on VitaBench to evaluate three agents under four compound styles.
To reduce the gaps, they further propose Instruction Style Alignment (ISA), a lightweight BART-based rewriter trained with supervised warm-up and REINFORCE-style reinforcement learning, which converts user instructions into the style that the agent handles best and uses sampling-level fusion of two LoRA models to control the target style.
Critical strengths
-
The motivation is clearly explained, and the pilot study in Section 3.1 (Table 1) gives initial empirical evidence that changing only the style of an instruction can change agent behavior.
-
The authors release their code, which can support reproduction and future work.
Critical weaknesses
-
Mis-framed problem and limited novelty. The paper treats any style-induced performance difference as a fairness issue and claims to be the first to identify expression style as a bias dimension. However, fairness is defined with respect to sensitive or protected attributes[1], whereas the instruction is the main input of the agent. The phenomenon is better described as robustness to semantic-preserving input variations, and the effect of politeness and emotional expression on LLM performance has already been studied [2, 3]. Since the
novel bias dimensionis the central selling point of the paper, this substantially weakens its main contribution. -
Insufficient evidence for both the measured bias and its mitigation. The stylized instructions are checked only by LLMs. Additionally, a single model plays the agent, the user simulator, and the evaluator (and apparently also the rewriter). The test set contains only 50 samples in a single-round setting, and ISA is evaluated on only one agent. Moreover, some results in Table 4 contradict the claim that ISA
effectively eliminatesthe disparities (e.g., the Range on Pass^4 rises to 0.160). It is therefore unclear whether the reported disparities and their reduction reflect real agent behavior or artifacts of the evaluation pipeline.
[1] Machine Learning Testing: Survey, Landscapes and Horizons. TSE, 2020.
[2] Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance. SICon Workshop at ACL, 2024.
[3] Large Language Models Understand and Can Be Enhanced by Emotional Stimuli. arXiv:2307.11760, 2023.
Questions for authors
- Why should expression style be treated as a protected attribute rather than as an input variation that the agent should be robust to? Is there evidence that specific styles are strongly associated with specific user groups?
- Which model rewrites the instructions, and is it the same model as the agent and the evaluator? How do the authors rule out that the disparities come from the evaluator's own style preference or self-preference?
- What happens to instructions that still fail the Semantic Comparator or Style Checker after the maximum number of retries, and how many such cases are there? Has any human verification of semantic equivalence been performed?
- Why does ISA enlarge the Range on Pass^4 (0.160, +0.09), and why does the w/o Stage-2 variant achieve a higher Pass@4 Avg_styles (0.735) than the full ISA (0.650)? Which value of γ is used in the experiments?
- Please explain more clearly why the Big Five model is chosen as the theoretical basis and why only two of its five factors are kept. Moreover, the arousal axis is not actually a Big Five dimension: the paper says it uses
Neuroticism as a motivating personality correlate, but operationalize the axis as affective arousal. Please justify these choices (e.g., with user studies or with the literature on linguistic markers of personality), and discuss why other common style dimensions, such as formality, verbosity, or informal writing (slang, typos), are not considered. This relates to the logical rigor of the paper. The current presentation of dimension choices seems more like an ad-hoc approach.
13 comments
R1 (W1/Q1). Fairness Motivation versus Robustness (1)
We agree that expression-style sensitivity is closely related to robustness, but our fairness perspective focuses on whether such sensitivity leads to systematic disparities in the quality of service users receive despite equivalent task requirements.
Q1.1: Is expression style treated as a protected attribute?
Our emphasis on examining fairness beyond traditional demographic attributes may have unintentionally given the impression that we regard expression style as another protected attribute. We apologize for this lack of clarity. We do treat expression style as a protected attribute, but rather as a linguistic characteristic of user instructions that serves as a grouping dimension for fairness evaluation. Specifically, operationalizes this characteristic into four controlled expression conditions, allowing us to examine systematic disparities in agent service quality across styles under equivalent task requirements.
More broadly, while fairness concerning protected attributes is a central paradigm, fairness is not exclusively defined in terms of such attributes. For example, individual fairness emphasizes that individuals who are similar with respect to a task should be treated similarly (Dwork et al., 2012). Our study draws on this broader principle to motivate fairness-relevant service-quality consistency across expression styles, rather than treating expression styles as protected demographic attributes.
Q1.2: Why study expression-style induced performance disparities as a fairness issue rather than solely a robustness issue?
We agree that expression-style induced performance variation falls within the broader scope of prompt sensitivity and robustness. However, our primary concern is not merely whether different expressions affect agent performance, but whether particular expression styles are associated with lower average task performance for a given agent under equivalent task requirements. General prompt sensitivity does not necessarily imply a consistent disadvantage for any particular style, whereas our study specifically investigates such style-related disparities across repeated evaluations.
The fairness implications arise when expression styles are considered in relation to users. Expression style is not merely an arbitrary input variation; it can also reflect how users habitually communicate their requests. If particular styles consistently lead to poorer agent performance, users who tend to express themselves in those styles may systematically receive lower-quality service despite having equivalent task requirements. We therefore examine these disparities from a fairness perspective, focusing on whether differences in user expression styles are associated with disparities in average agent task performance under equivalent task requirements.
Importantly, fairness and robustness are complementary rather than mutually exclusive perspectives. Improving overall task performance does not necessarily reduce cross-style disparities. Accordingly, we evaluate both Average Performance and Cross-style Disparity, aiming to improve service-quality consistency without sacrificing overall task effectiveness. Related research has similarly examined linguistic variation through both fairness and robustness perspectives (Lin et al., 2025).
Q1.3: Is there evidence that specific styles are strongly associated with specific user groups?
Prior research has documented associations between linguistic politeness and social factors (Danescu-Niculescu-Mizil et al., 2013; van Dorst et al., 2024). However, we do not claim that our four categories correspond to specific demographic or protected groups, nor that the observed disparities establish demographic discrimination. Such associations are not a prerequisite for investigating systematic service-quality disparities across expression styles, although further evidence would be necessary to support stronger claims about disparities affecting specific demographic groups.
We will clarify these distinctions and the scope of our fairness claims in the revised manuscript.
R1 (W1/Q1). Fairness Motivation versus Robustness (2)
W1: Novelty and Contributions Relative to Prior Work.
We sincerely appreciate your observation regarding prior studies on the effects of politeness and emotion on LLM performance. We acknowledge that our original novelty claims may have been interpreted too broadly, and we will clarify their scope to distinguish our agent-level contributions from previously established effects on LLM performance.
However, our contribution is not simply to demonstrate that politeness or emotion affects LLM performance, but to investigate whether and how expression-style sensitivity extends to the actual task execution of tool-using LLM agents, and how the resulting disparities can be mitigated.
| Work | Target | Style | Agent Actions | Mitigation Focus |
|---|---|---|---|---|
| Yin et al. (2024) | LLM | Politeness | No | Not studied |
| Dobariya & Kumar (2025) | LLM | Politeness | No | Not studied |
| Li et al. (2024) | Generative AI | Emotion | No | Performance enhancement |
| Ours | Tool-using agents | Politeness × Arousal | Yes | Cross-style disparity reduction (ISA) |
① From LLM Responses to Agent Task Execution.
Prior studies primarily examine LLM response accuracy or generation quality, whereas tool-using agents additionally involve planning, decision-making, tool invocation, and environment interaction. Whether and how response-level style sensitivity extends to agent actions and end-to-end task completion cannot be directly inferred from these studies. We investigate this distinction by evaluating three tool-using agents and analyzing style-dependent tool-use behavior in Section 4.3.
② From Individual Style Dimensions to Controlled Compound-style Disparity Evaluation.
Rather than examining politeness or emotional cues individually, combines Politeness × Arousal into four controlled expression conditions. This enables systematic evaluation of cross-style performance disparities in agent task execution under equivalent task requirements, beyond isolated prompt-level performance effects.
③ From Performance-oriented Prompting to Disparity-oriented Mitigation.
Unlike prior approaches primarily aimed at improving model performance, ISA targets cross-style performance disparity reduction while preserving overall agent task performance. Since higher average performance does not necessarily imply smaller disparities, we evaluate both Average Performance and Cross-style Disparity to assess mitigation effectiveness.
We will revise the manuscript to better acknowledge prior work, clarify our contributions, and avoid overstating our novelty.
Response to Reviewer b1hE
We sincerely thank you for your thorough and insightful review. We have carefully read and reflected on every comment, and we are grateful for your recognition of our research motivation, pilot study, and code release efforts.
We will do our best to address your concerns directly and head-on. If any concerns remain or we have misunderstood your comments, we would sincerely welcome further feedback.