acceptodds
Withdrawn from ICLR 2027

EASE: Efficient Human-Agent Communication under Evolving User Preferences

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Agentic AI systems can execute increasingly complex real-world tasks, yet their transition from passive responders to active task coordinators raises a central communication challenge: how to efficiently incorporate necessary human input without overburdening the user. Existing systems often fail to adapt interaction forms and progress-reporting strategies as user preferences evolve, leading to redundant elicitation, over-automation, and verbose operational traces. In this paper, we propose EASE, an adaptive communication framework that enables granular control over user engagement, interaction form, and progress exposure. By mapping user preference stability, intervention signals, and task risk to a fine-grained automation spectrum, EASE elicits necessary user input with reduced user effort. To support timely intervention without cognitive overload, EASE surfaces only decision-relevant execution states while suppressing low-level operational details. Extensive experiments show that EASE consistently reduces time cost and cognitive load compared with strong baselines across no-history, stable-preference, and preference-drift settings. These results demonstrate that efficient human-agent communication is achieved not by maximizing autonomy alone, but by dynamically calibrating when, how, and to what extent users should be engaged.

Primary area:
optimization
Keywords:
Human-Agent Interaction, Adaptive Automation-Level Adjustment, Evolving User Preferences

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Y9yWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose EASE, an agentic communication framework that provides fine-grained control over user engagement, interaction form, and progress exposure. The paper demonstrate efficient human-agent communication that reduced human effort.

Critical strengths

The paper targets a practical problem in agent design: agents that ask too much waste the user’s time, while agents that act on stale preferences make mistakes. Splitting user involvement into fine-grained automation spectrum per field is a simple, interpretable way to handle this.

Critical weaknesses
  • Unclear value over capable general-purpose agents: EASE’s behaviors (reusing past choices, asking through options or yes/no confirmations, and giving short progress updates) could plausibly be obtained by prompting a strong LLM agent with the interaction history, but no such baseline is included. The two environments are also small (100 flight records; 30 contacts and 8 rooms), so it is hard to tell whether the gains hold beyond these toy settings.
  • Hand-tuned design limited to predefined tasks: EASE requires predefined fields for each task, along with manually set risk scores and default levels and five hyperparameters. It is unclear how it would transfer to open-ended agent tasks where the relevant fields are not known in advance.
Questions for authors
  • How does EASE compare with a strong general-purpose LLM agent (e.g., a frontier model) given the same interaction history and an instruction to avoid unnecessary questions?
  • How are the fields, risk scores, and default levels set for a new task? Can they be derived automatically when the relevant fields are not known in advance?
  • With a Qwen2.5-7B backbone, EASE reaches 95% task success on the τ-bench-derived airline tasks. Which steps are performed by the LLM, and which are scripted by the framework?
Reviewer_jkxrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores a problem on how an AI agent can calibrate user preference over time and present an EASE framework that maintains a dynamic preference, assign each task dimension to five automation levels, and filter to report selected decision-relevant workflow states. The EASE was evaluated with simulated flight booking environments, which shows lower interaction time and improved adaptation to preference changes.

Critical strengths

This paper explores an interesting problem of dynamically leveraging user preference over time.

Critical weaknesses

Although I appreciate an interesting problem and evaluation with simulation environment, I found the evidence on cognitive load and human outcomes are not evaluated well. Specifically, the direct human evaluation only include three participations, which is too long to derive conclusion and does not provide enough evidence. For the human evaluation, i wonder if there was any ethical review and approval.

Also, the proposed framework require predefined task dimensions and dimension-specific scores and parameters. Thus, it is not clear how this method can transfer to a new setting without these engineering.

Questions for authors

Please clarify whether the study was reviewed by an institutional ethics/IRB body?

Reviewer_h4VBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how AI agents should adapt the degree of human involvement as user preferences evolve over repeated interactions. The authors propose EASE, which decomposes tasks into individual decision dimensions and assigns each dimension one of five automation levels, ranging from human control to full automation. The central methodological component is a heuristic score (Equation 4) that adjusts automation levels based on historical preference stability, recent user interventions, and task risk. The framework also includes a progress-reporting mechanism that selectively exposes execution information to reduce communication overhead.

The authors evaluate EASE in flight-booking and meeting-scheduling environments, primarily using simulated users with stable or changing preferences. They compare their approach against several existing methods and report improvements in interaction time, communication volume, and task success. The paper also includes a small human evaluation involving three participants.

Critical strengths

I think the topic of this article is quite interesting and important. How we should think about human-AI collaboration, and particularly when an AI agent should prompt the human for input, is an important question as AI becomes more ubiquitous in our lives. The increasing ability of AI agents to execute tasks autonomously creates a genuine need to understand when human involvement is valuable and how this involvement should evolve as agents learn more about their users.

I also appreciate the paper's attempt to consider evolving user preferences, which is an important and relatively underexplored aspect of long-term human-AI interaction. The idea of adjusting the degree of human involvement at the level of individual decisions is intuitively appealing. The authors also make a reasonable effort to evaluate the framework across different preference settings, including several robustness experiments. I therefore think there is a potentially interesting research direction here, although I have significant reservations about the methodology and the evidence supporting the proposed approach.

Critical weaknesses

I have significant reservations about the methodology of this paper, and unfortunately I do not find the current experimental evidence sufficiently convincing to support its central claims.

1. The proposed heuristic score has no fundamental justification.

In essence, this paper proposes a five-level classification of automation, defines a heuristic score for moving between the different levels (Equation 4), and then runs experiments to justify that it works. My biggest concern is that this heuristic has essentially no theoretical or behavioral foundation. It is not derived from first principles, from an optimization problem, or from an established model of human decision-making. The authors simply propose a particular score and attempt to demonstrate empirically that it performs well.

More specifically, Equation 4 combines preference stability, intervention strength, and task risk through an additive linear function. While the directional intuition behind these terms is reasonable, there is no explanation for why these factors should enter additively, why the marginal effects should be constant, or why the risk penalty should be proportional to the previous automation level. The subsequent incremental update in Equation 5 is similarly heuristic. There is no characterization of the resulting policy, or explanation of why this particular functional form should produce an appropriate level of human involvement.

This is extremely problematic because there is no replicable philosophy behind how the score is constructed. Why are these particular behavioral factors included and others omitted? Why should a particular change in preference stability offset a change in intervention strength? How should the relative weights be chosen in a new application? Without a principled foundation, it is almost impossible to know how to generalize the proposed score to other tasks. The fact that a particular heuristic works in two constructed environments does not establish that it constitutes a generalizable methodology for human-AI collaboration.

I find this particularly disappointing because the underlying question lends itself naturally to a decision-theoretic formulation. One could explicitly model the trade-off between the cost of requesting human input and the expected loss from allowing an agent to act on an incorrect preference. Such a formulation would provide a principled basis for determining when human intervention is valuable. The authors instead introduce a collection of terms and coefficients without connecting them to any underlying objective.

The parameter sensitivity analysis in Appendix C does little to resolve this concern. It demonstrates that increasing automation can reduce interaction time while increasing the frequency of user corrections. This is an expected trade-off, but it does not explain why Equation 4 is the appropriate way to manage it, or how its parameters should be calibrated.

2. The experimental comparisons do not convincingly establish the value of the proposed methodology.

My second concern is that the empirical improvements may largely reflect the interaction interface and experimental design, as opposed to the particular adaptive automation algorithm proposed in this paper.

For example, one of the central mechanisms of EASE is to replace open-ended user questions with selection or confirmation as the agent accumulates information. The authors then evaluate user interaction time using a keystroke-level model, under which clicking a confirmation button is substantially less costly than typing an answer. Consequently, an important portion of the reported efficiency gains follows almost mechanically from the choice of interaction interface.

Similarly, the authors report substantial reductions in cognitive burden, but the primary proxy for cognitive load is the volume of text shown to the user. Since EASE explicitly includes a mechanism for suppressing low-level execution information, it is unsurprising that it produces fewer output tokens. Producing shorter responses, however, is not equivalent to reducing cognitive burden, especially when users may need to scrutinize an agent's actions to detect errors.

The comparisons with existing methods are relevant, particularly DTAC and CIPHER, but they do not adequately isolate the contribution of the proposed automation policy. The most important missing comparison, in my view, is a simple rule-based adaptive policy that remembers historical preferences, progressively moves from specification to confirmation or automation following repeated consistent responses, and reduces automation following user corrections. Such a policy could implement the core intuition of this paper without Equation 4 or its associated mathematical machinery. It is entirely unclear whether the proposed score would outperform this much simpler alternative.

The ablation study does not resolve this issue either. Removing the fine-grained automation mechanism demonstrates that having different levels of human involvement is useful, but does not establish that the particular rule used to move between those levels is valuable. I would have expected the authors to hold the interaction interface and progress-reporting mechanism fixed, and directly compare alternative policies for determining automation levels.

More broadly, the paper does not convincingly establish the relationship between communication efficiency and decision quality. A system can reduce user effort simply by asking fewer questions, but this may increase the probability of incorrect autonomous actions. The natural evaluation would examine whether a method achieves lower interaction costs for a given error rate, or lower error rates for a given interaction budget. Although Appendix C examines the relationship between automation and correction rates, the main comparisons do not provide a systematic evaluation of this trade-off.

3. The experimental evaluation relies too heavily on synthetic users.

The vast majority of the experiments use synthetic users with predetermined preferences and controlled preference changes. I understand the practical motivation for using simulation, but this is a particularly problematic setting in which to rely primarily on synthetic evaluation, because the effectiveness of the proposed method fundamentally depends on human behavior.

The simulated users are instructed to follow predefined preference values and provide corrections when the agent proposes an inconsistent value. However, actual users do not necessarily behave this way. They may overlook incorrect recommendations, interpret questions differently, fail to correct mistakes, or change their preferences without explicitly communicating those changes. These behaviors are central to the problem the paper claims to address.

Indeed, the proposed algorithm explicitly uses human corrections as signals for reducing automation. In the preference-drift experiments, the simulator reveals preference changes through corrections, and the algorithm is designed to reduce automation in response. Figure 6 therefore demonstrates that the proposed mechanism behaves as specified, but provides much weaker evidence that it can effectively identify and respond to evolving human preferences in realistic settings.

I am particularly concerned about the assumption in Section 3.2 that the absence of a correction can be treated as implicit confirmation of preference stability. This is a strong behavioral assumption that is not justified empirically. In real human-AI interactions, users may fail to correct the agent precisely because they are not paying attention, misunderstand what the agent is doing, or find intervention burdensome. Treating such behavior as evidence that the agent's preference estimate is correct could lead to precisely the kind of over-automation the paper seeks to prevent.

The appendix experiments on benchmark tasks and robustness are useful additions, and I acknowledge that the authors do examine several forms of interaction failure. However, these experiments continue to rely on simulated user behavior and controlled mechanisms for exposing errors and eliciting corrections. I therefore remain unconvinced that they establish the effectiveness of the proposed framework in realistic human-AI collaboration.

Although the authors include a human evaluation, it involves only three participants. This is simply not enough to draw meaningful conclusions about cognitive load, user effort, or actual human interaction patterns. I find this especially concerning given that the paper's central claims concern the behavior and experience of human users. A substantially larger human evaluation would be necessary to support these conclusions.

4. The paper is extremely poorly written and unnecessarily complicated.

Finally, I found the presentation of the paper extremely difficult to follow. The amount of mathematical notation introduced for what is ultimately a relatively simple idea does the paper no favors. The authors define an extensive collection of indices, states, functions, and update rules, but much of this notation appears to formalize straightforward engineering decisions without yielding corresponding theoretical insights.

Even the abstract is close to unreadable. It is difficult to identify the actual methodological contribution without repeatedly reading the same passages, and the extensive terminology often obscures what the system is doing. I had to spend considerably more effort than should have been necessary to understand that the central mechanism is essentially a heuristic for increasing or decreasing automation based on preference consistency and user corrections.

I strongly believe that, particularly in this day and age of AI, how much effort authors put into communicating their research is just as important as the results themselves. The purpose of writing a research paper is to communicate ideas clearly and enable other researchers to understand, evaluate, and build upon them. If a reader has to read the paper three times or resort to AI assistance simply to understand the basic idea, then the paper is failing at one of its most fundamental purposes.

Overall, while I find the research question interesting and important, I do not think the current paper provides a sufficiently principled methodology or convincing empirical evidence to support its claims. The combination of an essentially arbitrary heuristic, comparisons that do not isolate its contribution, and a heavy reliance on synthetic user behavior leaves me with substantial concerns about the paper's contribution and soundness.

Questions for authors
  1. Is there a principled derivation or behavioral justification for Equation 4? In particular, why should preference stability, intervention strength, and task risk enter through this specific additive functional form, and what justifies the interaction between task risk and the previous automation level? How should the score and its coefficients be determined when transferring the method to a new application?

  2. Can the authors compare EASE against a simple rule-based adaptive automation policy that increases automation after repeated consistent user responses and decreases it after corrections? I believe this is the most important missing baseline, because it would establish whether the proposed mathematical formulation offers meaningful advantages over a straightforward implementation of the same intuition. Ideally, this comparison would hold the interaction interface and progress-reporting mechanism fixed.

  3. How sensitive are the results to the assumptions about simulated user behavior, particularly the reliability of user corrections? The current simulator provides corrections when inconsistent preferences become visible. Would the proposed approach continue to perform well if users occasionally failed to notice incorrect actions, provided noisy feedback, or changed their preferences without explicitly intervening?

  4. Can the authors provide more substantive evidence that their approach reduces actual human cognitive burden, beyond reductions in output tokens and modeled keystroke costs? The three-participant human study is too small to support the paper's central human-centered claims.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.