acceptodds
Under review as a conference paper at ICLR 2027

The illusion of Task Success: Structured Behavioral Diagnosis in Multi-turn LLM Interactions

Abstract

Large language models (LLMs) achieve strong performance on standard benchmarks. However, existing evaluations remain dominated by aggregate metrics that reduce dynamic interactions to coarse scores. In this work, we show how conventional evaluation approaches may create an illusion of task success by masking severe late-stage failures across multi-turn interactions. We introduce RHCA, a structured framework informed by 126 literature sources that tracks model behavior across four core dimensions: reasoning transparency, helpfulness, consistency, and context alignment. To support RHCA at the trajectory level, we also introduce the Behavioral Trajectory Verifier (BTV), which is designed to capture severe failures and performance drops across turns. Our evaluation includes an in-depth music programming study and cross-domain evaluations using tasks created by 16 AI practitioners. While conventional approaches, including automated LLM judges, may miss critical late-stage failures, BTV can flag failures and score drops across turns. These findings highlight the value of structured trajectory evaluation for assessing LLMs in multi-turn interactions.

Primary area:
datasets and benchmarks
Keywords:
Multi-Turn LLMs, LLM Evaluation, Interactive AI, Model Reliability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nZxhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies behavioral evaluation in multi-turn LLM interactions. It introduces RHCA, a turn-level rubric covering reasoning transparency, helpfulness, consistency, and context alignment, and BTV, a trajectory-level metric that penalizes low scores and declines across turns. Evaluation includes 13 music-programming scenarios, controlled late-failure trajectories based on 16 practitioner-defined tasks, and selected real-model conversations. The results show discrepancies between LLM judges and human ratings, and cases where BTV assigns failing scores to trajectories that pass under arithmetic averaging.

Critical strengths
  • The paper addresses an important evaluation problem: task success alone may overlook behavioral failures in multi-turn LLM interactions, motivating assessment across multiple dimensions.
  • The RHCA dimensions are reasonable, and BTV is motivated by the concern that averaging can mask severe failures and quality declines across turns.
  • The experiments illustrate applications of RHCA and demonstrate how BTV captures failures masked by averaging in both controlled trajectories and real-model conversations.
Critical weaknesses

The distinction from prior evaluation methods needs clarification.

Prior work has motivated fine-grained evaluation of multi-turn dialogues [a,b], including minimum-based aggregation [a] and assessment across multiple quality dimensions [b]. The paper contributes a literature- and interaction-informed RHCA rubric and a trajectory metric combining sensitivity to low scores with a decline penalty. However, it should clarify what these choices add beyond existing evaluation frameworks and aggregation methods.

The assumptions underlying the decline penalty are unclear.

Later turns may receive lower scores simply because the user asks harder questions or introduces a new subtask. It is unclear why such drops should receive an additional penalty. BTV also gives the same score of 0.8 to [3, 1, 3] and [3, 3, 1], although the first interaction recovers from its error and the second ends in failure. The paper should explain when this penalty is appropriate and whether recovery should affect the final score.

The validation of BTV is limited.

Figure 4 and Table 3 use constructed final-turn failures that closely match Proposition 1. These results do not show how often BTV incorrectly rejects acceptable interactions. Comparison with mean- and minimum-based aggregation against independent human judgments of whole conversations would better establish its value.


[a] Bai et al. “MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues.” ACL, 2024.

[b] Zhang et al. “FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation.” EMNLP, 2022.

Questions for authors

See weaknesses

3 comments

Thank you for your review. Our replies follow.

W1. We revised Section 2 to clarify two main differences from MT-Bench-101 (Bai et al., 2024) and FineD-Eval (Zhang et al., 2022). (a) Evaluation focus: these methods assess general dialogue quality, while RHCA uses criteria grounded in learning and professional workflows. (b) Trajectory scoring: MT-Bench-101 takes the minimum turn score, overlooking repeated low scores and score trends. FineD-Eval combines dimension-level sub-metrics for the whole dialogue, without tracking how quality changes across turns. BTV accounts for low-scoring turns and drops from earlier peaks.

W2. Our 1-3 rubric scores response quality, not task difficulty: accurate, appropriate responses to harder questions or new subtasks still score 2 or 3. The decline penalty targets drops in response quality, such as losing a constraint or context that the model handled earlier. A score of 1 marks a serious failure, such as forgotten constraints, hallucinations, or misleading guidance, whose costs remain even after recovery. Averaging passes [3, 1, 3] at 2.33, while BTV's strict default flags it. However, recovery may deserve more credit in trial-and-error tasks such as debugging or brainstorming. We plan to explore recovery-aware variants (Section 6.2).

W3. We agree that these failures are constructed and that their detection follows from BTV's definition. We have softened the claim and revised the Table 3 caption and the discussion of Figure 4 accordingly. We also assessed one flagged case, Gemini 3 Flash (Table 4): a domain evaluator who did not see scores, BTV values, or model names reviewed the full transcript and judged it a failure, consistent with BTV. In Setting 1 (Table 9), BTV also reverses the mean's preference for Model A in Scenario S3, matching the annotators' preference for Model B. In Appendix Table 12, we also add a comparison of BTV with other aggregators on the four real-model trajectories. Assessing the remaining flagged cases and measuring how often BTV rejects acceptable interactions, both against independent task-success labels, are in our plans (Section 6.2).

Thank you for the reply. Here are my thoughts:

  1. [Partially addressed] The paper’s focus on workflows and its approach to trajectory aggregation are clearer. However, multidimensional and dialogue-level evaluation still overlap with prior work, and MT-Bench-101 also covers multi-turn reasoning.
  2. [Partially addressed] I understand the intent to penalize severe failures, but it remains unclear when an additional decline penalty is appropriate, particularly across subtask changes or after successful recovery.
  3. [Partially addressed] The added comparisons and independent assessment of one case are helpful. However, validation against independent whole-conversation judgments, including successful and recovered interactions, is still needed.

I therefore maintain my current rating.

Thank you for the further comments.

  1. Multidimensional and dialogue-level evaluation are established, and we build on them. Our additions are practitioner-defined criteria for professional workflows and a trajectory rule that responds to both low scores and declines (Section 2). MT-Bench-101’s minimum rule ignores repeated low scores and trends, and in Appendix Table 12 it passes the Gemini 3 Flash trajectory that BTV flags.
  2. The decline penalty targets drops in response quality, such as losing earlier context or constraints; a well-handled new subtask triggers none. In professional workflows, users may act on an incorrect intermediate answer before it is corrected, so BTV does not let later recovery cancel a severe failure. For trial-and-error tasks such as debugging, we plan recovery-aware variants (Section 6.2).
  3. Whether BTV’s flags match expert judgments of whole conversations is the next test: we plan to collect expert whole-conversation labels and report BTV’s agreement with them at 2.0 and nearby thresholds, and how often BTV fails conversations that experts label successful (Section 6.2), with recovered interactions included.
Reviewer_SePXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyzes a phenomenon called “illusion of success”: a standard evaluation can make an LLM look successful in a multi-turn conversation while hiding serious failures in later turns. The authors traces back the “illusion of task success” due to running the code/ pass@k, averaging the turn scores, LLM-as-a-judge, etc.

The authors provide RHCA, as a structured, extensible evaluation framework for multi-tern LLMs. Along the four core dimensions Reasoning, Helpfulness, Consistency, Context Alignment. In addition, for the multi-turn interactions phase, the authors choose to score each turn and combine the turn scores with harmonic mean, so the BTV will pull down the assessments by low scores. The authors define pass threshold to >= 2.0.

For evaluation, the authors evaluate the approaches in music programming and cross-domain tasks. For the music programming tasks, the authors show several interesting observations: pass/fail check can’t tell the models apart,LLM judge tends to pick the opposite winners from humans, RHCA saw some aspects of where B is better, BTV also catches what averaging misses. For cross-domain tasks, the authors also show that BTV captures critical failures that are injected in the conversations and succeeded in flagging the failing trajectories for LLM-as-a-judge scenario.

Critical strengths

(1) Novel metrics. The paper defines a metric that is frequently overlooked in multi-turn evaluations of LLMs. Instead of averaging turns or pass@k, LLM-as-a-judge, the metrics BTV is more consistent with human evaluators with a harmonic mean that weights low-scoring turns heavily and a drop penalty for declines from an earlier high point. The Proposition 1 provides a formal guarantee for BTV.

(2) While RHCA concept is not new to the community, RHCA’s 2×2 grid separates single-turn qualities (Reasoning, Helpfulness) from conversation-level qualities (Consistency, Context Alignment). That makes it specifically suited to multi-turn and agentic evaluation.

(3) The authors focus on important problem in nowadays LLM evaluation: "illusion of success", which is that common evaluation methods can hide critical late-turn failures and even reverse model rankings. The newly designed rubrics are lightweight and can function as a easy plug-in for many task evaluations.

(3) Large-scale experiment designs with human annotators: For the music programming tasks, the scenarios come from more than 1,400 consented interactions from 81 real users (under IRB approval), not synthetic prompts. For both scenarios, the human annotators judge these interactions with a dedicated calibration process.

(4) Domain experts define failure: In scenario 2, 16 industry practitioners wrote their own criteria and 1/2/3 score descriptions. Participants had at least one year of experience building or evaluating interactive LLM systems and direct involvement in release decisions. This makes the human annotations and rubrics more reliable for the experiments.

(5) The authors have also defined controlled tests, in addition real runs. The cross-domain study combines controlled failure injection at conversation various length turns, isolating how averaging becomes more forgiving as conversations get longer, with real runs of three frontier models on long practitioner tasks.

(6) Many levels of controls during the experiment, for example, scenario 1 compares two versions of the same base model. In scenario 2. the judge and the humans use the same rubric.

(7) The authors use rank-based tests for skewed 1–3 ratings. With additional statistics (Fleiss’ and Cohen’s) to measure the alignment between humans as well as how LLM judges aligned with humans.

Critical weaknesses

(1) Figure 1 is inconsistent with the paper’s own formula. It shows “FAIL (Score = 1.0)” with “Context Alignment: 1/3”. Under equal weights that turn would score about 2.5, not 1.0?

(2) BTV still weakens with more turns and longer trajectories. Table 3 shows the trend as BTV gradually rises from 0.89 to 1.42 as conversations lengthen. The claim that BTV “naturally scales to longer dialogues” should be qualified.

(3) Proposition 1 is close to trivial as it follows directly from the formula.

(4) There is an obvious additional baseline: an LLM judge that scores the whole conversation. Currently, all LLM-judge baselines score individual turns in the paper. The obvious alternative to BTV is to give the judge the full conversation and ask whether it fails the practitioner’s criterion. In addition, the authors can also consider last-turn score, recency-weighted mean, min + mean and τ\tau-bench’s pass@k.

(5) Setting 2 doesn’t actually test RHCA. Section 3.3.2 says the second study is meant “to evaluate whether RHCA generalizes beyond music programming.” But each practitioner task uses one criterion mapped to one dimension.

(6) Figure 3 could be revised to be more readable: there appears to be mixes two units on one y-axis (“Score / Decline Difference”) and two sign conventions: score differences are B − A, but the decline difference is A − B.

Questions for authors

(1) On the rubric, 2 means acceptable. Yet BTV fails a conversation that goes from 3 to 2: for example, [3, 2] gives a BTV = 1.90 and fail. [3, 3, 2, 2] also gives a BTV of 1.9 and fail. [2,2,2] gives 2.0 and passes. So a model that is consistently mediocre passes, while one that starts excellent and settles to acceptable fails?

(2) Why is the BTV threshold set to 2.0? The 2.0 pass threshold comes from the rubric’s “acceptable” level, which applies to turn scores. There’s no calibration showing that BTV = 2.0 means anything comparable.

(3) Does the16 practitioner domains matter in the injected-failure test?

(4) The four dimensions may overlap? One of the authors’ selection criteria was distinct failure type, but there is no analysis of correlation between dimensions. The rubric also mixes them: helpfulness level 1 penalizes only focuses on task completion (a teaching concern), and misleading suggestion is hard to separate from “too generic”.

3 comments

Thank you for your thorough review and encouraging comments. Our responses follow.

W1, W6. In Figure 1, the FAIL comes from a critical failure (Context Alignment = 1), which we report alongside BTV (Section 3.2), and we relabeled the figure to show this. In Figure 3, we unified all differences to a consistent (Model B - Model A) sign convention and clarified the axis caption.

W2. We have softened this claim in Section 3.2. The rise from 0.89 to 1.42 in Table 3 happens because one low turn counts for less in longer trajectories (the mean also rises, from 2.06 to 2.50). By Property 1, a final-turn collapse to 1.0 still stays below 2.0 at every length, with a smaller margin. We plan to test longer trajectories (Section 6.2).

W3. We agree that it follows directly from the formula, and we renamed it Property 1.

W4 (Baselines). We added last-turn, recency-weighted mean, and (mean+min) baselines on the four real trajectories (Appendix Table 12); they give the same verdicts as the mean and min, passing Gemini 3 Flash, which BTV flags. The per-turn LLM judges we tested were not reliable: in Setting 1, GPT-4o and Claude Sonnet 4.6 showed near-zero agreement with human raters (Section 5.1; Appendix E.4), and in Setting 2 the judge gave 3.0 to 74 of 78 turns, but BTV still flagged the Claude Haiku trajectory that human evaluators rated as failing (Section 5.3; Table 4). A whole-conversation LLM judge is in our plans (Section 6.2).

W5 (Role of Setting 2). We agree that Setting 2 does not test all four dimensions. Our setting is that each of the 16 practitioners chose the dimension critical to their domain (Table 2 and Appendix Table 13): Context Alignment (10/16), Consistency (3/16), Reasoning (2/16), Helpfulness (1/16). Setting 2 therefore tests BTV across domains and trajectory lengths (T = 3-12), while Setting 1 evaluates all four dimensions. We have updated Section 3.3.2 accordingly.

Q1 (Why [3, 2] scores 1.90 while [2, 2, 2] scores 2.0). This is by design: BTV counts a fall from an earlier high point against the trajectory, even when the new level is acceptable. [3, 2] has a harmonic mean of 2.4 and a drop of 1, while [2, 2, 2] has no drop and scores 2.0. A fall from 3 to 2 can mean the model is losing context or consistency, which can be an early warning of later failures.

Q2 (Choice of 2.0 threshold). The 2.0 threshold comes from the rubric's "Acceptable" level: a stable trajectory at that level, such as [2, 2, 2], scores V = 2.0, while unrecovered drops or scores below 2 fall below it (Section 3.2). This is a design choice, and we plan to calibrate it with independent task-success labels (Section 6.2).

Q3 (Role of the 16 domains). In the injected-failure test, BTV sees only the scores, so the domain does not change its result. The domains matter for the failure criteria that practitioners define and for the real-model runs.

Q4 (Dimension overlap). Section 5.2, Figure 7 and Appendix E.1-E.2 show distinct behavioral profiles across dimensions. We also added Spearman correlations on our human-annotated calibration set (Appendix Table 6): the dimensions are related but not redundant. Reasoning is nearly uncorrelated with Context Alignment and Consistency (0.04 each) and weakly correlated with Helpfulness (0.27); Context Alignment and Consistency correlate moderately (0.49). In collaborative tasks such as debugging, writing, or analysis, giving a solution without explanation or guidance makes it hard for users to check or build on the output, limiting its usefulness. "Misleading" means the advice contains factual or logical errors that push the user in the wrong direction; "too generic" means the advice is factually safe but ignores the user's specific context and constraints (Context Alignment).

Thank you for the responses. The fixes to Figures 1 and 3, the renaming of Property 1 and the added correlations address those points. Two of my previous concerns still remain:

  1. Previously W4: The paper claims BTV catches failures that existing evaluation misses. Per-turn judges failing does not show that a judge reading the full conversation would also fail. This is cheap to test on your existing trajectories (the four human-rated conversations and the 78-turn runs). If such a judge catches the same failures, what does BTV add? If it doesn’t, that directly supports BTV.

  2. Previously Q1 & Q2: The 2.0 threshold is borrowed from the per-turn “acceptable” level, and outcomes like [3, 2] failing while [2, 2, 2] passes are justified only by author's design choice. One suggestion is to let human experts judge each existing conversation as pass or fail, and report how often BTV agrees with them at 2.0 and at nearby thresholds. This would show whether 2.0 is a good cutoff and how often BTV wrongly fails acceptable conversations. This requires future work with added analysis and human reviewer.

Overall, I think this paper is valuable as a paper for LLM evaluation. I will keep my positive scoring.

Thank you for the follow-up and for your positive assessment.

W4. We agree this is a useful test. Our turn-level judge sees the history through the target turn (Appendix C) but scores one turn at a time. We plan to compare BTV with whole-conversation LLM judges on our existing trajectories (Section 6.2), using a fixed protocol and several judge models. Even if such a judge flags the same conversations, RHCA and BTV show where and how the failure arises.

Q1 & Q2. We agree this is the right next test for BTV’s default 2.0 threshold, and it is in our plans (Section 6.2): experts label each existing conversation as successful or not, and we report BTV’s agreement with them at 2.0 and nearby thresholds, including how often BTV fails conversations that experts label successful.

Reviewer_UiQSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes RHCA, a four-dimensional behavioral assessment framework, and BTV, a trajectory score for quality drops. They evaluate those metrics on multi-turn music-programming tasks and practitioner-defined cross-domain scenarios, showing that structured behavioral evaluation can reveal failures obscured by average scores or missed by LLM judges.

Critical strengths

RHCA provides an interpretable rubric for diagnosing behavioral failures in multi-turn interactions, with substantial reported agreement among human annotators. This supports its practical usefulness beyond task-success scores.

Critical weaknesses
  1. The contribution beyond existing work is unclear. Closely related studies such as AgentBoard, AgentRewardBench, and AgentProcessBench already examine intermediate progress, trajectory evaluation, and step-level failures, but are not discussed. Although their settings differ, the paper should explain what limitations remain and how RHCA and BTV address them.

  2. The experiments do not clearly show that BTV’s failure judgments are correct. The controlled experiments assign the lowest score to the final turn, so detecting these failures is largely expected from BTV’s formula. Human evaluation covers only four real-model conversations and provides turn-level ratings, but no clear independent assessment of overall task success to verify the additional failures flagged by BTV.

Questions for authors
  1. BTV assigns the same score to [3,3,1] and [3,1,3], although the latter could represent successful recovery from an intermediate mistake. Why is treating these trajectories equally appropriate?

  2. What specific benefits do RHCA and BTV offer over existing process and trajectory evaluation approaches, and which results in the paper demonstrate these benefits?

3 comments

Thank you for your review and constructive suggestions. Our responses follow.

W1. We added AgentBoard, AgentRewardBench, and AgentProcessBench to Section 2. These benchmarks offer useful ways to evaluate task progress, web-agent outcomes, and step-level effectiveness. Our focus is complementary: open-ended, domain-specific interactions in professional workflow, where task completion or a single reference answer may not fully capture interaction quality. RHCA uses practitioner-defined criteria to assess reasoning transparency, helpfulness, consistency, and context alignment. Separately, even context-aware turn ratings can hide important declines when simply averaged. BTV addresses this aggregation issue by penalizing low scores and drops from earlier peaks.

W2: We agree that the failures in Table 3 and Figure 4 are injected at the final turn, so BTV's detection follows from its formula. We have softened our claims and now present these experiments as sensitivity checks for late-stage failures. In Setting 1 (Table 9), BTV reverses the mean's preference for Model A in Scenario S3, matching the annotators' preference for Model B. We also checked one flagged case: a domain evaluator reviewed the full Gemini 3 Flash conversation in Table 4 without seeing the turn scores, BTV scores, or model names, and judged it a failure, consistent with BTV, although both the human-score mean and minimum indicated a pass. We use this assessment as supporting evidence. Human ratings cover four real-model trajectories (34 turns), while real-model runs cover 78 turns (Section 5.3). We plan to collect independent end-of-conversation task-success labels for more trajectories (Section 6.2).

Q1.BTV gives [3, 3, 1] and [3, 1, 3] the same score because they have the same harmonic mean and maximum drop. BTV prioritizes detecting severe failures and does not explicitly reward recovery: an RHCA score of 1.0 indicates a severe failure, such as misleading novices or losing core context (Section 3.3.2). These failures can impose cognitive and corrective costs even if later replies improve. Minor declines, such as [3.0, 2.5, 3.0], still pass (Table 10). We plan to study recovery-aware variants (Section 6.2).

Q2. RHCA gives interpretable, rubric-based evaluations where no reference answer exists (Sections 3.1, 5.2). BTV, our trajectory verifier, flags late failures and slow declines that a mean passes (Table 11). In Tables 4 and 9, it also flags human-rated trajectories that the mean passes.

Thank you for the responses. The rebuttal addresses some of my concerns, but I remain concerned about the validity of the proposed metric. In long-horizon agentic tasks, successful recovery is generally preferable to an unresolved failure, but BTV assigns the same score to trajectories such as [3,1,3] and [3,3,1], as mentioned in my comments. I find this difficult to justify and therefore maintain my current score.

Thank you for the follow-up. One addition on the example: [3, 1, 3] has a harmonic mean of 1.8, so BTV fails it at the 2.0 threshold even without the decline penalty. A variant of BTV that measures the drop only to the final turn would rank it above [3, 3, 1], but would still fail it. For long-horizon agentic tasks, we will evaluate recovery-aware BTV variants against expert labels that include recovered interactions (Section 6.2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.