acceptodds
Under review as a conference paper at ICLR 2027

You Changed Your Mind, The Model Didn’t: Demystifying Intent in Multi-Turn Dialogue

Abstract

When a large language model handles a multi-turn task and a user proposes a change but ultimately rejects it, the model should continue as if nothing changed. We find a surprising failure: merely mentioning a rejected change can derail task execution, even when the user's final intent remains unchanged. To systematically study language model behavior under evolving user intent, we introduce Intent-Eval, a controlled benchmark spanning tool actions, code, databases, and mathematics. Across diverse tasks, models are vulnerable to both rejected proposals and superseded requirements, consistent with mentioned-as-in-effect confusion: conversational content is treated as active requirements even after it has been rejected or replaced. Accuracy degradation can deepen or persist as interaction continues, highlighting the need to distinguish what has been mentioned from what remains in effect. Building on this insight, we propose Intent-OPSD, a decision-conditioned on-policy self-distillation framework with Teacher and Student initialized from the same model. The frozen Teacher provides active-intent supervision from the complete task matching the user's decision, training the Student on the full dialogue to follow active requirements reflecting user intent.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Multi-Turn Dialogue, User Intent, On-Policy Distillation, LLM Evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5xwJOpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether LLMs can tell what has been said in a multi-turn dialogue apart from what actually remains in effect. It introduces Intent-Eval, a benchmark across Actions, Code, Database and Math, where the same task is run under different conditions: no interruption, a neutral clarification, a proposed change that is rejected, and the same change accepted. Matched single-turn versions separate multi-turn effects from task difficulty, and longer dialogues test whether the interference grows or persists. The authors find that models lose accuracy even from neutral clarifications, and more so when a change is proposed, whether it is rejected or accepted. Their errors often contain content that was rejected or replaced. To address this, they propose Intent-OPSD, a Teacher-Student self-distillation method where the Teacher sees the single-turn task matching the user’s final decision and the Student learns from the full dialogue. It improves over the base models and over standard fine-tuning.

Critical strengths

[1] The paper takes a focused look at a problem that earlier work only touches on. Prior work such as “LLMs get lost in evolving user intent” treats argument revision as one setting among several. Here, comparing clarification, rejection and acceptance on the same tasks, with matched single-turn controls, isolates whether a mentioned change derails the task that should remain in effect. This matters because it separates real tracking failures from task difficulty.

[2] The research questions are well designed and the method follows from them. The RQs are clearly articulated and answer the questions a reader is likely to have. RQ3 (does interference accumulate or persist) and RQ4 (can distillation help) are the most interesting and important. The Teacher-Student setup, with the Teacher matched to the active task, offers a way for the model to follow what is in effect without a Teacher at inference time. The table comparing single-turn and multi-turn conditions makes the results transparent.

Critical weaknesses

The realism of the inserted clarification and revision events is not fully established. Dialogues run about 4-8 turns, which is typical of real interactions. However, the proposals, rejections and clarifications are template-based, with LLM-drafted wording and only a human review. These events need to be examined to make sure they are natural and not out of order, since awkward insertions could confuse the model for reasons unrelated to intent tracking. The benchmark also covers only one revised requirement per task.

Questions for authors

[1] Have you encountered scenarios where the model provided a solution to both the changes that are in effect and the changes that were requested before but revised later? Will such kind of a response be considered incorrect in the current experimental setting?

[2] Can you explain why there is a relative increase in performance as event blocks k increases from 3 to 4 in Figure 5?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 5xwJ

Dear Reviewer 5xwJ,

Thank you for the careful review. We address your concerns as follows.

W1. Event realism and the scope of revisions.

  • Event naturalness and ordering. We agree that awkward or misplaced events could confuse models for reasons unrelated to intent tracking. As described in Appendix A.1, we reviewed each selected revision for clarity and consistency with the source task, and manually reviewed the clarification, proposal, and decision templates before use. Event ordering is enforced by construction: each event follows the introduction of the relevant requirement, and each proposal precedes its acceptance or rejection.
  • Interaction coverage. Changing one requirement at a time lets us examine an individual decision while keeping the other task requirements fixed. We agree that this covers only part of real-world interaction. As discussed in Appendix F (Limitations and Discussion), extending the benchmark to multiple interacting requirements and more varied user behavior remains future work.

Q1. Responses containing both active and superseded solutions

Yes, we observed this behavior, and a final response that executes both the active and superseded requests is scored incorrect. For example, after a user changed a factorial request from 5 to 7, the model included calls for both in its final answer. Since the accepted revision replaced the original request, the additional call for 5 violated the active requirements.

Q2. The increase from k=3 to k=4

The increase most likely comes from the added block prompting the model to revisit which requirement remains active, which on some tasks lets it correct an earlier mistake. This increase is small and does not appear consistently across models and domains. Despite these local increases, average accuracy at k=4 remains below the corresponding k=0 baselines.


Thank you again for your positive assessment and constructive feedback. Your suggestions have helped us clarify the benchmark’s design, scoring criteria, and interpretation of the results. We hope these responses address your concerns and are happy to discuss any remaining questions.

Best,

The Authors

Reviewer_5xwJOpenReview ↗

Reply to Rebuttal

Thank you for your reply. I will keep my scores.

Thank you for taking the time to consider our responses. We appreciate your thoughtful review and feedback.

Reviewer_FDoNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper finds the phenomenon that the agent fails to realize the recovered context. Using the LiC benchmark, this paper propose an evaluation framework to test the llms in such situations and a new OPD based learning algorithm as well. Experimental results shows that frontier llm struggles in such situations

Critical strengths
  • This paper tackles some important topic that goes beyond multi turn interaction of user and agent

  • The experiment design seems proper. Neutral, revised, retained score comparison is helpful to understand the results

Critical weaknesses
  • As far as I know, the main claim of this paper is: “When the user proposes a state change and then reverts it, the agent often fails to recognize it.” However, Table 1 shows that the gap between Revised and Retained is only about 2%p. To justify that this result is significant, the authors should report the number of trials per task, since multi-turn conversation settings usually show high variance across trials (which is why LiC uses n = 10 trials). If the authors ran only one trial per task, they should extend it to at least four.

  • Intent-OPSD seems limited to this benchmark. To address this, the authors should at least run inference with the trained model in a cross-domain setting (e.g., train on Code and infer on Actions), or, if that is not feasible, run some other OOD experiments.

  • Original OPD should be added as a baseline in table 2.

  • The method needs ablation studies.

Questions for authors
  • Comparing M_Orig, M_Ret, and M_Rev aligns with the main motivation of this paper, but I’m not sure why the single-turn experiments were added. Could the authors explain the reasons specifically?

  • As I understand it, Revised errors have a much lower proportion of inactive content than Retained errors, yet the number of new errors is almost the same. Then what was the main reason they failed?

5 comments
AuthorsOpenReview ↗

Response to Reviewer FDoN (Part I)

Dear Reviewer FDoN,

Thank you for the thoughtful and constructive feedback. We address each concern below.

W1. Main claim, effect size, and repeated evaluations

  • Main claim and effect size: Our claim concerns whether models still follow the original task when a proposed change is rejected before taking effect. We therefore compare Original, which contains no proposed change, with Retained, which includes the rejected proposal while preserving the same active requirements. Table 1 shows a substantial 8.06 pp decline in Retained accuracy. The roughly 2 pp Revised–Retained gap answers a different question: it compares executing an accepted revision with retaining the original task after rejection, so the final active requirements differ.
  • Repeated evaluations: To assess the reliability of our findings, we conducted four evaluation runs in total: the original evaluation reported in the paper and three additional runs. Across the eight models, the mean Original–Retained gap is 8.19 pp, close to the originally reported 8.06 pp, providing further support for the reported overall decline. In addition, 28 of 32 model–condition cells have four-run mean accuracies within 2 percentage points of the original results.

Each table cell shows mean accuracy ± standard deviation (%); parentheses indicate changes from the original accuracy in percentage points.

ModelOriginalNeutralRevisedRetained
Qwen3.6-27B61.23 ± 0.94 (−0.12)59.66 ± 0.88 (+0.97)57.79 ± 1.33 (−0.91)55.13 ± 0.80 (+0.54)
Qwen3-8B48.91 ± 0.24 (+0.12)45.65 ± 1.04 (0.00)44.75 ± 0.91 (+0.79)29.89 ± 0.64 (−0.30)
Llama3.1-8B34.90 ± 2.02 (+0.60)31.22 ± 1.01 (−0.66)26.45 ± 1.66 (+0.60)25.36 ± 1.96 (+1.45)
Gemma4-26B-A4B56.46 ± 0.50 (−0.30)51.57 ± 2.85 (+2.78)51.51 ± 1.77 (−1.39)45.17 ± 1.23 (0.00)
GPT-5.6-Luna64.07 ± 0.75 (+1.03)62.32 ± 1.72 (+1.45)60.14 ± 0.95 (+0.48)53.74 ± 0.94 (−0.12)
Gemini-3.7-Flash59.72 ± 1.95 (−2.60)54.39 ± 1.89 (+1.73)44.93 ± 1.38 (−0.72)59.12 ± 0.46 (−0.30)
DeepSeek-V4-Flash-073157.85 ± 2.82 (+2.05)53.80 ± 1.22 (−1.51)51.15 ± 0.69 (+0.91)51.63 ± 2.05 (−1.99)
Claude-Sonnet-561.47 ± 0.83 (+0.36)52.84 ± 0.99 (−1.27)59.90 ± 1.82 (−2.17)59.06 ± 1.22 (+0.85)

W2. Cross-domain generalization

Following your suggestion, we evaluated Code→Actions transfer across all four RQ4 models using four evaluation runs (seeds 0–3), without target-domain fine-tuning. The results are mixed. In our Code tasks, accepting a revision changes only the required function name, while preserving the function’s intended functionality. Learning to update function names may offer limited transfer to Actions, where the model must select tool-call argument values that match the active requirements.

We also evaluated Math→Actions, Math→Code, and Math→Database. Mean accuracy improves over Base in all 12 model–target combinations. Unlike function renaming, Math revisions change numerical information and require the answer to be updated accordingly. This may provide more transferable practice in identifying which requirements remain active and using them to produce the final answer.

Each cell shows the accuracy change relative to Base, averaged across four conditions and four runs, in percentage points.

Training → EvaluationQwen3-8BQwen2.5-7BLlama3.1-8BGemma4-26B-A4B
Code → Actions+0.60−1.20+3.85−2.34
Math → Actions+5.95+0.30+7.63+12.14
Math → Code+2.81+1.31+0.81+2.75
Math → Database+3.86+1.34+17.46+10.28
AuthorsOpenReview ↗

Response to Reviewer FDoN (Part II)

W3. Original OPD baseline

Thank you for suggesting this comparison. Our original experiments compared Intent-OPSD with Base and matched SFT. We have now added a FiC-style OPSD baseline following its concatenated-view Teacher design [1]. The Teacher receives all user messages concatenated in chronological order into a single-turn prompt, while Intent-OPSD provides the complete task matching the user’s decision to accept or reject the proposed change. Both retain our Student-on-policy distillation procedure.

We conducted four evaluation runs per trained checkpoint across four models and four domains. Mean accuracy is 51.73% for Intent-OPSD versus 49.53% for the concatenation baseline. The table below reports condition-level differences. (Δ Accuracy = Intent-OPSD − User-concat (pp), averaged over four domains and four evaluation runs.)

ModelOriginalNeutralRetainedRevised
Qwen3-8B+1.01+2.02+1.55+1.91
Qwen2.5-7B−1.26−2.43+0.76−0.77
Llama-3.1-8B+9.13+4.72+6.17+10.48
Gemma-4-26B-A4B+2.02−0.08+0.27−0.21

W4: Ablation studies

We examine two aspects of the method:

  • Training scheme: Under shared active-task supervision, Intent-OPSD improves mean accuracy over matched SFT by 3.73 pp (Table 2). This supports on-policy distribution matching as an effective alternative to directly learning hard Teacher targets.
  • Teacher input: We use the FiC-style concatenation baseline introduced in W3 (User-concat) to examine the Teacher-input design. It presents task details and the user’s decision in a single-turn prompt, while Intent-OPSD directly provides the task specification resolved according to that decision. Both use our Student-on-policy distillation procedure. Intent-OPSD achieves 2.20 pp higher mean accuracy, with higher Retained accuracy across all four models (+0.27–6.17 pp). These results are consistent with a benefit from explicitly resolving active requirements before distillation.

Q1. Purpose of the single-turn experiments

The single-turn experiments help explain why multi-turn performance drops. As described in Section 3.3, they serve two purposes:

  • Check task difficulty: Original vs. Direct Revised (both single-turn) tests whether the revision itself makes the task harder. Without this comparison, lower Revised accuracy could simply reflect increased task difficulty.
  • Examine decision-induced confusion: Original vs. Retained and Direct Revised vs. Revised (all single-turn) test whether proposals and decisions affect performance while keeping the active requirements unchanged. Together with the multi-turn comparisons in RQ1, these results show that decision-induced confusion occurs in both single-turn and multi-turn settings.

Q2. Main reason for Revised failures

Thank you for raising this question. Appendix E.2 reports 649 new Revised errors and 748 new Retained errors. Inactive content appears in 30.97% of these Revised errors, compared with 59.63% of the Retained errors.

In Retained, using the rejected requirement can leave detectable inactive content in the answer. In Revised, our inspection found two recurring failure patterns: some responses omit the accepted change, while others apply it but miss other active requirements or produce incomplete answers. These failures can occur without inactive content, explaining why Revised’s lower inactive-content rate does not imply a comparable reduction in new errors.

We will acknowledge in the Limitations that this analysis captures one error pattern and does not fully explain the remaining failures.

Reference

[1] Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap. NeurIPS 2026.


Thank you again for your constructive feedback, which has helped us strengthen our baseline comparisons and cross-domain evaluation. We hope the additional experiments and clarifications address your concerns and would appreciate your reconsideration of our submission. We are happy to discuss any remaining questions.

Best,

The Authors

Thank you for the thorough responses. I appreciate the effort, and I’m sorry to say I will keep my score. The following are just suggested revisions for the camera-ready version.

  1. The paper currently reports results from a single trial (n_trial = 1). I suggest updating all reported results to the four-run setting.

  2. Clarify the role of the single-turn experiments. I had many questions about the single-turn experiments. At first, I wondered: if Revised and Retained also degrade performance in the single-turn setting, shouldn’t that be improved as well? And if so, why is the title of this paper limited to multi-turn dialogue? On reflection, however, I think a real user would rarely produce a Revised or Retained situation within a single turn, so it is reasonable that this setting does not need to be improved. Still, readers may question the paper’s framing. I suggest stating explicitly that the single-turn setting is used only for comparison and is not targeted for improvement. This would reduce confusion for readers.

Reviewer_ncqKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies whether LLM in multi-turn dialogue act on and what is currently in effect, or on everything that was mentioned, or act on the requirements that remain in effect after a user proposes a change and then accepts or rejects it. The authors built a customized dataset of sharded tasks from LiC and for each task that receives one atomic, verifiable revision and rendered in eight forms. The authors further propose an on policy self distillation method and shows improved accuracy over base models and models with SFT.

Critical strengths

The paper is generally well written and easy to follow, the motivation is clear.

The experiment design is well carried for the stated question and motivation, with "retained" and "revised" sharing the identical proposal with different decision of whether the user accepts it. The single-turn controls show the revised tasks are easier or about the same as the original; Edits are confirmed by the verifier. Together these design makes the results clear and interpretable.

The single-turn decision controls also produce direct result in the paper by isolating the real effect without simulator noise; S_Ret and S_Rev drops relative to the same active task stated directly with fixed prompts and no multi-turn sampling.

Critical weaknesses

It seems that Intent-OPSD does not fix the failure the paper is about: The gains are largest on Original and Neutral, where nothing is proposed or rejected in this case, and smallest gain on Retained, the gap between Original and Retained grows after training. It seems what this method really shows is that it narrows the general multi-turn gap, while there are already other works targeting this gap, and the paper does not compare against them either.

The multi-turn decision effect is small compared to the matched control: against Neutral, which adds the same number of turns, rejection and acceptance difference are both low, also the direction is not consistent across all the models.

Since each condition’s dialogue is sampled independently with a stochastic user simulator, and there are no reruns or confidence intervals, the headline claim may not hold for several of the models evaluated.

Questions for authors
  1. In evaluation; do the four conditions share the pre-event dialogue prefix or is each sampled independently?

  2. How do you demonstrate that the increasing Original to Retained gap with the claim that Intent-OPSD helps the student follow active requirements despite rejected content?

3 comments
AuthorsOpenReview ↗

Response to Reviewer ncqK (Part I)

Dear Reviewer ncqK,

We sincerely appreciate your time and insightful feedback! We have carefully addressed your concerns as follows:

W1 & Q2: Intent-OPSD’s effectiveness, the widening gap, and baseline comparisons

Thank you for highlighting the widening Original–Retained gap and the need for comparisons with existing methods.

  • Effectiveness and the widening gap: The 7.11 pp gain over Base on Retained (Table 2) supports our claim that Intent-OPSD improves execution of active requirements even when rejected content remains in the dialogue. Revised accuracy also improves by 9.97 pp. The Original–Retained gap widens because Original improves more than Retained. Intent-OPSD aligns Teacher supervision with the complete task matching the user’s decision to accept or reject the proposed change. However, it supervises final answers without separate decision-level supervision, which may contribute to smaller gains when competing requirements must be resolved.

  • Comparison with existing methods: Following FiC’s concatenated-view Teacher design [1], we evaluated a User-concat baseline that concatenates all user messages into a single-turn Teacher prompt while retaining our Student-on-policy distillation procedure. Across four models, four domains, and four evaluation runs per checkpoint, mean accuracy is 51.73% for Intent-OPSD versus 49.53% for User-concat. Mean Retained accuracy is higher for all four models (+0.27–6.17 pp), while differences in other conditions are model-dependent. These results are consistent with a benefit from explicitly resolving active requirements before distillation.

    Each entry in the table below shows Δ Accuracy = Intent-OPSD − User-concat in percentage points, averaged over four domains and four evaluation runs. Positive values favor Intent-OPSD.

    ModelOriginalNeutralRetainedRevised
    Qwen3-8B+1.01+2.02+1.55+1.91
    Qwen2.5-7B−1.26−2.43+0.76−0.77
    Llama-3.1-8B+9.13+4.72+6.17+10.48
    Gemma-4-26B-A4B+2.02−0.08+0.27−0.21
  • Positioning and future work: Our main contributions are the benchmark, the empirical findings, and Intent-OPSD as an approach motivated by these findings. We acknowledge that its gains over Base are smaller on Retained than on Original and Neutral. As discussed in Appendix F (Limitations and Discussion), further improving responses to rejected changes and extending the method to more varied interactions remain directions for future work.

AuthorsOpenReview ↗

Response to Reviewer ncqK (Part II)

W2: Comparisons with Neutral and reliability across repeated evaluations

Thank you for raising this concern. We address it through repeated evaluations, model-level comparisons against Neutral, and an analysis of how models respond to Neutral clarification.

  • Reliability across repeated evaluations. We added three evaluation runs, giving four runs in total, indexed by seeds 0–3, with seed 0 corresponding to the original paper. The four-run means remain close to the original results: 28 of 32 model–condition cells differ by at most 2 percentage points, supporting the reliability of the reported pattern. We report mean ± standard deviation below.

  • Overall tendency relative to Neutral. We agree that the effect varies across models. Nevertheless, Retained accuracy is lower than Neutral for six of eight models, by 2.17–15.76 percentage points, and Revised accuracy is lower for seven of eight models. These results show an overall downward tendency, with differences in magnitude and direction across models.

  • Interpreting the comparisons. We compare Neutral with Original to assess the conversational overhead of added clarification while keeping the active task unchanged. Some Claude and Gemini Code dialogues under Neutral end with explanations rather than executable code, lowering Neutral accuracy and potentially narrowing or reversing the Retained–Neutral gap. The overall declines reported under “Decision-induced confusion” in the Introduction compare Retained and Revised with Original, while comparisons with Neutral assess rejection and acceptance relative to clarification.

Each cell in the table below shows mean accuracy ± standard deviation (%) across seeds 0–3; parentheses indicate changes from the original paper in percentage points.

ModelOriginalNeutralRevisedRetained
Qwen3.6-27B61.23 ± 0.94 (−0.12)59.66 ± 0.88 (+0.97)57.79 ± 1.33 (−0.91)55.13 ± 0.80 (+0.54)
Qwen3-8B48.91 ± 0.24 (+0.12)45.65 ± 1.04 (0.00)44.75 ± 0.91 (+0.79)29.89 ± 0.64 (−0.30)
Llama3.1-8B34.90 ± 2.02 (+0.60)31.22 ± 1.01 (−0.66)26.45 ± 1.66 (+0.60)25.36 ± 1.96 (+1.45)
Gemma4-26B-A4B56.46 ± 0.50 (−0.30)51.57 ± 2.85 (+2.78)51.51 ± 1.77 (−1.39)45.17 ± 1.23 (0.00)
GPT-5.6-Luna64.07 ± 0.75 (+1.03)62.32 ± 1.72 (+1.45)60.14 ± 0.95 (+0.48)53.74 ± 0.94 (−0.12)
Gemini-3.7-Flash59.72 ± 1.95 (−2.60)54.39 ± 1.89 (+1.73)44.93 ± 1.38 (−0.72)59.12 ± 0.46 (−0.30)
DeepSeek-V4-Flash-073157.85 ± 2.82 (+2.05)53.80 ± 1.22 (−1.51)51.15 ± 0.69 (+0.91)51.63 ± 2.05 (−1.99)
Claude-Sonnet-561.47 ± 0.83 (+0.36)52.84 ± 0.99 (−1.27)59.90 ± 1.82 (−2.17)59.06 ± 1.22 (+0.85)

Q1: Do the four conditions share the pre-event dialogue prefix?

No. At evaluation, each condition is generated independently to assess task performance over complete conversations. The source tasks and condition-specific event definitions remain fixed, and the user simulator is instructed to preserve task information and follow the specified user decisions. We evaluate 414 tasks across four domains and eight models to reduce dependence on particular tasks, domains, or model families. In W2, we further assess the reliability of our results through four evaluation runs per model, reporting mean accuracy and standard deviation.

Reference

[1] Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap. NeurIPS 2026.


Thank you again for your thoughtful feedback, which has helped us clarify our claims and strengthen our evaluation. We hope the additional results and clarifications address your concerns and would appreciate your reconsideration of our submission. We are happy to discuss any remaining questions.

Best,

The Authors

AuthorsOpenReview ↗

Kind Request for Reviewer's Attention and Feedback

Dear Reviewer ncqK,

We sincerely thank you for your valuable and constructive feedback. Since the End of author/reviewer discussions is coming soon, may we know if our response addresses your main concerns? If so, we kindly ask for your reconsideration of the score. Should you have any further advice on the revised paper or our rebuttal, please let us know and we will be more than happy to engage in more discussion and paper improvements.

Thank you so much for devoting time to improving our paper!

Best regards,

The Authors

Comments on the paper

2 comments

Dear Reviewers,

Thank you for your reviews and contributions. As we are now halfway through the discussion period, I would like to encourage you to carefully read the authors' responses and engage in further discussion where appropriate. Please also indicate whether the rebuttal has addressed your concerns or changed your assessment.

Thank you for your time and effort!

AuthorsOpenReview ↗

General Response

Dear AC and Reviewers,

We sincerely thank you for your time and effort. It is encouraging to see that all reviewers have recognized positive aspects of our manuscript, such as an important and focused problem (Reviewers FDoN, 5xwJ), clear motivation (Reviewer ncqK), well-designed and interpretable experiments (Reviewers ncqK, FDoN, 5xwJ), clearly articulated research questions (Reviewer 5xwJ), a well-motivated Teacher–Student approach (Reviewer 5xwJ), and clear presentation (Reviewers ncqK, 5xwJ).

We have responded to each reviewer in detail. Below we summarize the main clarifications and revisions. Changes to the manuscript are highlighted in purple.

  • Evaluation reliability. In response to feedback from Reviewers ncqK and FDoN, we ① clarified that the original evaluation covered 10 models across four domains, giving our findings a broad empirical basis, and ② added three evaluation runs to further assess the stability of the multi-turn results. We updated the multi-turn accuracy results in Tables 1 and 2 and the corresponding discussion in Section 4 to reflect the four-run evaluation. The updated values are close to those from the original evaluation and support the same findings (see Appendix B.1).
  • Baselines, ablations, and cross-domain transfer. In response to Reviewers ncqK and FDoN, we ① clarified that the existing matched SFT comparison tests the training scheme under the same task supervision. Intent-OPSD achieved higher overall accuracy (see Table 2). ② We added the User-concat baseline, which uses the same distillation procedure and combines all user messages into one Teacher prompt. Intent-OPSD performed better overall, with higher Retained accuracy on all four models (see Appendix E.5). ③ Without target-domain fine-tuning, Math-trained models improved over Base in all 12 tested model–domain pairs, while Code-to-Actions results were mixed (see Appendix E.6).
  • Presentation and discussion. Following feedback from Reviewers ncqK, FDoN, and 5xwJ, we ① clarified that single-turn experiments serve as diagnostic comparisons and that Intent-OPSD targets multi-turn performance (Section 2.3). ② We positioned Intent-OPSD as an approach motivated by our Intent-Eval findings. Retained accuracy improves over Base, with larger gains on Original widening the gap. Handling rejected proposals in more varied dialogues remains future work (Appendix F). ③ We clarified that detecting superseded content captures only one type of error (Appendix F). ④ Our replies address event realism and revision scope and offer a possible explanation for the small accuracy gain from three to four event blocks (see Appendix E.3).

We’re grateful that all three reviewers took the time to acknowledge our responses. Reviewer ncqK raised the rating after the rebuttal. Reviewer 5xwJ gave the paper a positive assessment at the highest confidence level and maintained it after the discussion. We also appreciated Reviewer FDoN’s follow-up suggestions, both of which are addressed in the revised PDF.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.