acceptodds
Under review as a conference paper at ICLR 2027

Context Mismatch in LLMs: Stale Interaction Policies Across Task Boundaries

Abstract

As interactions with large language models unfold across multiple turns, a conversation establishes not only facts but also an interaction policy about who should propose, verify, constrain, and decide. Prior work has examined how information and user preferences persist across turns, but has largely overlooked whether such policies expire when the conversation moves to a new task. We show that they can remain behaviorally active. We call this failure context mismatch: a model reuses a prior task's governance state even when the new task requires a different allocation of decision rights. Our account of boundary-conditioned state reuse predicts that errors arise when the carried state persists, the new task reads it differently, and the resulting shift crosses the current decision margin. Using paired evaluations across six reasoning benchmarks that hold current evidence and answer options fixed, we find a task-dependent crossover: obedience histories impair independent verification but help authorized delegation, with failures concentrated near the decision boundary. Within each of two architectures, bidirectional state exchanges transport both the failure and its rescue, supporting a causal role for the carried state. This mechanism motivates Composite Directional Governance Editing (C-DGE), which amortizes the causally identified matched-state displacement: from one observed run, two mutually exclusive history-task routes predict input-dependent corrections in protected, gradient-defined low-rank subspaces. On a locked final test, C-DGE recovers 30.36% of the mismatch-induced margin gap and rescues 32 of 135 induced decision flips while leaving all 2,856 protected controls unchanged on the locked test. These results identify reversible governance-state interference as a source of task-transition errors and show that it can be repaired selectively without a counterfactual rerun.

Primary area:
interpretability and explainable AI
Keywords:
large language models, multi-turn interaction, context mismatch, task switching, mechanistic interpretability, activation steering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Fo17OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how behaviors learned in context from earlier tasks in a conversation, referred to as the history, affect performance on a new task. Each task consists of a binary-choice reasoning problem, the user's answer to that problem, and an instruction specifying whether to follow the user's answer or to verify it independently. The authors examine whether each task instruction is followed under obedience and verification histories, and propose several hypotheses to explain the cases where it is not. Based on these hypotheses, C-DGE is proposed and empirically validated that it corrects decisions made incorrect by mismatched histories.

Critical strengths

None

Critical weaknesses
  1. The core phenomenon is not convincingly established, which undermines both the mechanistic analysis and C-DGE built upon it.

In task-aligned accuracy, mismatched histories change performance by only about 1 - 5% relative to matched histories as shown in Table 9. Matched accuracy under independent verification is 51.82%, which is almost random chance for a binary choice. The reported effects therefore sit on a task the model barely solves and a task leaving little room for change.

Moreover, the paper assumes that the "obedience" and "verification" histories create the corresponding interaction policies, but never verifies this. The observed margin differences are explained by these policies without direct evidence that the histories actually lead to obedient or verifying behavior.

The evaluation protocol also limits what can be concluded. Because decisions are restricted to a forced A/B choice, the most desirable behavior, complying with the delegated choice while flagging the factual error, cannot be observed. As a result, correcting the user's mistake is counted as a failure.

Since the causal localization and the repair method both presuppose a real and meaningful failure mode, this weakness propagates to the entire paper. It is reflected in C-DGE's marginal practical gains: it improves mismatch accuracy by only about one percentage point on the locked test, as the gap it targets is itself small.

  1. The claims far exceed the experimental scope, so the results cannot support the generality the paper claims.

The paper frames the problem as stale interaction policies in general, i.e., who should propose, verify, constrain, and decide, yet only a single axis, verification versus obedience, is tested. All histories are derived from one templated cost-comparison record repetition, with only two policies and two surface styles. Such histories are unlikely to occur in real conversations, and varying the source benchmarks does not diversify them. Furthermore, only two models from the same family are evaluated. Given one behavioral axis, one synthetic history template, and one model family, the findings may reflect artifacts of this specific construction rather than a general property of multi-turn LLM interaction.

Questions for authors

Please see the weaknesses.

4 comments

We thank you for this thoughtful comment. We understand your concern that the conclusions should be supported by evidence appropriate to the scope of the evaluation. Our claim is specific: an interaction state established during one task can persist across a task boundary and influence the next task when its requirements differ.

Our study follows a common approach in mechanistic and intervention-based research: isolate a defined behavior, test it under controlled conditions, and evaluate a targeted intervention. For example, CAA ([ACL 2024] evaluates selected behaviors using multiple-choice questions and open-ended generation, while CAST ([ICLR 2025]) investigates conditional refusal across seven models. LoReFT ([NeurIPS 2024]) and RePS ([NeurIPS 2025]) likewise study representation interventions on defined tasks or behavioral targets.

Nevertheless, we reconsidered the paper from your perspective and added analyses to address the concerns about behavioral evidence, history diversity, and model coverage:

  1. We directly measured the history-induced preference for the user-selected answer, as detailed in the response to Reviewer 6uhm.

  2. We added a cross-family behavioral analysis on MiniCPM5-2B. As shown in Appendix D.2, the mismatch-gap reduction was 17.03%, which provides evidence that the measured task-dependent history effect is not confined to the Qwen family.

  3. We evaluated C-DGE after ten consecutive benchmark-task turns. After the model completed a sequence of ten benchmark questions, we evaluated the subsequent task transition on 192 held-out test items. C-DGE achieved 17.61% NGR, compared with 9.60% for RePS. This tests the effect of context accumulated through successive benchmark tasks, beyond a single repeated history instance.

We would also like to clarify the interpretation of the 51.82% figure and of the relatively small change in aggregate accuracy. The effect is intuitive at the item level: very easy items may remain correct under either history, while items already beyond the model’s capability may remain incorrect under either history. In both cases, a history-induced shift in the model’s interaction state may not change the final answer. The effect is therefore most visible near the decision boundary, where a change in the model’s preference can change the selected answer. This is why we evaluated the full benchmark distribution rather than constructing a test set consisting only of items expected to flip.

Our locked-test analysis supports this account. Among matched-correct decisions, the low-versus-high-margin flip-rate difference was 44.4 percentage points for independent verification and 4.8 points for delegated choice; none of the 582 high-margin cases flipped. The margin bands were determined using the discovery split and frozen before evaluation on the prospective split. Thus, aggregate accuracy depends in part on how a benchmark’s items are distributed across easy, difficult, and decision-boundary cases. Averaging over all items can dilute a real, task-dependent effect concentrated near that boundary; it does not by itself show that the effect is absent.

This also motivates the practical question behind our study. In everyday use, people may recognize experiences such as an assistant seeming not to follow a new request, or seeming to defer too readily to a prior preference. These experiences motivate our investigation; they are not themselves evidence for the mechanism. Our experiments test whether a specific task-transition effect can be observed in open-source benchmarks and whether it can be reduced through intervention. The effect is measurable in the full benchmark evaluation and becomes clearer in the boundary-risk analysis. We therefore view the contribution as identifying when this failure is likely to affect an answer and testing whether it is repairable, rather than claiming a large uniform accuracy gain on every benchmark item.

Accordingly, this paper identifies and tests a specific form of context mismatch: an interaction state established during one task can persist across a task boundary and influence behavior when the next task requires a different policy. By measuring this effect across benchmark tasks and history conditions, examining where it changes decisions, and evaluating a targeted intervention, we characterize when this failure emerges and show that it can be reduced. We again thank you for raising these concerns. We hope the added analyses and clarifications address them and make the scope and contribution of the work clearer.

Otherwise, we would also like to clarify two aspects of your characterization. First, it is accurate that our central behavioral contrast is verification versus delegation; this is the specific transition the paper sets out to test. However, the complete evaluation is not limited to one repeated history template: it also varies history style and length, includes six naturalistic task scenarios, and evaluates sequential benchmark-task histories. Second, Qwen3-8B and Qwen3.5-9B belong to the same model family, but are distinct model architectures. And the MiniCPM analysis further extends the behavioral evidence to another model family.

We hope our current clarification addresses your concerns, and we thank you again for your suggestions on this work.

Reviewer_Fo17OpenReview ↗

Response to the authors response

Thanks for the response. However, I will maintain my score for the following reasons:

  1. Even with the added scenarios, the study tests a single, unrealistic behavioral axis under a forced A/B protocol, whereas the cited works, e.g., CAA with seven behaviors and CAST with multiple refusal categories, cover far broader and realistic cases.
  2. The 44.4% flip rate is reported only in aggregate across benchmarks, so it is unclear whether the effect generalizes or is driven by specific benchmarks.
  3. Boundary-case analysis is beneficial to the research community only when the failure is substantial.

We understand your concern. You question whether the effect observed in the forced A/B contrast reflects a genuine failure rather than an artifact manufactured by the design itself. This is precisely why we chose this design, rather than something we avoided.

We should first be clear that a forced A/B design does not mean there is no substantive failure. The binary format is an identification device, not a claim about the richness of real interaction. The failures we report are real model errors that occur in real task transitions. The A/B format simply makes them measurable.

These flips are not imposed by the design. They are genuine model errors that would occur in real task transitions.

First, the design does not manufacture flips. It only makes them observable. In real use, models already switch between independently judging the evidence and following a user's delegation. Our design holds the question and answer options fixed and changes only the preceding policy. If a flip occurs, it can only come from the model's own decision process, not from differences in the question, options, or labels. The design itself does not produce errors. It merely exposes errors that would otherwise be masked by item differences.

Second, real-world cases confirm that these flips are genuine failures. We not only report flip rates under controlled conditions, but also test whether the same failure reappears after ten benchmark tasks executed in sequence, that is, after genuinely accumulated task history (C-DGE 17.61% NGR vs. RePS 9.60%). If the flips were merely a constructive artifact, they should not reappear after sequential, real task execution. The positive crossover interaction across all six benchmarks in both discovery and replication shows that this is reproducible, genuine model behavior, not a product of the evaluation format.

Third, the design is precisely what distinguishes genuine failures from design failures. We observe 0/266 flips among high-margin decisions and 91/205 flips among low-margin decisions. If the design were manufacturing failures, it would not land so precisely on the decision boundary. This selectivity itself shows that what we capture are genuine judgment errors that occur when evidence and delegation conflict, not artifacts imposed by the evaluation format. What we report are meaningful failure cases, not a failure of the design.

Finally, we agree that supplementing the controlled contrast with studies of failures on real open-ended tasks would help complete the picture of how this mechanism behaves in practice. We would note, however, that such supplementation would come at a cost to effect observation. Real tasks introduce uncontrolled variation in wording, difficulty, and context, which is exactly the confound our paired design removes. The two approaches are therefore complementary. The controlled design identifies the mechanism, and naturalistic tasks characterize its expression, but they serve different inferential goals. We sincerely appreciate the opportunity to clarify this motivation. If there are specific analyses or clarifications that would help address your remaining concerns or reconsider your assessment, we would be grateful for your comments.

Reviewer_6uhmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how earlier conversation history affects model decisions after switching to a new task. It compares histories that encourage following the user’s choice with histories that encourage independent verification.

The experiments use A/B tasks built from six reasoning benchmarks. Compared with verification-oriented history, obedience-oriented history harms independent verification but helps the model record an authorized user choice.

C-DGE learns low-rank activation corrections from runs with matching and mismatched histories. At inference, it corrects the current run without running the model again with a different history.

Critical strengths
  • Controlled experiments support the history effect: comparisons keep the current evidence and answer options fixed. Label swaps, neutral histories, and reset conditions address several alternative explanations. Changing internal states in both directions also supports their causal influence on decisions.
  • The error pattern is tested on separate data: Among examples initially answered correctly under matching history, Table 1 shows that mismatched history mainly causes errors when the gap between answer scores is small. No answers flip in the high-margin group.
Critical weaknesses
  • The opposite effects on verification and delegation do not establish explanation: following the user’s choice is correct for delegation but wrong for verification. A general tendency to favor the user’s choice could therefore produce this pattern even if history changes that preference equally in both tasks. The state-changing experiments do not rule out this explanation.
  • The baseline comparison does not clearly show which parts of C-DGE provide the benefit: LoReFT and RePS use different ranks and training objectives. It is therefore unclear whether C-DGE performs better because of its specific design, greater capacity, or different training.
Questions for authors
  • Does C-DGE still perform better when the comparison method has similar capacity, the same training objective, and the same rules for deciding when to edit or leave outputs unchanged?
  • How much does history change the model’s preference for the user-selected answer in each task? Please measure this preference in the same direction for both tasks and report a paired confidence interval for the difference between the two effects.
3 comments
AuthorsOpenReview ↗

Add a Matched-Capacity Comparison and Measure History-Induced User-Choice Preference

Q1:Does C-DGE still perform better when the comparison method has similar capacity, the same training objective, and the same rules for deciding when to edit or leave outputs unchanged?

Response: We agree that the earlier baseline comparison did not isolate the effect of parameter capacity, training objective, and edit policy. To address this, we extended the evaluation across NPU and CUDA platforms: the original \CDGE final-test evaluation was conducted on NPU, and we ran a matched-capacity comparison on CUDA using the same 6,144 evaluation items. In the CUDA comparison, we aligned the training objective and the rule for when to edit or leave an output unchanged, and closely matched the parameter budgets: 25,641 for \CDGE, 24,578 for LoReFT, and 24,576 for RePS. \CDGE achieved 33.27% NGR (95% CI [32.10, 34.40]), compared with 27.89% for LoReFT (95% CI [26.98, 28.79]) and 28.42% for RePS (95% CI [27.42, 29.42]). Thus, \CDGE retained a higher NGR when capacity, training objective, and edit policy were controlled. This supports the conclusion that its performance advantage is attributable to its mechanism design rather than a larger parameter budget.

Location and change: We added a cross-platform, matched-capacity comparison to Section 5.2 under “Cross-platform matched-capacity evaluation.” The new paragraph reports the platforms, matched evaluation conditions, parameter budgets, and NGR results for \CDGE, LoReFT, and RePS.

Q2:How much does history change the model’s preference for the user-selected answer in each task? Please measure this preference in the same direction for both tasks and report a paired confidence interval for the difference between the two effects.

Response: We agree that the accuracy crossover alone does not show whether history changes the model’s preference for the user-selected answer in the same way across tasks. To directly test this, we evaluated 600 paired items across three fit seeds and defined the user-choice preference margin in the same direction for both tasks: positive values always indicate greater preference for the user-selected answer. The obedience-minus-verification history effect was +2.229+2.229 (95% CI [2.112,,2.346][2.112,,2.346]) for independent verification and −0.788-0.788 (95% CI [−0.864,,−0.710][-0.864,,-0.710]) for delegated choice. Their paired difference (delegation minus verification) was −3.017-3.017 (95% CI [−3.163,,−2.869][-3.163,,-2.869]; paired item-cluster bootstrap). The estimated history effects therefore have opposite signs, rather than being equal across tasks. This result addresses the proposed equal history-induced shift as an account of the crossover.

Location and change: We added this explanation to Section 4.2, under “H1: The crossover is task-dependent.”

Thank you for addressing my concern. I’m keeping my rating at 2 for now, but I’d appreciate clarification on why obedience history favors the user’s choice in both delegation splits in Table 3 while the new analysis finds an effect in the other direction.

Thank you for pointing out the apparent discrepancy between Table 3 and our 600-item analysis. After carefully rechecking the experiments, we found that our previous response omitted a controlled follow-up on history wording, leaving the 600-item result without important context. We have corrected this in the revised version.

The original 600-item analysis with three seed used only 8-exchange lexical-matched histories. With positive values defined as greater preference for the user-selected answer, the obedience-minus-verification history effect was +2.229 (95% CI [2.112, 2.346]) for independent verification and −0.788 (95% CI [−0.864, −0.710]) for delegated choice. Their paired difference (delegation minus verification) was −3.017 (95% CI [−3.163, −2.869]; paired item-cluster bootstrap).

To investigate why this differs from Table 3, we used the original crossover runner to compare natural and lexical-matched histories on the same 600 items. For delegated choice (one seed), lexical-matched wording produced effects of −0.770 (95% CI [−0.849, −0.690]) at 8 exchanges and −0.954 (95% CI [−1.027, −0.881]) at 32 exchanges. Natural wording produced effects of +3.213 (95% CI [+2.962, +3.472]) and +4.209 (95% CI [+3.981, +4.440]), respectively. Thus, the negative estimate in the original 600-item analysis is specific to its lexical-matched wording. At 32 exchanges, equally averaging the two wording conditions gives a delegated-choice effect of +1.628 (95% CI [+1.518, +1.743]). This is close to the positive delegation effects reported in Table 3: +1.526 for the discovery split and +1.386 for the replication split.

The new results are particularly interesting . One possibility is that more explicit authority-allocation language elicits a countervailing, reactance-like response. Separately, our state-intervention experiments show that prior-context states, including KV states, can causally affect behavior. The current experiments establish that the history wording changes the preference shift,but further research is needed to establish why this occurs and identify the underlying psychological or computational mechanism.

Location and change: We added the history-style comparison and clarified interpretation to Section 4.2, under “H1: The crossover is task-dependent.”

Reviewer_pDBSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether an interaction policy established earlier in a conversation (obey the user vs. verify independently) keeps influencing behavior after the task changes. Paired A/B forced-choice items are built from six reasoning benchmarks, with evidence, candidates and labels fixed and only the preceding history varied. On Qwen3-8B, an obedience history lowers the correct-answer margin when independent verification is required and raises it when delegated choice is required (match advantage about +2.2 to +2.4 logits); flips from correct to wrong concentrate in the low-margin band. Swapping cached state (Qwen3.5-9B) and residual/component activations (Qwen3-8B) between matched and mismatched runs rescues or induces the effect, which the authors take as causal evidence for a carried "governance state". They then propose C-DGE, a single-run activation editor (history/task router, two low-rank experts, veto gate). On a locked test it recovers 30.36% of the margin gap, rescues 32/135 induced flips, and leaves 0/2,856 protected controls changed.

Critical strengths
  1. Rigorous, well-controlled identification design. Disjoint discovery/replication splits, margin bands frozen before the prospective split, item-cluster bootstrap CIs, a one-shot locked test, identity and fixed-answer controls, and reset/length-matched histories. This makes the behavioral crossover and the rescue/induction results credible, which is the main reason I lean toward acceptance.
  2. Informative, honest baseline comparison. CAA, CAST, LoReFT and RePS close at most about 10% of the gap, and prefix isolation closes more but degrades matched cells. The paper reports this rather than hiding it, and the paired design isolates a phenomenon (task-dependent crossover of history effects) that is of real practical relevance for deployed assistants.
Critical weaknesses
  1. The "governance state" mechanism is not identified. The decisive evidence is full recurrent-state/KV exchange, which swaps the entire prefix computation including content differences, so it cannot single out a governance variable. Single-site residual patches recover little (Table 2: +0.11 at layer 19, +0.35 at layer 27, vs. about +1.5 for the joint cache swap), and the linear readout model (Eq. 3) is never tested directly. The simpler reading, that the model weighs an earlier in-context norm ("defer to the user") against a later instruction, is not ruled out; the natural-history test (n=72 per cell, six tasks) covers only the verification side. Since the paper's framing and title rest on this mechanism, the claim needs scoping or direct evidence.
  2. The repair is narrow and its headline selectivity is largely by construction. The veto returns bit-identical outputs when closed, so "0/2,856 protected rows changed" mostly reflects the gate (without it the core edits 963/2,856). Gains are modest (mismatch accuracy 74.48% to 75.68%; 24% of induced flips rescued; prefix isolation achieves higher NGR, 40.96%), evaluated only on next-token A/B decisions with templated histories and one model family. Whether it helps on free generation, OOD histories or non-Qwen models is unknown.
Questions for authors
  1. How much of the 0.457 margin recovery comes from routing/gate accuracy vs. expert corrections (oracle routing vs. learned routing)?
  2. Can Eq. 3 be tested directly, e.g., extract z_O - z_V and show that adding it (rather than full KV) reproduces the task-dependent sign?
  3. Does the crossover hold in genuine multi-turn conversations with a real task change, including the delegation side?
  4. How does C-DGE compare with a simple prompt-level fix (explicit new-task marker or summarized history) at equal cost?
2 comments

We sincerely appreciate your thoughtful questions. Your feedback has been extremely helpful in improving this paper.

Q1:How much of the 0.457 margin recovery comes from routing/gate accuracy vs. expert corrections (oracle routing vs. learned routing)?

Response: We agree that the overall margin recovery alone does not separate the contribution of routing from that of the correction experts. To quantify their respective roles, we compared learned and oracle routing while holding the experts fixed on the original 6,144 locked test rows. The learned route selected the correct direction on all mismatched rows and abstained on all matched rows. Replacing it with oracle routing changed margin recovery by only −0.00008-0.00008 (95% CI [−0.00045, 0.00028][-0.00045,\,0.00028]), while applying the expert correction under oracle routing recovered 0.4790.479 margin points (95% CI [0.462, 0.496][0.462,\,0.496]). The decomposition therefore indicates that nearly all measured margin recovery comes from the expert corrections.

This does not make the learned route redundant. It selects the task-appropriate directional expert and restricts edits to mismatched cases, while the application veto protects matched cases. The component ablations support these roles: on the 6,144-row CUDA final-test evaluation, removing structural routing reduced NGR from 33.27% to 12.54%; using a shared single expert, a positive-only expert, or a negative-only expert yielded 17.93%, 18.27%, and 15.00%, respectively. Removing the application veto left NGR unchanged, while on the development protected controls the veto reduced edits from 963/2,856 to 0/2,856. Oracle routing adds no detectable margin on the original locked test because the learned route already selects the correct direction and abstains on matched cases.

The archived original estimate is 0.457, while the paired replay estimate is 0.479 because of cross-runtime numerical drift.

Location and change: The main-text mechanism-ablation summary is in Section 5, “Relation-Conditioned Activation Repair with C-DGE,” under “Experimental Evaluation and Analysis,” immediately after the locked final-test results.

Q2:Can Eq. 3 be tested directly, e.g., extract z_O - z_V and show that adding it (rather than full KV) reproduces the task-dependent sign?

Response: We clarify that Eq. 3 is a mechanistic hypothesis that factorizes the task-dependent behavioral crossover; it is not the inference-time editing rule, nor does it identify zHz_H with a single residual vector. The latent factorization is coordinate-dependent: for any invertible transform AA, replacing zHz_H by AzHAz_H and rTr_T by A−⊤rTA^{-\top}r_T leaves the predicted margin unchanged. So the equation does not imply that zO−zVz_O-z_V is directly observable as a single raw residual difference. Given the above reasons, direct formula verification is difficult; however, following the your suggestion, we operationalized zO−zVz_O-z_V as the layer-27 boundary-state difference and tested its direct injection at the corresponding boundary and C-DGE edit site. These tests did not produce a clear, consistent repair effect. We therefore do not identify the raw residual difference as the latent coordinate in Eq. 3; the result does not rule out a distributed or projected latent representation.

In the implemented editor, Eq. 3 motivates the design principle that history information should be interpreted in light of the current task. C-DGE implements this principle operationally: it reads the history-boundary and current-task states, routes the example to one of two directional experts, and applies the learned correction at the layer-27 MLP output for the final token of the current turn. The applicability veto protects rows that should remain unchanged, and the correction is norm-clipped. We therefore present Eq. 3 as the mechanism hypothesis motivating the editor, while describing the routed, gated expert correction as the implemented editing mechanism

Q3:Does the crossover hold in genuine multi-turn conversations with a real task change, including the delegation side?

Response: We thank you for raising this question. To test whether history effects persist across an actual change of task, we added an evaluation in which the model first answered ten consecutive questions from six distinct benchmarks—ARC-Challenge, BBH, GSM8K, Math-500, MMLU-Pro, and MuSR—with each answer retained in the conversation history. We then gave it a held-out question requiring either independent verification or delegated choice. We tested both transition directions: delegation-to-verification and verification-to-delegation.

The raw model showed a history-dependent shift in both directions. Relative to matched verification history, delegation history lowered the verification-task margin by 2.178 (95% CI [−2.978, −1.380]). In the reverse transition, verification history changed the delegated-choice margin by +2.114 relative to matched delegation history (95% CI [+1.515, +2.729]). The effects therefore were not identical across the two tasks; in particular, the delegated-choice result is a positive margin shift, not evidence of an accuracy drop on that side.

We then fully retrained C-DGE on the development set for this setting. It increased the task-aligned margin by +0.406 for delegation-history-to-verification (95% CI [+0.368, +0.448]) and by +0.263 for verification-history-to-delegation (95% CI [+0.218, +0.308]). C-DGE achieved 17.61% NGR, 8.01 percentage points above RePS, the strongest external baseline (paired 95% CI for the difference [+5.91, +10.26]). These results show that history effects and C-DGE’s intervention extend to both directions of a real benchmark-task transition.

Location and change: We added this evaluation to Section 4.2, under “Empirical evaluation after ten consecutive benchmark tasks.” The new paragraph reports both transition directions, their margin effects, and the C-DGE results. Table \ref{tab:ten-turn-baselines} reports NGR and paired comparisons with the baselines.

Q4: How does C-DGE compare with a simple prompt-level fix (explicit new-task marker or summarized history) at equal cost?

We evaluated both prompt-level fixes on the same 6,144 locked-test identities used in the original main experiment. In the CUDA platform evaluation, C-DGE achieved 33.27% NGR (95% CI [32.10, 34.40]). An explicit new-task marker achieved 30.26% (95% CI [25.62, 34.87]), and summarized history achieved 32.23% (95% CI [21.09, 43.33]).

The aggregate prompt-level results conceal substantial benchmark heterogeneity. The explicit marker’s NGR ranges from 19.68% to 42.75% across the six benchmarks, while summarized history ranges from −8.83% to 62.92%. Thus, the 32.23% aggregate estimate for summarized history does not reflect a consistently effective repair: it helps substantially on some benchmarks but widens the gap on MuSR. These prompt fixes were evaluated in our deliberately controlled main setting, where task boundaries and history formats are explicit. In natural task sequences, relevant prior content and decision authority can be interleaved with the new request, making a fixed marker or summary less predictable.

C-DGE offers a task-adaptable alternative: its editor can be fitted on development examples from the target setting to learn when to intervene and the context-dependent correction to apply. In a separate, more demanding test, we first presented the model with ten consecutive benchmark tasks and then evaluated the next task across 192 held-out items. C-DGE achieved 17.61% NGR (95% CI [15.84, 19.45]), compared with 9.60% for the strongest listed baseline, RePS. This shows that C-DGE also produces measurable repair after a substantially longer sequence of task transitions. Together, these findings suggest that prompt-level fixes can be effective in a controlled setup but vary considerably by benchmark, while C-DGE can be fitted to the target interaction setting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.