acceptodds
Under review as a conference paper at ICLR 2027

Statecraft: Revoking Accumulated Instructions by Appending a System Message

Abstract

Agents now run long sessions, and the application around them often changes their instructions midway: it switches the reply language, lifts the rule to ask before each file edit, or lowers the reasoning effort. Editing the system prompt for each change throws away the prefix cache, while appending only the change leaves the earlier instructions in the context, and no message lists all of those still in force. We propose STATECRAFT, an append-only control plane for these instructions: after each change, the application appends the full, versioned list of current instructions, so the latest list alone says what applies. As in any revised list of instructions, one is revoked simply by leaving it off. This keeps the cache but asks the model for something that ordinary instruction following does not: to stop following an instruction that is no longer listed. We build Statecraft-200 to test this, with multi-turn, tool-using tasks that check both that the current instructions are followed and that revoked ones are no longer followed. Frontier models such as Claude Sonnet 5 and Opus 5, DeepSeek V4 Pro, Kimi K3 and GLM 5.3 still follow a revoked instruction in 32 to 65% of later checks, even though the prompt states that anything left off the list no longer applies. We therefore train Qwen3.8-27B with DPO on pairs of its own answers to the same conversation, chosen by an automatic check that rewards following the latest list and penalizes following a revoked instruction. Training removes 81% of its failures on Statecraft-200 overall, enough to lift its overall score past every frontier model we test, and raises its success on the tasks where the instructions change most often from 52% to 82%. The gains hold in unseen workspaces, on kinds of instruction held out of training, after long real conversations and on conversations from the external SEQUOR benchmark whose constraints arrive as snapshots, and scores on IFBench, GSM8K and Multi-IF do not fall.

Primary area:
reinforcement learning
Keywords:
behavioral control plane, instruction following, instruction revocation, preference optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9Z4DOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Nowadays, the agents work through long sessions with instructions that change midway, asking them to change their reasoning, ask for permission, and so on. The paper discusses that in order to apply this change, the first natural solution is the system prompt, but since the game prefix chase can't be used, changing the system prompt forces the whole conversation to be recomputed. Instead, a API can accept a system message in the middle of the conversation while keeping the case by only appending the changes. However, blind appending results in a long list of interfering instructions that the agent needs to infer what they all mean together. The contributions of this paper are as follows: 1) Providing a method for appending such that revokes some interaction and determines which is in force without removing the cache. 2) A benchmark that grades what a model stopped doing. 3) A set of instructions to make the protocol reliable.

Critical strengths

The paper is very well written. The downsides of changing the system instruction and appending method is very well discussed in both the introduction and related work. The problem definition is well motivated. The experiments are fully design to provide insights about the claims of the paper and the results are fully explained

Critical weaknesses

I do appreciate the authors acknowledging the limitation on using only Qwen3.8-27B. I do believe that it would be interesting to see the performance on the smaller models as well as other family of models such as Gemma family. Although the paper talks in detail about the implemented method, there is little justification on some design choices such as why DPO was used.

Questions for authors

As mentioned in the weaknesses, what do you expect would be the performance on smaller/other families of LLMs? justify design choices of the method such as DPO and finally, I would like to see some prints on the snapshots during training and how some of the instructions are revoked.

2 comments

Thank you for your thoughtful review and positive assessment of our problem formulation, protocol design, and experimental analysis. Revisions are highlighted in blue in the updated manuscript, and our code and benchmark are available at https://anonymous.4open.science/r/Statecraft-Review-7C4D/. Below we address each of your questions in order.

1. Smaller models and other model families, including Gemma (Section 6.5, Table 5; Appendix E.4)

We applied the same self-sampled LoRA DPO recipe, sampling preference pairs from each model's own responses under its native chat template, to three additional open-weight models across families and parameter scales:

ModelFamilyBaseDPOSingle Revocation
gpt-oss-20bGPT-OSS46.2%58.3%44.4% -> 83.3%
Qwen3-8BQwen 330.0%36.1%25.9% -> 46.3%
Gemma-4-E4BGemma 427.8%35.8%11.1% -> 27.8%

Self-sampled DPO improves overall accuracy and roughly doubles single-revocation compliance across all three models, confirming that the approach transfers beyond the Qwen family. Their post-DPO ceilings are lower than on Qwen3.8-27B (80.6% -> 96.4%) because 4B–20B base models start much weaker on multi-turn tool use and multi-key compliance even before any instruction changes. Since DPO teaches which snapshot applies on a turn rather than adding general tool-use ability, weaker base capability both limits the task ceiling and produces fewer passing rollouts (R=1) per prefix on multi-version sequences, which is also why gains on single revocations are much larger than on the full benchmark (Section 6.5, Appendix E.4). While academic GPU limits prevented training models above 27B, stronger base models produce more passing rollouts and should benefit even more clearly (Section 7).

2. Why offline DPO over SFT and online RL (Section 5, Section 6.2, Appendix E.1)

When an instruction is revoked by leaving it out of the new snapshot, earlier assistant replies in the prompt still show the old behavior, so the model has to stop doing something it was doing in previous turns (Section 5). SFT trains only on passing responses, which shows a valid next reply but applies no direct penalty against continuing the revoked behavior. DPO instead pairs a passing response and a failing response generated from the same conversation prefix and workspace state, directly penalizing the old behavior when it should stop. Compared with online RL, offline DPO also needs no live rollouts during training, while matching online GRPO on our held-out evaluation matrix at much lower compute cost (Section 6.2, Appendix E.1).

To test SFT against DPO head-to-head at full data scale, we trained both methods under identical settings on the full preference dataset (Section 6.2, Appendix E.1). DPO outperforms SFT by nearly 11 points on Statecraft-200 (96.2% vs. 85.2%). Both methods learn single-step revocations, but on 4–6-version sequences SFT drops to 43.3% whereas DPO reaches 81.1%. Even starting from the trained SFT checkpoint, a short DPO continuation lifts accuracy to 92.7%, compared with 86.1% for continued SFT.

3. Training <snapshot> examples and revoking a subset of instructions (Section 5, Appendix C.1)

Section 5 (An example training pair) shows the snapshots from the training pair illustrated in Figure 3(a):

<snapshot>{"state": {"message_header": "The first characters of each of your replies must be [ACK]."}, "version": 1}</snapshot>
... (three completed tasks in history, each answered with [ACK]) ...
<snapshot>{"state": {}, "version": 4}</snapshot>

Version 4 omits message_header, revoking the prefix rule while three [ACK] replies remain in the conversation history. From the same post-revocation state, the chosen response answers directly, whereas the rejected response still begins with [ACK].

When multiple instructions are active and only some are revoked, the new snapshot keeps the remaining active keys and omits the revoked ones (Equation 1). For example, if version 2 sets both reply_language and message_header and version 5 retains only reply_language, then message_header is revoked while reply_language stays in force. Additional multi-key and forged-snapshot examples appear in Appendix C.1 and training_example.json in the repository.

I thank the authors for their rebuttal which clarified my questions. I will discuss this further with other reviewers with higher confidence to decide on increasing the score.

Reviewer_6HGSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes to address the instruction revocation problem in the long-running LLM agents, where a model may continue following outdated instructions. The proposed method Statecraft enables LLM agents to revoke outdated instructions by versioned system snpashots and preserve the prefix caching at the same time. Furthermore, a verifier-guided DPO improves the instruction following accuracy from 80.6% to 96.4%.

Critical strengths
  1. The practical motivation is the most salient strength of the paper. The following of outdated instructions is a crucial practical problem that deserves research attention.
  2. The empirical results are significant, and overall benchmark accuracy improves from 80.6% to 96.4% while preserve 95-98% of the cached prefix.
  3. The ablation studies are also meaningful.
Critical weaknesses
  1. the model coverage is limited, and only trained and evaludated at Qwen3.8-27B model.
  2. limited evaluations. Most of the experiments rely on simulated workspaces, limiting real-world applicability.
  3. the proposed append-only method will improve the context cost.
Questions for authors
  1. because each update will append a full snapshot, how does the method prevent unlimted context windwo growth?
  2. Missing reference:

Li, M., Chen, P., Wang, C., Zhao, H., Liang, Y., Hou, Y., ... & Zhou, T. (2025, July). Mosaic-IT: Cost-free compositional data synthesis for instruction tuning. In Findings of the Association for Computational Linguistics: ACL 2025 (pp. 25287-25318).

Jia, Q., Shen, Y., Song, X., Zhang, K., Wang, S., Pei, D., ... & Zhai, G. (2026, July). One battle after another: Probing llms’ limits on multi-turn instruction following with a benchmark evolving framework. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 9574-9590).

2 comments

Thank you for your constructive feedback on our motivation, prefix-cache reuse, and ablations. Revisions are highlighted in blue in the updated PDF, and our code and benchmark are at https://anonymous.4open.science/r/Statecraft-Review-7C4D/. Below we address each concern in order.

1. Model coverage across families and scales (Section 6.5, Table 5; Appendix E.4)

Beyond the seven frontier API models in Table 1, we trained three open-weight models spanning distinct families and scales on pairs self-sampled from each model's own responses:

ModelFamilyBaseDPOSingle Revocation
gpt-oss-20bGPT-OSS46.2%58.3%44.4% -> 83.3%
Qwen3-8BQwen 330.0%36.1%25.9% -> 46.3%
Gemma-4-E4BGemma 427.8%35.8%11.1% -> 27.8%

All three models improve in overall accuracy and roughly double single-revocation compliance, showing that the method transfers across families. Post-DPO ceilings remain below Qwen3.8-27B (80.6% -> 96.4%) because 4B–20B base models are weaker at multi-turn tool use and multi-key compliance before any state change, which both caps overall task success and yields fewer passing rollouts (R=1) per prefix on multi-version sequences, explaining why single-revocation gains are much larger than overall gains (Section 6.5, Appendix E.4).

2. Real-world multi-file repository evaluation (Section 6.5, Table 4; Appendix E.3)

Statecraft-200 uses self-contained workspaces to isolate instruction drift from open-ended debugging errors (Section 4), alongside real WildChat and SEQUOR dialogues (Table 1, Appendix E.2). To test real repositories directly, we added a three-stage workflow on 94 held-out issues across nine Python repositories in SWE-bench Verified (Section 6.5, Table 4; Appendix E.3).

Inside clean containers with bash and file tools, Stage 1 sets a snapshot requiring approval before any file edit, Stage 2 keeps that rule active on a follow-up turn, and Stage 3 appends a higher-version snapshot omitting the rule so the agent can patch the codebase and run upstream tests:

CheckpointRepair (/94)Two-Stage Approval (/94)Joint (/94)Edit Violations
SFT (220 steps)311067
DPO (220 steps)2819131
DPO + first-edit mix (+50 steps)3721151

Once editing is unlocked, DPO resolves a similar number of repository bugs to SFT while nearly doubling two-stage approval compliance, reducing unauthorized edits from 7 issues to 1, and more than doubling joint completions. Continuing DPO with single-step edit examples from disjoint projects further raises repair accuracy while maintaining high approval compliance.

3. Context overhead and preventing unbounded window growth (Section 3, Appendix A.2, Section 7)

Section 3 (Context overhead and compaction) and Appendix A.2 examine per-turn overhead and long-session scaling. First, a snapshot is appended only when standing instructions change, so ordinary turns append nothing and add no tokens. Even when instructions do change, a snapshot only lists the user's active instructions and remains small in practice (depending on user settings, a median of 40 tokens in Statecraft-200, or under 3% of the final prompt).

Second, in long sessions, snapshots do not accumulate indefinitely. When dialogue and tool outputs fill the context window and trigger standard context compaction (which already rebuilds the prompt and resets the KV cache), the host keeps only the latest snapshot in the compacted context and drops earlier ones. Since a trained model follows the latest snapshot by default, dropping older snapshots at compaction keeps snapshot overhead bounded without hurting accuracy, while retaining them between compactions preserves prefix-cache reuse (Appendix A.2). STATECRAFT-27B also transfers well to multi-thousand-token WildChat dialogues beyond its training length (Table 1), and Section 7 discusses scaling to larger models and longer windows.

4. Missing references (Section 2)

Thank you for pointing out both papers. Section 2 now cites and contrasts EvolIF (Jia et al., ACL 2026), where instruction additions, updates, and deletions are stated explicitly in user turns rather than revoked by omission in system snapshots, and Mosaic-IT (Li et al., ACL Findings 2025), which composes static multi-constraint tuning data rather than dynamic mid-session transitions.

Together, these results support our core takeaway: appending full-state snapshots (keeping only the latest one at compaction) lets agents update or revoke rules without breaking the prefix cache, and same-prefix offline DPO teaches models across families to stop following revoked instructions reliably, including on real repositories.

Dear Reviewer 6HGS,

Thank you again for your time and constructive feedback on our submission. As the author–reviewer discussion deadline (Nov 18 AoE) approaches, we wanted to gently check in to see whether our rebuttal and revised manuscript have addressed your concerns and might impact your rating.

Please let us know if any question remains or if there is anything else we can clarify before the discussion period closes. We would greatly appreciate your feedback!

Reviewer_6rNeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

In long-running agent sessions applications often need to change standing instructions (e.g., reasoning effort, reply language, approval policies) mid-conversation without re-writing the whole system prompt, since that would invalidate the KV/prefix cache. The authors propose STATECRAFT, a protocol in which the application appends a new, fully-versioned snapshot of all currently active instructions after each change. In statecraft a instruction is revoked simply by omitting it from the newest snapshot, rather than issuing an explicit cancellation. To measure this, the authors build Statecraft-200, a benchmark of multi-turn, tool-using episodes with deterministic, code-based checks for whether revoked and changed instructions are actually dropped and current ones are followed. They alsp fine-tune Qwen3.8-27B with DPO on self-sampled response pairs, scored automatically by the same checks, and show this removes most of the model's own failures.

Critical strengths

[+] The benchmark itself is the paper's most valuable contribution. It isolates a failure mode, revocation by omission that existing IF benchmarks (Multi-IF, SEQUOR, IFEval) do not test, since those grade instructions that are added or explicitly replaced, not instructions that must be stopped because they were silently dropped from a restated list. Showing that every frontier model tested fails this at a high rate (32–65%), even when the protocol states explicitly that omission means revocation is useful for the community.

[+] The verification methodology is comprehensive. Checks are deterministic and derived from tool-call reproducible rules rather than LLM judges in the main results. The authors cross-validate with a second independent implementation.

Critical weaknesses

[-] Data and benchmark materials are not available with the submission. The reproducibility statement says code, checks, the will be released "upon acceptance" but none of it is provided for review. For a paper whose central claims rest entirely on a newly constructed benchmark and a newly constructed training pipeline, this is a significant obstacle to reviewing. It is not possible to inspect the actual episodes, verify that the deterministic checks are not gameable or miscalibrated, check for leakage between the training distribution.

[-] Its episodes take place in three simulated projects, an inventory tracker, a rainfall log and a link shortener. Given we don't know much about the complexity of these workspaces - I'm not sure how much the models were "put to test".

Questions for authors

Why not use the https://anonymous.4open.science/ or any of the open-source, free, and well accepted sources where you can respect the double blind policy while sharing datasets, so we can review/analyze the evals data - and confirm training is sufficiently deduped?

2 comments

Thank you for your detailed review and for highlighting Statecraft-200's isolation of revocation by omission across frontier models. We agree that benchmark tasks, verifiers, and deduplication checks should be inspectable during review, and have released our anonymous repository alongside blue-highlighted clarifications and new experiments in the updated PDF.

1. Anonymous code and benchmark release (Section 4, Reproducibility statement)

We have released the full Statecraft-200 benchmark, workspace environments, dual checker implementations, training and evaluation code, and replay scripts at https://anonymous.4open.science/r/Statecraft-Review-7C4D/ (linked in Section 4, footnote 1, and the Reproducibility statement), allowing both our aggregate tables and individual tool-using episodes to be inspected and rerun directly.

2. Why deterministic checks are neither gameable nor miscalibrated (Section 4, Appendix B.1)

Section 4 (Decision points) explains how our verifiers guard against superficial patterns or verbal claims. A decision point passes only when all currently active controls hold, behaviors from revoked or forged user-text snapshots are absent, and default behavior resumes once a rule is revoked (Appendix B.1).

For action-level rules such as file-edit approval, checkers grade executed tool calls and the resulting workspace state rather than assistant text: claiming an edit without calling write_file fails, calling write_file while approval is active fails, and our audit of passing file-edit turns confirms that they modify only the requested target file and content while leaving the rest of the workspace unchanged. Cross-checking against an independent second checker implementation and a manual audit (Appendix B.1, with no false passes or false fails observed) further supports checker reliability.

3. Train–evaluation deduplication (Section 6.2, Appendix B.2, Appendix E.1)

Training and evaluation use disjoint workspaces, held-out control types, and conversation histories, with no prompt overlap against our validation set or WildChat dialogues (Appendix B.2, Appendix E.1). Checking individual user turns across Statecraft-200 found only five generic requests (such as asking what a project does from its README) that also occurred in training dialogues on different workspaces. To check whether these generic requests affected our results, we removed all training pairs containing a matching turn and retrained from scratch: Statecraft-200 accuracy remained essentially unchanged relative to removing the same number of random pairs (93.9% vs. 94.1%; Section 6.2, Appendix E.1), showing that the gains come from learning the snapshot protocol rather than prompt overlap.

4. Workspace design and real-world repositories (Section 4, Section 6.5, Appendix E.3)

In Statecraft-200, each workspace combines code, data, notes, and policy documents under deterministic tools to isolate instruction transitions from open-ended coding failures (Section 4); even in this controlled setting, frontier API models fail most 4–6-version sequences (Table 1).

To evaluate real multi-file codebases directly, we added a three-stage workflow across 94 held-out issues from nine Python repositories in SWE-bench Verified, running inside clean containers with bash, file tools, and upstream test suites (Section 6.5, Table 4; Appendix E.3). Stage 1 sets an edit-approval requirement via snapshot, Stage 2 retains it on a follow-up turn, and Stage 3 revokes it by omission so the agent can patch the repository and run tests. Once editing is unlocked, our DPO model resolves 37 of the 94 real bugs while maintaining two-stage approval compliance on 21 issues and making only 1 unauthorized edit attempt across all 94 issues.

Together, these checks, deduplication runs, and SWE-bench experiments confirm that our gains come from learning the protocol rather than prompt overlap or toy setups: current models keep following revoked instructions when earlier turns still show the old behavior, and pairing passing and failing responses from the same prefix under offline DPO makes append-only, cache-preserving instruction revocation reliable in real coding workflows.

Thank you for the detailed review. I have looked at the repository and given my analysis I have updated my score.

I would also strongly recommend the reviewers to include examples (ie snippets), from both https://anonymous.4open.science/r/Statecraft-Review-7C4D/check_records.jsonl and https://anonymous.4open.science/r/Statecraft-Review-7C4D/data/bench/statecraft200.v1.json in the paper to demonstrate the diversity of tasks (language, format, code, etc).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.