acceptodds
Withdrawn from ICLR 2027

Probing and Steering Constraint Following in Multi-Turn Dialogue

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Why do language models stop satisfying an initial constraint as a dialogue grows? We construct a multi-turn evaluation set from IFEval-inspired instructions and Dolly questions: 23 constraint types, 230 sessions and 20 turns per session. Outcomes use repository-specific checks, not official IFEval scoring; some checks approximate rather than fully validate the requested constraint. Across Qwen2.5 3B, 7B, and 14B and Llama3 8B, we extract 18,400 pre-response states, excluding each current answer. Session-disjoint linear probes identify the constraint on 48.3–72.2% of subsequently checker-failed turns; all four family-adjusted intervals exceed the 4.35% uniform-label reference. This establishes decodability, not functional use. We compare concept activation vector (CAV) and instruction-directed attention steering in 108 common-cap cells totaling 496,800 response-condition evaluations. At unit strength, combined steering improves checker pass rates by 21.7–29.8 percentage points over baseline, with subadditive gains. Across all 36 model–strength combinations in the complete joint grid, adding CAV to attention both gains and loses successes; losses exceed gains in 24. Modest CAV steering adds value to weak attention, but no tested addition to stronger attention has a positive pointwise 95% interval excluding zero. Individual rescues therefore do not guarantee net joint benefit. The findings concern this exploratory evaluation protocol, not a demonstrated natural failure mechanism or semantic answer quality.

Primary area:
interpretability and explainable AI
Keywords:
large language models, multi-turn dialogue, instruction following, constraint following, linear probing, activation steering, attention intervention

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_BugaOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies whether an initial constraint remains decodable as dialogue adherence declines, and how two steering methods interact over multiple turns. It constructs 230 sessions of 20 turns covering 23 constraint types, evaluates four Qwen and Llama models, and trains session-disjoint linear probes on states extracted before each answer.

Critical strengths
  1. The primary probe avoids direct contamination from the current answer: The extraction excludes that answer, and PCA and classifier fitting use session-disjoint training folds (Section 4.2; Appendix W). This supports the narrow finding that constraint-type information is available before many checker failures, while the paper correctly avoids equating decodability with functional use.

  2. The intervention analysis examines both benefits and damage: The study reports the complete integer strength grid, random and wrong-constraint controls, and paired whole-session intervals (Section 6; Appendix I). Distinguishing subadditivity from marginal benefit, and reporting lost as well as gained successes, makes the comparison useful for assessing whether separately helpful controls combine successfully.

Critical weaknesses
  1. The task and labels do not consistently measure persistent adherence. Some prompts refer only to "the response," although every later answer is scored against the initial target (Section 4.1). Table 4 also includes constrained-choice prompts without choices scored using a 50-word limit, and JSON checks that accept arbitrary values or fenced blocks despite requests for an object. These problems affect both intervention effects and failed-turn probe subsets, and no corrected-scoring sensitivity analysis is provided (Appendix V).

  2. The interventions lack independent evaluation. Qwen’s layers and head counts were selected by maximizing checker performance on the same 230 sessions later used for evaluation (Appendix F). Session-specific steering targets and attention calibration also use these sessions' recorded trajectories, so the bootstrap intervals do not establish that the selected controls transfer to fresh conversations.

  3. Pervasive truncation limits the meaning of the gains. Between 62.35% and 80.20% of baseline answers reach the 256-token cap, while steering substantially changes stopping rates (Section 6.5). Without a larger-budget comparison and an answer-quality assessment, gains on JSON, closing quotations, and ending phrases may partly reflect whether responses finish, leaving their practical value unresolved.

Questions for authors
  1. How many instruction variants explicitly require adherence on every turn, and do the main findings persist on that subset with faithful checks and malformed constrained-choice cases excluded?
  2. Do the intervention gains and joint-interaction patterns replicate on untouched sessions, with steering sites and calibration chosen using separate data?
  3. Do these findings persist with a response budget that allows most answers to finish, and do the rescued answers still answer the content questions correctly?
Reviewer_gCL3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies why LLMs fail to comply with a constraint given in the initial instruction as the conversation grows. To study this question, the paper introduces an evaluation dataset that adapts 23 constraint types from IFEval and pairs them with questions from Dolly. Using linear probes, the paper shows that even when the LLMs fail to comply with the constraint, the information about which constraint was given is still detectable from the model's hidden state before it answers. The paper then tries two intervention methods, steering either the hidden states or the attention to the instruction, on four LLMs, and shows that each intervention improves the compliance rate. Activating both interventions at the same time, however, leads to less than the sum of their individual gains.

Critical strengths
  • The studied problem - LLMs fail to comply with instruction constraints - is crucial and well motivated
  • The paper provides insights into model behaviors - the constraints remain decodable from the hidden state even when failing to comply with the constraints.
Critical weaknesses
  • The paper has major presentation and clarity issues, making it hard to understand the true contribution and results. For example, in Section 3 (Theoretical Motivation), the paper does not explicitly explain what the notations (e.g., n, s, t) used in the equations represent. In the same section, the paper describes " μt\mu_t is the unit-normalized per-constraint mean cached residual", a long phrase with many modifiers that is hard to parse, and the details of how to compute it are still unclear. In Section 4 (Dataset and Experimental Setup), it is unclear where the ten variants per constraint type come from. These are only some examples. I believe the paper requires a substantial revision of the writing for its contributions to be properly assessed.
  • The designed checker, used in the paper to check if a response complies the constraint, is also questionable. For example, in Table 4, for "Format: constrained response", the paper mentions "At most 50 whitespace words. No choice-membership test; the instruction templates do not supply answer options. This is a proxy mismatch, not valid constrained-choice verification.". This rule (check for 50 whitespaces words) seems to be really loose for the corresponding constraint.
Questions for authors

Besides the points in Weaknesses, I have the following questions:

  • In the CAV extraction, can you justify using "Wikipedia neutral text + filler" as the "without" baseline? For example, could the Wikipedia text shift the representation toward something else, rather than serving as a "null" (no-instruction) version of the input? Also, how was the Wikipedia text selected?
  • Giving the constraint only at turn 1 may lead to two interpretations: whether the constraint should apply to every turn or only to the current turn. What happens if all instructions are changed to explicitly state that the constraint applies to all turns?
  • In the paper’s setup, 20 unrelated questions are used as the 20 turns in each conversation, which may not reflect realistic multi-turn use. Would the setting be more valid if 20 related questions (e.g., follow-ups on the same topic) were used?
Reviewer_as7sOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates why models fall off over long contexts in following constraints for tasks. Specifically, they analyze this by taking data samples from the IFEval and Dolly datasets and using constraint types such as casing (e.g. upper vs lower), format, keywords, length, etc., and using activation probes to measure if representations of the constraints are still present at each step of the conversation as they add many turns of unrelated QA in the context (instruction given at turn 1 and then 19 turns of unrelated questions). They test this over 4 models (3 Qwen models and 1 Llama model) and compliance to constraints is deterministically checked via a rule-based checker. Findings-wise, they find via probing that the information is still present in the model's residual stream representations, but the model just ignores it. For steering, they find that if you steer this representation of instruction-constraint following, it can boost compliance.

Critical strengths

The problem is timely and important to analyze and understand. Designing a mechanistic mitigation for this would be interesting

Critical weaknesses
  • The probe results are by construction, and I don’t understand what the purpose of those experiments is supposed to show or how it is a contribution. Although the probe can get good accuracy in reading the representation from the residual stream, this is by construction, since the instruction text for the constraint is literally still in the context, so of course the constraint would still be readable from the residual stream with probes. This decodability seems trivial.
  • The authors themselves acknowledge that the instruction remains visible, so decodability doesn’t show the model actually uses this information. That leaves the main probing result showing little beyond the instruction being in context. Moving the probe to pre-response only removes leakage from the current answer, not the instruction itself, so the result is still near-trivial.
  • The probing and steering halves of the paper are disconnected. The steering direction is not the probe direction, and for Llama and Qwen 14B the steering layers are not even the probe layers. So it’s unclear how the probing results inform or motivate the interventions.
  • The steering method seems to be just a naive application of residual stream activation steering on a constraint direction, and the attention steering is essentially PASTA. The authors frame their contribution as the joint behaviour of the two methods. However, two interventions pushing toward the same goal being subadditive, and strong steering breaking the model, seem expected rather than surprising or interesting.
  • Simply restating the instruction at every turn beats CAV steering and the combined steering policy at unit strength in every model, and beats every steering setting for Llama. Only high-strength attention steering alone outperforms it in the Qwen models, which further undercuts the case for the combined steering approach.
  • The compliance checker is a weak proxy for several constraints. For example, “constrained response” is scored only as being 50 words or fewer, and the JSON check accepts any JSON value. Every aggregate result depends on these checks, and no corrected-checker sensitivity analysis was run.
  • The steering layers and heads for Qwen were selected by compliance sweeps on the same 230 sessions used for evaluation, and Llama’s layers were not swept at all. The intervention results are therefore in-sample with respect to site choice.
  • The wrong-constraint direction recovers much of the CAV gain (+9 to +17 pp), and the CAV-minus-wrong difference is not significant for Llama. This suggests the steering effect is largely generic rather than constraint-specific.
  • Steering changes output length, truncation and repetition substantially (e.g. duplicate-trigram rates up to ~68–80% at high CAV strength). This confounds the compliance gains, especially for length- and format-based checks.
  • All results use a single seed with greedy decoding on one previously examined session bank, so nothing here is independently confirmed.
  • Writing is highly unclear. There are constant “X, not Y” sentence structures, e.g. “This establishes decodability, not functional use.” “This motivates a control, not a verified decay law.” “This is a policy test, not identified mediation.” “Exact accounting, not mediation.” There is lots of hedging that makes it hard to understand what claims are being made and what is supported vs not supported.
  • Overall, the organization of the paper and results seem highly exploratory, and it’s unclear what the contributions or claims are from the authors.
Questions for authors

refer to weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.