acceptodds
Under review as a conference paper at ICLR 2027

Lost with a Map: Conversational State and Behavioral Reliability in Language Models

66 views

Abstract

Task-oriented dialogue requires maintaining and updating information across turns, yet language models expose no explicit belief-state object. We study how conversational state is represented, updated, and used inside eight instruction-tuned language models from four families on MultiWOZ and SGD. Structure and values separate: which domains, slots, and requests are active is linearly readable just before the model acts, whereas exact values are far more readable where the user stated them. After a user changes a value, both values remain accessible at their mentions, and causal interventions show that both continue to influence the model's action. In natural closed-loop interaction, query failures separate into cases of weak structural support, incorrect value resolution, and failure to deploy otherwise-supported constraints, with targeted interventions producing systematically different repair behavior across these cases. These findings motivate a state–action controller that starts from the base model action and selectively edits it using structural readouts, without requiring a complete predicted belief state as an intermediate representation. On held-out MultiWOZ interaction across five models, it raises the base model exact-query accuracy from .318 to .621 and task success from .272 to .371 at negligible added cost. Overall, reliable interaction requires not only retaining conversational information, but resolving which available constraints currently apply and ensuring that they govern action.

Primary area:
interpretability and explainable AI
Keywords:
conversational AI, conversational state, language models, interpretability, multi-turn dialogue, dialogue state tracking, task-oriented dialogue, multi-turn interaction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RYH3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how language models retain, update, and use conversational state when generating executable database queries directly from dialogue. Analyses across eight models compare state readouts at different positions, intervene on historical sources through blocking and counterfactual substitution, and classify natural omissions using structural signals and conditional value preferences. The resulting State–Action Controller, SAC, edits an existing query using lightweight structural readouts and explicit user-text evidence. In the five-model, single-executable-domain MultiWOZ closed-loop evaluation, mean exact-query accuracy increases from .318 to .621 and task success from .272 to .371. Task success is close to that of T5-DST trained on the same decision examples, while SAC better preserves correct drafts and requires no additional language-model generation calls.

Critical strengths
  • The distinction between readable information, causal influence, and behavioral competence is a substantial strength. Larger models can select current values correctly on examples where the binding probe performs poorly. Retaining these results prevents the representation analysis from treating a weak readout as proof that the model lacks the relevant ability.
  • The source interventions have informative controls. Matched dialogues preserve source content and position while changing validity, and the blocking and substitution studies include length-matched non-value spans, unrelated content, and wrong-position controls. Fixed-orientation probability contrasts and the natural stale-error analysis jointly support the claim that historical sources affect action and can remain influential after supersession.
  • The controller evaluation measures both benefit and harm. Fixed-history ablations preserve the resolver, safeguards, and compiler while changing the selection signal. Closed-loop results then report fidelity, success, preservation, repair, and destructive edits. This provides a more convincing account of selective correction than an aggregate success rate alone.
  • The main interactive comparison handles trajectory divergence carefully. Methods are paired by initial goal, while query statistics use the turns actually encountered by each method. Human-history replay and naturalized simulator language add useful coverage, with clear qualifications about single-domain execution and reliance on explicit user evidence.
Critical weaknesses
  • The binary correctness contrast in Eq. (1) is given a stronger source-influence interpretation than it supports. The correct candidate reverses between the matched conditions. An identical intervention-induced change in fixed-candidate preference can therefore hurt correctness in one condition and improve it in the other, producing a positive contrast without a difference in source-effect strength. The statistic remains useful for comparing correctness effects. The fixed-orientation graded analysis in Table 13 supports the stronger validity claim, and the paper should distinguish these interpretations.
  • The temporal presentation can suggest within-event state decay despite changing eligible cohorts. The number of events falls from 727 to 334 across lags, and later restatements remain available. Figure 2b consequently describes dependence on the original sources in each lag's eligible population; it does not isolate decay in total current-state dependence within a fixed cohort. The appendix already recognizes this limitation, which should also appear in the main text and caption.
  • Table 23 resamples omissions even though a goal can contribute several turns and slots. Its intervals may therefore understate uncertainty over goals, particularly for the small binding groups. The point-estimate differences remain useful, but the interval interpretation should match the sampling unit. This concern does not apply in the same way to the goal-level main closed-loop analysis.
  • The T5 memory figure needs a clear accounting definition. The reported 60,506,624 float32 parameters require approximately 242 MB for weights alone if resident on the measured GPU, whereas Table 28 lists 40 MB beyond the resident backbone. Whether weights were already in the baseline, excluded, or placed elsewhere materially changes the comparison. This issue does not negate SAC's lack of extra generation calls or its reported low latency.
Questions for authors
  • Is the loss in Eq. (1) pre-intervention minus post-intervention correctness? How does the interpretation separate reversal of the correct label from a change in fixed-candidate source-effect strength, and should Table 13 be the primary evidence for the latter?
  • What is the implemented resampling unit for Table 23: individual omissions, initial goals, or dialogues? Please clarify the population those intervals are intended to describe.
  • What T5 objects are included in the 40 MB entry in Table 28? Please specify model residency before measurement, parameter placement, and the peak-reset baseline so that the figure can be reconciled with the parameter count and precision.
1 comment
AuthorsOpenReview ↗

Response to Reviewer RYH3

We thank the reviewer for the review. All four points are addressed below.

W1 and Q1. Yes. As implemented, Lac=Y(H)−Y(Ha)L^{c}_{a} = Y(H) - Y(H_a), the pre- minus post-intervention value of Y=1[the greedy query emits the currently valid value]Y = \mathbb{1}[\text{the greedy query emits the currently valid value}]. We will replace "favors" with "emits" in §4.2. The reviewer is correct that because the valid value changes between twins, this contrast can be positive without a difference in source-effect strength. Table 13 is the evidence for the stronger claim. Recomputed with the fixed outcome Y′=1[the query emits v1]Y' = \mathbb{1}[\text{the query emits } v_1] in both twins, the adjusted effect remains positive and significant on 7 of 8 models (macro +.558+.558 vs +.582+.582). The exception is OLMo-7B-IT, which emits v1v_1 in only 3.7% of current twins. Masking the v1v_1 source removes v1v_1 in .52–.86 of current twins and at most .061 of superseded twins, while the random mask changes YY by at most .005, so the superseded term sits near a floor. We will make the graded Table 13 analysis the primary evidence for validity-dependent source influence in §4.2, interpret the original binary contrast only as a correctness effect and report the fixed-candidate binary analysis separately.

W2. Agreed, and we have run the fixed-cohort analysis. On the 333 events present at every lag, the new-source advantage DD declines from lag 1 to lag 4 (Qwen +1.87→−0.37+1.87 \to -0.37, Llama −0.59→−1.75-0.59 \to -1.75). We then split the events by whether vnewv_{new} is restated between the update and the lag. Where it is not restated, DD stays positive on Qwen (+5.21→+2.57+5.21 \to +2.57) and is not distinguishable from zero on Llama (+1.04→+0.04+1.04 \to +0.04). Where it is restated, DD is negative throughout. Restatement status therefore materially affects the original-source contrast, but the contrast declines even without restatements. These analyses measure the relative influence of the original mentions. They do not establish decay in total current-state dependence or a return to the old value, and we will not claim either. We will state in §4.3 and the Figure 2b caption that the curve describes dependence on the original mentions within each lag's eligible population, and add the fixed-cohort and restatement results to the main text.

W3 and Q2. The reviewer is right. The implemented unit in Table 23 was individual omissions, and a goal can contribute several turns and slots, so the intervals understate uncertainty over goals. We recomputed them with resampling by initial goal on all seven checkpoints. The goal-level intervals are about 1.3×1.3\times wider for Binding and 2.0×2.0\times wider for Deployment in the pooled rows (per-checkpoint factors range from 1.0×1.0\times to 2.2×2.2\times). Binding and Deployment intervals still do not overlap on six of the seven checkpoints or in the pooled row. They overlap only on OLMo-32B-IT, whose Binding cell has 5 omissions from 3 goals. The goal-level interval for the Deployment−Binding difference itself is +.702+.702 [+.623,+.771][+.623, +.771] pooled, and it excludes zero on six of the seven checkpoints, again with OLMo-32B-IT as the exception. We will report goal-level intervals in the revision, state the resampling unit in the caption, mark Binding cells with fewer than ten omissions as descriptive, and keep the omission-level intervals in the appendix for comparison.

W4 and Q3. The reviewer’s arithmetic is right. T5-small has 60,506,624 float32 parameters, which occupy approximately 242 MB. These weights were resident on the GPU before the peak-reset baseline. The entry reported as “40 MB” in Table 28 corresponds to 40.14 MiB (approximately 42 MB) of peak additional call-time allocations, excluding the resident model weights. We re-measured with the baseline taken before T5 is loaded, under the Table 28 protocol. Peak additional GPU memory is approximately 284 MB on both backbones, comprising 242 MB of resident weights and 42 MB of call-time allocations, compared with 8 MB for the controller. We will replace the Table 28 entry with this accounting and state the residency convention and units in the caption. The latency and extra-call comparisons are unchanged.

We hope these address the concerns and welcome any that remain.

Reviewer_aXASOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how instruction-tuned language models represent and use conversational state without an explicit belief-state object. Across eight models on MultiWOZ and SGD, the authors find that structural information is accessible near the decision site, whereas exact values are more readily decoded at their historical mentions. Source interventions examine how current and superseded information influences subsequent actions. These findings motivate a State–Action Controller (SAC), which combines structural readouts with evidence from visible user utterances to selectively correct executable queries. The controller improves query fidelity and task success while keeping the underlying language model frozen.

Critical strengths
  1. The empirical investigation is broad and carefully controlled. The representation study covers eight models from four families and two dialogue datasets. Dialogue-disjoint splits, matched evaluation populations, rank-matched probe controls, and dialogue-level bootstrap intervals strengthen the evidence that the reported accessibility patterns are consistent across the tested settings.
  2. The paper makes a useful distinction between accessibility and behavioral use. It goes beyond probing accuracy through source masking, matched substitutions, and interventions on naturally occurring errors. The authors also acknowledge that weak binding readouts do not imply absent model competence and report nonlinear probes and slot-key-conditioned value recovery. These qualifications make the interpretation more credible.
  3. The controller produces substantial practical improvements. In Table 1, SAC-P raises mean exact-query accuracy from .318 to .621 and task success from .272 to .371. Although its task success is similar to T5-DST (.372), SAC-P achieves higher exact-query accuracy (.621 versus .549), better preservation of correct drafts (.965 versus .654), and fewer destructive corrections (.074 versus .650 per goal). These are meaningful advantages for a system intended to selectively repair an existing model’s actions.
  4. The efficiency, ablations, and robustness checks support the controller’s practical value. SAC adds no generation call and approximately 5–7 ms per decision in the two measured backbone configurations. Appendix E.2 includes resolver-only, observable-text, shuffled-readout, and rate-matched controls, while the naturalized-user and human-authored-dialogue experiments provide evidence beyond the primary template-based simulator.
Critical weaknesses
  1. The specific benefit of accessing the acting model’s internal representations remains insufficiently isolated. The selector ablations show that the learned readout outperforms the tested rules and randomized alternatives. However, the observable-text selector in Appendix E.2 is a deterministic heuristic, rather than a supervised slot-presence classifier trained on the same examples and labels. T5-DST also predicts complete slot–value states and uses a different query-construction interface. Consequently, the comparisons do not establish whether a trained text-based or external-encoder classifier, inserted into the same SAC editing pipeline, could obtain similar fidelity and preservation. This matters to the claimed connection between internal-state analysis and behavioral improvement.
  2. The binary validity statistic does not by itself establish validity-dependent source influence. In Equation (1), (Y) measures preference for the currently valid value, whose identity changes between the current and superseded twins. Consider a model that ignores the update cue, prefers (v_1) in both twins, and switches to (v_2) after the target source is masked, with random masking having no effect. Masking then decreases correctness in the current twin and increases it in the superseded twin, producing a positive (D_{\mathrm{validity}}) despite identical source dependence in both conditions. The binary contrast therefore needs a fixed candidate orientation or a narrower interpretation. This concern does not invalidate the graded analysis in Table 13, which consistently measures support for the same target value.
  3. The broad interpretation of conversational-state updating should be more tightly delimited. As Section 3.3 acknowledges, historical mention representations cannot be revised by later tokens in a causal transformer. Their persistence therefore does not distinguish unsuccessful updating from a system that retains history while representing the updated state elsewhere. Similarly, the substantial improvement in value readability after supplying the slot key demonstrates dependence on the measurement interface. The empirical characterization is useful, but the introduction’s “append–select–deploy” framing should more clearly distinguish these observations from claims about how current state is internally maintained.
  4. The demonstrated controller scope remains narrow. Closed-loop evaluation uses single-domain goals within three MultiWOZ domains, and value resolution depends on a known database vocabulary, frozen matching rules, and explicit user evidence. The naturalized-user and human-authored replay evaluations are useful, but do not establish transfer to new schemas or reliable handling of implicit revisions and values introduced through system utterances. Preserving the base model’s domain also leaves domain-selection errors outside the controller’s repair capability. These limitations constrain the broader conversational-reliability implications.
Questions for authors
  1. How does SAC compare with a supervised slot-presence classifier using dialogue text or an external encoder, trained on the same structural labels and inserted into the identical resolver, protection, and validation pipeline? A comparison of query fidelity, preservation, and incremental latency would help isolate the benefit of the internal readout.
  2. Could you clarify the exact definition of the losses in Equation (1), report the condition-specific effects, and recompute the interaction using a fixed binary outcome such as preference for (v_1)? Does the conclusion about validity-dependent influence on binary decisions survive this change?
  3. Which findings distinguish the proposed account from immutable historical representations combined with retrieval and an updated state represented elsewhere? How should the “append–select–deploy” interpretation be qualified in light of the slot-key-conditioned recovery results?
  4. In the existing human-authored evaluation, what proportion of remaining failures arises from missing or implicit user evidence, unrecognized updates, and incorrect base-model domains? This breakdown would clarify which limitations are intrinsic to the current controller interface.
1 comment
AuthorsOpenReview ↗

Response to Reviewer aXAS

We thank the reviewer for the review.

W1 and Q1. The paper's main question is how conversational state is represented and used inside the acting model itself: Is the map there and does the model follow it? The linear probe readout measures the first (representation), and the controller tests the second (deployment) by steering the same model with its own measured state (§6–7). A dialogue-text classifier cannot answer that question, but it does test the practical claim in §6.1, so we ran the requested comparison.

We trained two slot-presence selectors on the probe readout's own data and labels (same 22,083 training and 2,441 dev sites, same calibration): tf-idf 1–2-gram logistic regression over the 64-line window, and a frozen all-mpnet-base-v2 encoder with logistic heads (dev micro-F1 .893 / .894; readout .922). Each replaces psp_s in the otherwise unchanged SAC pipeline. We also inserted T5-DST's predicted slot presence as an external-encoder selector.

Against both text classifiers, the probe readout gives higher exact-query accuracy on simulator prefixes (+.033 to +.045) and on the 4,490 human-authored turns (+.014 to +.033), higher repair in all eight comparisons, and a +.069 / +.082 exact-query lead in closed loop, with the encoder adding 13–14 ms per decision. The text selectors edit less and therefore preserve slightly more on simulator turns. A 60M-parameter T5 encoder trained to emit the full state, inserted as the selector, edges the readout on human text (.377/.280 vs .341/.259) at ~100 ms and a second model call.

So the interpretability result stands: the belief-state structure (which slots are active) is present in the model's own activations, is recoverable there more cheaply than from any text model tested, and steers the model within the controller's 5–7 ms per decision. We will revise §6.1: the gain comes from a trained predictor of which slots are active. The internal readout is the cheapest such predictor we tested.

W2 and Q2. The reviewer is correct: because the identity of the valid value changes between twins, the binary statistic as written can be positive without a difference in source influence. As implemented, Lac=Y(H)−Y(Ha)L^{c}_{a} = Y(H) - Y(H_a) with Y=1[the greedy query emits the currently valid value]Y = \mathbb{1}[\text{the greedy query emits the currently valid value}]. Using Y′=1[the query emits v1]Y' = \mathbb{1}[\text{the query emits } v_1] in both twins, the adjusted effect remains positive and significant on 7 of 8 models (macro +.558 vs +.582 original). The exception is OLMo-7B-IT, which emits v1v_1 in only 3.7% of current twins, so decreases in this binary endpoint have little headroom. Masking the v1v_1 source removes v1v_1 in .52–.86 of current twins but in at most .061 of superseded twins. Table 13 already uses a fixed orientation (z=log⁡P(v1)−log⁡P(v2)z = \log P(v_1) - \log P(v_2) in both twins) and is significant on every model (−14.66 vs −6.31 nats). It becomes the primary evidence in §4.2. The binary contrast becomes secondary and is reported with fixed orientation.

W3 and Q3. We agree the framing should be delimited. No finding of ours distinguishes the account from immutable mentions plus retrieval plus a state represented elsewhere. §3.3 already notes that mention persistence follows from the causal architecture, and Table 6 shows value readability depends on probe position. The paper’s claims are behavioral: decision-site readouts, linear and nonlinear, do not expose current binding (App. B.4). Superseded sources remain causally influential, more so while current (Table 13; D.4). Natural stale errors are repaired by masking the stale source but not by controls (D.4). We will reword §1: append–select–deploy describes where failures arise in the model's use of available information, not how current state is maintained internally.

W4 and Q4. On the 4,490 human-authored turns, 2,959 (Qwen) and 3,328 (Llama) SAC-P queries remain inexact. Almost all were already Base failures. The main causes (Qwen / Llama) are:

1- the reference value never appears in a user turn, mostly system-introduced names, .483.483 / .359.359;

2- the base draft names a wrong or unparsable domain, which the controller keeps by design, .119.119 / .262.262;

3- the draft keeps a non-canonical form of the right value ("centre of town"), .171.171 / .191.191;

4- the readout decides wrongly despite available evidence, .117.117 / .085.085;

5- the resolver misses an alias, .069.069 / .070.070.

(1–2 are the interface limits the reviewer names, 3 is normalization, 4–5 are fixable errors.)

We also tested a fixed-prefix multi-domain extension of the controller, with a domain head and one query line per domain, on the 1,686 excluded multi-domain turns. On multi-domain turns it improves over the zero-shot base (.010→.088.010 \to .088, .026→.117.026 \to .117). We will state the single-domain scope in the main paper and add the full multi-domain evaluation to the appendix.

We welcome any remaining concerns.

Reviewer_wA6WOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper tries to study how models store states for conversations using two benchmarks MultiWOZ and SGD. It shows that even after user updates a value in the state, the older value still influences the future actions. They come up with state action controller which doesn't require explicit state belief as intermediate representation. This improves turn accuracy and task success rates over the baseline model.

Critical strengths

The direction that the author take is very promising, it is interesting to understand how models react and represent information during conversations. They evaluate on wide variety of models.

Critical weaknesses

The paper is hard to understand. Look at the questions.

Several key details about the probes are missing such as explicit equations, objectives, training-set sizes, hyper parameter searches, and a worked example showing the input, extracted activation, target, and prediction for each experiment

The authors need to properly mention the approach that they take to study this.

Questions for authors

What do you mean by Conversational failure therefore cannot be reduced to information loss alone

Why does the model need to generate a query all the time, some of these might just be asking followup questions: "language-model policy produce an executable query qt = fθ (Ht)."

What is SEARCH-PRE in L121?

What are readouts?

What do you mean by "parts of the conversational state are active is linearly accessible"? how can you say that? what analysis did you do? the methodology is not clear at all.

when providing a fixed history that doesn't come from model responses, does the way the model stores the information change? Do you provide it as templates or as raw context, how would that make a difference?

Why do you use specific rank 32, what happens at other ranks?

4 comments
AuthorsOpenReview ↗

Response to Reviewer wA6W

Each of the seven reviewer questions and the overall study approach (setting §3.1, causal interventions §4, natural failures §5, controller §6) is answered in the submission. We give the section (§), page (p.) and ruler-line (L) location of each answer and restate it. The probe details are in App. A.7, E.10, B.4 and Table 8. Q2 restates them and adds the objective.

Q1: What is SEARCH-PRE (L121)?

Defined in §3.1, p. 3, L120–122: It is the decision site, the final context position before the query is generated. It is the last prompt token after the [SEARCH] instruction (App. F.1, p. 38) and before the first query token.

Q2: What are readouts?

Defined at §3.1, p. 3, L122–123, and specified in App. A.7 (p. 18–19, L959–995). They are linear probes on the residual-stream representation h at a fixed layer and token position, fit on dialogue-disjoint training data:

Structural readouts (Table 4, p. 21): for each slot, domain and request, a linear readout (sigmoid for domains and requests, softmax per slot) that reads the hidden state at the analysis layer (Table 2) and predicts whether that item is active. Trained on 56,728 turns, evaluated on 7,371 turns from disjoint dialogues (App. B, p. 20).

Controller slot-presence readout (§6.1, p. 7, L357–359; A.7, p. 18, L967–971): one head per slot, p(s active | h) = σ(w_sᵀh + b_s), trained with binary cross-entropy and L2 regularization selected by development loss. Fit with L-BFGS on 22,083 decision sites, tuned on 2,441 (A.3, p. 17, L879–881), and Platt-calibrated.

Q3: “Linearly accessible”: what analysis supports this?

Explained in §3.2, p. 3, L125–130. A linear probe (readout) at the decision site predicts annotated belief-state structure (domain F1 .926–.976, slot F1 .752–.808, request F1 .662–.802; Table 4, p. 21, column “Full”) and beats rank-matched PCA, shuffled-supervision and random-subspace controls on every model and task (App. B.1, p. 20, L1057–1069).

Q4: Why rank 32? What happens at other ranks?

The primary results are full-dimensional (App. A.7, p. 18, L962–963; Table 4, column “Full”). Rank 32 only tests whether a compact component exists (App. B.1, p. 20, L1058–1062).

New analysis: r ∈ {1, …, 256} on all eight models with PCA and shuffled controls at each rank. Supervised projections exceed both controls from r = 4 upward on every model. Rank 32 retains 50.9–57.9% of full-dimensional slot F1 and r = 128 retains 75.6–86.0%.

Q5: Why generate a query every turn? Some turns are follow-up questions.

The query is a database lookup performed before the response, not the response itself (App. A.4, p. 17, L891–899), following the standard task-oriented dialogue pipeline in which a belief state is updated before every database call and response (§1, p. 1, L030–033). A domain-only query returns the unconstrained set (App. F.1.1, p. 38, L2034–2035), so the response that follows may be a question, an offer or an answer.

Q6: Fixed histories vs. model-generated histories, templates or raw context?

The history is always plain text: the last 64 dialogue lines, rendered as “User: …” / “System: …” (A.4, p. 17, L891–892; F.1, p. 38). The representation analyses use MultiWOZ/SGD dialogues (A.2, p. 16), and the closed-loop experiments use the model’s own responses (A.4, p. 17).

New analysis: the same probe readout scores micro-F1 .922 on development sites from dataset dialogues and .845 / .851 (Qwen / Llama) on logged closed-loop decision sites with model-generated histories. Every closed-loop result in §6 was obtained in that second setting.

Q7: “Conversational failure cannot be reduced to information loss alone”.

The sentence summarizes §3–5 through the append–select–deploy pattern defined at §1, p. 1, L048–053: we identify failures in which the relevant information remains accessible, but the model fails to select or deploy it.

The information is present: active slots are linearly readable via probes at the decision site (slot F1 .752–.808, §3.2, p. 3, L125–130) and values at their mentions (.961–.993) and at the key position (.618–.942; Table 6). Selection fails: the model emits a superseded value although the user stated the current one. Masking the stale mention repairs 17/28 (Qwen) and 47/121 (Llama) of these decisions (§5, p. 7, L338–344; App. D.4). Deployment fails: 197 Qwen-7B-IT omissions have a slot the readout probe marks active and a value the model itself prefers, yet absent from the query (§5, p. 6, L308–310; Table 16); appending only the slot key yields the correct value in .934 of them (Table 17).

Worked example

At the decision site for the turn ending “User: I would prefer Italian.” (Qwen-7B-IT, layer 28), the annotated target is restaurant {pricerange: cheap, area: north, food: Italian}. The readout gives p_pricerange = .992, p_area = .910, p_food = .843, p_name = .007. The base query is already correct and the controller proposes no edit. One such example per experiment will be added to the paper.

Thank you for providing the additional experimental results and for offering more detailed explanations in your rebuttal. My concerns still remain. The sections mentioned in the response are exactly where I do not understand what does it mean. It is very unclear and difficult to read and understand. I feel that the initial statement, that all questions were already addressed in the submission, does not fully resolve concerns regarding the accessibility and clarity of these explanations. It the current writing is extremely difficult to understand even for someone who is aware of the benchmarks and the methods used.

To further strengthen the paper, I suggest consolidating the methodology into a more coherent and way. Additionally, including a worked example of a failure case, clearly linking the dialogue history, extracted activation, probe target and prediction, erroneous query, and the subsequent correction, would be very helpful.

I read the updated draft, and i belief the improvements take the paper to a better direction.

Can you provide some failure case? the example you provide gives me no information.

AuthorsOpenReview ↗

Response to Reviewer wA6W

Thank you for reading the revised draft and for the concrete suggestion.

On the methodology, we rewrote Section 3.1 in the revised PDF so that the method sits in one place. It now opens with a short example dialogue and the query the model must write. It then defines the four terms the paper relies on (mention site, decision site, linear probe, readout) in a list, and gives the two probe forms next to them. Appendix A.11 and Table A.1 list every probe with its input, target and training setup. Appendix F.3 holds the worked examples.

You asked for a failure case that links each step. We give one below, and we will move it into Section 6.1 of the main text, next to the controller definition, in the next revision.

Failure case (Qwen-7B-IT, first turn of a simulated dialogue)

  1. Dialogue history. One line: "User: Hello, can you help me find a restaurant for my upcoming trip to Cambridge?"
  2. What the model should do. The user has named a domain (restaurant) and no constraint yet. The correct database query is "domain: restaurant".
  3. What the model does. It writes "domain: restaurant; area: Cambridge;". This is the erroneous query. It matches no entity in the database, so the model has nothing to offer the user.
  4. Extracted activation. We take the model's hidden state at the last prompt token before it writes the query (the decision site), at layer 28. This is a single vector.
  5. Probe target and prediction. The slot-presence probe is one logistic classifier per slot on that vector. Its target is whether the user has constrained that slot. Here the target is "not constrained" for every slot. The probe outputs food .056, pricerange .024, area .056, name .023. So the model's own state says the area slot is not active, even though the model wrote an area constraint.
  6. Correction. The controller deletes a slot from the query when the probe probability is below .50 and no user utterance supports a value for it. Both conditions hold for area, so it deletes "area: Cambridge".
  7. Executed query. "domain: restaurant", which matches the reference.

The opposite failure is a missing constraint. In another dialogue (Appendix F.3, slot forcing), the reference state includes "internet: yes" and the model writes "domain: hotel; stars: 4". The probe gives the internet slot probability .843. The state marks the slot as active, yet the query omits it. When the slot key "internet:" is appended, the model itself completes "internet: yes".

If any step above is still unclear, please tell us which one and we will rewrite it. If the failure case and the revised Section 3.1 resolve your clarity concern, we would be grateful if you would consider updating your presentation and overall scores to reflect the revision. If a soundness concern remains, please tell us what it is so that we can address it before the discussion period ends.

AuthorsOpenReview ↗

Response to Reviewer wA6W

A short addition to our reply above, in case it helps.

The method in four steps.

  1. The model reads the dialogue and writes a database query.
  2. Just before it writes, we take one hidden vector from the model. We call this position the decision site.
  3. One linear classifier per slot reads that vector and outputs the probability that the user has constrained that slot.
  4. We compare this probability with the query the model wrote. The failures we study are the disagreements. The controller deletes a slot that the query contains when the probability is below .50, and adds a slot that the query lacks when the probability is above .70. It takes the value from the user's own words.

Our first example showed a deletion. Here is the opposite direction, a missing constraint (Appendix F.3, natural omission profiles, Qwen-7B-IT).

  • Reference state. The restaurant price range is "cheap".
  • Erroneous query. The model writes "domain: restaurant; area: city centre; food: italian; type: cheap". It puts "cheap" under the wrong key, so the price range constraint is missing.
  • Extracted activation. The hidden vector at the decision site, layer 28.
  • Probe prediction. The probe gives the price range slot probability .933. The model's state marks the slot as active.
  • Value check. When we append the key "pricerange:", the model prefers "cheap" over every other value by a wide margin (+14.38 in log probability). So the model holds both the slot and the value, and still fails to write them.
  • Correction. This is the pattern the controller's addition rule targets, a slot absent from the query with probability above .70.

Section 3.1 of the revised PDF defines the terms used here, with a running example. If any of the four steps above is still unclear, please tell us which one.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Version Summary of Revisions

Summary of revisions

We thank the reviewers and the AC. We have uploaded a revised PDF. It contains our changes. Table and figure numbers from the submitted version (Tables 1–30) are unchanged. Added tables are numbered by appendix section (Tables A.1, B.1, C.1–C.2, E.1–E.5). Every section, appendix, table, figure, and equation number cited in our responses is unchanged, because new appendix material was added at the end of each appendix section. Page and line numbers have shifted. All locations below refer to the revised PDF.

Reviewer wA6W

  • As our response notes, each question is answered in the submitted version. To make those answers easier to find, §3.1 now states the definitions and probe equations up front with a running example, and Appendix A.11 (Table A.1) collects the probe details in one table.
  • The additions promised in our response are in: the rank sweep (Appendix B.6, Table B.1), probe accuracy on model-generated histories (Appendix E.13), and one worked example per experiment (Appendix F.3).

Reviewer aXAS

  • W1/Q1 (is the internal probe necessary?): we replaced the probe with two text classifiers trained on the same sites and labels, and with T5-predicted slot presence, keeping the rest of the SAC pipeline unchanged. We report fixed-prefix accuracy, preservation and repair, closed-loop task success, and cost. The probe leads both text classifiers in exact-query accuracy on simulator prefixes (+.033 to +.045) and on human-authored turns (+.014 to +.033). T5-predicted slot presence exceeds the probe on human-authored text at the cost of a second model call. §6.1 now says that the gain comes from a trained predictor of which slots are active. See §6.1, §7, and Appendix E.14, Tables E.1–E.3.
  • W2/Q2 (binary validity statistic): §4.2 defines L as Y(H) − Y(H_a) and uses "emits" in place of "favors". The graded analysis (Table 13) is now the primary evidence, and the binary contrast is read only as a correctness effect. The fixed-outcome version is reported separately. With the fixed outcome, the adjusted effect stays positive with intervals excluding zero on 7 of 8 models (macro +.558). The exception is OLMo-7B-IT. See §4.2, Table 14 (new D′ column), and Appendix C.3, C.5, Table C.1.
  • W3/Q3 (scope of append–select–deploy): the framing is limited to where failures arise in the model's use of available information. See §1 and §7.
  • W4/Q4 (controller scope and remaining errors): §6.2 and §7 now state the single-domain scope. Appendix E.15 (Table E.4) breaks down why SAC-P queries on human-authored text remain inexact. Appendix E.16 (Table E.5) adds a fixed-prefix multi-domain extension.

Reviewer RYH3

  • W1/Q1 (Eq. 1): addressed together with aXAS W2/Q2 above. See §4.2, Table 14, and Appendix C.5, Table C.1.
  • W2 (populations change across lags): §4.3 and the Figure 2b caption now state that each lag is measured on its own eligible events. §4.3 adds the fixed-cohort result and the restatement split, detailed in Appendix C.6, Table C.2.
  • W3/Q2 (resampling unit in Table 23): intervals now resample initial goals, the caption states the unit, Binding cells with fewer than ten omissions are marked as descriptive, and the omission-level intervals are kept for comparison. Table 23 (Appendix E.4).
  • W4/Q3 (T5 memory): the Table 28 entry now includes T5's weights (242 MB of weights plus 42 MB of activations, 284 MB), and the caption states the residency convention and units.

We are happy to address any remaining concerns before the discussion closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.