acceptodds
Withdrawn from ICLR 2027

Knowing What to Change Is Not Enough: State-Transition Locality in Foundation Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Foundation models increasingly update persistent structured state from new evidence. A reliable transition must change the variables supported by the evidence and preserve the rest. Across 205,920 controlled outputs from 13 open foundation models, these capabilities diverge: models with near-perfect target correctness can still have poor complement retention. Qwen3.8-27B, for example, reaches 99.5% joint target correctness but only 29.7% conditional retention. We formalize this failure as update spillover and separate support recovery, value prediction, and complement preservation. We then run a controlled interface intervention across ten models, varying whether transition support is predicted or provided and whether the model emits a full vector or a local update. Predicted sparse deltas underperform free full-vector decoding for nine of ten models. The oracle-local interface preserves the complement by construction and yields 86.5–100% exact transition accuracy. Explicit deterministic slot–value evidence nearly closes the gap. The hard regime is semantic support inference followed by scoped update execution, making transition locality a distinct reliability criterion for models that maintain persistent state.

Primary area:
foundation or frontier models, including LLMs
Keywords:
state-transition locality, foundation models, structured state updates, structured prediction, model reliability

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_22coOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the "update overflow" problem when the basic model updates the persistent structure state: even if the model correctly identifies the target field and its change direction, it may also modify the irrelevant field incorrectly. The author divides the status update into update range recognition, target action prediction and non target field maintenance, and proposes evaluation indicators such as conditional maintenance rate. This paper analyzes 205920 outputs of 13 models, and then examines the role of the update range and output interface through the "prediction/real update range × full/local output" experiment on ten models. The results show that correct prediction of the target does not mean that the remaining states can be maintained, and sparse output itself is not sufficient to ensure local updates.

Critical strengths
  1. The thesis keeps a clear distinction between the correct target modification and the rest of the fields to avoid a single accuracy rate masking incidental modifications. Condition retention rate, overflow quantity and accurate transfer accuracy rate jointly describe different error sources, which is helpful to evaluate the model of maintaining external state.

  2. The author cross controls whether the update range is provided, whether the output is localized, and compares the four protocols on the same sample and model. The experiment shows that providing the real range and limiting the execution range play a role respectively, while directly switching to sparse output may reduce the performance. This result is instructive in practice.

  3. The paper covers multiple models, and adds experiments to clarify field evidence, maintain instruction strength, field serialization and interpretation stability. Self help sampling based on basic state clustering also considers the dependence between repeated observations.

Critical weaknesses
  1. Security, activity, boredom and pedestrian facilities are semantically related, and the evidence describing the change of one attribute may support the change of other attributes at the same time. The generation process only specifies one target, which is not enough to prove that other changes are not supported by evidence. It is suggested to manually verify the unique update range of semantic evidence, and repeat the experiment on mutually independent fields to distinguish between true spillovers and reasonable association inference.

  2. The complete output protocol also requires returning the main target field, while the sparse protocol requires generating a field list, and the local real range protocol further restricts the fields and value fields. Therefore, protocol differences involve not only the output range, but also field selection, format burden and task difficulty. It is suggested to supplement the comparison between unified semantic tasks and output constraints, and separately evaluate the prediction range of the model before implementing the two-stage scheme locally.

  3. The experiment focused on the text tasks with eleven city attributes, single target and mainly extreme direction changes, and the output was also an increase or decrease action rather than a complete new value; All models use low bit quantization version. There is still a lack of verification of multi-objective, no update, continuous multiple rounds of accumulation and real state maintenance tasks.

Questions for authors
  1. Have you checked the semantic evidence manually to support only one field change? For the associated fields such as safety and pedestrian facilities, activity and boredom, how consistent are the labels with the update range?

  2. After all protocols unify the main target prediction requirements, field representation and decoding constraints, will sparse output still be weaker than full output on the nine models? What is the result of local update after updating the range of independent forecast?

3.Can the separation of target correctness and retention rate be reproduced in multiple targets, no update or continuous multiple rounds of status updates? Does the validation of a small number of non quantitative models get the same trend?

Reviewer_kw9FOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether LLMs, when updating a structured state from new evidence, change only the field the evidence supports. Using an 11-field probe in which exactly one field changes, the authors show that target accuracy and preservation of the other fields diverge across 13 models (e.g., Qwen3.8-27B: 99.5% target correctness but 29.7% retention). A controlled comparison of four output interfaces shows that providing the target field and asking only for its value works best, and that explicit slot–value evidence nearly removes the problem.

Critical strengths
  • Clear metrics. CRR separates "got the target right" from "kept the rest".
  • Clean controlled study. Sparse delta output is worse than full output, which is surprising.
Critical weaknesses
  • The fields are related (lively vs. boring, safe vs. pedestrian infra). The spillover pattern follows these pairs. So some "errors" may be correct inferences. No human check of the labels.

  • One small synthetic domain. Only single-target, extreme edits.

  • Protocol D is good by construction. It is just direction accuracy.

  • No test on DST benchmarks (e.g. MultiWOZ).

  • All models evaluated are relatively very small.

Questions for authors
  1. Did humans check that the evidence implies only one change?
  2. What is FoundationProbe v3?
  3. In protocol C, are errors extra fields or missed targets?
  4. Does the observation transfer to larger or frontier models? E.g., How does GPT-6-Astra perform?
Reviewer_TWZsOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates "update spillover", where states different from the target state can go awry as foundation models track the states. Across 13 evaluated open models, accuracy of the target state and the preservation of other fields hardly correlate. The paper found that an effective way to remedy this is to tell the model "which fields change".

Critical strengths

Evaluated 13 open source models.

Critical weaknesses
  1. The full paper has significant traces of AIGC without human intervention. The paper is very poorly written, and the AIGC flow is not that compatible with human readers.
  2. The paper is limited in significance. It diagnoses the "update spillover" problem but does not give any remedy. The only thing that work is providing the model with additional context on which states change.
Questions for authors
  1. Could the authors rewrite the paper in a way that is more understandable to human readers?
  2. Could the authors open source the dataset they use to grant more reproducibility?
  3. Could the authors also evaluate frontier, closed-source models to see how they differ from these 13 open-source models.
Reviewer_2vayOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether foundation models can update persistent structured state by changing only the variables supported by new evidence while preserving everything else. The authors call the updating of non-supported variables "update spillover" and use the term "state-transition locality" to characterize the supported vs non-supported variable update gap.

Critical strengths

The problem of evaluating and evaluating hallucinated field updates is important as spillover may be an important mechanism to control. Especially in long running models where compound error can propagate.

I appreciate the statistical strength of the analysis.

Critical weaknesses

The paper's entire contribution rests on the claim that non-target field changes represent errors ("update spillover"). This claim depends on an assumption that state fields are independent given a single-target edit. Using their own example, a model observes that the road surface has deteriorated and then also lowers safety. The authors score this as spillover error even though road surface quality causally affects pedestrian safety. A model that updates both fields is arguably producing a more accurate state than one that only touches road surface. The evaluation has no way to distinguish legitimate downstream inference from genuine error, because it never evaluates whether the complement field changes are factually appropriate or semantically related (e.g. liveliness and boredom). This would explain their observation that high accuracy models have high spillover in their analysis. I would argue the models making correct field changes and the authors have just not properly characterized the state-space.

The ground truth is constructed by a deterministic single-field edit operator with no domain model, no causal structure, and no human validation of whether the imposed independence is realistic. The metrics (CRR, ETA, CSC) therefore measure compliance with an artificial independence assumption which makes the results unreliable.

Questions for authors

Have you explored the question of whether the spillover is actually unwanted or negative behavior vs positive?

Does reducing spillover by your measurement improve model performance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.