acceptodds
Under review as a conference paper at ICLR 2027

Family-Dependent Changes in Token-Level Prompt Dependence across Post-Training Checkpoints

Abstract

Post-training is intended to make language models follow their inputs more closely, but whether it changes how strongly a stated fact supports the model’s answer, and in which direction, remains unclear. Response-level evaluations do not answer this question, because they do not show how individual answer-token probabilities respond when only the stated fact changes. We study it with Token-Level Matched-Substitution Sensitivity: holding the gold response prefix and target token fixed, we replace the factual prompt with a relation-matched alternative and average the resulting log-probability difference over the answer span to obtain Answer Prompt Dependence (Answer PD). We compare Base and post-trained checkpoints from four model families (Qwen3-1.7B, Mistral-7B, SmolLM2-1.7B, and OLMo 2 7B) on CounterFact and ParaRel under one fixed plain-text prompt format. The direction of change depends on the family: on CounterFact, Qwen3 decreases Answer PD (Δ = −1.81), whereas Mistral, SmolLM2, and OLMo 2 increase it (+2.30, +0.75, and +3.78). Across OLMo 2’s released Base, SFT, DPO, and Instruct (RLVR) checkpoints, Answer PD rises at every stage on both datasets, and about two-thirds of the total increase occurs at SFT. In a paraphrase control that preserves the fact, wording sensitivity changes in the opposite direction to Answer PD in all three evaluated families, with paired intervals excluding zero for Qwen3 and SmolLM2. Leave-one-relation-out analyses preserve every evaluated aggregate direction. Post-training should therefore not be assumed to change factual prompt dependence uniformly; our results characterize released checkpoints under a fixed protocol, not causal effects of individual training stages.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Post-Training, Prompt Dependence, Prompt Sensitivity, Token-Level Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6JbLOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose a new metric, Answer Prompt Dependence (Answer PD), which measures how the probability of a particular model answer changes between pre-training and post-training. The authors examine several relatively small models (1.7B–7B parameters) on two datasets. They analyze the results for OLMo 2 7B in greater detail because checkpoints from different stages of training are available for this model. The main conclusion of the paper is that the proposed Answer PD metric differs across models and, in some cases, may even change its sign.

Critical strengths

None.

Critical weaknesses
  1. It is unclear what specific problem this work helps solve and what practical knowledge can be gained from it. The paper does not explicitly discuss the potential impact of the research. For a reader who is not closely familiar with the topic, it is not obvious how the findings should be interpreted or how they could be applied in future research without additional explanation. This is further exacerbated by the complete absence of any discussion of future work. Moreover, the paper lacks scientific novelty and merely applies already known approaches to a small number of models. These experiments do not support any broad conclusions, and the impact of this work remains unclear.
  2. The quality of the manuscript itself is very poor. It is extremely difficult to read, lacks a clear structure, and the main point of many paragraphs is unclear, making it very difficult to develop an understanding of what the paper is about, even after careful reading. The low quality of the paper is particularly evident in its figures and tables, which are even more difficult to understand than the text and appear very poorly prepared. The manuscript appears to require substantial restructuring and careful proofreading before its contributions can be properly evaluated.
Questions for authors
  1. Apart from the questions arising from the weaknesses listed above, which the authors are unlikely to be able to address during the rebuttal period, there is also a question regarding the applicability of the proposed approach. Have the authors conducted any additional experiments on other models? In particular, it would be interesting to know whether the proposed approach is applicable to models with reasoning capabilities.
7 comments
AuthorsOpenReview ↗

Response: Research Contribution, Model Applicability, and Presentation

We thank the reviewer for the comments on the motivation, contribution, and presentation of our paper. We understand the concern that the original submission did not sufficiently explain why measuring Answer PD is useful. Our main question is whether post-trained checkpoints differ in how strongly a stated fact supports a particular answer token, when the preceding answer context is held fixed. This is different from asking whether a model can produce the correct answer, and we have revised the introduction and methodological explanation to make this distinction clearer.

The main contribution of this work is to provide a controlled way to examine how factual-prompt dependence changes across released post-training checkpoints, together with empirical evidence that these changes are not uniform across the evaluated models.

Our method compares two prompts that express the same factual relation but refer to different subjects and answers. By holding the gold response prefix and target token fixed, we measure how much the stated fact changes the model's conditional support for that token. This separates the factual substitution being tested from differences in generated responses or answer wording.

The distinction matters because conventional answer-level accuracy can identify whether a model answers correctly, but cannot directly show how strongly the stated fact supports a particular answer token. Our experiments use this controlled comparison to reveal differences between checkpoints that would otherwise be difficult to characterize from response-level results alone.

We view this as an empirical measurement contribution. It does not establish the internal mechanism behind the observed differences or the causal effect of individual training stages, but it provides a more specific basis for studying how factual-prompt dependence varies across checkpoints.

The experiments illustrate why this distinction matters. Under the same plain-text scoring protocol, Answer PD decreases by 1.81 nats for the evaluated Qwen3-1.7B Base-to-Post pair, but increases by 2.30, 0.75, and 3.78 nats for Mistral, SmolLM2, and OLMo 2, respectively, on CounterFact. The same directions are observed on ParaRel. For OLMo 2, the additional released checkpoints show that the largest adjacent increase occurs between Base and SFT, accounting for approximately 68% of the total Base-to-Instruct increase on CounterFact.

These results show that post-trained checkpoints do not exhibit a uniform change in factual-prompt dependence under the tested protocol. We do not interpret the OLMo 2 trajectory as evidence that SFT or any subsequent training stage caused the observed changes, since the released checkpoints do not provide controlled causal comparisons.

Regarding the applicability of the method, our main study already includes four model families. Following the review, we also conducted additional experiments with Qwen3-4B under the original plain-text protocol (W0) and a native-chat protocol (W3). On CounterFact, the Base-to-Post Answer PD changes for Qwen3-4B are +2.042 under W0 and +4.218 under W3. Both are positive, unlike the negative W0 change observed for Qwen3-1.7B.

These additional results indicate that the direction of change depends on the specific checkpoint pair and input protocol, and should not be generalized across all Qwen3 model sizes.

The scoring method can in principle be applied to autoregressive language models for which token-level conditional probabilities are accessible. However, we have not systematically evaluated reasoning-mode behavior or larger reasoning models. Our current results therefore do not establish whether the same patterns hold during multi-step reasoning or free generation. We consider this an important limitation of the present study.

We also appreciate the reviewer's comments on the presentation and readability of the paper. In response, we have revised all figures throughout the main text and appendix, with particular attention to layout, labeling, and the clarity of the information presented. Figures 1 and 2 now provide a clearer explanation of the overall evaluation framework and the fixed-prefix scoring procedure. We have also improved the presentation of the experimental results to make the comparisons across models and checkpoints easier to follow.

In addition to these revisions, we have added two new analyses to the appendix: a robustness evaluation across different prompt wrappers (Figure B10, p. 23) and a comparison of factual and wording sensitivity on the same subset of examples across four model families (Figure B11, p. 24).

We thank the reviewer again for the careful assessment and constructive criticism. Your comments helped us identify parts of the original manuscript that needed clearer explanations and presentation. We hope these revisions make the research question, methodology, and findings easier to understand and evaluate.

I have read the authors' response and examined the revised manuscript. Unfortunately, neither has addressed my main concerns. It remains unclear what meaningful conclusions can be drawn from the reported results, how they should be interpreted, or what practical scientific value the proposed metric provides. The additional experiments do not resolve these issues.

The presentation also remains very poor. In particular, the figures are still difficult to interpret and appear to be AI-generated, with insufficient attention to clarity and readability. Overall, I do not believe the manuscript has reached a level of clarity and scientific maturity that would allow for a meaningful evaluation of its contribution.

I therefore maintain my original assessment and rating.

AuthorsOpenReview ↗

Response (1/2): What We Measure and What the Results Tell Us

Thank you for reading our previous response and the revised manuscript. We understand that your main concern is what can actually be learned from our experiments. We would like to explain the method and the meaning of our findings more directly.

Our study asks whether post-training changes how a model responds to factual information when assigning probability to an answer. More specifically, when we replace one factual statement with another that expresses the same relation but has a different correct answer, does the model's probability for the original answer change differently before and after post-training?

Consider a simple example. The capital of France is Paris, while the capital of Japan is Tokyo. We construct two prompts: “The capital of France is ...” and “The capital of Japan is ...”. Instead of asking the model to generate two different answers, we keep the preceding answer text and the target answer “Paris” fixed. We then calculate the log probability assigned to “Paris” under each prompt.

The difference between these two log probabilities is what Answer PD measures. It tells us how much more or less the original factual prompt supports “Paris” than the matched substitute does. We perform exactly the same comparison for the Base and post-trained versions of a model.

How should we interpret a change in Answer PD? If Answer PD increases after post-training, the relative probability support for the original answer shifts in favor of the original factual prompt. If it decreases, the relative support shifts toward the substituted prompt. Thus, the comparison tells us how the model's response to a factual substitution changes between its two versions. It does not, by itself, tell us whether the model has become more accurate.

Our experiments reveal that these changes are not uniform. On CounterFact, Answer PD decreases by 1.81 nats for Qwen3-1.7B. This means that, after post-training, the original factual prompt provides less relative log-probability support for the fixed answer than it did in the Base version. In contrast, Answer PD increases by 2.30, 0.75, and 3.78 nats for Mistral, SmolLM2, and OLMo 2, respectively. For these evaluated model versions, the relative support shifts in the opposite direction. The same four directions are observed on ParaRel.

The OLMo 2 results provide a further insight. Because several intermediate model versions are available, we can examine where the measured change appears along the released sequence. Answer PD increases from Base to SFT, from SFT to DPO, and from DPO to Instruct. The largest increase occurs between Base and SFT, accounting for approximately two-thirds of the total increase on CounterFact. This identifies the model-version comparison in which the largest difference appears, without establishing that SFT itself caused it.

The central finding is therefore not simply that some scores increase while others decrease. It is that, under the same evaluation protocol, different post-trained model versions can show opposite changes in how a factual substitution affects their probability for the same answer. We therefore cannot assume that post-training uniformly increases or decreases this measured dependence on factual prompts.

This matters because two model versions may produce the same final answer while responding differently, in their conditional token probabilities, when the factual prompt changes. Answer PD makes that difference observable. Our findings are limited to the evaluated models and protocols; they do not imply changes in overall factual accuracy or establish causal effects of individual training stages.

Reviewer_yUgVOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces Answer Prompt Dependence (Answer PD). Under teacher forcing, with a fixed gold response prefix (“The answer is …”), it is the log-probability of the gold answer tokens given the original factual prompt minus their log-probability given a relation-matched substitute prompt (France→Japan, still scoring “Paris”), averaged over the answer span.

The authors compare Base and post-trained checkpoints for Qwen3-1.7B, Mistral-7B-v0.3, SmolLM2-1.7B, and OLMo 2 7B (Base/SFT/DPO/Instruct) on 252 CounterFact pairs and 230 ParaRel pairs, all under one plain-text wrapper. main findings are:

  • Answer PD drops for Qwen3 and rises for the other three families.
  • Most of OLMo 2’s increase appears at the SFT checkpoint.
  • A paraphrase-sensitivity control moves in the opposite direction to Answer PD.
  • The results survive leave-one-relation-out and multi-donor checks.
Critical strengths
  • Clean paired design with very careful scoping. Holding the prefix and target fixed while swapping only the subject is a tidy comparison. The decomposition in equations 1 and 2 shows clearly that the prefix-only term cancels out the effect.

  • Replication across datasets.

  • Strong reproducibility reporting. Revision hashes, seeds, bootstrap details, and code excerpts are all provided.

  • Useful staged series. The OLMo 2 Base→SFT→DPO→RLVR comparison makes good use of a publicly staged release.

Critical weaknesses
  • The “family dependence” may be template dependence. Every post-trained checkpoint is scored without its chat template. Although it is a defensible choice for isolating weight changes, it is unclear whether the small models suffer from this change of chat templates.
  • The scale is limited relative to the claims. CounterFact alone has about 21k records, but the study uses 252 and 230 pairs with no justification provided.
  • The significance and takeaway are unclear. The main conclusion is essentially a negative one: post-training does not change PD uniformly. The paper doesn’t link Answer PD to anything downstream, such as factual accuracy, hallucination rate, or knowledge-conflict behavior, so it’s hard to tell why a practitioner or researcher should care about the direction of ΔPD.
  • Bad paper presentation; plots are not made visually well, overuse of numbers in text.
Questions for authors
  1. Could you report results under all wrappers they evaluated, plus some native chat template sensitivity analysis?
  2. Does the Qwen3 reversal persist at other Qwen3 sizes?
3 comments
AuthorsOpenReview ↗

Response (1/2): Prompt Formats and Qwen3 Results

We thank the reviewer for the detailed comments, particularly the concerns about prompt formatting, sample selection, and the interpretation of Answer PD. We agree that these issues are important for understanding what our results establish. In response, we tested additional prompt formats, extended the Qwen3 comparison to a second model size, and clarified the scope of our evaluation. The additional results also revealed some important limitations, which we discuss below.

1. Robustness across plain-text wrappers

We first examined whether the results depend on the wording of the prompt wrapper. We evaluated three shared plain-text formats—W0, W1v2, and W2v2—on all four model families using the same 252 CounterFact and 230 ParaRel examples. W0 is the original Exact Plaintext B format, W1v2 changes the task instruction, and W2v2 changes both the instruction and field labels. All other evaluation settings were kept fixed.

Across these three formats, the mean Answer PD changes retain the same direction in all eight model–dataset combinations, although their magnitudes differ. One qualification is important: for SmolLM2 on ParaRel under W2v2, the change is only +0.011 (95% CI [−0.195, +0.216]), so the interval includes zero. These results support directional consistency across the tested plain-text formats, but not independence from prompt formatting in general. The full results and exact templates are now included in Appendix B.14 (Figure B10, p. 23).

2. Native-chat formatting and Qwen3 size

We further examined whether the Qwen3-1.7B result holds under native-chat formatting and for another model size. We compared Qwen3-1.7B and Qwen3-4B using W0 (the original plain-text protocol) and W3 (native-chat formatting). The observed Post-minus-Base changes in Answer PD are:

ModelFormatCounterFactParaRel
Qwen3-1.7BW0−1.795−1.771
Qwen3-1.7BW3+1.266+1.345
Qwen3-4BW0+2.042+1.953
Qwen3-4BW3+4.218+4.220

These results answer the reviewer's question directly: the negative change observed for Qwen3-1.7B under W0 does not generalize to the evaluated Qwen3-4B checkpoint pair. Qwen3-4B shows positive changes under both protocols. Moreover, Qwen3-1.7B changes from negative under W0 to positive under W3.

This distinction matters. Our shared plain-text wrapper experiments support directional consistency across the three tested wording variants, but the native-chat results show that this consistency does not extend to every input protocol. We therefore restrict the original Qwen3 finding to the evaluated Qwen3-1.7B checkpoint pair under the primary plain-text protocol, rather than treating it as a property of the entire Qwen3 family.

3. Verification limits of the additional comparisons

These additional comparisons also require some qualification. The Qwen3-4B results were independently checked against archived sample-level and token-level outputs. For Qwen3-1.7B, the available W0 records reproduce the negative direction, although the reconstructed means are not identical to those in the original submission. The W3 results are supported by archived paired summaries, but the underlying token-level scoring records and exact tokenizer-revision linkage are incomplete.

Importantly, W3 changes the serialized input and includes an assistant prefix. The archived Base and Post chat templates for Qwen3-1.7B also differ. We therefore cannot isolate native-chat serialization as the sole cause of the observed sign change, nor can we interpret the difference between 1.7B and 4B as a controlled model-size effect. These results are reported here as supplementary evidence; Figure B10 in the revised manuscript covers only the three shared plain-text wrappers.

AuthorsOpenReview ↗

Response (2/2): Sample Selection and Interpretation

4. Sample selection

We agree that the evaluation subsets are limited relative to the full benchmarks. Our aim was to construct relation-matched factual substitutions that could be scored consistently, rather than to evaluate every possible pair in the source datasets.

For CounterFact, we required each pair to preserve the relation while changing both the subject and answer. We also checked that the statements were compatible and unambiguous, that the target answer was not directly revealed in the prompt, and that the answer tokens could be reliably aligned. The final set contains 252 examples covering 24 relations. For ParaRel, we applied corresponding validity and leakage checks, excluded facts already used in CounterFact, and retained ten examples from each of the 23 shared relations, yielding 230 examples.

The CounterFact candidate pool was bounded during construction, so the final 252 examples should not be interpreted as all valid pairs available in the original dataset. Nor do we claim that they form a statistically representative random sample. Our conclusions are therefore limited to the evaluated subsets. To check whether any single included relation drives the results, we performed leave-one-relation-out analyses. Removing any one relation preserves the aggregate direction of the evaluated contrasts. This supports the stability of the findings within the selected relations, although it does not establish generalization to the full benchmarks. The construction criteria are documented in Appendix B.2 (p. 16).

5. Interpretation and scientific value of Answer PD

We agree that the connection between Answer PD and observable model behavior needs to be stated carefully. The purpose of Answer PD is not to replace accuracy-based evaluation, but to examine a different aspect of model predictions.

Consider two checkpoints that produce the same correct answer to a factual question. Their response-level accuracy would be identical, but this does not tell us whether the stated fact provides the same degree of conditional support for the answer token in both models. By keeping the target token and gold response prefix fixed and replacing the factual statement with a relation-matched alternative, Answer PD measures this difference directly.

This makes the diagnostic useful for comparing how released checkpoints respond to factual substitutions, including cases where conventional answer-level evaluations do not reveal the same information.

At the same time, Answer PD remains a conditional-probability measure. A change in Answer PD does not by itself imply a change in free-generation accuracy, hallucination rates, or model reliability. Establishing such relationships would require additional behavioral evaluations, which are beyond the evidence provided by the present experiments. We have clarified this distinction in the revised manuscript.

6. Presentation

Finally, we have revised the main methodological figures and their explanations to make the evaluation procedure easier to follow. The revised figures distinguish the factual substitution, fixed response prefix, token-level scoring, and checkpoint comparison more clearly. We have also added the wrapper definitions and robustness results in Appendix B.14.

Thank you again for the careful review. Your comments prompted us to examine assumptions that were not sufficiently tested in the original submission. The additional experiments do not remove every limitation, but they provide a clearer picture of which findings hold under the tested conditions and where further evaluation is needed.

The authors addressed several issues and acknowledged the limitations of sample selection. I have raised my rating, however, I suggest the author to revise the paper in another iteration.

Reviewer_jmQdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Post training forces the model follow their inputs closely. It is unclear whether it strengthens or weakens the degree to which its alignment of stated fact to the given answer token. The authors argue that response-level accuracy evaluations are not granular enough to shed light at this as they collapse token-level conditional probabilities.

The authors propose a diagnostic method called Token-Level Matched-Substitution Sensitivity. For a fixed gold prefix and fixed target token they swap the factual prompt for a relation-matched alternative and measure the signed log-prob difference, averaged over gold answer span. The isolation of factual-substitution sensitivity with answer context held fixed is a cleaner estimate than prior prompt-sensitivity work, which varies wording and scores whole responses. Scoring uses a single shared plain-text wrapper applied identically to base and post-trained checkpoints, making sure that the weight changes are not confounded with serialization changes.

Prompt Dependence metric measures sensitivity to subject substitution within a fixed relation, not prompt dependence in general.

Critical strengths

Clear well written paper. The idea of entity-substitution knowledge-conflict probe to capture factual-substitution sensitivity when answer context held fixed is a cleaner estimate in the area of prompt-sensitivity work.

Critical weaknesses

None

Questions for authors

The factual vs wording move oppositely claim is supported by Qwen3 and Smol2, what about other models?

1 comment
AuthorsOpenReview ↗

Response: Factual and Wording Sensitivity Across Model Families

We thank the reviewer for the positive assessment and the helpful question. Our additional analysis provides statistical support for opposite-direction changes in factual and wording sensitivity in Qwen3-1.7B and SmolLM2. For Mistral and OLMo 2, however, the changes in wording sensitivity remain inconclusive. We have included the four-family comparison in Appendix B.15 (Figure B11, p. 24).

To make the comparison directly comparable, we evaluated factual sensitivity (Answer PD) and wording sensitivity (Answer PS) on the same 83 CounterFact examples covering 17 relations. For Qwen3-1.7B, Answer PD changes by −1.952 nats, while Answer PS changes by +0.456 nats. SmolLM2 shows the reverse pattern: Answer PD changes by +0.795 nats, while Answer PS changes by −0.443 nats. In both families, the paired 95% confidence intervals for both measures exclude zero.

For Mistral, Answer PD increases by 2.072 nats, but the change in Answer PS is small (−0.035 nats, 95% CI [−0.275, +0.212]). For OLMo 2, Answer PD increases by 3.889 nats and Answer PS has a positive point estimate of +0.165 nats, but its confidence interval also includes zero. Thus, although Mistral has opposite point-estimate signs and OLMo 2 has same-sign point estimates, neither provides clear statistical evidence for a corresponding change in wording sensitivity.

These findings indicate that the relationship between factual and wording sensitivity differs across the evaluated model families. The opposite-direction pattern is supported for Qwen3-1.7B and SmolLM2, while the results for Mistral and OLMo 2 remain inconclusive with respect to wording sensitivity. We have clarified this distinction in the revised manuscript, including the provenance limitation of the OLMo 2 extension. We thank the reviewer again for this suggestion, which helped us extend the comparison and state our conclusions more precisely.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.