acceptodds
Under review as a conference paper at ICLR 2027

STANCE DRIFT: HOW AI-MEDIATED COMMUNICATION DISTORTS OUR MESSAGE

Abstract

Large language models (LLMs) increasingly mediate human communication, from drafting emails to summarizing scientific reports, yet whether they faithfully preserve a speaker's position remains largely untested. We model AI-mediated communication as a two-step generation-extraction pipeline: one LLM produces an argument from a specified stance, and a second LLM extracts the stance from that argument. We represent the pipeline as a probabilistic state transition over five Likert-type stance categories and define the stance preservation rate (SPR) as the average probability that the extracted stance matches the initial stance. Across 112 debate propositions, none of the nine LLMs tested exceeded an SPR of 0.7 under the default configuration. Three drift patterns accounted for most of the drift: polarization, deviation from neutrality, and flipping. Among the mitigation strategies tested, including in-context learning, multiple extraction with shuffled options, assertion, and reflection, only adding medium reasoning effort to a reflection prompt for GPT-5.4 substantially improved the SPR, to 0.775, yet polarization remained the largest pattern, with 0.119 of the transition mass. An exploratory comparison with human extraction on a single proposition suggests that drift arises at both the generation and the extraction stage. These results point to a fidelity gap in AI-mediated communication, with implications for journalism, policy deliberation, scientific communication, and other domains where opinion-laden messages pass through language models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, AI-mediated communication, stance preservation, stance drift, communication fidelity

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3JXNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces stance drift, the phenomenon where an intended opinion or position changes as it passes through an AI-mediated communication pipeline. The authors model communication as a two-stage process: (1) an LLM generates an argument from a specified stance, and (2) another LLM extracts the stance from that argument. They define Stance Preservation Rate (SPR) as the probability that the final extracted stance (over 5-point Likert scale) matches the original one. Across 112 debate propositions and nine LLMs, no model achieves SPR above 0.7 under the default setup. The authors identify three dominant failure modes: polarization (moderate stances becoming extreme), deviation from neutrality, and stance flipping. The best mitigation strategy is GPT-5.4 with third-person reflection and medium reasoning effort, reaching SPR = 0.775, but substantial drift remains.

Critical strengths
  1. Novel problem formulation. The paper studies communication fidelity rather than generation quality, alignment, or stance classification in isolation.
  2. Clear metric and analytical framework. The transition-matrix formulation and SPR provide a simple, interpretable measure of preservation failure. The decomposition into polarization, flipping, and neutrality drift is particularly useful.
  3. Thorough intervention analysis. The paper identifies which drift modes dominate and tests several interventions, finding that most common prompting tricks help little.
Critical weaknesses
  1. Heavy reliance on LLM-as-judge. The primary measurement uses an LLM extraction stage. If extraction itself is noisy or biased, observed drift may partially reflect evaluator limitations rather than communication failure. The human study is interesting but limited to a single proposition.
  2. Artificial communication setup. Real-world AI-mediated communication often involves editing, summarization, translation, and iterative interaction. The proposed generation--extraction pipeline may not fully capture these settings.
  3. Limited theoretical insight into causes. The paper quantifies drift very well, but provides less explanation of why specific models drift, whether generation or extraction dominates across models, or what architectural properties predict drift.
Questions for authors

Q1: Are the results robust across communication tasks, e.g. email drafting, summarization, translation? Q2: Can a more fine-grained failure mode analysis be done with respect to the specific component (generator, extractor)? Q3: Is there a stronger mitigation strategy for stance drift?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 3JXN(1)

We thank the reviewer for the careful reading and the comments, and respond to the weaknesses and questions below.

W1. "Heavy reliance on LLM-as-judge. ... If extraction itself is noisy or biased, observed drift may partially reflect evaluator limitations rather than communication failure. The human study is interesting but limited to a single proposition."

We would like to clarify a misunderstanding. The extraction stage models the receiver, a person who reads the message with the help of an LLM (Figs. 1 and 3); it does not score the generated argument. Drift caused by biased extraction is therefore part of the communication failure we study, since end-to-end drift is what the receiver experiences. The results of Section 3 and the appendix, measured by the SPR, the success rate per initial stance, the proportion of propositions whose confidence interval includes 1 and the pass rate at thresholds from 0.5 to 1, indicate that stance drift is pervasive rather than noise.

The human-extraction study is exploratory: on one proposition, it examines whether human readers recover the initial stance from the generated arguments and at which stage drift arises (Section 3.5). The revision states its scope in a new limitations subsection (Section 4.3):

Our conclusion is descriptive: it characterizes stance drift on the propositions of the debate speeches dataset and under the prompts we used, and it does not isolate a causal source. The two exploratory studies are limited in scope and serve as proofs of concept. The human-extraction study covers one proposition and one model due to costly labeling work (Section 3.5), and shows how drift can be located at the generation or the extraction stage. The email-writing task was run with one model (Appendix A.13) and explores drift in a concrete communication task.

W2, Q1. "Real-world AI-mediated communication often involves editing, summarization, translation, and iterative interaction ... Are the results robust across communication tasks, e.g. email drafting, summarization, translation?"

Realistic communication is also the motivation of our study. For email drafting, we added an exploratory run with GPT-5.4 as a proof of concept, in which the generation prompt asks for a short email to a colleague in place of the argument and the extraction prompt is unchanged. Drift remains, with an SPR of 0.713. Polarization is still the largest pattern, and an email written for "Agree" is read as "Agree strongly" in 0.494 of the trials. The revision describes the run in a new Appendix A.13 and Table S14:

The email-writing task keeps the prompts of Appendix A.3 except for three lines of the generation prompt: the task instruction, the output instruction and the final cue are replaced by the following ones.

Based on the specified letter, write a short email of at most {words_limit} words to a colleague in which the writer states a position on the proposition that is consistent with the corresponding stance.

Only output the email itself and do not directly quote or explicitly mention the option or its label.

Email:

The extraction prompt is unchanged. The task was run with GPT-5.4, as an exploratory check of a second generation task, with the other settings of the default configuration. The SPR is 0.713, against 0.593 for the argument, and polarization remains the largest pattern (Table S14).

Table S14: SPR and the components of the SDR by drift pattern for GPT-5.4 with the argument and with the email-writing task. As Table S5: mean (SE) over the 112 propositions. The row GPT-5.4 is the default configuration, in which the model writes an argument.

ModelSPRPolarizationModerationModeration to neutralFlippingDeviation from neutrality
GPT-5.4.593 (.008).328 (.006).002 (.001).000 (.000).018 (.004).059 (.006)
GPT-5.4 (email task).713 (.010).206 (.006).002 (.001).000 (.000).011 (.003).068 (.006)

However, real communication varies in setting, prompt and text length, so neither a single setup nor a small set of setups covers all of it. As a first study of stance drift in AI-mediated communication, we reduce communication to a generation-extraction pipeline that keeps its essence, a sender who expresses a stance and a receiver who reads it, and that can be quantified. For summarization, Liu et al. (2024) find that systems frequently alter the political stance of a news article (Section 1). Iterative interaction corresponds to the multi-hop communication chains proposed in Section 4.2, for which the transition matrix of one step is the building block. Covering more tasks, such as translation, is a valuable direction for future work.

AuthorsOpenReview ↗

Response to Reviewer 3JXN(2)

(Part 2 of 2)

W3, Q2. "... provides less explanation of why specific models drift, whether generation or extraction dominates across models, or what architectural properties predict drift." and "Can a more fine-grained failure mode analysis be done with respect to the specific component (generator, extractor)?"

The paper is descriptive by design, and the new limitations subsection states that the conclusion does not isolate a causal source (quoted under W1). On the specific points:

  • Generator or extractor. Separating the two stages needs a reference independent of the extractor, which human labeling, as in the human-extraction study, provides at a high cost.
  • Models and architecture. The models were reached through APIs and differ in size, family and training at once, so the results do not identify which property of a model, architectural or other, predicts drift (Section 3.1).

Both are valuable directions, for example through analyses of the attention mechanism, but each calls for a standalone study. Nevertheless, we consider the description a contribution in its own right, for three reasons.

  • It is an empirical finding enabled by a reusable evaluation framework. The transition matrix, SPR and the drift patterns apply to any model, and the same measurement detects the gain from reasoning effort (Q3), suggesting it can serve as the target of mitigation work.
  • The finding is consequential and has several facets. The paper gives a systematic evaluation of end-to-end stance preservation, and none of the nine models exceeds an SPR of 0.7. The transition matrices separate the drift into five patterns, of which polarization, deviation from neutrality and flipping carry most of the drift of every model (Table 2).
  • The phenomenon is robust and persistent. It holds across nine models from several families, over 112 propositions, under the robustness checks and mitigation strategies of Section 3.4, and in the exploratory email-writing task.

Q3. "Is there a stronger mitigation strategy for stance drift?"

The strongest effect we found is that of reasoning effort on GPT-5.4: medium reasoning effort raises the SPR from 0.593 to 0.686 with the default prompts and from 0.637 to 0.775 with third-person reflection (in which each stage is asked to check its output against the stance from the perspective of an impartial third-party reviewer). In both cases it lowers polarization and flipping but raises deviation from neutrality, and at the highest SPR, 0.225 of the transition mass still drifts. After reasoning effort, the largest effect is that of in-context learning, which raises the SPR of GPT-5.4 to 0.673 and, unlike reasoning effort, also raises the success rate for "Neutral", from 0.703 to 0.831; polarization remains its largest pattern. Five rows of Table S5, mean (SE) over the 112 propositions, with the new rows in blue:

ModelSPRPolarizationModerationModeration to neutralFlippingDeviation from neutrality
GPT-5.4.593 (.008).328 (.006).002 (.001).000 (.000).018 (.004).059 (.006)
GPT-5.4 (in-context)\color{blue}{\text{GPT-5.4 (in-context)}}.673 (.008)\color{blue}{\text{.673 (.008)}}.280 (.007)\color{blue}{\text{.280 (.007)}}.002 (.001)\color{blue}{\text{.002 (.001)}}.000 (.000)\color{blue}{\text{.000 (.000)}}.010 (.003)\color{blue}{\text{.010 (.003)}}.034 (.005)\color{blue}{\text{.034 (.005)}}
GPT-5.4 (third-person reflection).637 (.011).240 (.007).003 (.001).001 (.000).063 (.008).056 (.006)
GPT-5.4 (medium reasoning)\color{blue}{\text{GPT-5.4 (medium reasoning)}}.686 (.013)\color{blue}{\text{.686 (.013)}}.197 (.008)\color{blue}{\text{.197 (.008)}}.011 (.002)\color{blue}{\text{.011 (.002)}}.001 (.000)\color{blue}{\text{.001 (.000)}}.010 (.004)\color{blue}{\text{.010 (.004)}}.097 (.007)\color{blue}{\text{.097 (.007)}}
GPT-5.4 (third-person reflection, medium reasoning).775 (.012).119 (.007).019 (.003).001 (.000).008 (.004).079 (.007)

The run with the default prompts and the in-context learning run are new, and the revision adds them to Section 3.4:

To separate the reasoning effort from the reflection prompt, we also ran the prompts of the default configuration with medium reasoning effort (Fig. S5). Relative to the default configuration, the SPR rises from 0.593 to 0.686 and polarization falls from 0.328 to 0.197, but the success rate for "Neutral" falls from 0.703 to 0.517.

Meanwhile, although in-context learning made no meaningful difference for GPT-4o-mini, it raises the SPR of GPT-5.4 to 0.673, close to medium reasoning effort, and raises the success rate for "Neutral" to 0.831; about a third of the transition mass still drifts, mostly through polarization (0.280).

Thank you to the authors for the comprehensive response. The rebuttal substantially addresses the strongest validity concerns. While I still have reservations regarding generalization and causal understanding, I now believe the paper makes a meaningful empirical contribution and provides a useful evaluation framework, and I increased the score.

Reviewer_g1GbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper investigates whether an intended stance can be faithfully preserved when one AI agent expresses it in an argument and another infers the stance from the resulting text. The authors formulate this process as a two-stage generation–extraction pipeline using five stance categories: strongly agree, agree, neutral, disagree, and strongly disagree. Given a proposition and a specified stance, a generation model produces an argument, and an extraction model then identifies the stance conveyed by that argument. Experiments across 112 debate propositions and nine language models show that all evaluated models achieve stance preservation rates below 70% under the default configuration. Polarization emerges as the predominant drift pattern for most models, with moderate agreement or disagreement shifting toward stronger positions on the same side.

Critical strengths
  1. The paper addresses a meaningful question about the fidelity of AI-mediated communication. As AI systems are increasingly used for both content creation and evaluation, understanding whether a receiving AI accurately recovers the creator’s original intent has clear practical relevance.

  2. Broad coverage of model families. The experiments evaluate a reasonably comprehensive range of language models from multiple families, providing evidence that stance drift is a recurring phenomenon across different models rather than an issue confined to a single model family.

  3. The paper is clearly written and well organized. The experimental framework, evaluation metrics, and main findings are presented in an accessible manner, making the study easy to follow.

Critical weaknesses
  1. The prompts in Appendix A do not clearly define the distinction between moderate and strong agreement or disagreement. Some reported polarization may therefore reflect inconsistent interpretations of these labels across generation and extraction. Although polarization exceeds moderation, this asymmetry does not rule out systematic differences in how the two stages interpret stance intensity.

  2. The exploratory human study covers only one proposition, limiting its ability to validate the labels across topics. In Table S12, an argument suggesting that abandoning AI may be premature and advocating regulation is labeled Neutral by humans but Disagree by the AI. The latter is a reasonable interpretation: discussing both benefits and risks does not necessarily imply neutrality toward abandoning AI. This makes the attribution to extraction error debatable.

  3. The study provides a useful evaluation perspective, but its contribution is primarily descriptive, using a straightforward generation–extraction protocol and an exact-match metric. Although increased reasoning effort yields a meaningful improvement, the evidence remains confined to a controlled debate setting. Given the unresolved measurement concerns and limited broader validation, I am not yet convinced that the overall empirical contribution is strong enough to support acceptance.

Questions for authors
  1. Is there any misunderstanding in the weaknesses?

  2. What reasoning-effort settings were used in the default, reflection, and third-person reflection configurations of GPT-5.4? Have you evaluated the default prompts with medium reasoning effort to determine whether reflection provides additional benefits when reasoning effort is held fixed?

4 comments
AuthorsOpenReview ↗

Response to Reviewer g1Gb (1)

We thank the reviewer for the careful reading and the comments, and respond to the weaknesses and questions below.

W1. "The prompts in Appendix A do not clearly define the distinction between moderate and strong agreement or disagreement ... this asymmetry does not rule out systematic differences in how the two stages interpret stance intensity."

We agree that the asymmetry between polarization and moderation may have several underlying mechanisms, including a systematic difference in how the two stages interpret stance intensity.

On the one hand, the framework uses the same model in both stages, so such a difference lies between the generation and the extraction of one model and is part of the pipeline we measure. On the other hand, the five labels are self-explanatory to human readers, as in Likert scales, which are commonly administered without definitions of the levels, and the human-extraction study supports this. The nine annotators saw the same undefined labels, and their majority label was "Disagree" for all 20 arguments generated from "Disagree" and "Disagree strongly" for 19 of the 20 generated from "Disagree strongly". On this proposition, the intensity written at the generation stage is therefore recoverable without definitions, whereas the LLM extractor gave "Disagree strongly" in all 360 votes on these 40 arguments.

Whether it arises from a difference between human and AI readers or between the two stages, such an inconsistency, together with its cause, is pervasive and directly affects communication, so it counts as stance drift. We thank the reviewer for raising this point, which underlines the need to study stance drift and offers a starting point for future work on its underlying mechanism.

W2. "The exploratory human study covers only one proposition ... This makes the attribution to extraction error debatable."

We agree that the human-extraction study is limited, and we present it as a proof of concept of the decomposition into stages. It covers one proposition and one model because unaided human labeling is costly in time and effort and crowdsourcing platforms risk responses written with AI help (Section 3.5).

The argument of Table S12 that the reviewer mentions was generated from the initial stance "Neutral". Eight of the nine annotators labeled it "Neutral" and one "Disagree", and all nine AI votes were "Disagree". The revision adds these vote counts to Table S12, next to each majority label: Neutral (8 of 9)\color{blue}{\text{Neutral (8 of 9)}} and Disagree (9 of 9)\color{blue}{\text{Disagree (9 of 9)}}. The near-unanimous human vote suggests that the annotators shared an understanding of "Neutral" and supports the human-majority label as the reference, although the sample of nine annotators is small. Moreover, whether the stance of the argument is "Neutral" or "Disagree" decides only the stage to which the change is assigned, not whether stance drift occurs.

W3. "its contribution is primarily descriptive, using a straightforward generation-extraction protocol and an exact-match metric ... the evidence remains confined to a controlled debate setting."

The paper is descriptive by design, and we agree that the scope of its conclusion should be stated, which the revision does in a new limitations subsection (Section 4.3):

Our conclusion is descriptive: it characterizes stance drift on the propositions of the debate speeches dataset and under the prompts we used, and it does not isolate a causal source. The two exploratory studies are limited in scope and serve as proofs of concept. The human-extraction study covers one proposition and one model due to costly labeling work (Section 3.5), and shows how drift can be located at the generation or the extraction stage. The email-writing task was run with one model (Appendix A.13) and explores drift in a concrete communication task.

Nevertheless, we consider the description a contribution in its own right, for three reasons.

  • It is an empirical finding enabled by a reusable evaluation framework. The transition matrix, SPR and the drift patterns apply to any model, and the same measurement detects the gain from reasoning effort (Q2), suggesting it can serve as the target of mitigation work.
  • The finding is consequential and has several facets. The paper gives a systematic evaluation of end-to-end stance preservation, and none of the nine models exceeds an SPR of 0.7. Beyond this exact-match score, the transition matrices separate the drift into five patterns, of which polarization, deviation from neutrality and flipping carry most of the drift of every model (Table 2).
  • The phenomenon is robust and persistent. As the reviewer notes, the models come from several families, which provides evidence that stance drift is a recurring phenomenon across different models. Drift also holds over 112 propositions, under the robustness checks and mitigation strategies of Section 3.4, and in the exploratory email-writing task.
AuthorsOpenReview ↗

Response to Reviewer g1Gb (2)

(Part 2 of 2)

Q1. "Is there any misunderstanding in the weaknesses?"

Our clarifications are given above.

Q2. "What reasoning-effort settings were used ... when reasoning effort is held fixed?"

The default, reflection and third-person reflection configurations do not specify the reasoning effort, so they run with the default, none. The revision states this in Appendix A.11:

The two configurations with medium reasoning effort use the prompts of the default configuration and the third-person reflection prompts, respectively. The default, reflection and third-person reflection configurations of GPT-5.4 leave the reasoning effort at its default, none.

We have now run the default prompts with medium reasoning effort, and the revision adds the run to Section 3.4, Fig. S5 and the appendix tables. Section 3.4 now reads:

To separate the reasoning effort from the reflection prompt, we also ran the prompts of the default configuration with medium reasoning effort (Fig. S5). Relative to the default configuration, the SPR rises from 0.593 to 0.686 and polarization falls from 0.328 to 0.197, but the success rate for "Neutral" falls from 0.703 to 0.517.

Three rows of Table S5, mean (SE) over the 112 propositions, with the new row in blue:

ModelSPRPolarizationModerationModeration to neutralFlippingDeviation from neutrality
GPT-5.4.593 (.008).328 (.006).002 (.001).000 (.000).018 (.004).059 (.006)
GPT-5.4 (medium reasoning)\color{blue}{\text{GPT-5.4 (medium reasoning)}}.686 (.013)\color{blue}{\text{.686 (.013)}}.197 (.008)\color{blue}{\text{.197 (.008)}}.011 (.002)\color{blue}{\text{.011 (.002)}}.001 (.000)\color{blue}{\text{.001 (.000)}}.010 (.004)\color{blue}{\text{.010 (.004)}}.097 (.007)\color{blue}{\text{.097 (.007)}}
GPT-5.4 (third-person reflection, medium reasoning).775 (.012).119 (.007).019 (.003).001 (.000).008 (.004).079 (.007)

Medium reasoning effort raises the SPR from 0.593 to 0.686, but it also raises deviation from neutrality from 0.059 to 0.097, which is the fall of the success rate for "Neutral". Adding third-person reflection at the same reasoning effort lowers deviation from neutrality to 0.079 and raises the SPR further, to 0.775. With the reasoning effort held fixed, reflection therefore gives an additional benefit.

AuthorsOpenReview ↗

Follow-up on rebuttal

Thank you again for your review. As the discussion period closes today, we wanted to check whether our responses address your main concerns. In particular, Section 3.4 now reports the default prompts of GPT-5.4 with medium reasoning effort, which separates the effect of reasoning effort from that of reflection (Q2), Table S12 gives the vote counts behind each majority label (W2), and the new Section 4.3 states the scope of our conclusion (W3). Please let us know if any points remain unclear.

Thank you for the detailed responses and additional experiments. The methodological clarifications and more complete experimental evaluation have addressed my concerns regarding soundness. In particular, the added control separating reasoning effort from reflection and the clarification of the human annotation results are helpful. However, I remain unconvinced that the scope and depth of the empirical findings, together with their broader implications, establish a sufficiently substantial contribution to support acceptance. I will therefore increase my soundness score while maintaining my overall rating.

Reviewer_Ub5ROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The study systematically examines the stance drift that appears in AI-assisted human-human communication. The authors set up an argument generation and extraction pipeline that simulates communication with AI-assistance. They propose the stance preservation rate (SPR) as a simple evaluation metric. Analyses show that for most models, polarization is the predominant pattern. The authors tested different mitigation strategies and found that reflection improved the SPR for GPT-5.4. However, these effects were not observed when testing GPT-4o-mini.

Critical strengths

This paper systematically examines shifts in stance that occur in AI-assisted communication. It is well-written, and the experiments are straightforward. The results demonstrate that stance shifts, particularly in the form of polarization, are common patterns that occur when LLMs revise texts or perform tasks such as generating arguments or summarizing text. The results are of interest for a variety of applications.

Critical weaknesses

My main critique of this work is that it fails to measure the actual stance shifts that would occur in human-to-human communication because the prompts used do not resemble the ones people would use when completing tasks, such as revising emails. Nevertheless, the experiments demonstrate that these shifts occur across all models, suggesting that the same would happen during everyday communication. Furthermore, the authors experiment with mitigation strategies that seem ineffective and are only tested on one model. It would be interesting to see if these strategies have different effects on different models.

The use of the SPR is understandable, but in my opinion, there are patterns that are more serious (“flipping”) than, for example, polarization, while maintaining the original underlying stance. How critically something should be viewed naturally depends on the context.

Questions for authors
  • How well does the benchmark predict fidelity in realistic communication tasks?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Ub5R (1)

We thank the reviewer for the careful reading and the comments, and respond to the weaknesses and the question below.

Realism of the pipeline. "... fails to measure the actual stance shifts that would occur in human-to-human communication ... such as revising emails." and "How well does the benchmark predict fidelity in realistic communication tasks?"

Realistic communication is also the motivation of our study. To see how drift behaves in a realistic task, we added an exploratory run with GPT-5.4 as a proof of concept, in which the generation prompt asks for a short email to a colleague in place of the argument and the extraction prompt is unchanged. Drift remains, with an SPR of 0.713. Polarization is still the largest pattern, and an email written for "Agree" is read as "Agree strongly" in 0.494 of the trials. The revision describes the run in a new Appendix A.13 and Table S14:

The email-writing task keeps the prompts of Appendix A.3 except for three lines of the generation prompt: the task instruction, the output instruction and the final cue are replaced by the following ones.

Based on the specified letter, write a short email of at most {words_limit} words to a colleague in which the writer states a position on the proposition that is consistent with the corresponding stance.

Only output the email itself and do not directly quote or explicitly mention the option or its label.

Email:

The extraction prompt is unchanged. The task was run with GPT-5.4, as an exploratory check of a second generation task, with the other settings of the default configuration. The SPR is 0.713, against 0.593 for the argument, and polarization remains the largest pattern (Table S14).

Table S14: SPR and the components of the SDR by drift pattern for GPT-5.4 with the argument and with the email-writing task. As Table S5: mean (SE) over the 112 propositions. The row GPT-5.4 is the default configuration, in which the model writes an argument.

ModelSPRPolarizationModerationModeration to neutralFlippingDeviation from neutrality
GPT-5.4.593 (.008).328 (.006).002 (.001).000 (.000).018 (.004).059 (.006)
GPT-5.4 (email task).713 (.010).206 (.006).002 (.001).000 (.000).011 (.003).068 (.006)

However, real communication varies in setting, prompt and text length, so neither a single setup nor a small set of setups covers all of it. As a first study of stance drift in AI-mediated communication, we reduce communication to a generation-extraction pipeline that keeps its essence, a sender who expresses a stance and a receiver who reads it, and that can be quantified. Covering more real-life scenarios is a direction for future work. The revision states the scope of the conclusion in a new limitations subsection (Section 4.3):

Our conclusion is descriptive: it characterizes stance drift on the propositions of the debate speeches dataset and under the prompts we used, and it does not isolate a causal source. The two exploratory studies are limited in scope and serve as proofs of concept. The human-extraction study covers one proposition and one model due to costly labeling work (Section 3.5), and shows how drift can be located at the generation or the extraction stage. The email-writing task was run with one model (Appendix A.13) and explores drift in a concrete communication task.

As the reviewer notes, the shifts occur across all models, which suggests that the same would happen during everyday communication; they also remain under the robustness checks and mitigation strategies of Section 3.4. We hope that the exploratory run, together with these results, addresses the concern.

AuthorsOpenReview ↗

Response to Reviewer Ub5R (2)

(Part 2 of 2)

Mitigation. "... only tested on one model. It would be interesting to see if these strategies have different effects on different models."

We agree that the effect of the strategies on other models is of interest. Given the short discussion period, we ran multiple extraction and in-context learning on GPT-5.4. With the existing reflection runs, three strategies can now be compared between GPT-4o-mini and GPT-5.4.

Eight rows of Table S5, mean (SE) over the 112 propositions, with the new rows in blue:

ModelSPRPolarizationModerationModeration to neutralFlippingDeviation from neutrality
GPT-4o-mini.570 (.009).246 (.011).022 (.004).009 (.002).097 (.012).057 (.005)
GPT-4o-mini (multiple extraction).569 (.009).218 (.009).050 (.004).011 (.002).095 (.011).057 (.005)
GPT-4o-mini (in-context).573 (.011).271 (.010).011 (.003).003 (.001).088 (.012).052 (.006)
GPT-4o-mini (reflection).586 (.010).204 (.012).032 (.005).008 (.002).123 (.013).047 (.005)
GPT-5.4.593 (.008).328 (.006).002 (.001).000 (.000).018 (.004).059 (.006)
GPT-5.4 (multiple extraction)\color{blue}{\text{GPT-5.4 (multiple extraction)}}.609 (.008)\color{blue}{\text{.609 (.008)}}.305 (.006)\color{blue}{\text{.305 (.006)}}.006 (.001)\color{blue}{\text{.006 (.001)}}.000 (.000)\color{blue}{\text{.000 (.000)}}.022 (.005)\color{blue}{\text{.022 (.005)}}.056 (.006)\color{blue}{\text{.056 (.006)}}
GPT-5.4 (in-context)\color{blue}{\text{GPT-5.4 (in-context)}}.673 (.008)\color{blue}{\text{.673 (.008)}}.280 (.007)\color{blue}{\text{.280 (.007)}}.002 (.001)\color{blue}{\text{.002 (.001)}}.000 (.000)\color{blue}{\text{.000 (.000)}}.010 (.003)\color{blue}{\text{.010 (.003)}}.034 (.005)\color{blue}{\text{.034 (.005)}}
GPT-5.4 (reflection).613 (.009).283 (.007).002 (.001).000 (.000).042 (.007).060 (.006)

The strategies act differently on the two models. The revision adds the two runs to the appendix tables and figures and the in-context learning run to Section 3.4, which now reads:

Meanwhile, although in-context learning made no meaningful difference for GPT-4o-mini, it raises the SPR of GPT-5.4 to 0.673, close to medium reasoning effort, and raises the success rate for "Neutral" to 0.831; about a third of the transition mass still drifts, mostly through polarization (0.280).

Weighting of patterns. "... there are patterns that are more serious ("flipping") than, for example, polarization ... How critically something should be viewed naturally depends on the context."

We agree that this depends on the context. The paper therefore reports the five patterns separately (Table 2) to give a full view, in which flipping, for example, can be read on its own. As Section 2.3 explains, SPR is derived from the L1L_1 distance between the transition matrix and the identity matrix and serves as a general baseline that requires no assumption on the weights. Users can compute a weighted version from the same transition matrix, with weights set by the importance of each pattern in their context.

AuthorsOpenReview ↗

Follow-up on rebuttal

Thank you again for your review. As the discussion period closes today, we wanted to check whether our responses address your main concerns. In particular, Appendix A.13 and Table S14 now report an exploratory email-writing task with GPT-5.4, in which stance drift remains, and Section 3.4 and Table S5 add multiple extraction and in-context learning on GPT-5.4, so that the mitigation strategies can be compared between two models. Please let us know if any points remain unclear.

Reviewer_Ub5ROpenReview ↗

Answer to Response to Reviewer Ub5R

Dear authors,

thank you very much for your detailed response and for providing an additional example of realistic AI usage. While it is not possible to cover all aspects of real communication, this does provide a useful indication of the transferability of the results. I will increase my initial score accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.