acceptodds
Under review as a conference paper at ICLR 2027

Can LLMs Separate Pasted Artifacts from User Speech? Absorption at Unmarked Prompt Seams

Abstract

Large language models (LLMs) receive each user message as plain text, even when it combines text from different sources. For example, a user may paste text into a prompt and keep typing a comment directly below it. We study absorption: a phenomenon where the model treats a trailing user comment as part of the pasted text, returning it inside the edited text. This happens even though the user did not intend the comment to become part of that text. Existing instruction-data separation benchmarks tell the model which text is instruction and which is data, then test whether it obeys that separation. They do not test harmless user speech following an unmarked paste. We introduce SEAM, a controlled benchmark of 300 editing examples. Each example is tested under six matched conditions that vary how the boundary between pasted text and later user speech is expressed. Across 20 models, absorption at a bare newline ranges from 7.7% to 66.7%. Adding a blank line does not significantly reduce absorption in any model, while boundary markers reduce it in 19 of 20 models. Comments that fit the pasted text, such as a code comment typed after code, are absorbed significantly more often in 17 of 20 models. Models often fail to separate pasted material from later user speech, and explicit boundaries reduce but do not remove this failure.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
instruction-data separation, prompt boundaries, absorption, LLM evaluation, benchmark, source attribution, instruction following, chat interfaces, controlled evaluation, robustness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qDskOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines absorption which is a failure mode in which an LLM incorporates a user's comment into text that the user intended only to edit. In this context, the authors introduced SEAM which is a benchmark of 300 editing examples tested under six matched conditions. They then evaluate a set of language models on this dataset and find that absorption occurs across all tested models.

Critical strengths
  • The paper is easy to follow and it identifies an interesting and underexplored failure mode with practical implications for everyday LLM use.
Critical weaknesses
  • Perhaps my main concern with the paper is that it falls short of what it can bring. Particularly, the paper identifies and measures the problem but stops short of developing a method to prevent it (beyond explicit boundary markets). The contribution will be much more meaningful if beyond identifying the problem, the authors also propose a method or ways to circumvent this by the model through specific training or training data to be included.

  • Secondly, the relationship between the semantic content of the artifact and the trailing user comment needs further investigation. How much does semantic similarity, or the degree to which the comment naturally fits the artifact, explain absorption? This is important for understanding when the failure is most likely to occur. From my personal interaction with LLMs, I have never observed absorption and for that reason I worry about in what type of cases does this comes up.

Questions for authors
  • How does the semantic similarity between the pasted artifact and the trailing comment affect absorption? Can you isolate this factor from whether the comment fits the artifact's genre?

  • Beyond adding explicit boundary markers, have you explored methods that could help models reliably distinguish the artifact from the user's instructions or comments?

Reviewer_7YNaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper defines a new phenomenon absorption, i.e when LLMs cannot identify the boundary between a pasted artifact (e.g. code snippet or an email) and a casual user comment following it. The paper designs experiments to measure the conditions under which the LLMs can identify this boundary (newline, explicit boundary markers etc..).

Experiments are run on 20 models (including frontier models like GPT 5.6 sol, Opus 4.8, gemini 3.1 pro.). They find that absorption rate is high (20-30 range for these above frontier models) when using a fairly implicit boundary like newline, but explicit boundary markers and mitigation reduces this substantially.

Critical strengths

It is well-known that today's benchmarks do not capture the actual settings in which users use LLMs. In benchmarks, the inputs are well-formatted and task instructions are clear, whereas this is not the case in real world. This paper touches upon an interesting feature of user-generated queries -- they are not well-formatted and often do not sufficiently distinguish between pasted artifacts and surrounding text. However, I have some concerns about the controlled experiment setup. I have elaborated on them in the weaknesses section.

Critical weaknesses
  1. The paper frames absorption as a particular failure mode for LLMs. However, looking at the data construction, I am not convinced that it is an actual failure mode, as for a number of settings, the "correct" action to take is very ambiguous and open to interpretation. For e.g., the task instruction is "improve clarity..." --> I do not consider including the user's casual comment "I still need to ..." in the final response and applying the task instruction on it as well as incorrect, if there are no explicit markers separating the two.

For me, the task is only unambiguous, i.e. absorption is a clear failure, for the boundary (explicit boundary markers) and mitigation (instruction specifying to leave the line out) cases. For these, the observed absorption rate for the frontier models is actually quite low.

  1. Their result shows that absorption rate is higher for text edit tasks and almost negligible for code sources. This only supports point 1 above. For code, the task is unambiguous but for prose, the "correct" action is genuinely ambiguous.

  2. A more convincing setting will be one where the user comment contains some clear signals that it is not a part of the pasted artifact. This could be a continuation of the task instruction (e.g. "clarify this..." <pasted artifact> "btw also ensure proper spacing"). Otherwise, we run the risk of unfairly criticizing models for reasonable decision.

  3. The paper studies the absorption phenomenon in a very controlled setting. This controllable nature allows for an in-depth analysis, but the paper only includes a very high-level analysis. The paper can be strengthened by adding the following analysis: (1) LLMs may fail to recognize the boundary under the above setting, but can they recognize the boundary when explicitly asked? (2) Are certain user comments more likely to result in absorption than others? Are their any semantic/syntactic properties that affect it? Beyond the casual / artifact-native categorization. (3) Can we identify in-the-wild examples of inputs where the boundaries are not clearly stated, and test LLMs against that. It would help address the concerns about the artificial nature of the experiment setup.

Questions for authors

My main criticism is point 1 of the weakness above. I am not convinced that the "failure" mode being studied is a genuine failure vs genuine ambiguity in what the correct action is.

Please also address the questions in weakness 4 above.

Reviewer_1fL2OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Introduce a benchmark to test Absorption, where the model includes text that is not part of the intended artifact to be revised or changed. Experiment on 300 samples on 6 different conditions showed that simple separation like newline or blank line does not enough, where almost all type of models from frontier proprietary ones to small open models still has quite noticeable absorption rate (ranging from 7.7%-66.7%). The absorption rate significantly decrease for most models when provided with clear boundaries between the intended text to edit and the user comment, with further decrease until 0% absorption rate when the comment is replaced by a mitigation instruction. However, when the comment are replaced with something similar to the text-to-edit (called artifact-native), all models still suffer with quite high absorption rates.

Critical strengths

Overall, I highlight the grounding to actual use case from LLM users as the strengths of the paper

  • I found the setup to be highly related to what most LLM users encountered in their daily usage of LLMs, which makes the benchmark practical and grounded.
  • The findings on two setup, specifically newline and one blank line, where results show it does not have statistically significant reduction (even increase at some models), is very useful specifically for LLM users where I believe lots of people use it this way (one blank line)
Critical weaknesses
  • Limited impact and importance from scientific and modeling perspectives, with novelty and contributions incremental relative to other works in the literature. It also lacks mechanistic novelty, to understand further on the investigated behavior of the LLM experimented in the benchmark (specifically open models) on why it happened in the first place and what action could solve this problem (apart from UI and prompting solutions)
  • Ambiguity of the unmarked seams, specifically on the artifact-native setup. A lot of cases happened here actually does not have a theoretically identifiable boundaries, especially when the text-to-edit also has things like newlines inside them. Without human baseline verifying the ambiguity of such cases, I think it kinda confuse the target construct the benchmark wants to elicit and fix from LLMs
  • Although its clear the paper wants to aim specifically on absorption, absorption only explain the separation ability and not the edit-quality metric itself which is missing. If the paper wants to focus on the separation ability, then I think there are much better cases rather than current position (separating user comment from text-to-edit artifact) such as separating scenes from movie scripts/conversations, separate codes into files from a stacked one-file code, etc. As I mentioned in the first point, current position makes the paper kinda limited and feels incremental to prior works
Questions for authors
  • What would be the follow up of the benchmark from the current findings of the paper? For example, how can model developers improve their models on this absorption issue?
  • How clear the boundary between the user prompt/comment and the artifact? Is actual human can always tell the right boundary all the time? How about with the artifact-native cases?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.