acceptodds
Under review as a conference paper at ICLR 2027

Do You Copy? When Language Models Overwrite Observed Values with Remembered Defaults

14 views

Abstract

Coding agents copy values from tool output into the files they write. We show that when the output names a well-known service on a non-default port, such as redis on 6392, language models overwrite the observed port with the default they remember. We call this failure default capture. Both values are true, so counterfactual knowledge-conflict benchmarks cannot measure it. We isolate it by renaming the service: the name effect is the real name's rate of writing the remembered default minus the rate under a made-up name of the same length. Under made-up names six instruction-tuned models copy the port from command output almost without exception, so a default written under the real name is attributable to the name. Default capture concentrates where agents meet it: returning command output as a tool result instead of a chat message triples the average effect, while authored configuration files stay near zero. In the preregistered chat test a docker ps row exceeds a .env line, and three of six models pass the per-model criterion. A header naming the source of the text, as in retrieval-augmented generation, lowers the effect below chat delivery, and under tool delivery dropping the service name from the line being written removes most of it. In a sandboxed agent loop, Qwen3.5-9B with thinking on sets the documented default in 24 percent of real-name episodes and 1 percent under made-up names, DeepSeek V4.1-Flash in under 1 percent of either. Yang (2026) found the same pull in Python import aliases; the six models repeat it, as a preregistered test predicted. Training toward the model's own behaviour under made-up names cuts the effect on a 7B model from 28.8 to 1.0 points under tool delivery on held-out services and layouts, with recall of remembered defaults within 3 points. Every repair trained only toward the page, and every inference-time method we compared that lowers the effect, trades copying the observed value against a second request: what port the service normally uses. Supervising that request too, on prompts rotated through raw, chat and tool form, lets a 1.5B model answer it at 76 to 83 percent in every delivery, against 31 to 47 for the frozen model, with the name effect near zero (exploratory).

Primary area:
foundation or frontier models, including LLMs
Keywords:
knowledge conflict, context faithfulness, LLM Agents, tool use, parametric knowledge

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FpjXOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates Default Capture, a failure mode where language models replace externally observed values with default values stored in parametric memory. A controlled real-name vs. made-up-name comparison is introduced to isolate this effect across models and input settings. The paper further proposes Name-Conditioned Self-Distillation (NCSD) to mitigate Default Capture while preserving default-value knowledge. Experiments demonstrate the prevalence of this issue, particularly in tool-output settings, and the effectiveness of NCSD.

Critical strengths
  1. Interesting and practically relevant failure mode. The paper identifies a meaningful failure mode where parametric knowledge overrides observed values, particularly relevant to coding agents that rely on environment observations.

  2. Careful experimental design and analysis. The real-name vs. made-up-name comparison effectively isolates prior-knowledge interference. Experiments across six models, 48 services, and multiple settings provide convincing empirical evidence.

Critical weaknesses
  1. Limited generalization beyond default-value copying.

The experiments primarily focus on service ports with well-known defaults. It remains unclear whether Default Capture generalizes to more complex coding scenarios involving API parameters, version dependencies, or dynamic configurations.

  1. Limited validation in realistic coding agent workflows.

Most experiments involve generating a single value in prefilled code rather than realistic agent workflows. Although Appendix U includes agent-loop experiments, only two models are evaluated, with substantially different results. The impact on end-to-end task success remains unclear.

  1. Limited methodological novelty of NCSD.

NCSD closely resembles existing consistency training or context distillation approaches. Its main modifications involve constructing teacher targets using made-up names and applying KL anchoring, making its standalone methodological contribution relatively limited.

Questions for authors
  1. Generalization. Does Default Capture extend beyond service ports to more complex coding scenarios, such as API versions or dynamic configurations? Does it require a widely known default value?

  2. Realistic agent workflows. Does Default Capture remain significant when coding agents can verify and correct configurations through execution feedback? Can NCSD improve end-to-end task success rates?

  3. Methodological novelty and knowledge preservation. What distinguishes NCSD from existing consistency training or context distillation methods? Does it genuinely improve the model's ability to balance contextual observations and parametric knowledge, rather than simply increasing reliance on context?

Reviewer_Vo9gOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies "default capture." A coding model is shown command output with a service on a non-default port, such as redis on 6392, and writes the port it remembers instead, such as 6379.

It isolates this with a renaming control. Each real-name prompt is paired with three made-up names of the same length. Under made-up names, the models copy the observed port almost perfectly, so the extra default answers under the real name can be attributed to the name.

The preregistered study covers six open instruction-tuned models, 48 services and 17 layouts. The effect is small in chat and about three times larger when the output comes back as a tool result. It stays near zero for authored files like .env. Most of it is carried by the service name in the line being written.

Extra checks include an agent loop, where Qwen3.5-9B shows a large effect and DeepSeek V4.1-Flash shows none. There is also an API model on a probability scale, and a repeat of the import-alias result in code.

The repair trains a LoRA adapter toward the model's own behaviour under made-up names, anchored so it keeps remembered defaults. This removes the effect on held-out services but hurts answers to "what is the default port" requests. Supervising that second request fixes most of the trade-off, but this part is exploratory.

Critical strengths

The renaming control is clean and convincing - Made-up names copy the port on nearly all 36,288 tool prompts, so the effect is clearly tied to the name and not to general copying errors.

The research is transparent - The study is preregistered, failed tests are reported openly, deviations are listed, and the delivery steps are separated one at a time. Even where the effect is small, the claims are carefully qualified.

Critical weaknesses

Practical significance and generality are limited. The confirmatory effect is small and holds in only three of six models. The large numbers come from exploratory conditions, a prefilled single line, or one model in the agent loop. DeepSeek checks the port and shows no effect, and the path fields show nothing. So it is unclear how much this matters for real, stronger agents.

The repair evidence is weak for an ICLR-level method claim. Its main runs use a small model with a tiny base effect, the 7B result is one fold and one seed, held-out evaluation reuses training templates, and general ability is not tested. The best version, supervising the second request, was designed after seeing the trade-off and is fully exploratory.

Questions for authors

how much of the tool-delivery rise is template markup, such as Qwen's empty reasoning block or Gemma continuing inside the tool turn? Could a template-matched control rule this out?

does the effect hold with realistic docker ps output only, and with the 7B repair across both folds and more than one seed?

what happens to general capabilities after NCSD on standard coding or instruction benchmarks, not just the anchor families?

why was Qwen2.5-1.5B chosen for the main repair when it is not diagnosed and shows the opposite delivery pattern?

test stronger frontier agents with reasoning on, and fields other than ports where models do hold strong conventions.

Reviewer_ksHCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Coding agents copy values from tool output into the files they write. This papers show that when the output names a well-known service on a non-default port, such as redis on 6392, language models overwrite the observed port with the default they remember. The paper calls this failure default capture.

Training toward the model’s own behaviour under made-up names cuts the effect on a 7B model from 28.8 to 1.0 points under tool delivery on held-out services and layouts, with recall of remembered defaults within 3 points.

Critical strengths
  1. The identified issue on overwrite observed values with remembered defaults can happens frequently by next token prediction mechanism.

  2. Experiments validate the issue and the paper shows SFT with made-up name can improve the accuracy.

  3. The failures and side effects on reduced accuracy when users explicitly request the default are also analyzed.

Critical weaknesses
  1. The effect of LLM reasoning effort can be explored. Stronger LLMs may also have reduced such issue.

  2. Standard SFT might be sufficient instead of label-free self-distillation.

  3. The transferability of the training method on unseen tasks can be provided.

Questions for authors

As in weaknesses. Deeper mechanism analysis may further enhance the quality of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.