- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether coding models retain task information in reusable internal states that separate function, programming language, and implementation strategy. It first screens 26 models for their ability to generate code after attention to the task specification is removed. It then uses donor-derived state interventions to transport, install, and compose task factors across contexts in selected models. Additional experiments examine a synthetic source with known structure, cross-language transfer from program examples, and Python-only fine-tuning. The central finding is that persistent task states support substantial cross-context reuse, although the interpretation of these states as abstract programs is stronger than what some of the controls establish.
Critical strengths
-
The intervention design provides meaningful evidence of reusable task information. Estimating a function edit outside the recipient language, and a language edit outside the recipient function, is substantially more informative than showing that task identity can be decoded from activations. Composing these edits in an otherwise underspecified prompt is especially compelling. The execution-based evaluation and negative controls make the main result difficult to dismiss as a generic perturbation effect. The distinction between two-factor composition and the more difficult three-factor setting is also useful rather than being hidden in an aggregate score.
-
The computational-modifier experiments strengthen the contribution beyond task swapping. Appendix A.14.2 estimates modifications while excluding the recipient’s base function, language, and wording. Successful transfer in this setting is a stronger test of reuse than installing a function that already appears in the donor pool. The nontrivial success rates across all four tested models, together with unsuccessful random and wrong-modifier controls, provide a useful empirical result even without accepting a globally additive account of program representations.
Critical weaknesses
-
Section 5.1 overstates what the experiments establish about the content of the retained state. Non-identical generated code in Table 1 rules out straightforward verbatim reproduction, but it does not rule out a compressed task description or an algorithmic sketch that the model subsequently realizes as code. The strategy-transfer example likewise shows that donor information can influence computational organization, not that a prompt-summary explanation has been excluded. The operational definition in Section 6—state from one context supporting predictable reuse in another—is supported. The stronger statement that the state is neither a prompt summary nor a copy of an implementation needs a discriminating test or narrower wording.
-
The generalization evidence should be interpreted more narrowly. The high transplant success rates are conditional on finding donors whose task information survives masking. For example, Table 5 reports 13/90 qualifying pairs for Qwen on faromin, followed by 13/13 successful transplants. That is strong conditional reuse, but not broad unconditional transfer. The fine-tuning experiments also do not clearly distinguish learning a new computation from binding a new name to a familiar computation: vantril counts distinct digits, while thulep computes the difference between the largest and smallest digit. These experiments remain informative, but they provide weaker evidence for acquisition of previously unknown algorithms than the main narrative suggests.
Questions for authors
-
Which result is intended to distinguish a reusable computational representation from a compressed instruction or algorithm-description state? If no existing experiment separates these explanations, would the authors narrow the conclusion in Section 5.1 to the operational claim made in Section 6?
-
How should the SFT results be interpreted relative to the “untouched base, described in plain words” comparison in Figure 13? Do the results demonstrate acquisition of a computation the model previously could not implement, or mainly cross-language transfer of a newly learned name-to-computation association? This distinction affects the significance of the generalization claim.
Response to Reviewer yvRq
We appreciate the reviewers' positive comments about the intervention/experimental design of our study, particularly with respect to the strength of our evidence when composing representation. We also think it’s quite compelling that we can estimate the edits without using the recipient’s context in the donor sequences!
The referee points out that some elements of section 5 are confusingly worded, and potentially obfuscate some results. We agree and propose to make significant changes to the section to remedy this. We will start by renaming 5.1 from “what is an abstraction?” to “How abstract are these program representations?” to better represent the actual content of the section. Additionally, we plan to rewrite the full section so that our exact contributions and experiments are clear. The role that our experiments play in establishing a general picture of “abstraction” will be confined to the conclusion section to clearly differentiate motivation/discussion from concrete results. Excepts of these rewrites can be found in the Overall Response.
The reviewer also correctly points out that not all tests show strong transplant success. However, we interpret the fact that the failures are correlated to donors that survive masking as a feature. If the donor doesn’t survive the masking operation, then it likely is not forming the structured reusable representation for the given computation (and unmasked generation would work likely primarily through re-reading the provided program and, perhaps, translating it). In light of this, we think of this evidence as supporting the generalization behavior being dependent on the existence of the representation. This point should be made clearer, and will use the extra room in the revision for it. As an additional point, the results of the fine tuning experiments depend strongly on how well the fine-tune tune is optimized. We have not optimized our fine-tune, since our point is simply that it is possible to generalize with very little data– not to develop a useful low-data training algorithm. Thus, we caution against interpreting the algorithm success rates as more than showing a qualitative pattern. This is discussed in the appendix, but we can use the extra room in the revision to clarify.
We discuss our thoughts about whether the model is truly learning a new computation versus binding an already established one in the question responses below.
Q1: We believe that the current tests do support that the representation is not a copy of the prompt (the prompt is 8X longer than the suffix, so it can't be a copy of the prompt). However, like the referee points out, we do not show that the representation is not a “summary” of the prompt. What is even meant by a summary is left somewhat open (admittedly by us, but also the reviewer). In fact, it might even be reasonably argued that an organized abstract representation itself is a type of summary (compression) of the prompt – or vice-a-versa, that a compressed summary can be considered an abstraction. As suggested, we will make the wording more narrow– as well as rewrite section 5.1 for clarity. While we plan more changes to section 5.1, the specific part in question will be edited as follows:
neither a prompt summarynot an exact copy of an implementation it has seen."Q2: Any good coding model should be able to perform any invented algorithm if it is clearly explained in a prompt. All of these programming languages are universal, so any new algorithm can be thought of as a combination of existing primitives. This is all to say that, while we agree that “vantril” and “thulep” can be described in terms of existing algorithms, we think it is pretty unlikely that they exist in enough density in the training data for a model to have this particular combination of primitives already established as an abstract computation in the same way that the “sum of two squares” likely would be. Our experiments go somewhat towards showing this, in fact. For example, models find the more complicated invented algorithms “kanex” and “faromin” more difficult to transfer than the simpler ones, despite them all using similarly simple primitives. We can interpret this as evidence that the models are building new representations rather than linking up existing algorithmic combinations. Overall we agree with the reviewer that there is a risk that generalization of simpler algorithms works primarily through binding a new name to an existing computation, which is why we include the more invented algorithms that are unlikely to already exist under a specific name. Although, we also note that the more complicated functions potentially need more training to learn so this could just as easily be due to poor training optimization for these particular tasks, as task complexity and how it bears on generalization.