- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates why decoder-only code models are brittle to variable renaming. It introduces TinyTracer to generate synthetic Python programs and execution traces under controlled distributions. Experiments vary the diversity of variable names in training and show improved robustness with greater diversity. The paper also studies a 158M-parameter StarCoder model trained on real Python code. Its error analysis identifies difficulties in generating unfamiliar names. Attention interventions provide evidence that particular heads contribute to predicting name endings in the tracing models and initial name tokens in StarCoder. The experiments connect controlled changes in training data with analyses of specific prediction failures.
Critical strengths
1. Controlled experiments on variable name diversity
The controlled experiments provide evidence that greater diversity of variable names in training improves robustness to renaming. The control experiment applies the same renaming procedure but draws names from a small pool. This helps separate the effect of diversity from the removal of semantic naming cues.
2. Causal evidence from attention interventions
The attention interventions provide causal evidence that the identified heads contribute to identifier prediction. In StarCoder, directing these heads to earlier occurrences of the target variable repairs many errors on the first token. The comparisons with random heads and other target variables strengthen this evidence.
Critical weaknesses
1. The specificity of retrieval and its link to name diversity remain unclear
Table 5 shows substantial repair when the selected heads attend to delimiters after other variables. On the same probes, this repairs 62–73% as many errors as attending to the matching variable. Although the matching variable is more effective, it remains unclear how much of the repair depends on identifying its boundary. A general signal to end the name may also contribute. The paper acknowledges this possibility. Reporting the overlap between the repaired cases, together with attention patterns before intervention, would help clarify the role of these heads.
The connection between name diversity and attention routing remains correlational. Similar repair rates in StarCoder show that both models benefit from forced retrieval. They do not explain why greater diversity improves predictions without intervention. Figure 7A reports average copy scores from only one training seed. An analysis of individual examples could test whether improved predictions are associated with stronger attention to the target variable before intervention. Repeating this analysis across all three training seeds would strengthen the evidence.
2. The effect of local repairs on task accuracy remains unclear
The error analysis shows that many failures begin during variable name generation. However, the interventions measure repairs at individual token positions. They do not show how often these repairs recover a correct identifier or execution trace. For StarCoder, the existing benchmark already scores complete identifiers under teacher forcing, requiring every token to be correct. Applying this scoring rule after intervention would connect the repair results to identifier accuracy. An additional evaluation of generated traces would help quantify how much of the accuracy loss caused by renaming can be recovered in the tracing model.
The mechanistic analysis focuses on identifier generation. The paper also reports benefits from greater name diversity on variable misuse localization and cloze prediction. These results do not establish that the same mechanism explains those benefits. The conclusions should distinguish the mechanism examined here from the broader effects of diversity training.
Questions for authors
1. Clarifying the role of attention
Please report the overlap between the two sets of repaired errors in Table 5. For StarCoder, please examine whether examples improved by greater name diversity also show stronger attention to the target variable before intervention.
2. Measuring the effect on task accuracy
For StarCoder, please apply the existing identifier scoring rule after intervention, requiring every token to be correct under teacher forcing. For the tracing model, please evaluate the accuracy of complete generated traces. Both evaluations should include any errors introduced on examples that were correct before intervention. Any use of reference information to select intervention positions or targets should be stated explicitly.
4 comments
Response Part 2
W1, second paragraph / Q1, second part (link between name diversity and attention; Fig 7A uses one seed; do improved examples show stronger attention before intervention?). We ran two new analyses during the discussion period to answer these two questions. For the submission, we had trained each of the two 158M models (baseline and high-diversity) three times, with three different training seeds, but Fig 7A showed the copy score for one of these seeds only. The new analyses use these six trained models, and no model was retrained. In the first analysis, we computed the copy score of Fig 7A for the two other seeds, and the observation of Fig 7A holds in all three seeds. In the second analysis, we measured, for each test example and before any intervention, how much attention the copy heads put on the earlier occurrence of the variable. Improved examples (examples that only the high-diversity model gets right) have stronger attention in the high-diversity model, when each model is measured with its own top-3 copy heads. Both results are correlations, and we agree that the link between diversity and attention is not shown to be causal.
- Fig 7A over three seeds. On renamed code, the copy score is .209 for the baseline and .256 for the high-diversity model. The copy score is higher for the high-diversity model in all three seeds. For each seed, the difference is the copy score of the high-diversity model minus the copy score of the baseline, so a positive value means that the copy heads of the high-diversity model attend more to the earlier occurrence of the name. The mean difference over the three seeds is +.046, with a 95% interval of [-.055, +.148]. The interval is wide and includes zero: the difference is positive in each of the three seeds, but three seeds are not enough to establish its size.
- Attention before intervention, per example. Within each model, examples with more attention to the earlier occurrence of the variable are more often correct (AUROC .80 to .86). We then took the examples that only the high-diversity model gets right. On these examples, we compare the attention that the top-3 copy heads of each model put on the target variable. The difference is the attention of the high-diversity model minus the attention of the baseline, so a positive value means that the high-diversity model attends more to the target variable on the examples that it alone gets right. The mean difference is +.073, and the difference is positive in all three seeds. This result is not robust to the choice of heads: with 12 heads the difference is close to zero, and with the baseline's heads used for both models it is negative in two of three seeds (the baseline then attends more). We have updated Section 4.5 and the caption of Fig 7 in the revised paper to reflect this. App. E.3 of the revised paper gives the three-seed values (new Table 7) and the per-example analysis.
Response Part 3
W2, first paragraph / Q2 (effect of the repairs on complete identifiers and on complete traces; errors introduced; use of the reference). We ran two new experiments during the discussion period to answer this question. The submitted paper measured a repair on a single token, and only at positions where the model fails. The new experiments measure whether complete outputs become correct, and whether the intervention introduces errors where the model was right. In both experiments, the intervention is the attention redirection of the submitted paper: we force the copy heads to attend to the earlier occurrence of the variable. We deduce three things from the two experiments. First, the effect of the repairs is not limited to one token: the repairs lead to complete outputs that are correct, for most identifiers in the 158M model and for a part of the failed traces in the tracing model. Second, the copy heads are needed to produce complete correct traces, not only to make one prediction. Third, the redirection helps where the model fails, but it can disturb predictions that were already correct (rarely in the 158M model, more often in the tracing model). The redirection is therefore a diagnostic of the mechanism, not a method to improve accuracy.
Experiment 1: 158M model, complete identifiers. We apply the redirection at all names of the renamed test code, including the names that the model already writes correctly. An identifier counts as correct only if every one of its tokens is predicted correctly.
- Complete identifiers recovered. The share of renamed identifiers that are completely correct rises from 32.2% to 70.7% for the baseline, and from 40.8% to 84.6% for the high-diversity model. The gain of the repair is therefore not limited to one token: it holds for complete identifiers.
- Errors introduced. Of the names that were correct before the redirection, 4% (baseline) and 6% (high-diversity) become wrong (counts pooled over the three seeds). The redirection therefore repairs many more names than it breaks.
- Use of the reference. The reference gives the position where each name starts and the earlier occurrences of the same variable (the tokens that the heads are forced to attend to).
Experiment 2: tracing model, complete generated traces. Each model generates the complete trace of 5,120 held-out renamed programs (greedy decoding). A trace counts as correct only if it is identical to the reference trace, so one wrong token makes the trace wrong. Below, each result is given first for the models trained with 0.5% diversity, then for 1% diversity (3 seeds each; accuracies are means over the seeds, and shares of traces use counts pooled over the seeds). Without intervention, 21.5% and 37.7% of the traces are correct; we call the other traces failed traces.
- Removing the three copy heads tests whether the model needs these heads to generate a complete trace. The share of correct traces falls to 1.2% and 0.3%: 95% and 99% of the correct traces become wrong. As a control, removing three random heads leaves 17.8% and 30.5% of correct traces on average. The effect of random heads varies widely between the 18 random draws (share of correct traces that become wrong: median 7%, from 2% to 100%).
- Redirection at every name ending, whether the model was right or wrong there. Of the failed traces, 10% and 20% become correct. Of the correct traces, 25% and 32% become wrong (errors introduced). The two effects nearly cancel: the share of correct traces becomes 23.8% and 37.9%, a small change on average, and the direction of the change varies across seeds.
- Redirection only at the name endings where the model makes an error (1.0 to 1.6 positions per failed trace on average). Of the failed traces, 13% and 28% become completely correct, and the share of correct traces rises to 31.7% and 54.9%. This intervention introduces no error by construction, because a correct trace contains no such position.
- Use of the reference. The reference trace gives the positions where a name ends and the delimiters to attend to. In the last intervention, the reference also tells where the model makes an error.
Response Part 4
What these results show. The three heads are needed to generate complete traces, not only to make one prediction: without them, almost no trace is correct. Repairing the failing name endings is enough to make 13% and 28% of the failed traces completely correct. The redirection is a diagnostic, not a way to improve accuracy: it uses the reference, and it replaces all the attention of the three heads, so at a position where the model was already right it can only keep or disturb the prediction. These results do not change the claim of Section 4.4, which is that the three heads contribute to the prediction that ends a name and that redirecting them repairs this prediction. App. E.1 of the submitted paper already stated that a repair is a local measure and does not imply a correct trace. In the revised paper, the results on complete identifiers are in Section 4.5 and App. E.3 (new Table 8), and the results on complete traces are in Section 4.4 and App. E.1 (new Table 5c).
W2, second paragraph (the conclusions should distinguish the mechanism examined here from the broader effects of diversity training). We agree. Our mechanistic analysis concerns the generation of identifiers only. The gains on misuse localization and on the cloze test are shown empirically, and we do not claim that the copy heads explain them. We have updated the conclusion of the revised paper to state this distinction.
Response Part 1
Thank you for the precise requests. Below, "baseline" is the model trained on the original code, and "high-diversity" is the model trained with our renaming. For the tracing model, "0.5% diversity" and "1% diversity" are the two lowest levels of name diversity, as in Table 5. Section and appendix numbers refer to the revised PDF.
Part 1
W1, first paragraph / Q1, first part (overlap between the two sets of repaired errors in Table 5; attention before intervention). The two sets overlap largely, and the overlap shows that both effects are present: a general signal to end the name, and a repair that needs the matching name. The submitted Table 5 gave only the number of repairs in each condition. During the discussion period, we ran the experiment of Table 5 again and recorded the outcome of each probe. The new run reproduces the matching-name and different-name counts of Table 5 exactly.