From Similar Code to Neural Correspondence: Causal Interventions Guided by Program Structure

Large language models can generate code that closely resembles existing implementations, but output similarity alone does not explain how similar code is supported by the model's internal computation. We study whether program structure can help identify internal states that play corresponding causal roles across similar programs. We introduce Program Structured Causal Alignment (PSCA), a framework that uses program structure to select corresponding positions across programs and evaluates the transferred internal states through causal intervention. PSCA evaluates this correspondence in two complementary ways. Intact causal compatibility asks whether transfers selected by program structure better support code prediction than comparison transfers in an otherwise intact computation. Causal recovery asks whether they restore part of the lost prediction after disruption and achieve greater recovery than comparison transfers. During intact computation, transfers selected by program structure generally support prediction better than comparison transfers. Across three model families and nine fine-tuned models, data-flow-aligned restoration consistently improves prediction over both the disrupted condition and mismatch restoration at the prespecified intervention layers, while control-flow recovery varies across models and intervention settings. Together, these results provide evidence that program structure can guide the identification of internal states with meaningful functional effects on code prediction.