MINAR: Mechanistic Interpretability for Neural Algorithmic Reasoning

The recent field of neural algorithmic reasoning (NAR) studies the ability of graph neural networks (GNNs) to emulate classical algorithms like Bellman-Ford, a phenomenon known as algorithmic alignment. At the same time, recent advances in large language models (LLMs) have spawned the study of mechanistic interpretability, which aims to identify granular model components like circuits that perform specific computations. In this work, we demonstrate the potential of circuit discovery techniques in NAR through two case studies, focusing on neuron-level circuits from algorithmic reasoning GNNs. Our study sheds new light on the process of circuit formation and pruning during training, as well as giving new insight into how GNNs trained to perform multiple tasks in parallel reuse circuit components for related tasks. Our GNN circuit discovery implementation, **M**echanistic **I**nterpretability for **N**eural **A**lgorithmic **R**easoning (MINAR), is available at https://anonymous.4open.science/r/MINAR-ECD1/.