VFL: Visual Function Localization for Autoregressive Image Generation

In autoregressive image generation, visual functions refer to hypothesized internal processes that support behaviors such as introducing attributes and refining local details. These processes remain poorly understood. Circuit analysis explains how internal computations contribute to behavior in language models, but extending it to image generation presents several challenges. Visual concepts typically span multiple image tokens, attention heads may interact, and intervention effects must be evaluated on the final image. We introduce VFL (Visual Function Localization), a framework for studying these functions through activation interventions, i.e., controlled changes to internal activations. VFL first identifies image-token positions where interventions most clearly produce the behavior associated with each visual function. It then learns how strongly to intervene at each attention head at those positions. We evaluate VFL on four visual functions using single-object images with simple backgrounds. Across all four functions, the interventions produce visual changes consistent with the corresponding functional hypotheses. The learned masks assign weight to multiple attention heads rather than concentrating on a single head.