Pointer–Payload–Handoff: Tracing How Visual Evidence Reaches the Answer in Vision–Language Models

A vision-language model can attend to the right region and still answer incorrectly. Spatial grounding identifies where a model looks, not how the selected fact reaches the computation that chooses the answer. In Qwen3.5, the strongest spatial pointer even appears after the image-to-question handoff, and transplanting the grounded attention contribution itself fails to repair answers. We trace one answer-determining visual fact by asking four questions: where it is first causally available, which computation moves it downstream, where the transferred content is carried, and through which state it reaches the output. Pointer–Payload–Handoff (PPH) answers these questions with three complementary interventions—restoration, blocking, and semantic interchange—at coordinates frozen before held-out confirmation. Independent analyses in Qwen3.5, Qwen3-VL, and Gemma 3 recover the same ordered role graph, S → W → C → R, at architecture-specific depths. Writers range from one dominant attention component to distributed modules, yet every family certifies the directed links S → W, S → C, and C → R. The W → C adjacency holds in Qwen3.5-9B, where it is measurable. The frozen carrier is causally usable: replacing it with a state built from correct visual evidence repairs 19–35% of initially incorrect real-image answers across the three families, against near-zero matched controls. With the task-relevant option text present, the carrier mediates 58–89% of the source effect across the three families. Replacing that block with equal-length unrelated text reduces mediation by 54 points in each family; instructing the model to ignore the options leaves the point estimate close to the original. These controls isolate the effect of the option text, not the answer word within it. Interpreting a vision-language model therefore requires stating the prompt condition under which its mechanism was recovered.