Encoded Where It Is Drawn, Absent Where It Is Read: Relational Evidence in Scientific Diagrams

Scientific diagrams make claims about processes: a food web can depict every organism correctly yet misstate energy flow if an arrow is reversed. Image generators can produce plausible diagrams with incorrect structure. We study Qwen-Image and Stable Diffusion 3.5 on controlled diagrams spanning seven scientific systems. Across layouts, a target arrow is pixel-identical in both roles, while a remote arrow determines whether its chain continues. Using the activation change caused by editing the target, we test whether the remote arrow’s direction is decodable where drawn, at the target, and whether two arrows contribute when both determine the role. Under low-noise reversal edits on short chains, a Qwen-Image target-region probe achieves 99.6% accuracy when entire scientific systems are held out, versus 50.0% for a position-only baseline. Increasing the remote arrow’s separation preserves the signal, whereas seven of eight redraws largely remove it. Blocking each target region’s attention to its own lane’s remote arrowhead nearly eliminates it in three tested layouts; blocking the other lane’s, or a comparable control arrowhead, does not. In three other settings, direction is nearly perfectly decodable locally but weakly at the target. In a two-arrow task, the target readout favors the last arrow at most late blocks, although at one block pooled accuracy exceeds the best single-arrow accuracy. In Stable Diffusion 3.5 the corresponding estimate stays near zero. Accurate decoding alone therefore cannot identify which visual evidence supports a prediction or establish that all arrows defining a relation contribute.