From Art to Engineering: An Architecture Evaluation Framework for Edge-Level Deep Graph Transformation

Deep graph transformation lacks controlled evaluation protocols for comparing architectures across structurally different transformation tasks. We introduce a controlled evaluation framework for edge-level deep graph transformation that places ten architectures and thirteen graph-computation and real-world tasks under a shared evaluation contract, yielding sixteen directly comparable experiments. The framework is open-source, provides reproducible baselines for analysing architecture–task interactions, together with controls over graph topology, feature provision and supervision redaction, and welcomes community contributions of new tasks, architectures, and evaluation settings. Across it, no architecture is consistently strongest, and three ablation studies show that changing the information supplied to the learner can change relative architectural performance while the transformation itself remains fixed. Dense architectures, which do not use the graph’s edges to determine which nodes exchange information, can also match or exceed graph-native ones, leading in eight of the sixteen experiments. We define a task’s information regime as the information conditions under which it is learned: what task-relevant information is supplied, the structural range over which it must be resolved, and whether predictions depend on one another. We use cross-task analysis to motivate the information regime as a fifth factor that extends an existing task categorisation framework.