- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces SIGMA, a retrieval task for graph sequences. A corpus sequence is relevant when its timesteps contain the query graphs in the right order, with the same node alignment across all matched timesteps. SIGMANET uses a GNN and causal transformers to represent each sequence. It then computes one shared node alignment and uses a staircase aligner to match timesteps. The authors build five benchmarks from reaction, event, and protein interaction data. SIGMANET has the best MAP on all five datasets in the main comparison, where the baselines use Soft-DTW.
Critical strengths
-
The problem is defined clearly. Definition 1 and Equations 1–6 explain why both the node mapping and the timestep mapping matter. Requiring one node mapping across time is reasonable when node identities persist.
-
The benchmarks cover several different kinds of graph sequences. Table 8 gives useful information about the query and corpus sets. The paper also says that code and data are provided as supplementary material.
-
The appendix contains useful additional experiments. It studies where to place the temporal transformer, shared versus separate node alignments, finite timestep gaps, a strict DP matcher, and runtime against an exact solver. Table 18 also shows that using one shared node alignment saves memory.
Critical weaknesses
-
Some ablation results are exactly the same to three decimal places on all five datasets. In Tables 11–14, “IsoNet + TFθ” has the same MAP and MRR values as “SIGMANET − SA”, while “IsoNet + SA” has the same values as “SIGMANET − TFθ”. If these configurations are actually identical, the paper should explain this relationship and state more clearly what SIGMANET adds to IsoNet. If they are different configurations, the authors should explain why the reported values coincide. This matters because the ablations are used to support the contribution of each component.
-
The main results may reflect the choice of sequence matcher. All baselines in Table 2 use Soft-DTW, which aligns full sequences and is not a natural choice for finding a contained subsequence. When the baselines use the proposed staircase aligner in Table 13, GEN + SA scores 0.822 on PaRoutes, slightly above SIGMANET’s 0.818. GraphSSM + SA scores 0.814 on FlowER, slightly above SIGMANET’s 0.812. These small differences need uncertainty estimates. The authors should make this comparison more central and consider a subsequence-DTW or DP matcher with representative baseline encoders. As it stands, I cannot tell how much of the main improvement comes from the graph encoder.
-
I am not fully convinced by the explanation of the alignment method. The paper starts from a joint optimization over node and timestep alignments, but the model first predicts the node alignment from the final node embeddings and only then computes the timestep alignment. The order penalty is produced by a CNN, rather than obtained by optimizing a dual variable. The paper already describes this as an approximation. In the main setting, (b_1=1) and (b_2=\infty), so the ordered timestep matching can be solved exactly for a given node alignment and pairwise costs by the DP in Appendix F.6. The no-λ model has MAP of 0.133 on PaRoutes and 0.144 on FlowER, while strict DP drops to 0.234 on FlowER (Tables 16 and 21). I would like to understand the large strict-DP drop on FlowER. More analysis of training and learned pairwise costs would help.
-
I could not find enough detail about how the ground-truth labels are produced, although Table 28 reports an exact SIGMA solver. RDKit and Glasgow solve graph matching problems for individual graph pairs, but SIGMA also requires one node alignment shared across the selected timesteps. The paper should explain how it checks this shared alignment and searches over ordered timestep choices. The main result tables also do not show variation across runs or splits, although Appendix F.3 reports paired tests over ten splits. This is especially relevant for PaRoutes, which has only 122 queries in total and roughly 30 test queries per split. MRR is already close to one for many methods on Ito and Uetz, so small MRR differences are hard to interpret. Finally, Table 9 counts only baseline encoder parameters, while Table 10 counts the whole SIGMANET model. Full model parameter counts and clearer information about baseline-specific tuning would make the comparison easier to judge.
Questions for authors
Please see above weaknesses.